Skip to content

Syntax

Assessment structure

The following excerpt shows the syntax of the ITL and designed tree structure of assessment. Symbol + represents the possibility of using one or several instructions/variables.

assessment+ "Assessment Title"
{
    metrics+ Metric_set_1
    {
        Metric_linear1+ = (v,a,b), if a<b
        Metric_linear2+ = (v,b,a), if b>a

        Metric_trapezoid1+ = trapezoid(v,a,b,c,d)
        Metric_trapezoid2+ = trapezoid("-",v,a,b,c,d)

        Metric_triangle1+ = triangle(v,a,b,c)
        Metric_triangle2+ = trinagle("-",v,a,b,c)

        Metric_gauss1+ = gauss(v,a,b)
        Metric_gauss2+ = gauss("-",v,a,b)

        Metric_gauss21+ = gauss2(v,a,b,c,d)
        Metric_gauss22+ = gauss2("-",v,a,b,c,d)

        Metric_sigmoid1+ = sigmoid(v,a,b)
        Metric_sigmoid2+ = sigmoid("-",v,a,b)   
    }

    # comment
    print Metric_set_1+;

    grade Metric_set_1;

    # Metric_linear1&2 currently supported
    draw metric Metric_1+ from Metric_set_1;

    grade comparative Metric_set_1, Metric_set_2;

    grade cumulative Metric_set_1+;

    grade singular Metric_set_1+;

    make excel report "Excel Report Title";
}

Syntax reference

assessment STRING

The assessment object creating instruction.

Type: Instruction
Identifier: @assess_obj
1st parent: None
1st child: @metrics_set
Parameters: STRING : string
Defines the assesment report title.
Returns: None
Limitations: None
assessment example

Code:

assessment "Example Report Title"
{    
    metrics Example_Metric_Set
    {
     }
}

Result:

> Code successufully executed.

metrics NAME_METRICS_SET

The metrics set object creating instruction.

Type: Instruction
Identifier: @metrics_set
1st parent: @assess_obj
1st child: @metric
Parameters: NAME_METRICS_SET : @var_metric_set
Metric set variable.
Returns: None
Limitations: None
metrics NAME example

Code:

assessment "Example Report Title"
{    
    metrics Example_Metric_Set
    {
     }
}

Result:

> Code successufully executed.

@var_metric_set

The @metrics_set variable containing the @metric_var instances.

Type: Variable
Identifier: @var_metric_set
1st parent: @metrics_set
1st child: None
Parameters: [@var_metric_set]
Any letter or digit combination excluding keywords and whitespace. Cannot start with the digit either.
Returns: None
Limitations: None
@var_metric_set example

Proper varible names: metrName, ebank1, Metr_1_ebank1

Improper variable names: 1MetricName,Metric name,metric

Code:

assessment "Example Report Title"
{    
 metrics Example_Metric_Set
  {
    example_metr_1 = (1,1,1)
    }
 print Example_Metric_Set;

 metrics Example_Metric_Set
  {
    example_metr_1 = (3,2,13)
    }
 print Example_Metric_Set;
}

Result:

    > Metric sets values:
    _____________________

    Metric: Example_Metric_Set
    Values: {'example_metr_1': {'type': 'linear', 'data': {0: Token(NUMBER, '1'), 1: Token(NUMBER, '1'), 2: Token(NUMBER, '1')}}}

    > Metric sets values:
    _____________________

    Metric: Example_Metric_Set
    Values: {'example_metr_1': {'type': 'linear', 'data': {0: Token(NUMBER, '3'), 1: Token(NUMBER, '2'), 2: Token(NUMBER, '13')}}}

    > Code successufully executed.

(v, a, b) - type 1

Rising slope linear fuzzy function - type 1.

Type: Function
Identifier: @linear1_fuz_f
1st parent: @metrics_set
1st child: None
Parameters: v : float
Value to be assessed by type 1 fuzzy function.
a : float
The worst performance of measured metric.Values below a (including a) will be given grade 0.
b : float
The best performance of measured metric. Values above b (including a) will be given grade 1.
Returns: @metric_obj
Limitations: a < b
Rising linear fuzzy example

Code:

assessment "Example Report Title"
{    
    metrics Example_Metric_Set
    {
      fuzzy_falling = (3.69,13,6)
     }

    grade Example_Metric_Set;
}

Result:

> Metric grade:
______________

Metric set: {'Example_Metric_Set': {'fuzzy_falling': 38.5}}
Grade: 38/100 points
Grade zone: red

> Code successufully executed.

(v, a, b) - type 2

Falling slope linear fuzzy function - type 2.

Type: Function
Identifier: @linear2_fuz_f
1st parent: @metrics_set
1st child: None
Parameters: v : float
Value to be assessed by type 2 fuzzy function.
a : float
The best performance of measured metric.Values below a (including a) will be given a grade 1.
b : float
The worst performance of measured metric. Values above b (including a) will be given a grade 0.
Returns: @metric_obj
Limitations: a > b
Falling linear fuzzy example

Code:

assessment "Example Report Title"
{    
    metrics Example_Metric_Set
    {
      fuzzy_rising = (3.69,0,13)
     }

    grade Example_Metric_Set;
}

Result:

> Metric grade:
______________

Metric set: {'Example_Metric_Set': {'fuzzy_rising': 36.077}}
Grade: 36/100 points
Grade zone: red

> Code successufully executed.

trapezoid(sign, v, a, b, c, d)

Trapezoidal fuzzy function.

Type: Function
Identifier: @trapezoid__fuz_f
1st parent: @metrics_set
1st child: None
Parameters: sign : string
Optional. Fuzzy function sign. Can be "-" or "+". Default value is "+"
v : float
Value to be assessed by trapezoidal fuzzy function.
a : float
The left side worst performance of measured metric. Values below a (including a) will be given grade 0.
b : float
Starting point of the best measured metric performance. Values between b and c will be given grade 1.
c : float
Ending point of the best measured metric performance.
d : float
The right side worst performance of measured metric. Values above d (including d) will be given a grade 0.
Returns: @metric_obj
Limitations: a < b < c < d
Type: Function
Identifier: @trapezoid__fuz_f
1st parent: @metrics_set
1st child: None
Parameters: sign = '-'
Fuzzy function sign.
v : float
Value to be assessed by trapezoidal fuzzy function.
a : float
The left side best performance of measured metric. Values below a (including a) will be given grade 1.
b : float
Starting point of the worst measured metric performance. Values between b and c will be given grade 0.
c : float
Ending point of the worst measured metric performance.
d : float
The right side best performance of measured metric. Values above d (including d) will be given a grade 1.
Returns: @metric_obj
Limitations: a < b < c < d
Trapezoidal fuzzy examples

Basic use:

assessment "Example Report Title"
{    
    metrics Example_Metric_Set
    {
    fuzzy_trap = trapezoid(3.69,1,6,9,11)
    }

    grade Example_Metric_Set;
}

Result:

> Metric grade:
______________

Metric set: {'Example_Metric_Set': {'trap': 53.8}}
Grade: 54/100 points
Grade zone: orange

> Code successufully executed.

Advanced use with defined sign parameter:

assessment "Example Report Title"
{    
    metrics Example_Metric_Set
    {
    fuzzy_trap = trapezoid("-",3.69,1,6,9,11)
    }

    grade Example_Metric_Set;
}

Result:

> Metric grade:
______________

Metric set: {'Example_Metric_Set': {'trap': 46.2}}
Grade: 46/100 points
Grade zone: orange

> Code successufully executed.

triangle(sign, v, a, b, c)

Triangular fuzzy function.

Type: Function
Identifier: @triangle__fuz_f
1st parent: @metrics_set
1st child: None
Parameters: sign : string
Optional. Fuzzy function sign. Can be "-" or "+". Default value is "+"
v : float
Value to be assessed by triangular fuzzy function.
a : float
The left side worst performance of measured metric. Values below a (including a) will be given grade 0.
b : float
The best performance of measured metric. Values equal to b will be given a grade 1.
c : float
The right side worst performance of measured metric. Values above c (including c) will be given a grade 0.
Returns: @metric_obj
Limitations: a < b < c
Type: Function
Identifier: @triangle__fuz_f
1st parent: @metrics_set
1st child: None
Parameters: sign = '-'
Fuzzy function sign.
v : float
Value to be assessed by triangular fuzzy function.
a : float
The left side best performance of measured metric. Values below a (including a) will be given grade 1.
b : float
The worst performance of measured metric. Values equal to b will be given a grade 0.
c : float
The right side best performance of measured metric. Values above c (including c) will be given a grade 1.
Returns: @metric_obj
Limitations: a < b < c
Triangular fuzzy examples

Basic use:

assessment "Example Report Title"
{    
    metrics Example_Metric_Set
    {
      fuzzy_tria = triangle(6.39,3,6,9)
    }

    grade Example_Metric_Set;
}

Result:

> Metric grade:
______________

Metric set: {'Example_Metric_Set': {'fuzzy_tria': 87.0}}
Grade: 87/100 points
Grade zone: green

> Code successufully executed.

Advanced use with defined sign parameter:

assessment "Example Report Title"
{    
    metrics Example_Metric_Set
    {
      fuzzy_tria = triangle("-",6.39,3,6,9)
    }

    grade Example_Metric_Set;
}

Result:

> Metric grade:
______________

Metric set: {'Example_Metric_Set': {'fuzzy_tria': 13.0}}
Grade: 13/100 points
Grade zone: red

> Code successufully executed.

gauss(sign, v, mean, sigma)

🧪 Experimental

Experimental Gaussian type 1 fuzzy function. For a given value calculates its closeness to the average value of the given (\(\mu\), \(\sigma\)) Gaussian distribution.

Type: Function
Identifier: @gauss2__fuz_f
1st parent: @metrics_set
1st child: None
Parameters: sign : string
Optional. Fuzzy function sign. Can be "-" or "+". Default value is "+"
v : float
Value to be assessed by gaussian fuzzy function.
mean : float
Parameter \(\mu\) of the given gaussian distribution.
sigma : float
Parameter \(\sigma\) of the given gaussian distribution.
Returns: @metric_obj
Limitations: None
Type: Function
Identifier: @gauss2_fuz_f
1st parent: @metrics_set
1st child: None
Parameters: sign = '-'
Fuzzy function sign.
v : float
Value to be assessed by gaussian fuzzy function.
mean : float
Parameter \(\mu\) of the given gaussian distribution.
sigma : float
Parameter \(\sigma\) of the given gaussian distribution.
Returns: @metric_obj
Limitations: None
Gaussian type 1 fuzzy examples

Basic use:

assessment "Example Report Title"
{    
    metrics Example_Metric_Set
    {
       fuzzy_gaus = gauss(3,6,3.36)
    }

    grade Example_Metric_Set;
}

Result:

> Metric grade:
______________

Metric set: {'Example_Metric_Set': {'fuzzy_gaus': 67.126}}
Grade: 67/100 points
Grade zone: orange

> Code successufully executed.

Advanced use with defined sign parameter:

assessment "Example Report Title"
{    
    metrics Example_Metric_Set
    {
      fuzzy_gaus = gauss("-",3,6,3.36)
    }

    grade Example_Metric_Set;
}

Result:

> Metric grade:
______________

Metric set: {'Example_Metric_Set': {'fuzzy_gaus': 32.874}}
Grade: 67/100 points
Grade zone: red

> Code successufully executed.

gauss2(sign, v, mean1, sigma1, mean2, sigma2)

🧪 Experimental

Experimental Gaussian type 2 fuzzy function. For a given value calculates its level of membership for the two combined (\(\mu_1\), \(\sigma_1\)), (\(\mu_2\), \(\sigma_2\)) Gaussian distributions.

Type: Function
Identifier: @gauss2_fuz_f
1st parent: @metrics_set
1st child: None
Parameters: sign : string
Optional. Fuzzy function sign. Can be "-" or "+". Default value is "+"
v : float
Value to be assessed by gaussian2 fuzzy function.
mean1 : float
Parameter \(\mu_1\) of the given gaussian distribution.
sigma : float
Parameter \(\sigma_1\) of the given gaussian distribution.
mean2 : float
Parameter \(\mu_2\) of the given gaussian distribution.
sigma2 : float
Parameter \(\sigma_2\) of the given gaussian distribution.
Returns: @metric_obj
Limitations: None
Type: Function
Identifier: @gauss2_fuz_f
1st parent: @metrics_set
1st child: None
Parameters: sign = '-'
Fuzzy function sign.
v : float
Value to be assessed by gaussian2 fuzzy function.
mean1 : float
Parameter \(\mu_1\) of the given gaussian distribution.
sigma : float
Parameter \(\sigma_1\) of the given gaussian distribution.
mean2 : float
Parameter \(\mu_2\) of the given gaussian distribution.
sigma2 : float
Parameter \(\sigma_2\) of the given gaussian distribution.
Returns: @metric_obj
Limitations: None
Gaussian type 2 fuzzy examples

Basic use:

assessment "Example Report Title"
{    
    metrics Example_Metric_Set
    {
       fuzzy_gaus2 = gauss2(4,6,3,8,3)
    }

    grade Example_Metric_Set;
}

Result:

> Metric grade:
______________

Metric set: {'Example_Metric_Set': {'fuzzy_gaus2': 80.074}}
Grade: 80/100 points
Grade zone: green

> Code successufully executed.

Advanced use with defined sign parameter:

assessment "Example Report Title"
{    
    metrics Example_Metric_Set
    {
      fuzzy_gaus2 = gauss2("-"4,6,3,8,3)
    }

    grade Example_Metric_Set;
}

Result:

> Metric grade:
______________

Metric set: {'Example_Metric_Set': {'fuzzy_gaus2': 19.926}}
Grade: 20/100 points
Grade zone: red

> Code successufully executed.

sigmoid(sign, v, center, width)

🧪 Experimental

Experimental Sigmoidal fuzzy function.

Type: Function
Identifier: @sigmoid_fuz_f
1st parent: @metrics_set
1st child: None
Parameters: sign : string
Optional. Fuzzy function sign. Can be "-" or "+". Default value is "+"
Parameters: v : float
Value to be assessed by sigmoidal fuzzy function.
center : float
The center value of the sigmoid, where it equals 0.5.
width : float
"Width" of the sigmoidal region about center. Defines also also which side of the function is open (sign). For width > 0 the left side approaches 0 and right side approaches 1. For width < 0 the right side approaches 1 and the lefts side approaches 0.
Returns: @metric_obj
Limitations: None
Type: Function
Identifier: @sigmoid_fuz_f
1st parent: @metrics_set
1st child: None
Parameters: sign = '-'
Fuzzy function sign.
Parameters: v : float
Value to be assessed by sigmoidal fuzzy function.
center : float
The center value of the sigmoid, where it equals 0.5.
width : float
"Width" of the sigmoidal region about center. Defines also also which side of the function is open (sign). For width < 0 the left side approaches 0 and right side approaches 1. For width > 0 the right side approaches 1 and the lefts side approaches 0.
Returns: @metric_obj
Limitations: None
Sigmoidal fuzzy examples

Basic use:

assessment "Example Report Title"
{    
    metrics Example_Metric_Set
    {
       fuzzy_sigmoid = sigmoid(6.93,6.9,9.6)
    }

    grade Example_Metric_Set;
}

Result:

> Metric grade:
______________

 Metric set: {'Example_Metric_Set': {'fuzzy_sigmoid': 67.252}}
 Grade: 67/100 points
 Grade zone: orange

> Code successufully executed.

Advanced use with defined sign parameter:

assessment "Example Report Title"
{    
    metrics Example_Metric_Set
    {
      fuzzy_sigmoid = sigmoid("-",6.93,6.9,9.6)
    }

    grade Example_Metric_Set;
}

Result:

> Metric grade:
______________

 Metric set: {'Example_Metric_Set': {'fuzzy_sigmoid': 32.748}}
 Grade: 33/100 points
 Grade zone: red

> Code successufully executed.

Prints the defined metrics raw data to the standard output and to the Excel report object.

Type: Instruction
Identifier: @print
1st parent: @assesment
1st child: None
Parameters: NAME_METRICS_SET : name
Name of the metrics set which metrics are going to be printed out. Can be any letter or digit combination excluding keywords and whitespace. Cannot start with the digit either.
Returns: None
Limitations: None
print example

Code:

assessment "Example Report Title"
{    
    metrics Example_Metric_Set
    {
        example_metric = (10,20,50)
     }
}       
print Example_Metric_Set;

Result:

> Metric sets values:
  ____________________

  Metric: Example_Metric_Set
  Values: {'example_metric': {'type': 'linear', 'data': {0: Token(NUMBER, '10'), 1: Token(NUMBER, '20'), 2: Token(NUMBER, '50')}}}
  > Code successufully executed.

grade NAME_METRICS_SET;

Calculates grade of the defined metric set.

Type: Instruction
Identifier: @grade
1st parent: @assesment
1st child: None
Parameters: NAME_METRICS_SET : name
Name of the metrics set to be graded. Can be any letter or digit combination excluding keywords and whitespace. Cannot start with the digit either.
Returns: None
Limitations: None
grade example

Code:

assessment "Example Report Title"
{    
    metrics Example_Metric_Set
    {
        example_metric = (35,20,50)
     }
}       
grade Example_Metric_Set;

Result:

> Metric grade:
______________

 Metric set: {'Example_Metric_Set': {'example_metric': 50.0}}
 Grade: 50/100 points
 Grade zone: orange

> Code successufully executed.

grade singular NAME_METRICS_SET+;

Individually grades defined metric sets. Prints out individual grades of all graded metrics as far as leaderboard containing descending list of metric set scores.

Type: Instruction
Identifier: @grade
1st parent: @assesment
1st child: None
Parameters: NAME_METRICS_SET+ : name
Name of the metrics set to be individually graded. Can be a list of any letter or digit combination excluding keywords and whitespace separated with comma. Cannot start with the digit either.
Returns: None
Limitations: None
grade singular example

Code:

assessment "Example Report Title"
{    
    metrics Example_Metric_Set_1
    {
        example_metric = (35,20,50)
     }
    metrics Example_Metric_Set_2
    {
        example_metric = (22,20,50)
     }
    metrics Example_Metric_Set_3
    {
        example_metric = (47,20,50)
     }      
    grade singular Example_Metric_Set_1,Example_Metric_Set_2,Example_Metric_Set_3;
}

Result:

> Individual grades:
  ___________________

  Metric set: Example_Metric_Set_1
  Grade: 50/100 points
  Grade zone: orange

  Metric set: Example_Metric_Set_2
  Grade: 3/100 points
  Grade zone: red

  Metric set: Example_Metric_Set_3
  Grade: 90/100 points
  Grade zone: green

 > Leaderboard:
 ____________________________________
 1. Metric set: ('Example_Metric_Set_3', 90.0)
 2. Metric set: ('Example_Metric_Set_1', 50.0)
 3. Metric set: ('Example_Metric_Set_2', 3.333)

 > Code successufully executed.

grade comparative NAME_METRICS_SET~2;

Grades two defined metric sets and compares their grades. Prints out additional information.

Type: Instruction
Identifier: @grade_com
1st parent: @assesment
1st child: None
Parameters: NAME_METRICS_SET : name
Name of the metrics set to be included for comparing the metrics set grades. Can be a list of any letter or digit combination excluding keywords and whitespace separated with comma. Cannot start with the digit either.
Returns: None
Limitations: None
grade comparative example

Code:

assessment "Example Report Title"
{    
    metrics Example_Metric_Set_1
    {
        example_metric = (35,20,50)
     }
    metrics Example_Metric_Set_2
    {
        example_metric = (26,20,50)
     }  
    grade comparative Example_Metric_Set_1,Example_Metric_Set_2;
}

Result:

> Comparative grade:
 ___________________

  Metric sets: {'Example_Metric_Set_1': {'example_metric': 50.0}, 'Example_Metric_Set_2': {'example_metric': 20.0}}

 > Result:
  The metric Example_Metric_Set_1 has 150% higer grade than the metric Example_Metric_Set_2.
  Grades: {'Example_Metric_Set_1': 50, 'Example_Metric_Set_2': 20}

 > Code successufully executed.

grade cumulative NAME_METRICS_SET+;

Cumulatively grades the list of defined metric sets. Prints out cumulative grade.

Type: Instruction
Identifier: @grade_cml
1st parent: @assesment
1st child: None
Parameters: NAME_METRICS_SET : name
Name of the metrics set to be included in the calculation of the average metric grade. List of any letter or digit combination excluding keywords and whitespace separated with comma. Cannot start with the digit either.
Returns: None
Limitations: None
grade cumulative example

Code:

assessment "Example Report Title"
{    
    metrics Example_Metric_Set_1
    {
        example_metric = (35,20,50)
     }
    metrics Example_Metric_Set_2
    {
        example_metric = (26,20,50)
     }  
    metrics Example_Metric_Set_3
    {
        example_metric = (47,20,50)
     }  
    grade cumulative Example_Metric_Set_1,Example_Metric_Set_2,Example_Metric_Set_3;
}

Result:

> Cumulative grade:
__________________

 Metric sets: {'Example_Metric_Set_1': {'example_metric': 50.0}, 'Example_Metric_Set_2': {'example_metric': 20.0}, 'Example_Metric_Set_3': {'example_metric': 90.0}}
 Grade: 53/100 points
 Grade zone: orange

> Code successufully executed.

draw metric NAME_METRIC+ from NAME_METRICS_SET;

Draws a membership function graph of the selected metrics from the defined metric set. Also, visually displays the positon of the assessed value.

Type: Instruction
Identifier: @grade_drw
1st parent: @assesment
1st child: None
Parameters: NAME_METRIC : name
The name of a metric from the set of metrics to be drawn. Can be a list of any letter or digit combination excluding keywords and whitespace separated with comma. Cannot start with the digit either.
NAME_METRICS_SET : name
Name of the metrics set from wich the metrcis are going to be drawn. Can be any letter or digit combination excluding keywords and whitespace separated with comma. Cannot start with the digit either.
Returns: None
Limitations: Before executing this instruction @metric_set has to be graded first (grade NAME_METRICS_SET;).
draw metric example

Code:

assessment "Example Report Title"
{    
    metrics Example_Metric_Set_1
    {
        example_metric_1 = (35,20,50)
        example_metric_2 = triangle(47,20,50,100)
     }

    grade Example_Metric_Set_1;
    draw metric example_metric_1 from Example_Metric_Set_1;         
}

Result:

> Metric grade:
______________

 Metric set: {'Example_Metric_Set_1': {'example_metric_1': 50.0, 'example_metric_2': 90.0}}
 Grade: 70/100 points
 Grade zone: green

Drawing metrics: Example_Metric_Set_1

example_metric_1 50.0

> Code successufully executed.

make excel report STRING;

Creates a MS Excel .xlsx document containing assessment data and results.

Type: Instruction
Identifier: @make_rep_exc
1st parent: @assesment
1st child: None
Parameters: STRING : string
Specifies a name of the MS Excel report document.
Returns: None
Limitations: Before executing this instruction ethier one or multiple following instructions have to be executed first:
  • grade singular NAME_METRICS_SET
  • grade comparative NAME_METRICS_SET~2
  • grade cumulative NAME_METRICS_SET+
make excel report example

Code:

assessment "Example Report Title"
{    
    metrics Example_Metric_Set_1
    {
        example_metric_1 = (35,20,50)
        example_metric_2 = triangle(47,20,50,100)
     }

     metrics Example_Metric_Set_2
    {
        example_metric_1 = (45,20,50)
        example_metric_2 = triangle(32,20,50,100)
     }

    grade cumulative Example_Metric_Set_1,Example_Metric_Set_2;
    make excel report "Example_Excel_Report";           
}

Result:

> Cumulative grade:
__________________

 Metric sets: {'Example_Metric_Set_1': {'example_metric_1': 50.0, 'example_metric_2': 90.0}, 'Example_Metric_Set_2': {'example_metric_1': 83.333, 'example_metric_2': 40.0}}
 Grade: 66/100 points
 Grade zone: orange

> Code successufully executed.

Notice

The development of the documentation section is in progress.