Syntax
Assessment structure
The following excerpt shows the syntax of the ITL and designed tree structure of assessment. Symbol + represents the possibility of using one or several instructions/variables.
assessment+ "Assessment Title"
{
metrics+ Metric_set_1
{
Metric_linear1+ = (v,a,b), if a<b
Metric_linear2+ = (v,b,a), if b>a
Metric_trapezoid1+ = trapezoid(v,a,b,c,d)
Metric_trapezoid2+ = trapezoid("-",v,a,b,c,d)
Metric_triangle1+ = triangle(v,a,b,c)
Metric_triangle2+ = trinagle("-",v,a,b,c)
Metric_gauss1+ = gauss(v,a,b)
Metric_gauss2+ = gauss("-",v,a,b)
Metric_gauss21+ = gauss2(v,a,b,c,d)
Metric_gauss22+ = gauss2("-",v,a,b,c,d)
Metric_sigmoid1+ = sigmoid(v,a,b)
Metric_sigmoid2+ = sigmoid("-",v,a,b)
}
# comment
print Metric_set_1+;
grade Metric_set_1;
# Metric_linear1&2 currently supported
draw metric Metric_1+ from Metric_set_1;
grade comparative Metric_set_1, Metric_set_2;
grade cumulative Metric_set_1+;
grade singular Metric_set_1+;
make excel report "Excel Report Title";
}
Syntax reference
assessment STRING
The assessment object creating instruction.
Type: | Instruction |
Identifier: | @assess_obj |
1st parent: | None |
1st child: | @metrics_set |
Parameters: | STRING : string |
Defines the assesment report title. | |
Returns: | None |
Limitations: | None |
assessment example
Code:
assessment "Example Report Title"
{
metrics Example_Metric_Set
{
}
}
Result:
> Code successufully executed.
metrics NAME_METRICS_SET
The metrics set object creating instruction.
Type: | Instruction |
Identifier: | @metrics_set |
1st parent: | @assess_obj |
1st child: | @metric |
Parameters: | NAME_METRICS_SET : @var_metric_set |
Metric set variable. | |
Returns: | None |
Limitations: | None |
metrics NAME example
Code:
assessment "Example Report Title"
{
metrics Example_Metric_Set
{
}
}
Result:
> Code successufully executed.
@var_metric_set
The @metrics_set variable containing the @metric_var instances.
Type: | Variable |
Identifier: | @var_metric_set |
1st parent: | @metrics_set |
1st child: | None |
Parameters: | [@var_metric_set] |
Any letter or digit combination excluding keywords and whitespace. Cannot start with the digit either. | |
Returns: | None |
Limitations: | None |
@var_metric_set example
Proper varible names: metrName
, ebank1
, Metr_1_ebank1
Improper variable names: 1MetricName
,Metric name
,metric
Code:
assessment "Example Report Title"
{
metrics Example_Metric_Set
{
example_metr_1 = (1,1,1)
}
print Example_Metric_Set;
metrics Example_Metric_Set
{
example_metr_1 = (3,2,13)
}
print Example_Metric_Set;
}
Result:
> Metric sets values:
_____________________
Metric: Example_Metric_Set
Values: {'example_metr_1': {'type': 'linear', 'data': {0: Token(NUMBER, '1'), 1: Token(NUMBER, '1'), 2: Token(NUMBER, '1')}}}
> Metric sets values:
_____________________
Metric: Example_Metric_Set
Values: {'example_metr_1': {'type': 'linear', 'data': {0: Token(NUMBER, '3'), 1: Token(NUMBER, '2'), 2: Token(NUMBER, '13')}}}
> Code successufully executed.
(v, a, b) - type 1
Rising slope linear fuzzy function - type 1.
Type: | Function |
Identifier: | @linear1_fuz_f |
1st parent: | @metrics_set |
1st child: | None |
Parameters: | v : float |
Value to be assessed by type 1 fuzzy function. | |
a : float | |
The worst performance of measured metric.Values below a (including a) will be given grade 0. | |
b : float | |
The best performance of measured metric. Values above b (including a) will be given grade 1. | |
Returns: | @metric_obj |
Limitations: | a < b |
Rising linear fuzzy example
Code:
assessment "Example Report Title"
{
metrics Example_Metric_Set
{
fuzzy_falling = (3.69,13,6)
}
grade Example_Metric_Set;
}
Result:
> Metric grade:
______________
Metric set: {'Example_Metric_Set': {'fuzzy_falling': 38.5}}
Grade: 38/100 points
Grade zone: red
> Code successufully executed.
(v, a, b) - type 2
Falling slope linear fuzzy function - type 2.
Type: | Function |
Identifier: | @linear2_fuz_f |
1st parent: | @metrics_set |
1st child: | None |
Parameters: | v : float |
Value to be assessed by type 2 fuzzy function. | |
a : float | |
The best performance of measured metric.Values below a (including a) will be given a grade 1. | |
b : float | |
The worst performance of measured metric. Values above b (including a) will be given a grade 0. | |
Returns: | @metric_obj |
Limitations: | a > b |
Falling linear fuzzy example
Code:
assessment "Example Report Title"
{
metrics Example_Metric_Set
{
fuzzy_rising = (3.69,0,13)
}
grade Example_Metric_Set;
}
Result:
> Metric grade:
______________
Metric set: {'Example_Metric_Set': {'fuzzy_rising': 36.077}}
Grade: 36/100 points
Grade zone: red
> Code successufully executed.
trapezoid(sign, v, a, b, c, d)
Trapezoidal fuzzy function.
Type: | Function |
Identifier: | @trapezoid__fuz_f |
1st parent: | @metrics_set |
1st child: | None |
Parameters: | sign : string |
Optional. Fuzzy function sign. Can be "-" or "+". Default value is "+" | |
v : float | |
Value to be assessed by trapezoidal fuzzy function. | |
a : float | |
The left side worst performance of measured metric. Values below a (including a) will be given grade 0. | |
b : float | |
Starting point of the best measured metric performance. Values between b and c will be given grade 1. | |
c : float | |
Ending point of the best measured metric performance. | |
d : float | |
The right side worst performance of measured metric. Values above d (including d) will be given a grade 0. | |
Returns: | @metric_obj |
Limitations: | a < b < c < d |
Type: | Function |
Identifier: | @trapezoid__fuz_f |
1st parent: | @metrics_set |
1st child: | None |
Parameters: | sign = '-' |
Fuzzy function sign. | |
v : float | |
Value to be assessed by trapezoidal fuzzy function. | |
a : float | |
The left side best performance of measured metric. Values below a (including a) will be given grade 1. | |
b : float | |
Starting point of the worst measured metric performance. Values between b and c will be given grade 0. | |
c : float | |
Ending point of the worst measured metric performance. | |
d : float | |
The right side best performance of measured metric. Values above d (including d) will be given a grade 1. | |
Returns: | @metric_obj |
Limitations: | a < b < c < d |
Trapezoidal fuzzy examples
Basic use:
assessment "Example Report Title"
{
metrics Example_Metric_Set
{
fuzzy_trap = trapezoid(3.69,1,6,9,11)
}
grade Example_Metric_Set;
}
Result:
> Metric grade:
______________
Metric set: {'Example_Metric_Set': {'trap': 53.8}}
Grade: 54/100 points
Grade zone: orange
> Code successufully executed.
Advanced use with defined sign parameter:
assessment "Example Report Title"
{
metrics Example_Metric_Set
{
fuzzy_trap = trapezoid("-",3.69,1,6,9,11)
}
grade Example_Metric_Set;
}
Result:
> Metric grade:
______________
Metric set: {'Example_Metric_Set': {'trap': 46.2}}
Grade: 46/100 points
Grade zone: orange
> Code successufully executed.
triangle(sign, v, a, b, c)
Triangular fuzzy function.
Type: | Function |
Identifier: | @triangle__fuz_f |
1st parent: | @metrics_set |
1st child: | None |
Parameters: | sign : string |
Optional. Fuzzy function sign. Can be "-" or "+". Default value is "+" | |
v : float | |
Value to be assessed by triangular fuzzy function. | |
a : float | |
The left side worst performance of measured metric. Values below a (including a) will be given grade 0. | |
b : float | |
The best performance of measured metric. Values equal to b will be given a grade 1. | |
c : float | |
The right side worst performance of measured metric. Values above c (including c) will be given a grade 0. | |
Returns: | @metric_obj |
Limitations: | a < b < c |
Type: | Function |
Identifier: | @triangle__fuz_f |
1st parent: | @metrics_set |
1st child: | None |
Parameters: | sign = '-' |
Fuzzy function sign. | |
v : float | |
Value to be assessed by triangular fuzzy function. | |
a : float | |
The left side best performance of measured metric. Values below a (including a) will be given grade 1. | |
b : float | |
The worst performance of measured metric. Values equal to b will be given a grade 0. | |
c : float | |
The right side best performance of measured metric. Values above c (including c) will be given a grade 1. | |
Returns: | @metric_obj |
Limitations: | a < b < c |
Triangular fuzzy examples
Basic use:
assessment "Example Report Title"
{
metrics Example_Metric_Set
{
fuzzy_tria = triangle(6.39,3,6,9)
}
grade Example_Metric_Set;
}
Result:
> Metric grade:
______________
Metric set: {'Example_Metric_Set': {'fuzzy_tria': 87.0}}
Grade: 87/100 points
Grade zone: green
> Code successufully executed.
Advanced use with defined sign parameter:
assessment "Example Report Title"
{
metrics Example_Metric_Set
{
fuzzy_tria = triangle("-",6.39,3,6,9)
}
grade Example_Metric_Set;
}
Result:
> Metric grade:
______________
Metric set: {'Example_Metric_Set': {'fuzzy_tria': 13.0}}
Grade: 13/100 points
Grade zone: red
> Code successufully executed.
gauss(sign, v, mean, sigma)
Experimental
Experimental Gaussian type 1 fuzzy function. For a given value calculates its closeness to the average value of the given (\(\mu\), \(\sigma\)) Gaussian distribution.
Type: | Function |
Identifier: | @gauss2__fuz_f |
1st parent: | @metrics_set |
1st child: | None |
Parameters: | sign : string |
Optional. Fuzzy function sign. Can be "-" or "+". Default value is "+" | |
v : float | |
Value to be assessed by gaussian fuzzy function. | |
mean : float | |
Parameter \(\mu\) of the given gaussian distribution. | |
sigma : float | |
Parameter \(\sigma\) of the given gaussian distribution. | |
Returns: | @metric_obj |
Limitations: | None |
Type: | Function |
Identifier: | @gauss2_fuz_f |
1st parent: | @metrics_set |
1st child: | None |
Parameters: | sign = '-' |
Fuzzy function sign. | |
v : float | |
Value to be assessed by gaussian fuzzy function. | |
mean : float | |
Parameter \(\mu\) of the given gaussian distribution. | |
sigma : float | |
Parameter \(\sigma\) of the given gaussian distribution. | |
Returns: | @metric_obj |
Limitations: | None |
Gaussian type 1 fuzzy examples
Basic use:
assessment "Example Report Title"
{
metrics Example_Metric_Set
{
fuzzy_gaus = gauss(3,6,3.36)
}
grade Example_Metric_Set;
}
Result:
> Metric grade:
______________
Metric set: {'Example_Metric_Set': {'fuzzy_gaus': 67.126}}
Grade: 67/100 points
Grade zone: orange
> Code successufully executed.
Advanced use with defined sign parameter:
assessment "Example Report Title"
{
metrics Example_Metric_Set
{
fuzzy_gaus = gauss("-",3,6,3.36)
}
grade Example_Metric_Set;
}
Result:
> Metric grade:
______________
Metric set: {'Example_Metric_Set': {'fuzzy_gaus': 32.874}}
Grade: 67/100 points
Grade zone: red
> Code successufully executed.
gauss2(sign, v, mean1, sigma1, mean2, sigma2)
Experimental
Experimental Gaussian type 2 fuzzy function. For a given value calculates its level of membership for the two combined (\(\mu_1\), \(\sigma_1\)), (\(\mu_2\), \(\sigma_2\)) Gaussian distributions.
Type: | Function |
Identifier: | @gauss2_fuz_f |
1st parent: | @metrics_set |
1st child: | None |
Parameters: | sign : string |
Optional. Fuzzy function sign. Can be "-" or "+". Default value is "+" | |
v : float | |
Value to be assessed by gaussian2 fuzzy function. | |
mean1 : float | |
Parameter \(\mu_1\) of the given gaussian distribution. | |
sigma : float | |
Parameter \(\sigma_1\) of the given gaussian distribution. | |
mean2 : float | |
Parameter \(\mu_2\) of the given gaussian distribution. | |
sigma2 : float | |
Parameter \(\sigma_2\) of the given gaussian distribution. | |
Returns: | @metric_obj |
Limitations: | None |
Type: | Function |
Identifier: | @gauss2_fuz_f |
1st parent: | @metrics_set |
1st child: | None |
Parameters: | sign = '-' |
Fuzzy function sign. | |
v : float | |
Value to be assessed by gaussian2 fuzzy function. | |
mean1 : float | |
Parameter \(\mu_1\) of the given gaussian distribution. | |
sigma : float | |
Parameter \(\sigma_1\) of the given gaussian distribution. | |
mean2 : float | |
Parameter \(\mu_2\) of the given gaussian distribution. | |
sigma2 : float | |
Parameter \(\sigma_2\) of the given gaussian distribution. | |
Returns: | @metric_obj |
Limitations: | None |
Gaussian type 2 fuzzy examples
Basic use:
assessment "Example Report Title"
{
metrics Example_Metric_Set
{
fuzzy_gaus2 = gauss2(4,6,3,8,3)
}
grade Example_Metric_Set;
}
Result:
> Metric grade:
______________
Metric set: {'Example_Metric_Set': {'fuzzy_gaus2': 80.074}}
Grade: 80/100 points
Grade zone: green
> Code successufully executed.
Advanced use with defined sign parameter:
assessment "Example Report Title"
{
metrics Example_Metric_Set
{
fuzzy_gaus2 = gauss2("-"4,6,3,8,3)
}
grade Example_Metric_Set;
}
Result:
> Metric grade:
______________
Metric set: {'Example_Metric_Set': {'fuzzy_gaus2': 19.926}}
Grade: 20/100 points
Grade zone: red
> Code successufully executed.
sigmoid(sign, v, center, width)
Experimental
Experimental Sigmoidal fuzzy function.
Type: | Function |
Identifier: | @sigmoid_fuz_f |
1st parent: | @metrics_set |
1st child: | None |
Parameters: | sign : string |
Optional. Fuzzy function sign. Can be "-" or "+". Default value is "+" | |
Parameters: | v : float |
Value to be assessed by sigmoidal fuzzy function. | |
center : float | |
The center value of the sigmoid, where it equals 0.5. | |
width : float | |
"Width" of the sigmoidal region about center. Defines also also which side of the function is open (sign). For width > 0 the left side approaches 0 and right side approaches 1. For width < 0 the right side approaches 1 and the lefts side approaches 0. | |
Returns: | @metric_obj |
Limitations: | None |
Type: | Function |
Identifier: | @sigmoid_fuz_f |
1st parent: | @metrics_set |
1st child: | None |
Parameters: | sign = '-' |
Fuzzy function sign. | |
Parameters: | v : float |
Value to be assessed by sigmoidal fuzzy function. | |
center : float | |
The center value of the sigmoid, where it equals 0.5. | |
width : float | |
"Width" of the sigmoidal region about center. Defines also also which side of the function is open (sign). For width < 0 the left side approaches 0 and right side approaches 1. For width > 0 the right side approaches 1 and the lefts side approaches 0. | |
Returns: | @metric_obj |
Limitations: | None |
Sigmoidal fuzzy examples
Basic use:
assessment "Example Report Title"
{
metrics Example_Metric_Set
{
fuzzy_sigmoid = sigmoid(6.93,6.9,9.6)
}
grade Example_Metric_Set;
}
Result:
> Metric grade:
______________
Metric set: {'Example_Metric_Set': {'fuzzy_sigmoid': 67.252}}
Grade: 67/100 points
Grade zone: orange
> Code successufully executed.
Advanced use with defined sign parameter:
assessment "Example Report Title"
{
metrics Example_Metric_Set
{
fuzzy_sigmoid = sigmoid("-",6.93,6.9,9.6)
}
grade Example_Metric_Set;
}
Result:
> Metric grade:
______________
Metric set: {'Example_Metric_Set': {'fuzzy_sigmoid': 32.748}}
Grade: 33/100 points
Grade zone: red
> Code successufully executed.
print NAME_METRICS_SET;
Prints the defined metrics raw data to the standard output and to the Excel report object.
Type: | Instruction |
Identifier: | |
1st parent: | @assesment |
1st child: | None |
Parameters: | NAME_METRICS_SET : name |
Name of the metrics set which metrics are going to be printed out. Can be any letter or digit combination excluding keywords and whitespace. Cannot start with the digit either. | |
Returns: | None |
Limitations: | None |
print example
Code:
assessment "Example Report Title"
{
metrics Example_Metric_Set
{
example_metric = (10,20,50)
}
}
print Example_Metric_Set;
Result:
> Metric sets values:
____________________
Metric: Example_Metric_Set
Values: {'example_metric': {'type': 'linear', 'data': {0: Token(NUMBER, '10'), 1: Token(NUMBER, '20'), 2: Token(NUMBER, '50')}}}
> Code successufully executed.
grade NAME_METRICS_SET;
Calculates grade of the defined metric set.
Type: | Instruction |
Identifier: | @grade |
1st parent: | @assesment |
1st child: | None |
Parameters: | NAME_METRICS_SET : name |
Name of the metrics set to be graded. Can be any letter or digit combination excluding keywords and whitespace. Cannot start with the digit either. | |
Returns: | None |
Limitations: | None |
grade example
Code:
assessment "Example Report Title"
{
metrics Example_Metric_Set
{
example_metric = (35,20,50)
}
}
grade Example_Metric_Set;
Result:
> Metric grade:
______________
Metric set: {'Example_Metric_Set': {'example_metric': 50.0}}
Grade: 50/100 points
Grade zone: orange
> Code successufully executed.
grade singular NAME_METRICS_SET+;
Individually grades defined metric sets. Prints out individual grades of all graded metrics as far as leaderboard containing descending list of metric set scores.
Type: | Instruction |
Identifier: | @grade |
1st parent: | @assesment |
1st child: | None |
Parameters: | NAME_METRICS_SET+ : name |
Name of the metrics set to be individually graded. Can be a list of any letter or digit combination excluding keywords and whitespace separated with comma. Cannot start with the digit either. | |
Returns: | None |
Limitations: | None |
grade singular example
Code:
assessment "Example Report Title"
{
metrics Example_Metric_Set_1
{
example_metric = (35,20,50)
}
metrics Example_Metric_Set_2
{
example_metric = (22,20,50)
}
metrics Example_Metric_Set_3
{
example_metric = (47,20,50)
}
grade singular Example_Metric_Set_1,Example_Metric_Set_2,Example_Metric_Set_3;
}
Result:
> Individual grades:
___________________
Metric set: Example_Metric_Set_1
Grade: 50/100 points
Grade zone: orange
Metric set: Example_Metric_Set_2
Grade: 3/100 points
Grade zone: red
Metric set: Example_Metric_Set_3
Grade: 90/100 points
Grade zone: green
> Leaderboard:
____________________________________
1. Metric set: ('Example_Metric_Set_3', 90.0)
2. Metric set: ('Example_Metric_Set_1', 50.0)
3. Metric set: ('Example_Metric_Set_2', 3.333)
> Code successufully executed.
grade comparative NAME_METRICS_SET~2;
Grades two defined metric sets and compares their grades. Prints out additional information.
Type: | Instruction |
Identifier: | @grade_com |
1st parent: | @assesment |
1st child: | None |
Parameters: | NAME_METRICS_SET : name |
Name of the metrics set to be included for comparing the metrics set grades. Can be a list of any letter or digit combination excluding keywords and whitespace separated with comma. Cannot start with the digit either. | |
Returns: | None |
Limitations: | None |
grade comparative example
Code:
assessment "Example Report Title"
{
metrics Example_Metric_Set_1
{
example_metric = (35,20,50)
}
metrics Example_Metric_Set_2
{
example_metric = (26,20,50)
}
grade comparative Example_Metric_Set_1,Example_Metric_Set_2;
}
Result:
> Comparative grade:
___________________
Metric sets: {'Example_Metric_Set_1': {'example_metric': 50.0}, 'Example_Metric_Set_2': {'example_metric': 20.0}}
> Result:
The metric Example_Metric_Set_1 has 150% higer grade than the metric Example_Metric_Set_2.
Grades: {'Example_Metric_Set_1': 50, 'Example_Metric_Set_2': 20}
> Code successufully executed.
grade cumulative NAME_METRICS_SET+;
Cumulatively grades the list of defined metric sets. Prints out cumulative grade.
Type: | Instruction |
Identifier: | @grade_cml |
1st parent: | @assesment |
1st child: | None |
Parameters: | NAME_METRICS_SET : name |
Name of the metrics set to be included in the calculation of the average metric grade. List of any letter or digit combination excluding keywords and whitespace separated with comma. Cannot start with the digit either. | |
Returns: | None |
Limitations: | None |
grade cumulative example
Code:
assessment "Example Report Title"
{
metrics Example_Metric_Set_1
{
example_metric = (35,20,50)
}
metrics Example_Metric_Set_2
{
example_metric = (26,20,50)
}
metrics Example_Metric_Set_3
{
example_metric = (47,20,50)
}
grade cumulative Example_Metric_Set_1,Example_Metric_Set_2,Example_Metric_Set_3;
}
Result:
> Cumulative grade:
__________________
Metric sets: {'Example_Metric_Set_1': {'example_metric': 50.0}, 'Example_Metric_Set_2': {'example_metric': 20.0}, 'Example_Metric_Set_3': {'example_metric': 90.0}}
Grade: 53/100 points
Grade zone: orange
> Code successufully executed.
draw metric NAME_METRIC+ from NAME_METRICS_SET;
Draws a membership function graph of the selected metrics from the defined metric set. Also, visually displays the positon of the assessed value.
Type: | Instruction |
Identifier: | @grade_drw |
1st parent: | @assesment |
1st child: | None |
Parameters: | NAME_METRIC : name |
The name of a metric from the set of metrics to be drawn. Can be a list of any letter or digit combination excluding keywords and whitespace separated with comma. Cannot start with the digit either. | |
NAME_METRICS_SET : name | |
Name of the metrics set from wich the metrcis are going to be drawn. Can be any letter or digit combination excluding keywords and whitespace separated with comma. Cannot start with the digit either. | |
Returns: | None |
Limitations: | Before executing this instruction @metric_set has to be graded first (grade NAME_METRICS_SET;). |
draw metric example
Code:
assessment "Example Report Title"
{
metrics Example_Metric_Set_1
{
example_metric_1 = (35,20,50)
example_metric_2 = triangle(47,20,50,100)
}
grade Example_Metric_Set_1;
draw metric example_metric_1 from Example_Metric_Set_1;
}
Result:
> Metric grade:
______________
Metric set: {'Example_Metric_Set_1': {'example_metric_1': 50.0, 'example_metric_2': 90.0}}
Grade: 70/100 points
Grade zone: green
Drawing metrics: Example_Metric_Set_1
example_metric_1 50.0
> Code successufully executed.
make excel report STRING;
Creates a MS Excel .xlsx document containing assessment data and results.
Type: | Instruction |
Identifier: | @make_rep_exc |
1st parent: | @assesment |
1st child: | None |
Parameters: | STRING : string |
Specifies a name of the MS Excel report document. | |
Returns: | None |
Limitations: | Before executing this instruction ethier one or multiple following instructions have to be executed first:
|
make excel report example
Code:
assessment "Example Report Title"
{
metrics Example_Metric_Set_1
{
example_metric_1 = (35,20,50)
example_metric_2 = triangle(47,20,50,100)
}
metrics Example_Metric_Set_2
{
example_metric_1 = (45,20,50)
example_metric_2 = triangle(32,20,50,100)
}
grade cumulative Example_Metric_Set_1,Example_Metric_Set_2;
make excel report "Example_Excel_Report";
}
Result:
> Cumulative grade:
__________________
Metric sets: {'Example_Metric_Set_1': {'example_metric_1': 50.0, 'example_metric_2': 90.0}, 'Example_Metric_Set_2': {'example_metric_1': 83.333, 'example_metric_2': 40.0}}
Grade: 66/100 points
Grade zone: orange
> Code successufully executed.