Explainability Formats
Image Classification
Modzy provides full support for explainability in image classification models. All that is necessary is that a model container returns the following information in the JSON object accompanying the results object:
"explanation": {
"maskRLE": [],
"dimensions": {}
}
In this explanation object, maskRLE refers to the column-major order (Fortran order) Run-Length Encoded (RLE) 2-dimensional binary mask (the size of the original input image) containing 1s in the region(s) within the image which were deemed by the employed explainability technique to be the most important in the model's prediction and 0s everywhere else. The dimensions object should contain the height and width of the original image in pixels.
Here is an example of the full model output containing explainability results from our Image Classification: Explainable model:
{
"data": {
"drift": null,
"explanation": {
"dimensions": {
"height": 426,
"width": 640
},
"maskRLE": [
[
156402,
6,
420,
6,
420,
6,
410,
16,
410,
16,
410,
16,
408,
20,
406,
20,
406,
20,
399,
27,
399,
27,
399,
27,
397,
29,
397,
29,
389,
37,
389,
37,
389,
37,
384,
42,
384,
42,
384,
42,
382,
47,
379,
47,
379,
47,
364,
3,
10,
49,
364,
3,
10,
49,
364,
3,
10,
49,
364,
3,
6,
63,
354,
3,
6,
63,
354,
3,
6,
63,
350,
7,
6,
63,
350,
7,
6,
63,
350,
7,
6,
63,
352,
81,
345,
81,
345,
81,
54,
2,
289,
81,
54,
2,
289,
81,
54,
2,
291,
78,
20,
16,
15,
13,
8,
6,
270,
78,
20,
16,
15,
13,
8,
6,
270,
78,
20,
16,
15,
13,
8,
6,
270,
78,
20,
16,
15,
30,
6,
4,
257,
78,
20,
16,
15,
30,
6,
4,
257,
78,
20,
16,
15,
30,
6,
4,
260,
84,
13,
16,
9,
46,
258,
84,
13,
16,
9,
46,
258,
84,
13,
16,
9,
46,
258,
84,
13,
19,
6,
42,
262,
84,
13,
19,
6,
42,
262,
84,
13,
19,
6,
42,
264,
82,
11,
69,
264,
82,
11,
69,
264,
82,
11,
69,
264,
166,
260,
166,
264,
167,
259,
167,
259,
167,
267,
161,
265,
161,
265,
161,
267,
159,
267,
159,
267,
159,
270,
158,
268,
158,
268,
158,
272,
158,
268,
158,
268,
158,
268,
164,
262,
164,
262,
164,
262,
164,
262,
164,
262,
164,
262,
164,
262,
164,
262,
168,
258,
168,
258,
168,
258,
169,
257,
169,
257,
169,
257,
170,
256,
171,
255,
171,
255,
171,
255,
171,
255,
171,
255,
171,
255,
171,
255,
171,
259,
168,
258,
169,
257,
169,
257,
169,
257,
169,
257,
169,
257,
169,
257,
169,
257,
169,
257,
169,
257,
169,
257,
169,
257,
169,
257,
169,
257,
169,
257,
169,
257,
169,
257,
169,
257,
169,
257,
169,
255,
171,
255,
171,
255,
173,
253,
173,
253,
173,
255,
167,
259,
167,
259,
167,
261,
163,
263,
163,
263,
163,
263,
162,
264,
162,
264,
162,
262,
164,
262,
164,
262,
164,
264,
160,
266,
160,
266,
160,
266,
160,
266,
160,
266,
163,
263,
163,
263,
163,
265,
163,
263,
163,
263,
163,
263,
161,
265,
161,
265,
161,
265,
161,
265,
161,
265,
161,
267,
159,
267,
159,
267,
159,
274,
151,
275,
151,
275,
151,
275,
52,
5,
80,
289,
52,
5,
80,
289,
33,
5,
8,
15,
25,
9,
4,
6,
32,
289,
33,
5,
8,
15,
25,
9,
4,
6,
32,
289,
33,
5,
8,
15,
25,
9,
4,
6,
32,
287,
33,
9,
6,
15,
17,
27,
32,
287,
33,
9,
6,
15,
17,
27,
32,
287,
33,
9,
6,
15,
17,
27,
32,
291,
2,
19,
4,
76,
32,
293,
2,
19,
4,
76,
32,
293,
2,
19,
4,
76,
32,
404,
19,
407,
19,
407,
19,
329,
1,
77,
3,
10,
4,
331,
1,
77,
3,
10,
4,
331,
1,
77,
3,
10,
4,
336,
2,
69,
3,
352,
2,
69,
3,
352,
2,
69,
3,
43286
]
]
},
"result": {
"classPredictions": [
{
"class": "pug, pug-dog",
"score": 1
},
{
"class": "Norwegian elkhound, elkhound",
"score": 0
},
{
"class": "Brabancon griffon",
"score": 0
},
{
"class": "chow, chow chow",
"score": 0
},
{
"class": "bull mastiff",
"score": 0
}
]
}
}
}
If the explainability output adheres to the required format, the explanation will be visualized in Modzy. Here is a screenshot of the rendered visualization corresponding with the example JSON results above:
This particular model uses Modzy's AXAI explainability technique, but you are free to use any explainability technique given that you can transform the results into the required format. More image classification usage information is available here.
Text Classification
Modzy provides full support for explainability in text classification models. All that is necessary is that a model container returns the following information in the JSON object accompanying the results object:
"explanation": {
"wordImportances": {},
"explainableText": {}
}
In this explanation object, wordImportances is an object in which keys are class names and values are arrays containing explanation entry objects. Each of these explanation entry objects contains the word, score, and optionally the index of the occurrence of the word in the text split on whitespace. If explainableText is provided, then index values must refer to the explainableText. If explainableText is not provided, then index values must refer to the original input text. If a score is negative, it means that the word contributed negatively to that class prediction.
Here is an example of the full model output containing explainability results from our Sentiment Analysis model:
{'data': {'drift': None,
'explanation': {'wordImportances': {'negative': [],
'neutral': [{'index': 4,
'score': -0.00433,
'word': 'the'},
{'index': 2,
'score': -0.00532,
'word': 'explainability'},
{'index': 0,
'score': -0.00612,
'word': 'i'},
{'index': 3,
'score': -0.00665,
'word': "it's"},
{'index': 5,
'score': -0.29722,
'word': 'best'}],
'positive': [{'index': 1,
'score': 0.3015,
'word': 'love'},
{'index': 5,
'score': 0.29722,
'word': 'best'},
{'index': 3,
'score': 0.00665,
'word': "it's"},
{'index': 0,
'score': 0.00612,
'word': 'i'},
{'index': 2,
'score': 0.00532,
'word': 'explainability'}]}},
'result': {'classPredictions': [{'class': 'positive',
'score': 0.677},
{'class': 'neutral',
'score': 0.323},
{'class': 'negative',
'score': 0}]}}}
If the explainability output adheres to the required format, the explanation will be visualized in Modzy. Here is a screenshot of the rendered visualization corresponding with the example JSON results above:
This particular model uses the LIME explainability technique, but you are free to use any explainability technique given that you can transform the results into the required format. More text classification usage information is available here.
Updated 12 months ago