Explainability

Overview

Explainability describes a model’s ability to return information describing how a model produced a given prediction, inference, or result. The details returned depend on the explainable input type. Modzy currently supports many types of explanability, but provides a graphical UI for image classification and text classification models. For this feature to work, models must include explainability code within the model container itself before it's deployed to Modzy. This is commonly called "White Box" Explainability.

These models return JSON outputs with mask values for explainable results. Mask values include the prediction results and the explainability results (pixel values that motivate the prediction made by the model).

How to Generate Explanations

When submitting a job to Modzy, you can request explainable results by adding "explain": "true" to the end of your job submission object. As mentioned above, this will only work for models that comply with Modzy's explainability requirements. See Explainability Formats for more details.

// Requesting an explanation of a prediction from a sentiment analysis model
{
  "model": {
    "identifier": "ed542963de",
    "version": "1.0.1"
  },
  "input": {
    "type": "text",
    "sources": {
      "us-travelers-are-back": {
         "input.txt": "This strong desire to travel has driven new trends in the industry — some of which may be here to stay. Like Burglewskis family, people are flocking to outdoor activities, rural areas and private vacation rentals, with less interest in hotels and international and urban destinations."
      }
    }
  },
  "explain": "true"
}

🚧

Explainability can be slow

Generating explainable results from machine learning models requires additional computation which can sometimes slow a model down. Keep that in mind when you're deciding when to request explainability and when not to.

Viewing Explainable Results

When you are viewing results directly in the UI, these models return JSON outputs with mask values for explainable results. Mask values include the prediction results and the explainability results (pixel values that motivate the prediction made by the model).

To find the results for your explainable job, navigate to the "Operations" section in the top header, and then the "Explainability" tab. This will display all explainable jobs/inputs run within your Team and corresponding results. Click into each one to see the details and image visualization.

List of all explainable jobs run within your teamList of all explainable jobs run within your team

List of all explainable jobs run within your team

Each results page will show the image along with the mask value laid on top of the image, which you can toggle on or off. It will display the top result, along with the percentage certainty that result is correct, along with the other potential top results with corresponding certainty. This then allows the user to hit the "correct" or "incorrect" button to provide the model feedback whether the prediction was accurate. This data is stored for future retraining efforts. The bottom of the page also includes the JSON output for use. You can toggle between inputs on the left navigation to vie results for the other inputs in the job.

Explainability result visualizedExplainability result visualized

Explainability result visualized


Did this page help you?