Model drift

Overview

The model operations service provides a clear insight into AI performance and usage across the installation. Its APIs report model usage (by model and by user), jobs, queue details, and model drift.

Model Ops monitors model performance metrics and alerts when retraining should be considered.

By monitoring the distribution of data feeding a model and detecting when it changes, you can learn if predictions are different than normal, faster. You can identify why models are making wrong predictions: if the models are getting wrong data or simply are no longer up to date.

The model drift object

{
  "modelIdentifier": "...",
  "modelVersion": "...",
  "modelName": "...",
  "lastActiveDatetime": "...",
  "driftCalculations": 1,
  "avgChiSquaredValue": 0,
  "avgPValue": 0,
}

Parameter

Type

Description

modelIdentifier

string

A model identifier provided by Modzy. It contains 10 random alphanumeric characters.

modelVersion

string

The model’s version number. It follows the semantic versioning format.

modelName

string

The model’s name. It may only contain alphanumeric, dash, and underscore characters.

lastActiveDatetime

string

The last time the model was used in ISO8601 (YYYY-MM-DDThh:mm:ss.sTZD) format.

driftCalculations

number

The number of drift calculations made for the model’s version.

avgChiSquaredValue

number

The average chiSquaredValue. The chiSquaredValue determines how likeliness that any differences arose by chance between the baseline dataset and a given inference dataset.

avgPValue

number

The average pValue. The pValue measures the strength of the evidence provided by the chiSquaredValue.