Prediction drift


The model operations service provides a clear insight into AI performance and usage across the installation. Its APIs report model usage (by model and by user), jobs, queue details, and model drift.

Model Ops monitors model performance metrics and alerts when retraining should be considered.

By monitoring the distribution of data feeding a model and detecting when it changes, you can learn if predictions are different than normal, faster. You can identify why models are making wrong predictions: if the models are getting wrong data or simply are no longer up to date.

The model drift object

  "modelIdentifier": "...",
  "modelVersion": "...",
  "modelName": "...",
  "lastActiveDatetime": "...",
  "driftCalculations": 1,
  "avgChiSquaredValue": 0,
  "avgPValue": 0,
modelIdentifierstringA model identifier provided by Modzy. It contains 10 random alphanumeric characters.
modelVersionstringThe model’s version number. It follows the semantic versioning format.
modelNamestringThe model’s name. It may only contain alphanumeric, dash, and underscore characters.
lastActiveDatetimestringThe last time the model was used in ISO8601 (YYYY-MM-DDThh:mm:ss.sTZD) format.
driftCalculationsnumberThe number of drift calculations made for the model’s version.
avgChiSquaredValuenumberThe average chiSquaredValue. The chiSquaredValue determines how likeliness that any differences arose by chance between the baseline dataset and a given inference dataset.
avgPValuenumberThe average pValue. The pValue measures the strength of the evidence provided by the chiSquaredValue.