5. Set Drift Baseline

👍

Self-Service Tutorial Contents

  1. Package Model
  2. Deploy Model
  3. Scale Model Up
  4. Run Model Inference
  5. :arrow-right: Set Drift Baseline
  6. Deploy Model to Edge Device

Tutorial Preparation

In the penulimate tutorial of this series, we will learn how to monitor our newly deployed model with Modzy's monitoring features. To follow along, you must have completed tutorials 2-4.

Note: If you have not run at least 30 inferences against your model (which we covered at the end of Tutorial #4, you will not have enough inferences to set a drift baseline.

📘

What you'll need for this tutorial

  • A valid Modzy account
  • Your newly-deployed model
  • At least 30 successful inferences against your model

Navigate to drift page

First, open the "Drift" page under the "Operations Tab". Here, you should see a list of models whose output meet Modzy's model drift format.

Drift Home Page

Drift Home Page

Then, find the row with your model and click into the drift page.

Drift Home Page

Drift Home Page

At the top of this page, you should see a message that indicates the need to configure drift for your model.

Drift Configuration Home Page for Model

Drift Configuration Home Page for Model

Set baseline and threshold configuration

Continue scrolling down until you see two configuration fields you can set, "Baseline Period" and "Thresholds". First, set the start and end dates for your Baseline Period to be a day before and after you ran inferences (Tutorial #4).

Baseline Period Configuration

Baseline Period Configuration

After selecting your dates, you will notice an inference count that appears under the end date field. If you closely followed the steps laid out in Tutorial #4, this number should read somewhere between 50-55.

Baseline Period Inference Count

Baseline Period Inference Count

Modzy will use this baseline period to measure a drift score as a comparison between all subsequent inferences and this baseline distribution.

Next, toggle the slider bars in the "Thresholds" section to determine the Nominal, Medium, and High drift categories. This is completely customizable based on your use case's drift requirements.

Drift Threshold Configuration

Drift Threshold Configuration

Finally, navigate to the "Results" tab in this model's drift page. Since you just set your baseline, your page should be cleared of any results and look similar to the below image.

Initial Drift Results

Initial Drift Results

Now, as you continue to submit inferences to this model over time, Modzy will compute a drift score derived from a Chi-square statistical test. Learn more about Modzy's drift features here.

Congratulations! You just successfully configured drift for your model. This means you will constantly be able to monitor it's performance over time. Explore the last tutorial to learn how to take your model and deploy it to an edge device.