5. Set Drift Baseline

👍

Self-Service Tutorial Contents

1. Package Model
2. Deploy Model
3. Scale Model Up
4. Run Model Inference
5. Set Drift Baseline
6. Deploy Model to Edge Device

Tutorial Preparation

In the penulimate tutorial of this series, we will learn how to monitor our newly deployed model with Modzy's monitoring features. To follow along, you must have completed tutorials 2-4.

Note: If you have not run at least 30 inferences against your model (which we covered at the end of Tutorial #4, you will not have enough inferences to set a drift baseline.

📘

What you'll need for this tutorial

• A valid Modzy account
• Your newly-deployed model
• At least 30 successful inferences against your model

Navigate to drift page

First, open the "Drift" page under the "Operations Tab". Here, you should see a list of models whose output meet Modzy's model drift format.

Then, find the row with your model and click into the drift page.

At the top of this page, you should see a message that indicates the need to configure drift for your model.

Set baseline and threshold configuration

Continue scrolling down until you see two configuration fields you can set, "Baseline Period" and "Thresholds". First, set the start and end dates for your Baseline Period to be a day before and after you ran inferences (Tutorial #4).

After selecting your dates, you will notice an inference count that appears under the end date field. If you closely followed the steps laid out in Tutorial #4, this number should read somewhere between 50-55.

Modzy will use this baseline period to measure a drift score as a comparison between all subsequent inferences and this baseline distribution.

Next, toggle the slider bars in the "Thresholds" section to determine the Nominal, Medium, and High drift categories. This is completely customizable based on your use case's drift requirements.

Finally, navigate to the "Results" tab in this model's drift page. Since you just set your baseline, your page should be cleared of any results and look similar to the below image.

Now, as you continue to submit inferences to this model over time, Modzy will compute a drift score derived from a Chi-square statistical test. Learn more about Modzy's drift features here.

Congratulations! You just successfully configured drift for your model. This means you will constantly be able to monitor it's performance over time. Explore the last tutorial to learn how to take your model and deploy it to an edge device.