2. Deploy Model

👍

Self-Service Tutorial Contents

1. Package Model
2. Deploy Model
3. Scale Model Up
4. Run Model Inference
5. Set Drift Baseline
6. Deploy Model to Edge Device

Prepare to deploy your model

In this tutorial, we will deploy a container directly from Dockerhub into our Modzy library in a few simple steps.

Note: If you did not complete Tutorial #1, you can still proceed with a Modzy-provided pre-built model container.

📘

What you'll need for this tutorial

• Your packaged model from Tutorial #1 or the provided example container
• A valid Modzy account
• A sample input.txt file to use for testing

Optional:

• Review our Model Deployment feature docs for more details

To get started, open two separate tabs in your browser:

1. Your Modzy account (e.g., https://my-modzy-domain.app.modzy.com)
2. The Dockerhub link to your model container. If you are using the model you created in Tutorial #1, the link would look something like https://hub.docker.com/repository/<your-docker-username>/tinybert-amd/general. Or, you can follow with the example we will use: https://hub.docker.com/repository/docker/modzy/tinybert-amd/general

Next, locate the input.txt file we created in Tutorial #1. It should be in the same directory that contains the Jupyter notebook we created.

If you did not follow Tutorial #1, simply create a text file and add this sentence: "This is my first time using Modzy!"

Deploy model to library

First, navigate to the "Models" page in your Modzy instance and click the "Add Model" button next to the search bar.

Next, select the "Import Container" option.

Provide the following information for your model:

• Model Name: We will name our model "Hugging Face TinyBERT (AMD)"
• Model Version: We will start with v1.0.0. Note: After deploying your model, you can add new versions

Click "Continue" and when prompted to confirm, select "Save and Continue".

First, select the "Choose Import Method" button to begin importing your container.

Next, select the "Docker Hub" option.

Because we pushed our model container to Dockerhub, we can simply copy a command directly from our container repository page and paste it into this field.

In the tab with Dockerhub pulled up, navigate to the "Tags" tab and copy the docker pull command on the right side of the row.

Now back in your Modzy tab, paste the command you just copied.

From here, click "Save and Continue". Modzy will immediately begin to download this container, which you can track via the progress bar on the top right of the screen.

While your container image downloads, you can begin step two - "Select hardware requirements". For this model, select the "1 core, 1 Gi RAM" option.

Next, scroll to the bottom of the page and set the Status and Run timeout values to 60.

Click "Save and Continue" to move onto the next step. Modzy will perform a test against the model container with the specified hardware.

This first test can take a few minute, so sit tight! Once it does complete, notice the "Verify Status Endpoint" progress bar on the right will be filled in with a green check mark. Now you can move onto step 3 - "Test model on sample input data".

Here, you need to upload the sample input.txt file you created in Tutorial #1. To do so, click the "Select File" button, find the location where you saved this file. Once selected, click the "Start Job" button.

Once you start this job, expect to see a series of messages that pop up as process progresses. The screen should look something like this:

After the job completes, you may view the sample input and output of your job. Verify your sample output looks similar to this:

{
  "jobIdentifier": "<uuid>",
  "total": "<number of inputs>",
  "completed": "<total number of completed inputs>",
  "failed": "<number of failed inputs>",
  "finished": "<true or false>",
  "submittedByKey": "<api key>",
  "results": {
    "<input-id>": {
      "model": null,
      "userIdentifier": null,
      "status": "SUCCESSFUL",
      "engine": "partial-model-run-g0h96fgwjq-1-0-0-775fbf4c57-9njfz",
      "error": null,
      "startTime": "2023-03-08T00:55:47.525+0000",
      "endTime": "2023-03-08T00:55:58.624+0000",
      "updateTime": "2023-03-08T00:55:58.624+0000",
      "inputSize": 0,
      "accessKey": null,
      "teamIdentifier": null,
      "accountIdentifier": null,
      "timeMeters": null,
      "datasourceCompletedTime": null,
      "explained": false,
      "elapsedTime": 11099,
      "results.json": {
        "data": {
          "result": {
            "classPredictions": [
              {
                "class": "joy",
                "score": 0.9277271032333374
              },
              {
                "class": "anger",
                "score": 0.06407234817743301
              },
              {
                "class": "fear",
                "score": 0.0030489922501146793
              },
              {
                "class": "sadness",
                "score": 0.002112747635692358
              },
              {
                "class": "surprise",
                "score": 0.0017078057862818241
              },
              {
                "class": "love",
                "score": 0.001331000356003642
              }
            ]
          },
          "explanation": null,
          "drift": null
        }
      }
    }
  }
}

Pending everything looks as expected, click "Save and Continue" to move onto the final stages of deploying your model.

📘

Note

You have now completed all mandatory steps required to deploy your model. You may optionally complete steps 4 and 5 to add custom documentation, performance metrics, and tags specific to your model.

Finally, you can optionally download the below image and add it to your model as the model card image.

Simply edit the circle photo to the left of your model title, select "Upload new photo", and add your image.

When you are happy with the state of your model, click "Publish Model ". View your newly deployed model in Modzy!

With your model now deployed, follow the next tutorial to learn how to scale it up and run inferences against it!