2. Deploy Model
GuidesRecipesAPI ReferenceChangelogDiscussions
Log In

2. Deploy Model

👍

Self-Service Tutorial Contents

  1. Package Model
  2. :arrow-right: Deploy Model
  3. Scale Model Up
  4. Run Model Inference
  5. Set Drift Baseline
  6. Deploy Model to Edge Device

Prepare to deploy your model

In this tutorial, we will deploy a container directly from Dockerhub into our Modzy library in a few simple steps.

Note: If you did not complete Tutorial #1, you can still proceed with a Modzy-provided pre-built model container.

📘

What you'll need for this tutorial

  • Your packaged model from Tutorial #1 or the provided example container
  • A valid Modzy account
  • A sample input.txt file to use for testing

Optional:

To get started, open two separate tabs in your browser:

  1. Your Modzy account (e.g., https://my-modzy-domain.app.modzy.com)
  2. The Dockerhub link to your model container. If you are using the model you created in Tutorial #1, the link would look something like https://hub.docker.com/repository/<your-docker-username>/tinybert-amd/general. Or, you can follow with the example we will use: https://hub.docker.com/repository/docker/modzy/tinybert-amd/general

Next, locate the input.txt file we created in Tutorial #1. It should be in the same directory that contains the Jupyter notebook we created.

If you did not follow Tutorial #1, simply create a text file and add this sentence: "This is my first time using Modzy!"

Deploy model to library

First, navigate to the "Models" page in your Modzy instance and click the "Add Model" button next to the search bar.

Models Page

Models Page

Next, select the "Import Container" option.

Model Deployment Home Page

Model Deployment Home Page

Provide the following information for your model:

  • Model Name: We will name our model "Hugging Face TinyBERT (AMD)"
  • Model Version: We will start with v1.0.0. Note: After deploying your model, you can add new versions
Model Name and Version

Model Name and Version

Click "Continue" and when prompted to confirm, select "Save and Continue".

First, select the "Choose Import Method" button to begin importing your container.

Model Deployment Workflow

Model Deployment Workflow

Next, select the "Container Registry" option.

Container Registry Import Option

Container Registry Import Option

Because we pushed our model container to Docker Hub, we can simply copy a command directly from our container repository page and paste it into this field.

In the tab with Docker Hub pulled up, navigate to the "Tags" tab and copy the docker pull command on the right side of the row.

Dockerhub Container Image Page

Docker Hub Container Image Page

Now back in your Modzy tab, paste the command you just copied. Modzy will automatically format this pull command correctly.

Docker Pull Copy-Paste

Docker Pull Copy-Paste

From here, click "Save and Continue". Modzy will immediately begin to download this container, which you can track via the progress bar on the top right of the screen.

Container Import In Progress

Container Import In Progress

While your container image downloads, you can begin step two - "Select hardware requirements". For this model, select the "1 core, 1 Gi RAM" option.

Hardware Requirements

Hardware Requirements

Next, scroll to the bottom of the page and set the Status and Run timeout values to 60.

Timeout Thresholds

Timeout Thresholds

Click "Save and Continue" to move onto the next step. Modzy will perform a test against the model container with the specified hardware.

This first test can take a few minute, so sit tight! Once it does complete, notice the "Verify Status Endpoint" progress bar on the right will be filled in with a green check mark. Now you can move onto step 3 - "Test model on sample input data".

Modzy Container Tests

Modzy Container Tests

Here, you need to upload the sample input.txt file you created in Tutorial #1. To do so, click the "Select File" button, find the location where you saved this file. Once selected, click the "Start Job" button.

Test with Sample Data Screen

Test with Sample Data Screen

Once you start this job, expect to see a series of messages that pop up as process progresses. The screen should look something like this:

Test Job Running

Test Job Running

After the job completes, you may view the sample input and output of your job. Verify your sample output looks similar to this:

{
  "jobIdentifier": "<uuid>",
  "total": "<number of inputs>",
  "completed": "<total number of completed inputs>",
  "failed": "<number of failed inputs>",
  "finished": "<true or false>",
  "submittedByKey": "<api key>",
  "results": {
    "<input-id>": {
      "model": null,
      "userIdentifier": null,
      "status": "SUCCESSFUL",
      "engine": "partial-model-run-g0h96fgwjq-1-0-0-775fbf4c57-9njfz",
      "error": null,
      "startTime": "2023-03-08T00:55:47.525+0000",
      "endTime": "2023-03-08T00:55:58.624+0000",
      "updateTime": "2023-03-08T00:55:58.624+0000",
      "inputSize": 0,
      "accessKey": null,
      "teamIdentifier": null,
      "accountIdentifier": null,
      "timeMeters": null,
      "datasourceCompletedTime": null,
      "explained": false,
      "elapsedTime": 11099,
      "results.json": {
        "data": {
          "result": {
            "classPredictions": [
              {
                "class": "joy",
                "score": 0.9277271032333374
              },
              {
                "class": "anger",
                "score": 0.06407234817743301
              },
              {
                "class": "fear",
                "score": 0.0030489922501146793
              },
              {
                "class": "sadness",
                "score": 0.002112747635692358
              },
              {
                "class": "surprise",
                "score": 0.0017078057862818241
              },
              {
                "class": "love",
                "score": 0.001331000356003642
              }
            ]
          },
          "explanation": null,
          "drift": null
        }
      }
    }
  }
}

Pending everything looks as expected, click "Save and Continue" to move onto the final stages of deploying your model.

📘

Note

You have now completed all mandatory steps required to deploy your model. You may optionally complete steps 4 and 5 to add custom documentation, performance metrics, and tags specific to your model.

Finally, you can optionally download the below image and add it to your model as the model card image.

Hugging Face Logo for Model Card

Hugging Face Logo for Model Card

Simply edit the circle photo to the left of your model title, select "Upload new photo", and add your image.

Upload Photo to Model Card

Upload Photo to Model Card

When you are happy with the state of your model, click "Publish Model ". View your newly deployed model in Modzy!

Newly Deployed Model Page

Newly Deployed Model Page

With your model now deployed, follow the next tutorial to learn how to scale it up and run inferences against it!


What’s Next