How to Import a Container

Deploy OMI-compliant model container

This page will guide you through the process of importing your model container into Modzy. Throughout this process, Modzy will perform security scans on your container to check for any vulnerabilities, run functional tests to ensure your container meets the proper specification, and you will have the opportunity to document your model in detail.

🚧

Container Check

To follow this guide, verify that your model meets the Open Model Interface (OMI) specification. You can either manually build your container or leverage Chassis to automatically containerize your model code into an OMI-compliant container.


:beginner: Getting Started

Begin importing your container by navigating to the models page within your Modzy account and select Add New Model.

Figure 1. Models PageFigure 1. Models Page

Figure 1. Models Page


Next, click the Import Container option to launch the model deployment process.

Figure 2. Deployment Home PageFigure 2. Deployment Home Page

Figure 2. Deployment Home Page


Give your model a name and version. Note: After deploying your model, you can add new versions incrementally according to the semantic versioning system.

Figure 3. Name and version your modelFigure 3. Name and version your model

Figure 3. Name and version your model


:arrow-double-down: Import Container and Run Tests

Begin importing your model container by starting the Choose Import Method step.

Figure 4. Deployment Process Driver PageFigure 4. Deployment Process Driver Page

Figure 4. Deployment Process Driver Page


Here, you will have two choices for importing your container:

  1. Docker Hub: import from hub.docker.com
  2. Docker: import from another Docker registry
  3. Upload .tar file: import from saved tar archive version of Docker container
Figure 5. Choose Model Import MethodFigure 5. Choose Model Import Method

Figure 5. Choose Model Import Method

If you select the Upload .tar file option, ensure you created your .tar file using the following command:

docker save -o my-container.tar your-container-image

If you select the Docker option, fill in the three fields as follows:

Docker registry URL
You must convert your docker container image URL into the Docker Registry HTTP API v2 protocol. Regardless of the registry you are using (Dockerhub, AWS ECR, Azure or GCP Container Registry, etc.), the URL Modzy expects will follow this general structure:

https://<registry-url>/v2/<username>/<container-repository-name>/manifests/<image-tag>

For example, Modzy's Echo Model hosted on Dockerhub (https://hub.docker.com/r/modzy/grpc-echo-model) converts to https://registry.hub.docker.com/v2/modzy/grpc-echo-model/manifests/1.0.0 in HTTP API v2 protocol format.

Credentials
If your container image repository is public, then you can uncheck the "Docker registry requires authentication" box and continue. Otherwise, fill in the Username and Password fields with your registry credentials. For Dockerhub, these values are your username and password, but for cloud registries, these values correspond to your cloud provider Access Key (Username) and Secret Access Key (Password).

Figure 6. Docker Import MethodFigure 6. Docker Import Method

Figure 6. Docker Import Method


If you specify a valid Docker registry link and credentials (if needed), the Import model progress bar on the right of the screen will display the status of the model import.

Figure 7. Model Import In-ProgressFigure 7. Model Import In-Progress

Figure 7. Model Import In-Progress


While Modzy is downloading your model container file, you can start step two: Select hardware requirements. On this page, select the hardware option that best suits your model. As a general rule of thumb, select an option with RAM that exceeds the size of your Docker container. Further, if your model is enabled to parallelize the processing of data while making predictions, select the number of CPU cores and/or GPUs that most appropriately fit your model's intended use.

The Timeouts section allows you to specify timeouts Modzy will use as validation checks during two phases of your model's inference lifecycle in production:

  • Status timeout: corresponds to your model spinning up and performing any required model instantiation
  • Run timeout: corresponds to a single inference run (includes real-time or batch inference calls)

As a general rule of thumb, set these values high enough to account for large batches of data (if relevant for your model), but not too high where a user will wait an unreasonable amount of time. Click Save and Continue when complete to kick off the first of two functionality tests Modzy will execute.

Figure 8. Hardware RequirementsFigure 8. Hardware Requirements

Figure 8. Hardware Requirements


After the "Import Model" progress bar completes, Modzy will move onto the next step in the process: spinning up your model container and checking the Status() remote procedure call. As this test is occurring, you will notice the second progress bar for the "Verify Status Endpoint" will start to complete.

Figure 9. Verify Status EndpointFigure 9. Verify Status Endpoint

Figure 9. Verify Status Endpoint


Pending a successful status message, start the Test model on sample input data step and upload a sample data file. This file should be representative of the data you would expect end users to run through your model. After uploading this file, select Start Job, which will kick off the second functionality test Modzy executes. For this test, Modzy will pass the sample input file through your model for predictions and ensure the container can return the model output through Modzy's APIs.

Figure 10. Test Inference JobFigure 10. Test Inference Job

Figure 10. Test Inference Job


As Modzy is executing this test a progress bar will indicate the status of the job. Note that inference times can vary between models and depend entirely on the size of the model, optimization of the model, and the size of the input data. Pending a successful test job, you will see the full sample API request and responses.

Figure 11. Successful Test Job ResultsFigure 11. Successful Test Job Results

Figure 11. Successful Test Job Results


:page-with-curl: Document Model

Lastly, complete steps four and five - "Add Documentation" and "Assign Tags and Categories" - to fully document your model. In these steps, you can add a descriptions, performance metrics, detailed technical information, and anything else relevant an end user would need to know before using your model. Additionally, you can assign tags to your model to promote the discoverability of your model across your entreprise.

Figure 12. Tags and CategoriesFigure 12. Tags and Categories

Figure 12. Tags and Categories

You are now ready to publish your model and begin submitting production inferences! After clicking Publish, view your model's details page and let your team know they can begin leveraging it in their applications.

Figure 13. Publish Model into ProductionFigure 13. Publish Model into Production

Figure 13. Publish Model into Production


What’s Next
Did this page help you?