Import a Container

This page will guide you through the process of importing your model container into Modzy. With this guide, you'll learn to download and import your model, configure hardware settings for your model, test your model to ensure it meets your specifications, and document your model in detail.

🚧

Container Check

To follow this guide, verify that your model meets the Open Model Interface (OMI) specification. You can either manually build your container or leverage Chassis to automatically containerize your model code into an OMI-compliant container.

Getting Started

Begin importing your container by navigating to the models page within your Modzy account and select Add New Model.

Next, click the Import Container option to launch the model deployment process.

Give your model a name and version. Note: After deploying your model, you can add new versions incrementally according to the semantic versioning system.

Import Container and Run Tests

Begin importing your model container by starting the Choose Import Method step.

Here, you will have three choices for importing your container:

1. Container Registry: import from a container registry such as Docker Hub
2. Upload .tar file: import from saved tar archive version of Docker container

If you select the Upload .tar file option, ensure you created your .tar file using the following command:

docker save -o my-container.tar your-container-image

If you select the Container Registry option, fill in the fields as follows:

Docker registry URL
Simply copy and paste the pull command for the container image that you'd like to import into Modzy. Most registries, such as Docker Hub, provide a convenient copy link. Once copied, you can paste that value directly into the URL text field within Modzy and it will automatically format the docker pull command for you.

Credentials
If your container image repository is public, then you can uncheck the "Docker registry requires authentication" box and continue. Otherwise, fill in the Username and Password fields with your registry credentials.

📘

Importing from registries with delegated auth coming soon

Support for registries like Amazon's ECR, Azure's ACR, and Google's GCR registries use delegated authentication. Support for importing images from these registries is not yet supported, but will be coming soon.

If you specify a valid Docker registry link and credentials (if needed), the Import model progress bar on the right of the screen will display the status of the model import.

While Modzy is downloading your model container file, you can start step two: Select hardware requirements. On this page, select the hardware option that best suits your model. As a general rule of thumb, select an option with RAM that exceeds the size of your Docker container. Further, if your model is enabled to parallelize the processing of data while making predictions, select the number of CPU cores and/or GPUs that most appropriately fit your model's intended use.

The Timeouts section allows you to specify timeouts Modzy will use as validation checks during two phases of your model's inference lifecycle in production:

• Status timeout: corresponds to your model spinning up and performing any required model instantiation
• Run timeout: corresponds to a single inference run (includes real-time or batch inference calls)

As a general rule of thumb, set these values high enough to account for large batches of data (if relevant for your model), but not too high where a user will wait an unreasonable amount of time. Click Save and Continue when complete to kick off the first of two functionality tests Modzy will execute.

After the "Import Model" progress bar completes, Modzy will move onto the next step in the process: spinning up your model container and checking the Status() remote procedure call. As this test is occurring, you will notice the second progress bar for the "Verify Status Endpoint" will start to complete.

Pending a successful status message, start the Test model on sample input data step and upload a sample data file. This file should be representative of the data you would expect end users to run through your model. After uploading this file, select Start Job, which will kick off the second functionality test Modzy executes. For this test, Modzy will pass the sample input file through your model for predictions and ensure the container can return the model output through Modzy's APIs.

As Modzy is executing this test a progress bar will indicate the status of the job. Note that inference times can vary between models and depend entirely on the size of the model, optimization of the model, and the size of the input data. Pending a successful test job, you will see the full sample API request and responses.

Document Model

Lastly, complete steps four and five - "Add Documentation" and "Assign Tags and Categories" - to fully document your model. In these steps, you can add a descriptions, performance metrics, detailed technical information, and anything else relevant an end user would need to know before using your model. Additionally, you can assign tags to your model to promote the discoverability of your model across your entreprise.

You are now ready to publish your model and begin submitting production inferences! After clicking Publish, view your model's details page and let your team know they can begin leveraging it in their applications.