This guide demonstrates the process of automatically containerizing your MXNet model.

📘

What you will need

  • Dockerhub account
  • Connection to running Chassis.ml service (either from a local deployment or via publicly-hosted service)
  • Trained MXNet model that can be loaded into memory or code to train a MXNet model from scratch
  • Python environment

NOTE: To follow along, you can reference the Jupyter notebook example and data files here.

Set Up Environment

👍

We recommend you follow this guide using a Jupyter Notebook. Follow the appropriate install instructions based on your environment.

Create a Python virtual environment and install the python packages required to load and run your model. At a minimum, pip install the following packages:

pip install chassisml modzy-sdk

If you would like to follow this guide directly, pip install the following additional packages:

mxnet>=1.9.0
matplotlib>=3.5.1
numpy>=1.22.3

Load Model into Memory

If you plan to use the Chassis service, you must first load your model into memory. If you have your trained model file saved locally (.pth, .pkl, .h5, .joblib, or other file format), you can load your model from the weights file directly, or alternatively train and use the model object.

import cv2
import chassisml
import numpy as np
import getpass
import json
import mxnet as mx
import mxnet.image as mxnet_img
from mxnet import gluon, nd
from mxnet.gluon.model_zoo import vision
import matplotlib.pyplot as plt

ctx = mx.cpu()
mobileNet = vision.mobilenet0_5(pretrained=True, ctx=ctx)

# load imagenet labels for postprocessing
imagenet_labels = np.array(json.load(open('data/image_net_labels.json', 'r')))
print(imagenet_labels[4])

# load sample image
filename = "data/dog.jpg"

# visualize image to test
image = mx.image.imread(filename)
plt.imshow(image.asnumpy())

def predict(model, image, categories, k=3):
    predictions = model(transform(image)).softmax()
    top_pred = predictions.topk(k=k)[0].asnumpy()
    probs = []
    labels = []
    for index in top_pred:
        probability = predictions[0][int(index)]
        probs.append(probability.asscalar())
        category = categories[int(index)]
        labels.append(category)

    return probs, labels

After testing the model, we get the following printed to our console:

# test model
probs, labels = predict(mobileNet, image, imagenet_labels, 3)
print(probs)
print(labels)
>>> [0.84015656, 0.13626784, 0.006610237]
>>> ['boxer', 'bull mastiff', 'Rhodesian ridgeback']

Define process Function

You can think of this function as your "inference" function that will take input data as raw bytes, process the inputs, make predictions, and return the results. This method is the sole parameter required to create a ChassisModel object.

def process(input_bytes):
    # read image bytes
    img = mxnet_img.imdecode(input_bytes)

    # run inference
    probs, labels = predict(mobileNet, img, imagenet_labels, 3)

    # structure results
    inference_result = {
        "classPredictions": [
            {"class": labels[i], "score": probs[i]}
        for i in range(len(probs)) ]
    }

    structured_output = {
        "data": {
            "result": inference_result,
            "explanation": None,
            "drift": None,
        }
    }
    return structured_output

Create ChassisModel Object and Publish Model

First, connect to a running instance of the Chassis service - either by deploying on your machine or by connecting to the publicly hosted version of the service). Then, you can use the process function you defined to create a ChassisModel object, run a few tests to ensure your model object returns the expected results, and finally publish your model.

chassis_client = chassisml.ChassisClient("http://localhost:5000")
chassis_model = chassis_client.create_model(process_fn=process)

Define sample file from local filepath and run a series of tests.

NOTE: test_env method is not available on publicly-hosted service.

sample_filepath = './data/dog.jpg'
results = chassis_model.test(sample_filepath)
print(results)

test_env_result = chassis_model.test_env(sample_filepath)
print(test_env_result)

Define your Dockerhub credentials and publish your model.

dockerhub_user = <my.username>
dockerhub_pass = <my.password>

response = chassis_model.publish(
   model_name="MXNET MobileNet Image Classifiction",
   model_version="0.0.1",
   registry_user=dockerhub_user,
   registry_pass=dockerhub_pass
)

job_id = response.get('job_id')
final_status = chassis_client.block_until_complete(job_id)

You have successfully completed the packaging of your MXNet model. In your Dockerhub account, you should see your new container listed in the "Repositories" tab.

Figure 1. Example Chassis-built ContainerFigure 1. Example Chassis-built Container

Figure 1. Example Chassis-built Container

Congratulations! In just minutes you automatically created a Docker container with just a few lines of code. To deploy of your new model container to Modzy, follow the Import Container guide.






🚧

Programmatic Model Deployment (Beta)

If you prefer to automate the deployment of your model programmatically instead of following the above guide, follow the below instructions. Note: this feature includes limited error handling and does not allow you to configure the hardware options or documentation for your model.

To continue, you will need valid Modzy Credentials (instance URL and API Key, e.g., https://app.modzy.com and q4jp1pOZyFTddkFsOYwI.flHw34veJgfKu2MNzAa7)

First, modify your publish method to include your Modzy credentials.

dockerhub_user = <my.username>
dockerhub_pass = <my.password>
modzy_url = <modzy.base.url> # E.g., https://app.modzy.com/api
modzy_api_key = <my.modzy.api.key> # E.g., 8eba4z0AHqguxyf1gU6S.4AmeDQYIQZ724AQAGLJ8

response = chassis_model.publish(
   model_name="MXNET MobileNet Image Classifiction",
   model_version="0.0.1",
   registry_user=dockerhub_user,
   registry_pass=dockerhub_pass,
   modzy_url=modzy_url,
   modzy_api_key=modzy_api_key,
   modzy_sample_input_path=sample_filepath
)

job_id = response.get('job_id')
final_status = chassis_client.block_until_complete(job_id)

Executing this code will not only build and push a new container image to your Dockerhub repository, but it will also programmatically deploy the model to Modzy. However, as noted above, you have less flexibility to configure custom hardware options, and you will need to go back into your model to add documentation after the fact.

Once your model is deployed and you navigate to the new model page, click "Edit Model" to add documentation and tags to your newly deployed model. This will ensure other team members within your organization can discover this model and decide whether or not it fits their use case.

Figure 2. Edit ModelFigure 2. Edit Model

Figure 2. Edit Model

Edit the "Add documentation" section to add the following documentation to your model:

  • Description: Few sentence summary of your model. Include the task your model accomplishes and brief information about expected inputs and outputs.
  • Performance Overview: Few sentence overview of how your model was evaluated and any relevant performance metrics captured during the training and validation processes.
  • Performance Metrics: Add metrics you computed during training that will be displayed on your model home page.
  • Transparency and bias reporting: Technical details for your model, such as information about your model’s design, architecture, training data, development approach, etc.
Figure 3. Add documentationFigure 3. Add documentation

Figure 3. Add documentation

When you have finished, move on to the "Assign tags and categories" section. Adding tags will make your model more accessible and discoverable within your organization's Modzy model library.

Figure 4. Assign tagsFigure 4. Assign tags

Figure 4. Assign tags

Congratulations! In just minutes, you have successfully packaged, deployed, and documented your machine learning model. Check out our Running Inferences guides to start using your model right away.


What’s Next
Did this page help you?