Hugging Face
This guide demonstrates the process of automatically containerizing your Hugging Face model.
What you will need
- Dockerhub account
- Connection to running Chassis.ml service (either from a local deployment or via publicly-hosted service)
- Transformers library
- Python environment
NOTE: To follow along, you can reference the Jupyter notebook example and data files here.
Set Up Environment
We recommend you follow this guide using a Jupyter Notebook. Follow the appropriate install instructions based on your environment.
Create a Python virtual environment and install the python packages required to load and run your model. At a minimum, pip install the following packages:
pip install chassisml modzy-sdk
If you would like to follow this guide directly, pip install the following additional packages:
torch
transformers
numpy
Download Model with Transformers
There are several ways to interact with and download pre-trained models from the Hugging Face model hub, and in this guide, we will use the transformers
library to download and save a model. Specifically, we will download this DistilBERT model.
First, select the "Use in Transformers" button on your model card page

Figure 1. Hugging Face Model Card
Next, copy the transformers code at the top of this window.

Figure 2. Use in Transformers
Now, download and save this pre-trained model to your desktop in your notebook.
# Import required packages
import os
import torch
import chassisml
import numpy as np
# Download model with the transformers library
from transformers import AutoTokenizer, AutoModelForSequenceClassification
tokenizer = AutoTokenizer.from_pretrained("bhadresh-savani/distilbert-base-uncased-emotion")
model = AutoModelForSequenceClassification.from_pretrained("bhadresh-savani/distilbert-base-uncased-emotion")
# Save model locally so we can load it into memory for use with Chassis package
tokenizer.save_pretrained("./model")
model.save_pretrained("./model")
Load Model into Memory
If you plan to use the Chassis service, you must first load your model into memory. If you have your trained model file saved locally (.pth
, .pkl
, .h5
, .joblib
, or other file format), you can load your model from the weights file directly, or alternatively train and use the model object.
# Load model into memory from local file
distilbert_tokenizer = AutoTokenizer.from_pretrained("./model")
distilbert_model = AutoModelForSequenceClassification.from_pretrained("./model")
# define labels from model config
labels = model.config.id2label
Define process
Function
process
FunctionYou can think of this function as your "inference" function that will take input data as raw bytes, process the inputs, make predictions, and return the results. This method is the sole parameter required to create a ChassisModel
object.
# define process function that will serve as our inference function
def process(input_bytes):
# decode and preprocess data bytes
text = input_bytes.decode()
inputs = distilbert_tokenizer(text, return_tensors="pt")
# run preprocessed data through model
with torch.no_grad():
logits = distilbert_model(**inputs).logits
softmax = torch.nn.functional.softmax(logits, dim=1).detach().cpu().numpy()
# postprocess
indices = np.argsort(softmax)[0][::-1]
results = {
"data": {
"result": {
"classPredictions": [{"class": labels[i], "score": softmax[0][i]} for i in indices]
}
}
}
return results
Create ChassisModel
Object and Publish Model
ChassisModel
Object and Publish ModelFirst, connect to a running instance of the Chassis service - either by deploying on your machine or by connecting to the publicly hosted version of the service). Then, you can use the process
function you defined to create a ChassisModel
object, run a few tests to ensure your model object returns the expected results, and finally publish your model.
chassis_client = chassisml.ChassisClient("http://localhost:5000")
chassis_model = chassis_client.create_model(process_fn=process)
Define sample file from local filepath and run a series of tests.
NOTE: test_env
method is not available on publicly-hosted service.
sample_filepath = './data/input.txt'
results = chassis_model.test(sample_filepath)
print(results)
test_env_result = chassis_model.test_env(sample_filepath)
print(test_env_result)
Define your Dockerhub credentials and publish your model.
dockerhub_user = <my.username>
dockerhub_pass = <my.password>
modzy_url = <modzy.base.url> # E.g., https://trial.app.modzy.com/api
modzy_api_key = <my.modzy.api.key> # E.g., 8eba4z0AHqguxyf1gU6S.4AmeDQYIQZ724AQAGLJ8
response = chassis_model.publish(
model_name="Hugging Face DistilBERT",
model_version="0.0.1",
registry_user=dockerhub_user,
registry_pass=dockerhub_pass
)
job_id = response.get('job_id')
final_status = chassis_client.block_until_complete(job_id)
You have successfully completed the packaging of your Hugging Face model. In your Dockerhub account, you should see your new container listed in the "Repositories" tab.
Figure 1. Example Chassis-built Container
Congratulations! In just minutes you automatically created a Docker container with just a few lines of code. To deploy your new model container to Modzy, follow one of the following guides:
Updated 2 months ago