MLflow

Import your model directly from MLflow

This guide walks through the process of importing your MLflow model directly into your Modzy instance.

👍

Supported MLflow Model Types

  • Tabular: any model trained on tabular data using an MLflow model flavor saved with a model signature. Learn more here
  • Non-tabular: any model trained on non-tabular data (unstructured) and saved using an MLflow model flavor format

:file-folder: Saving Model Artifacts

After training your model, you must save your model weights and any other inference-required dependencies along with a metadata file to your cloud storage of choice.

Resources

The following metadata file, i.e., model.yaml file, must be included to import your MLflow model.

specification: "0.3"
type: "file"
source: "Custom Model"
version: "0.0.1"
name: "Sample Model Name"
author: "Sample Model Author"
description:
  summary: "Short description of model."
  details: "Longer description of what the model does, what inputs it expects, and what outputs it returns."
  technical: |-
    #OVERVIEW:
    A continuation of "details" section that may include any information about model architecture, use cases, and any other general information.
    #TRAINING:
    Training-specific information, including links to the training dataset, a list of specific training techniques used, and any other development-specific information.
    #VALIDATION:
    Information about how the model was validated and any relevant dataset-specific information.
    #INPUT SPECIFICATION:
    Detailed description of expected inputs the model expects.
    #OUTPUT DETAILS:
    Detailed description of the output the model returns.
  performance: "Description about the model's performance on validation dataset. Include common machine learning evaluation metrics."
releaseNotes: "Short phrase describing new model version."

tags:[]
filters:
- type: "Task"
  label: "<task>"
- type: "Input Type"
  label: "<input data type>"
- type: "Subject"
  label: "<subject>"

metrics:
- label: "Probability"
  type: "percentage"
  value: 1.0
  description: |-
    Use case dependent
inputs:
  image:
    acceptedMediaTypes:
      - "image/jpeg"
      - "image/png"
      - "image/tiff"
    maxSize: 5M
    description: "Any jpg or png image for object detection"

outputs:
  results.json:
    mediaType: "application/json"
    maxSize: 128K
    description: "Classify objects in an image and give probabilities for each"

# replace size, cpu-count, and gpu-count values to reflect the resources your model needs to run
resources:
  memory:
    size: 6G
  cpu:
    count: 1
  gpu:
    count: 1

timeout:
  status: 20s # amount of time it will take your model to load and spin up
  run: 20s # amount of time it will take your model to complete inference 

# keep these values as are
internal:
  recommended: false
  experimental: false
  available: true
  active: true
  features:
    explainable: false
    adversarialDefense: false

After filling in this file with your model information, zip this file up as a tar archive named resources.tar.gz if you choose to use import your model directly within Modzy's user interface. Otherwise if you plan to use the SDK, there is a helper function that takes care of this for you.

Model Artifacts

Save your MLflow model (tabular or non-tabular) to a local directory. This directory should contain the contents of a properly saved MLflow model:

├── MLmodel
├── conda.yaml
└── data
    ├── model.pth
    └── pickle_module_info.txt



:construction: Executing Importer Job

When your model artifacts are saved in the appropriate cloud storage bucket, it is time to import your model to your Modzy instance. There are two methods for importing your model:

  1. Modzy's user interface
  2. Python SDK

User Interface

Navigate to models page within your Modzy account and select Add New Model.

1877

Figure 1. Models Page


On the home deployment page, select MLflow on the No-Code Model Importer section.

1791

Figure 2. Deployment Home Page


Fill in blob storage container, credentials to access blob storage (if private), model type, and paths to your model weights and resources archives.

823

Figure 3. Import MLflow Model


Wait until the converter job completes, and view your newly deployed model! Begin submitting jobs to this model using the Jobs API.

754

Figure 4. Converter Job Progress Bar


558

Figure 5. Converter Job Completion


Programmatic (SDK)

Import packages and set environment variables (MODZY_API_KEY, SP_ACCESS_KEY_ID and SP_SECRET_ACCESS_KEY) with your credentials.

import os
import json
import time

os.environ["MODZY_API_KEY"] = "modzy-api-key"
os.environ["SP_ACCESS_KEY"] = "cloud-storage-access-key"
os.environ["SP_SECRET_ACCESS_KEY"] = "cloud-storage-secret-key"

Use the Python SDK to set up an API Client to the Model Converter Service, which is the Modzy service that allows you to import your Sagemaker model directly.

from modzy.converter.model_converter import ModelConverter
from modzy.client import ApiClient

# Create a Modzy API client to interact with your Modzy envrionment
modzy_api_key = os.getenv("MODZY_API_KEY")
modzy_instance_base_url = "https://<your-modzy-instance-url>/api"
modzy_api_client = ApiClient(api_key=modzy_api_key, base_url=modzy_instance_base_url)

# Instantiate a Model Converter client with access to your Modzy environment
model_converter = ModelConverter(modzy_api_client)

Define blob storage provider and container. This container inside your preferred cloud storage provider is where you will upload your model artifacts.

blob_storage_provider = "S3"
blob_storage_container = "<path-to-your-s3-bucket>"

Define local paths to model.yaml, your model weights file(s), and any auxiliary files required for inference (e.g., labels.json for an image classification model).

from modzy.converter.utils import upload_resources

current_working_directory = os.getcwd()

# Define source platform and model type
source_platform = "mlflow"
model_type = "tabular"

# Paths to your local files
model_weights_path = os.path.join(current_working_directory, "local-file-path/weights.pt") # TODO check this type of file
model_yaml_path = os.path.join(current_working_directory, "local-file-path/model.yaml")

# Your local files will be archieved and stored in the following location:
weights_key = "cloud-storage-path-to-weights/weights.tar.gz"
resources_key = "cloud-storage-path-to-resources/resources.tar.gz"

# Utility function to wrap resources in tar archive and upload to specified cloud storage filepath
upload_resources(
    model_yaml_path, blob_storage_container, resources_key,
    os.getenv("SP_ACCESS_KEY_ID"), os.getenv("SP_SECRET_ACCESS_KEY"), blob_storage_provider
)

# Utility function to wrap model weights in tar archive and upload to specified cloud storage filepath
upload_model_dir(
    model_weights_path, blob_storage_container, weights_key,
    os.getenv("SP_ACCESS_KEY_ID"), os.getenv("SP_SECRET_ACCESS_KEY"), blob_storage_provider, "mlflow"
)

Kick off importer job.

# Now, provide the Model converter with information about your stored model assets and the credentials required
# to access them. The Model converter will do the rest of the work.

converter_output = model_converter.start(
    sp_access_key_id=os.getenv("SP_ACCESS_KEY_ID"),
    sp_secret_access_key=os.getenv("SP_SECRET_ACCESS_KEY"),
    blobstore_provider=blob_storage_provider,
    blobstore_container=blob_storage_container,
    weights_path=weights_key,
    resources_path=resources_key,
    platform=source_platform,
    model_type=model_type,
)

job_id = converter_output["jobId"]

Check initial status of converter job.

print("Model Converter Status: {} - {}".format(converter_output["status"], converter_output["message"]) + '\n' + "Model Converter Job ID: {}".format(job_id))

Wait until converter job completes. This process usually takes a few minutes.

# Block Model Converter Job until complete
start_time = time.time()
converter_result = model_converter.block_until_complete(job_id, timeout=600)
end_time = time.time()
model_id = converter_result["modelId"]
model_version = converter_result["modelVersion"]
print("Model Converter Job Status: {}".format(converter_result["message"]) + '\n' + "Completed in {} minutes".format((end_time - start_time)/60) + '\n' + "Find your newly deployed model here: {}/models/{}/{}".format(modzy_instance_base_url[:-4], model_id, model_version))

Click on the output link which will take you to your newly deployed model on Modzy. Start submitting jobs right away using the Jobs API.


What’s Next