Overview

This guide provides step-by-step instructions for deploying Modzy into an existing AWS VPC using the Modzy AWS Terraform module. Note that the VPC must meet the requirements for an AWS EKS cluster as outlined in the AWS documentation.

The guide assumes some familiarity with Terraform by Hashicorp/use of Terraform modules to provision infrastructure in the cloud.

Architecture

With the VPC and subnets created prior, executing the steps outlined in the guide builds the following Modzy environment in the AWS Cloud:

• A highly available architecture that spans three Availability Zones.

AWS Services

AWS Services used by Modzy

• CloudWatch
• Elastic Block Store (EBS)
• Elastic Compute Cloud (EC2)
• Elastic Kubernetes Service (EKS)
• Elastic Load Balancing (ELB)
• Key Management Service (KMS)
• Relational Database Service (RDS)
• AWS Simple Storage Service (S3)
• Secrets Manager
• Simple Email Service
• Virtual Private Cloud (VPC)

Deployment Cost

The overall cost of a Modzy environment will vary based on the type, quantity and purchase method of the underlying AWS resources. Prices are subject to change; for current pricing, visit each AWS service home page.

Modzy Platform

The Modzy Platform resources are the minimum required set of resources to be able to run models but not including any of the inference nodes used to run models. The pricing list is smaller than the services used list as some of the costs below, like Simple Storage Service (S3), could grow depending on the amount of customer data in a deployment. Other services like Simple Email Service give you the first 62k emails per month for free and are unlikely to ever add a significant cost to a deployment. If the service is going to cost less than $1/month and is unlikely to grow, it’s not been included in the pricing list.

Table 1. Modzy core platform monthly component costs w/default parameters and on-demand pricing in us-east-1

Modzy Inference Nodes

Modzy inference nodes run your AI/ML jobs. The type, size, and quantity of inference nodes are determined by you/configurable parameters for the deployment. EC2 pricing is by the second with a minimum run time of 10 minutes as the Modzy platform has a scheduling optimization to reduce result latency on a busy system by ensuring no other jobs need to be run for up to 10 minutes before terminating an inference node.

Table 2. Sample Modzy inference nodes hourly cost at on-demand pricing rates in us-east-1

Deployment Cost Examples

Monthly cost of 10 concurrent processing engines running 24x7 using small CPU inference nodes
$2000/month = ($0.096/hr x 730 hrs x 10 nodes) CPU inference nodes + $1300 Modzy platform

Monthly cost of 10 concurrent processing engines running 24x7 using small GPU inference nodes
$7870/month = ($0.90 x 730 hrs x 10 nodes) GPU inference nodes + $1300 Modzy platform

These estimates assume 24x7 operation of each inference node, but the platform will scale down any unused inference nodes that are not pre-provisioned as part of the model settings after 10 minutes of inactivity. Another option to reduce costs if the workload has at least a partial steady-state is to pre-purchase EC2 reserved instances where you pay upfront for future utilization in return for a discount.

Planning the Deployment

AWS Account

If you don’t already have an AWS account, create one at https://aws.amazon.com by following the on-screen instructions. Part of the sign-up process involves receiving a phone call and entering a PIN using the phone keypad.

Technical Requirements

Before provisioning the Modzy infrastructure as outlined in the steps that follow, check your account service quota limits at https://console.aws.amazon.com/servicequotas/home and ensure you have sufficient capacity available before launching the deployment, otherwise, the deployment will fail.

Table 3. Service Quota Utilization

Table 4. Other Technical Requirements

Deployment Steps

The following steps deploy Modzy in your existing AWS infrastructure, into a VPC that must be configured prior to the deployment. This VPC must meet the minimum requirement of having 3 public and 3 private subnets provisioned and configured.

Prerequisites

Domain and SSL Certificate

• Control of an internet domain. A fully qualified domain name (FQDN) for the planned Modzy instance is required for provisioning.
• Use a service (such as Let’s Encrypt) to request a certificate for the FQDN with the domain name and wildcard (Example: modzy.mycompany.com and *.modzy.mycompany.com). The certificate and key files are required for platform configuration after installation. This certificate will be used by the API and the image registry.
• Ensure your domain has been verified in AWS Simple Email Service (SES) in the region in which you deploy
  • SES is used to send user account registration emails

SAML Identity Provider Client Configuration

The following items are needed to configure the Modzy client in the SAML Identity Provider:

• SSL URL: https://FQDN/saml/SSO

• Audience URI (SP Entity ID): com.modzy.saml.sp

  Where FQDN is the fully qualified domain name for the Modzy instance (Ex.: mymodzy.mydomain.com) and must match the FQDN input parameter the Modzy instance is to be provisioned with.

Client Tools
The following tools are required on the client machine from which provisioning of Modzy environment will be executed:

• AWS CLI

• Terraform CLI

• kubectl kubernetes command-line tool

• kubectl kots plugin

  Shell

  curl https://kots.io/install | bash
  

  Alternatively, the 1.75.0 version (or higher) of the KOTS plugin may also be installed from here

The following files are needed to configure the Modzy instance :

• A Modzy license file

• The metadata (file) for the SAML Identity Provider to be used
  
  

Additionally, networking information (VPC Id, Subnet Ids) for the VPC to provision the Modzy environment into must be determined prior to execution. These and other parameter values will be needed to set up variables for the Modzy terraform module.

Provisioning Modzy Environment

Provisioning a Modzy Environment is comprised of the following steps:

• Provision Infrastructure
  • Setup Modzy module with input variables
  • Run Terraform Init, Plan and Apply to create the infrastructure

• Deploy Modzy Platform
  • Install Modzy Platform
  • Upload license, configure and deploy

Provision Infrastructure

1. Setup Modzy module with input variables

   • Clone the repository as shown below for sample files. The example creates the folder modzy (this can be any valid name for a directory) and clones the contents of the repository :

     Shell

     git clone https://github.com/modzy/terraform-aws-modzy-sample modzy
     

     This should create the directory with contents as shown below:

     Shell

     modzy
     │- providers.tf
     │- modzy.tf
     

   • Edit the modzy.tf template file to specify appropriate input variables for the Modzy environment to provision

     Note: Each variable is described in detail in the Modzy Terraform module documentation

2. In a command-line/shell, from within modzy directory,

   • Set the AWS credentials profile for the command-line/shell window

     Shell

     export AWS_PROFILE=your-aws-profile-name
     

   • Execute the following in turn, reviewing the output from each :

     • Init
       Shell

       terraform init
       

     • Plan
       Shell

       terraform plan
       

     • Apply

       terraform apply
       

     On successful completion of Terraform Apply, two local files will be created which will be used in the next section to deploy/configure the Modzy platform :

   • kubeconfig_installation-identifier

   • replicatedconfig__installation-identifier.yaml
     
     (where installation-identifier is the value supplied for the installation-identifier variable and a postfix of randomly generated string of characters)
     
     

     Note: In addition, secrets used by the installation are created in Secrets Manager in the form /modzy/{installation-identifier}/secret-type

Deploy Modzy Platform

• Install Modzy platform

  • Set the AWS credentials profile for the command-line/shell window

    Shell

    export AWS_PROFILE=your-aws-profile-name
    

  • Use the kubeconfig file generated above to access the cluster context as shown below :

    Shell

    export KUBECONFIG=$(pwd)/kubeconfig_<installation-identifier>
    

  • Verify you are in the desired kubernetes cluster context by executing

    Shell

    kubectl config current-context
    

    Expected output: eks_installation-identifier

  • Export install command from Terraform output with :

    Shell

    KOTS_CMD=$(terraform output -raw kots_install_cmd)
    

  • Execute the installation :

    Shell

    sh -c $KOTS_CMD
    

    (Note: There may be a need to execute the above with sudo for sufficient permissions)

    The following output should be displayed on completion of execution :

    Shell

    kubectl kots install modzy/stable \
    --namespace modzy \
    --shared-password "<password-value>" \
    --config-values ./replicated_config_<installation-identifier>.yaml \
    --port-forward=true
    • Deploying Admin Console
        • Creating namespace ✓
        • Waiting for datastore to be ready ✓
    • Waiting for Admin Console to be ready ✓
    
    • Press Ctrl+C to exit
    • Go to http://localhost:8800 to access the Admin Console
    

    (Note: The Admin Console Url and password-value for use in next steps. If not displayed, the password is also available in secrets manager /modzy/{installation-identifier}/app as config-password)

• Log in to Admin Console and Upload license

  • In a browser, navigate to the Admin Console URL as indicated at the bottom of the above output
    Log in to the Admin Console with the password-value displayed in the output and upload the license file

• Ensure the KOTS version indicated at the bottom of the page is >= 1.75.0

• After successfully validating the license, a configuration page with various sections will be presented. Specify the indicated items in the sections listed:

  • General: Choose AWS as the Installation Profile
  • Security: Choose between SAML and Cognito Identity Provider
  • Ingress Settings: Certificate and Private Key
  • Database Settings: Choose between embedded and your own database
  • Storage Settings: Leave defaults to use AWS S3
  • SSO Settings: Identity Provider Metadata
  • SMTP Settings: From Address (Ex. [email protected])

• On completion, click the Save button at the bottom of the page. This brings up the preflight checks. Wait for completion of the checks and click Continue

• A status page such as the example shown below should be displayed indicating successful initiation of deployment and configuration of containers in the cluster

• Configure DNS
  After the above configuration steps, you may end port-forwarding to the admin-console.

  • Obtain load balancer address
    While still in the kubernetes context for the cluster, execute the following to see an ingress listing for the Modzy environment
  Shell

  kubectl get ingress -n modzy
  

  Note the load balancer indicated in the Address property of the resulting listed ingresses - use this value to create DNS entries for

  • Fully qualified domain name (FQDN) for the Modzy Environment
  • registry.[FQDN for the Modzy Environment]

Deprovisioning a Modzy Environment

S3

The Modzy installation creates three S3 buckets:

• [installation-identifier]-model-assets-data
• [installation-identifier]-job-data
• [installation-identifier]-results-data

To successfully run terraform destroy to deprovision the Modzy Infrastructure, the buckets must be emptied.

Tip:
Using the AWS CLI, an S3 bucket can be emptied by executing:

aws s3 rm --recursive s3://[bucket name]

Alternatively, if there is a desire to preserve any of the buckets (and contents), you may remove the bucket from the Terraform state by executing the following

terraform state rm aws_s3_bucket.<bucket name>

RDS

The Modzy RDS Database is created with the Deletion Protection property enabled. This must be disabled in the AWS console in order for terraform to successfully remove the database if so desired.

You may also exclude the database from Terraform state management by executing the following

terraform state rm aws_db_instance.modzy

Deprovision Infrastructure

To remove the resources created by Terraform, execute the following

terraform destroy