AWS

Deployment Guide for Launching Modzy on AWS

Overview

This guide provides step-by-step instructions for deploying Modzy into an existing AWS VPC using the Modzy AWS Terraform module. Note that the VPC must meet the requirements for an AWS EKS cluster as outlined in the AWS documentation.

The guide assumes some familiarity with Terraform by Hashicorp/use of Terraform modules to provision infrastructure in the cloud.

Architecture

With the VPC and subnets created prior, executing the steps outlined in the guide builds the following Modzy environment in the AWS Cloud:

  • A highly available architecture that spans three Availability Zones.

AWS Services

AWS Services used by Modzy


Deployment Cost

The overall cost of a Modzy environment will vary based on the type, quantity and purchase method of the underlying AWS resources. Prices are subject to change; for current pricing, visit each AWS service home page.

Modzy Platform

The Modzy Platform resources are the minimum required set of resources to be able to run models but not including any of the inference nodes used to run models. The pricing list is smaller than the services used list as some of the costs below, like Simple Storage Service (S3), could grow depending on the amount of customer data in a deployment. Other services like Simple Email Service give you the first 62k emails per month for free and are unlikely to ever add a significant cost to a deployment. If the service is going to cost less than $1/month and is unlikely to grow, it’s not been included in the pricing list.

Table 1. Modzy core platform monthly component costs w/default parameters and on-demand pricing in us-east-1

Resource

Quantity

Unit Price/Month

Cost

Amazon Elastic Kubernetes Service (EKS) Control Plane

1

$73.00

$73.00

Amazon Elastic Compute Cloud (EC2) - Modzy Platform

3 x m5.8xlarge (4 vCPU + 16GiB RAM) + 100GB SSD Storage

$140.16 Platform Server + $10.00 storage volume

$450.48

Load Balancer

1

$16.42/month + bandwidth processed

$20

NAT Gateway

3

$32.85/month

$98.55

AWS Simple Storage Service (S3)

5GB in an empty environment

$0.023/GB/month

$0.12

RDS Postgres Database (Multi-AZ H/A Deployment)

1

$211.70 Server + 10GB storage 1.15

$212.85

Subtotal

$871.18


Modzy Inference Nodes

Modzy inference nodes run your AI/ML jobs. The type, size, and quantity of inference nodes are determined by you/configurable parameters for the deployment. EC2 pricing is by the second with a minimum run time of 10 minutes as the Modzy platform has a scheduling optimization to reduce result latency on a busy system by ensuring no other jobs need to be run for up to 10 minutes before terminating an inference node.

Table 2. Sample Modzy inference nodes hourly cost at on-demand pricing rates in us-east-1

Inference Node Type

GPU(s)

vCPU(s)

RAM (GiB)

Hourly Cost

Small CPU m5.large

2

8

$0.096

Medium CPU - m5.xlarge

4

16

$0.192

Large CPU - m5.2xlarge

8

61

$0.384


Deployment Cost Examples

Monthly cost of 15 concurrent processing engines running 24x7 using medium CPU inference nodes
$970/month = ($0.192/hr 730 hrs 15 nodes) CPU inference nodes + $870 Modzy platform

Monthly cost of 15 concurrent processing engines running 24x7 using medium GPU inference nodes
$134,898/month = ($12.24 730 hrs 15 nodes) GPU inference nodes + $870 Modzy platform

These are assuming 24x7 operation of the inference nodes, the platform will scale down any unused inference nodes that are not pre-provisioned as part of the model settings after 10 minutes of inactivity. Another option to reduce costs if the workload has at least a partial steady-state is to pre-purchase EC2 reserved instances where you pay upfront for future utilization in return for a discount.

Planning the Deployment

AWS Account

If you don’t already have an AWS account, create one at https://aws.amazon.com by following the on-screen instructions. Part of the sign-up process involves receiving a phone call and entering a PIN using the phone keypad.

Technical Requirements

Before provisioning the Modzy infrastructure as outlined in the steps that follow, check your account service quota limits at https://console.aws.amazon.com/servicequotas/home and ensure you have sufficient capacity available before launching the deployment, otherwise, the deployment will fail.

Table 3. Service Quota Utilization

Resource

Default Quota

Modzy Default Configuration

VPCs

5 per region

1

VPC Elastic IPs

5 per region

4

VPC security groups

2,500 per region

6

IAM roles

1,000 per account

3

Auto Scaling Groups

200 per region

More than 3
(3 for the platform + number of inference node groups configured)

m5.xlarge ec2 instances

3

Other ec2 instances

Number and types configured for inference node groups

Table 4. Other Technical Requirements

Requirement

Description

Regions

Amazon EKS and Amazon EFS aren’t currently supported in all AWS Regions. For a current list of supported Regions, see Service Endpoints and Quotas in the AWS documentation.

Key pair

Make sure that at least one Amazon EC2 key pair exists in your AWS account in the region for the deployment. Make a note of the key pair name as it’s a required parameter in the deployment launch screen. To create a key pair, follow the instructions in the AWS documentation. If the deployment is for testing or proof-of-concept purposes, we recommend that you create a new key pair instead of specifying a key pair that’s already being used by a production instance.

IAM permissions

To deploy Modzy, you must have credentials with IAM permissions for the resources and actions the templates will deploy. The AdministratorAccess managed policy within IAM provides sufficient permissions, although your organization may choose to use a custom policy with more restrictions.

Deployment Steps

The following steps deploy Modzy in your existing AWS infrastructure, into a VPC that must be configured prior to the deployment. This VPC must meet the minimum requirement of having 3 public and 3 private subnets provisioned and configured.

Prerequisites

Domain and SSL Certificate

  • Control of an internet domain. A fully qualified domain name (FQDN) for the planned Modzy instance is required for provisioning.
  • Use a service (such as Let’s Encrypt) to request a certificate for the FQDN with the domain name and wildcard (Example: modzy.mycompany.com and *.modzy.mycompany.com). The certificate and key files are required for platform configuration after installation. This certificate will be used by the API and the image registry.
  • Ensure your domain has been verified in AWS Simple Email Service (SES) in the region in which you deploy
    • SES is used to send user account registration emails

SAML Identity Provider Client Configuration
The following items are needed to configure the Modzy client in the SAML Identity Provider:

  • SSL URL: https://FQDN/saml/SSO

  • Audience URI (SP Entity ID): com.modzy.saml.sp

    Where FQDN is the fully qualified domain name for the Modzy instance (Ex.: mymodzy.mydomain.com) and must match the FQDN input parameter the Modzy instance is to be provisioned with.

Client Tools
The following tools are required on the client machine from which provisioning of Modzy environment will be executed:

  • AWS CLI

  • Terraform CLI

  • kubectl kubernetes command-line tool

  • kubectl kots plugin
    NOTE: The Modzy platform requires kots plugin version to be at most v1.50.0 which can be installed as follows:

    curl -LJ0 https://raw.githubusercontent.com/modzy/modzy-kots-install/v1.0.0/kots-install-v1.50.0.sh | bash
    

    (Ensure needed permissions for executing the script locally)

    Alternatively, the 1.50.0 version of the KOTS plugin may also be installed from here

The following files are needed to configure the Modzy instance :

  • A Modzy license file

  • The metadata (file) for the SAML Identity Provider to be used

Additionally, networking information (VPC Id, Subnet Ids) for the VPC to provision the Modzy environment into must be determined prior to execution. These and other parameter values will be needed to set up variables for the Modzy terraform module.

Provisioning Modzy Environment

Provisioning a Modzy Environment is comprised of the following steps:

  • Provision Infrastructure
    • Setup Modzy module with input variables
    • Run Terraform Init, Plan and Apply to create the infrastructure

  • Deploy Modzy Platform
    • Install Modzy Platform
    • Upload license, configure and deploy


Provision Infrastructure

  1. Setup Modzy module with input variables

    • Clone the repository as shown below for sample files. The example creates the folder modzy (this can be any valid name for a directory) and clones the contents of the repository :

      git clone https://github.com/modzy/terraform-aws-modzy-sample modzy
      

      This should create the directory with contents as shown below:

      modzy
      │- providers.tf
      │- modzy.tf
      
    • Edit the modzy.tf template file to specify appropriate input variables for the Modzy environment to provision

      Note: Each variable is described in detail in the Modzy Terraform module documentation

  2. In a command-line/shell, from within modzy directory,

    • Set the AWS credentials profile for the command-line/shell window
      export AWS_PROFILE=your-aws-profile-name
      
    • Execute the following in turn, reviewing the output from each :
      • Init
        terraform init
        
      • Plan
        terraform plan
        
      • Apply
        terraform apply
        

    On successful completion of Terraform Apply, two local files will be created which will be used in the next section to deploy/configure the Modzy platform :

    • kubeconfig_installation-identifier
    • replicatedconfiginstallation-identifier.yaml

      (where installation-identifier is the value supplied for the installation-identifier variable and a postfix of randomly generated string of characters)

    Note: In addition, secrets used by the installation are created in Secrets Manager in the form /modzy/{installation-identifier}/secret-type

Deploy Modzy Platform

  • Install Modzy platform

    • Set the AWS credentials profile for the command-line/shell window

      export AWS_PROFILE=your-aws-profile-name
      
    • Use the kubeconfig file generated above to access the cluster context as shown below :

      export KUBECONFIG=$(pwd)/kubeconfig_<installation-identifier>
      
    • Verify you are in the desired kubernetes cluster context by executing

      kubectl config current-context
      

      Expected output: eks_installation-identifier

    • Export install command from Terraform output with :

      KOTS_CMD=$(terraform output -raw kots_install_cmd)
      
    • Execute the installation :

      sh -c $KOTS_CMD
      

      (Note: There may be a need to execute the above with sudo for sufficient permissions)

      The following output should be displayed on completion of execution :

      kubectl kots install modzy/stable \
      --namespace modzy \
      --shared-password "<password-value>" \
      --config-values ./replicated_config_<installation-identifier>.yaml \
      --port-forward=true
      • Deploying Admin Console
          • Creating namespace ✓
          • Waiting for datastore to be ready ✓
      • Waiting for Admin Console to be ready ✓
      
      • Press Ctrl+C to exit
      • Go to http://localhost:8800 to access the Admin Console
      

      (Note: The Admin Console Url and password-value for use in next steps. If not displayed, the password is also available in secrets manager /modzy/{installation-identifier}/app as config-password)

  • Log in to Admin Console and Upload license

    • In a browser, navigate to the Admin Console URL as indicated at the bottom of the above output
      Log in to the Admin Console with the password-value displayed in the output and upload the license file
  • Ensure the KOTS version indicated at the bottom of the page is <= 1.50.0
  • After successfully validating the license, a configuration page with various sections will be presented. Specify the indicated items in the sections listed:

    • General: Choose AWS as the Installation Profile
    • Security: Choose between SAML and Cognito Identity Provider
    • Ingress Settings: Certificate and Private Key
    • Database Settings: Choose between embedded and your own database
    • Storage Settings: Leave defaults to use AWS S3
    • SSO Settings: Identity Provider Metadata
    • SMTP Settings: From Address (Ex. [email protected])
  • On completion, click the Save button at the bottom of the page. This brings up the preflight checks. Wait for completion of the checks and click Continue

  • A status page such as the example shown below should be displayed indicating successful initiation of deployment and configuration of containers in the cluster
  • Configure DNS
    After the above configuration steps, you may end port-forwarding to the admin-console.

    • Obtain load balancer address
      While still in the kubernetes context for the cluster, execute the following to see an ingress listing for the Modzy environment
    kubectl get ingress -n modzy
    

    Note the load balancer indicated in the Address property of the resulting listed ingresses - use this value to create DNS entries for

    • Fully qualified domain name (FQDN) for the Modzy Environment
    • registry.[FQDN for the Modzy Environment]

Deprovisioning a Modzy Environment

S3

The Modzy installation creates three S3 buckets:

  • [installation-identifier]-model-assets-data
  • [installation-identifier]-job-data
  • [installation-identifier]-results-data

To successfully run terraform destroy to deprovision the Modzy Infrastructure, the buckets must be emptied.


Tip:
Using the AWS CLI, an S3 bucket can be emptied by executing:

aws s3 rm --recursive s3://[bucket name]

Alternatively, if there is a desire to preserve any of the buckets (and contents), you may remove the bucket from the Terraform state by executing the following

terraform state rm aws_s3_bucket.<bucket name>

RDS

The Modzy RDS Database is created with the Deletion Protection property enabled. This must be disabled in the AWS console in order for terraform to successfully remove the database if so desired.

You may also exclude the database from Terraform state management by executing the following

terraform state rm aws_db_instance.modzy

Deprovision Infrastructure

To remove the resources created by Terraform, execute the following

terraform destroy

Did this page help you?