AWS

Deployment Guide for Launching Modzy on AWS

Overview

This Quick Start reference deployment guide provides step-by-step instructions for deploying Modzy. Modzy is the Enterprise AI platform and marketplace offering scalable, secure, and ready-to-deploy AI models.

Architecture

Deploying this Quick Start for a new virtual private cloud (VPC) using the default parameters builds the following Modzy environment in the AWS Cloud.

This Quick Start sets up the following:

  • A highly available architecture that spans three Availability Zones.
  • A VPC configured with public and private subnets according to AWS best practices, to provide you with a virtual network on AWS.

AWS Services

AWS Services used by Modzy

Deployment Cost

The AWS CloudFormation templates for this Quick Start include configuration parameters that you can customize. The overall cost of a Modzy environment will vary based on the type, quantity and purchase method of the underlying AWS resources. Prices are subject to change; for current pricing, visit each AWS service home page.

Modzy Platform

The Modzy Platform resources are the minimum required set of resources to be able to run models but not including any of the inference nodes used to run models. The pricing list is smaller than the services used list as some of the costs below, like Simple Storage Service (S3), could grow depending on the amount of customer data in a deployment. Other services like Simple Email Service give you the first 62k emails per month for free and are unlikely to ever add a significant cost to a deployment. If the service is going to cost less than $1/month and is unlikely to grow, it’s not been included in the pricing list.

Table 1. Modzy core platform monthly component costs w/default parameters and on-demand pricing in us-east-1

Resource

Quantity

Unit Price/Month

Cost

Amazon Elastic Kubernetes Service (EKS) Control Plane

1

$73.00

$73.00

Amazon Elastic Compute Cloud (EC2) - Modzy Platform

3 x m5.8xlarge (4 vCPU + 16GiB RAM) + 100GB SSD Storage

$140.16 Platform Server + $10.00 storage volume

$450.48

Load Balancer

1

$16.42/month + bandwidth processed

$20

NAT Gateway

3

$32.85/month

$98.55

AWS Simple Storage Service (S3)

5GB in an empty environment

$0.023/GB/month

$0.12

RDS Postgres Database (Multi-AZ H/A Deployment)

1

$211.70 Server + 10GB storage 1.15

$212.85

Subtotal

$871.18

Modzy Inference Nodes

Modzy inference nodes run your AI/ML jobs. The type, size, and quantity of inference nodes vary by workload. EC2 pricing is by the second with a minimum run time of 10 minutes as the Modzy platform has a scheduling optimization to reduce result latency on a busy system by ensuring no other jobs need to be run for up to 10 minutes before terminating an inference node.

Table 2. Modzy inference nodes hourly cost at on-demand pricing rates in us-east-1

Inference Node Type

GPU(s)

vCPU(s)

RAM (GiB)

Hourly Cost

Small CPU m5.large

2

8

$0.096

Medium CPU - m5.xlarge

4

16

$0.192

Large CPU - m5.2xlarge

8

61

$0.384

Deployment Cost Examples

Monthly cost of 15 concurrent processing engines running 24x7 using medium CPU inference nodes
$970/month = ($0.192/hr 730 hrs 15 nodes) CPU inference nodes + $870 Modzy platform

Monthly cost of 15 concurrent processing engines running 24x7 using medium GPU inference nodes
$134,898/month = ($12.24 730 hrs 15 nodes) GPU inference nodes + $870 Modzy platform

These are assuming 24x7 operation of the inference nodes, the platform will scale down any unused inference nodes that are not pre-provisioned as part of the model settings after 10 minutes of inactivity. Another option to reduce costs if the workload has at least a partial steady-state is to pre-purchase EC2 reserved instances where you pay upfront for future utilization in return for a discount.

Planning the Deployment

AWS Account

If you don’t already have an AWS account, create one at https://aws.amazon.com by following the on-screen instructions. Part of the sign-up process involves receiving a phone call and entering a PIN using the phone keypad.

Technical Requirements

Before you launch the Quick Start, check your account service quota limits at https://console.aws.amazon.com/servicequotas/home and ensure you have sufficient capacity available before launching the CloudFormation stack, otherwise, the deployment will fail.

Table 3. Service Quota Utilization

Resource

Default Quota

Modzy Default Configuration

VPCs

5 per region

1

VPC Elastic IPs

5 per region

4

VPC security groups

2,500 per region

8

IAM roles

1,000 per account

19

Auto Scaling Groups

200 per region

5

t3.small ec2 instances

1

m5.xlarge ec2 instances

3

p3.2xlarge ec2 instances

1

Table 4. Other Technical Requirements

Requirement

Description

Regions

Amazon EKS and Amazon EFS aren’t currently supported in all AWS Regions. For a current list of supported Regions, see Service Endpoints and Quotas in the AWS documentation.

Key pair

Make sure that at least one Amazon EC2 key pair exists in your AWS account in the region where you are planning to deploy the Quick Start. Make a note of the key pair name as it’s a required parameter in the deployment launch screen. To create a key pair, follow the instructions in the AWS documentation. If you’re deploying the Quick Start for testing or proof-of-concept purposes, we recommend that you create a new key pair instead of specifying a key pair that’s already being used by a production instance.

IAM permissions

To deploy the Quick Start, you must log in to the AWS Management Console with IAM permissions for the resources and actions the templates will deploy. The AdministratorAccess managed policy within IAM provides sufficient permissions, although your organization may choose to use a custom policy with more restrictions.

Deployment options

  • Deploy Modzy into a new VPC (end-to-end deployment). This option builds a new AWS environment consisting of the VPC, subnets, NAT gateways, security groups, bastion hosts, and other infrastructure components, and then deploys Modzy into this new VPC.
  • Deploy Modzy into an existing VPC. This option provisions Modzy in your existing AWS infrastructure, which will have to meet specific minimum requirements such as having 3 public and 3 private subnets provisioned and configured.
    The Quick Start provides separate templates for these options. It also lets you configure network CIDR blocks, instance types and quantities, and other Modzy platform configuration, as discussed later in this guide.

Deployment steps

Step 1. Sign in to your AWS account

  1. Sign in to your AWS account at https://aws.amazon.com with an IAM user role that has the necessary permissions. For details, see Planning the deployment earlier in this guide.
  2. Make sure that your AWS account is configured correctly, as discussed in the Technical requirements section.
  3. Use the region selector in the navigation bar to choose one of the following supported AWS Regions
    • us-east-1 (N. Virginia)
  4. Select the key pair that you created earlier. In the navigation pane of the Amazon EC2 console, choose Key Pairs, and then select your key pair from the list.

Step 2. Pre-deployment tasks

  • Have control of a an internet domain. We’ll use the fully qualified domain name (FQDN) in several of the launch steps
  • Use AWS Certificate Manager (ACM) to request a certificate with a primary name for the modzy platform and a secondary wildcard name which will be used by the admin-console
    • Example domain name: modzy.mycompany.com with an additional name of *.modzy.mycompany.com
    • Copy the certificate ARN which will be entered as a parameter when launching the stack
  • Use a service like Let’s Encrypt to request a certificate with the same two domain names and have the PEM key and certificate files ready for the platform configuration following the installation. This certificate will be used by the API and the image registry
  • Ensure your domain has been verified in AWS Simple Email Service (SES) in the region in which you deploy
    • SES is used to send user account registration emails

Step 3. Launch the stack

You are responsible for the cost of the AWS services used while running this Quick Start reference deployment. There is no additional cost for using this Quick Start. For full details, see the pricing pages for each AWS service you will be using in this Quick Start. Prices are subject to change.

  1. Choose either deploy to new VPC or deploy into an existing VPC discussed earlier in the Deployment Options section. Either option takes roughly one hour to complete.
  2. On the Select Template page, keep the default setting for the template URL, and then choose Next.
  3. On the Specify Details page, change the stack name if needed. Review the parameters for the template. Provide values for the parameters that require input. For all other parameters, review the default settings and customize them as necessary. In the following tables, parameters are listed by category and described separately for the two deployment options.

📘

Launch stack in a new VPC

Clicking this link while signed into your AWS account will take you to the CloudFormation console with the Launch in new VPC template pre-loaded

Option 1: Parameters for deploying Modzy into a new VPC

Table 5. Option 1. Deploying Modzy into a new VPC

Parameter Name

Default

Description

Cluster Name

None/Required

Kubernetes cluster name

AWS Certificate Manager Cert Arn

None/Required

Arn of ACM certificate used by the application

FQDN

None/Required

Fully qualified domain name (ex: modzy.yourcorp.com)

Kubernetes version

1.19

The Kubernetes control plane version

CPU Node Instance Type

m5.xlarge

CPU Node Instance Type

Min-Max CPU Node Count

4-8

The min and max number of on-demand CPU instances in the cluster ex: 4-8

Custom CPU Node AMI

None/Optional

Use a customized EKS AMI for CPU to support workloads

GPU Node Instance Type

p2.xlarge

GPU Node Instance Type

Min-Max GPU Node Count

0-4

The min and max number of on-demand CPU instances in the cluster ex: 0-4

Custom GPU Node AMI

None/Optional

Use a customized EKS AMI for GPU to support workloads

Node Volume Size (GB)

100

The size of node EBS volumes, in GB

Role to add to system:masters group

None/Optional

An optional IAM role arn to add to the administrator group

Base URL to download container images

Modzy Image Registry

A registry from which the platform will pull platform container images

Email address of the super-user account

None/Required

Admin account email

Public API Endpoint

Disabled

API endpoint managed through public DNS zone

Public API Endpoint Access CIDR

None/Required

CIDR range of IPs allowed to connect to the public api endpoint

SSH key name

None/Required

The name of an existing public/private key pair, which allows you to securely connect to your instance after it launches

Remote Access to Management Node

None/Required

The remote CIDR range for allowing SSH into the management instance

Availability Zones

None/Required

Select 3 availability zones to be used to construct a highly available VPC

VPC CIDR Prefix

10.210

The CIDR range prefix for the VPC

Database Instance Type

db.t3.large

Database EC2 Instance Type

High-Availability Database Configuration

true

Enhanced availability and durability

Quick Start S3 bucket name

aws-quickstart

S3 bucket name for the Quick Start assets

Quick Start S3 key prefix

quickstart-modzy/

Quick Start key prefix within the S3 bucket

Quick Start S3 bucket region

us-east-1

The AWS Region where the Quick Start S3 bucket (QSS3BucketName) is hosted

Option 2: Parameters for deployment of Modzy into an existing VPC

Table 6. Option 2. Deploying Modzy into an existing VPC

Parameter Name

Default

Description

Cluster Name

None/Required

Kubernetes cluster name

AWS Certificate Manager Cert Arn

None/Required

Arn of ACM certificate used by the application

FQDN

None/Required

Fully qualified domain name (ex: modzy.yourcorp.com)

Kubernetes version

1.19

The Kubernetes control plane version

CPU Node Instance Type

m5.xlarge

CPU Node Instance Type

Min-Max CPU Node Count

4-8

The min and max number of on-demand CPU instances in the cluster ex: 4-8

Custom CPU Node AMI

None/Optional

Use a customized EKS AMI for CPU to support workloads

GPU Node Instance Type

p2.xlarge

GPU Node Instance Type

Min-Max GPU Node Count

0-4

The min and max number of on-demand CPU instances in the cluster ex: 0-4

Custom GPU Node AMI

None/Optional

Use a customized EKS AMI for GPU to support workloads

Node Volume Size (GB)

100

The size of node EBS volumes, in GB

Kubernetes Namespace

modzy

The Kubernetes namespace into which Modzy will be installed

Role to add to system:masters group

None/Optional

An optional IAM role arn to add to the administrator group

Comma separated CIDR range(s) to add to the load balancer security group

None/Optional

Additional allow CIDR ranges for restricted access clusters

Base URL to download container images

Modzy Image Registry

A registry from which the platform will pull platform container images

Email address of the super-user account

None/Required

Admin account email

Private API Endpoint

Enabled

API endpoint managed through private DNS zone

Private API Endpoint Access CIDR

None/Required

CIDR range of IPs allowed to connect to the private api endpoint

Public API Endpoint

Disabled

API endpoint managed through pubilc DNS zone

Public API Endpoint Access CIDR

None/Required

CIDR range of IPs allowed to connect to the public api endpoint

SSH key name

None/Required

The name of an existing public/private key pair, which allows you to securely connect to your instance after it launches

Management Node State

Enabled

Bastion host contains useful cluster management tools

Management Node Instance Type

t3.small

Amazon EC2 instance type for the management node

Remote Access to Management Node

None/Required

The remote CIDR range for allowing SSH into the management instance

PublicSubnet1ID

None/Required

Public Subnet 1 ID

PublicSubnet2ID

None/Required

Public Subnet 2 ID

PublicSubnet3ID

None/Required

Public Subnet 3 ID

PrivateSubnet1ID

None/Required

Private Subnet 1 ID

PrivateSubnet2ID

None/Required

Private Subnet 2 ID

PrivateSubnet3ID

None/Required

Private Subnet 3 ID

VPCCIDR

None/Required

Full VPC CIDR ex: 10.210.0.0/16

VPCID

None/Required

ID of existing VPC

Database Instance Type

db.t3.large

Database EC2 Instance Type

High-Availability Database Configuration

true

Enhanced availability and durability

Quick Start S3 bucket name

aws-quickstart

S3 bucket name for the Quick Start assets

Quick Start S3 key prefix

quickstart-modzy/

Quick Start key prefix within the S3 bucket

Quick Start S3 bucket region

us-east-1

The AWS Region where the Quick Start S3 bucket (QSS3BucketName) is hosted

Step 4: Test the deployment

  • Wait until the main stack changes to CREATE_COMPLETE status (approx 1 hour)
  • Navigate to the EC2 >> Load Balancers console and locate the two load balancers created by the EKS stack
  • Create a DNS record that maps FQDN to the classic load balancer ex: modzy.mycompany.com, this is the API endpoint
  • Create a DNS record that maps .FQDN to the network load balancer ex: .modzy.mycompany.com, this is the admin console
  • Navigate to https://admin-console.$YOUR_MODZY_FQDN
    • You will see a login page to the admin console for the Modzy application
  • Go to the Secrets Manager and locate the application secret that matches your deployment stack
    • Naming convention is /modzy/STACK_NAME/app
  • Click the Retrieve Secret Value button and locate the modzy-config-password key and copy the value
    • All secrets are generated by the installer and are unique to each environment
  • Paste the config password into the admin-console screen and you’ll be presented with a license upload option
  • After successfully validating the license, additional configuration options will be presented and the cluster containers will be deployed and configured
Admin console password is auto-generated and can be found in Secrets Manager → /modzy/STACK_NAME/app → modzy-config-passwordAdmin console password is auto-generated and can be found in Secrets Manager → /modzy/STACK_NAME/app → modzy-config-password

Admin console password is auto-generated and can be found in Secrets Manager → /modzy/STACK_NAME/app → modzy-config-password

License upload screenLicense upload screen

License upload screen

Termination

Terminate the stack

  • Sign into your AWS account and ensure you’re in the same region you used to launch the stack
  • Navigate to the CLoudFormation console and locate the Modzy top-level stack
    • Sub-stacks will have a label that says Nested. You can use the View nested toggle to hide sub-stacks from the list
  • Select the top-level Modzy stack and click the Delete button from the CloudFormation console
  • Complete stack termination will take approximately 45 minutes

Remove all data

The Modzy stack will terminate all the resources it provisioned with the exception of those that store data. Follow the steps below to manually remove these data storage resources.

S3

There are three S3 buckets which name prefixes that match your stack in which job data, results data and model assets are stored.

RDS

The Modzy database will take a snapshot backup which is retained in RDS >> Manual Snapshots before terminating the database server


Did this page help you?