AWS
Deployment Guide for Launching Modzy on AWS
Overview
This guide provides step-by-step instructions for deploying Modzy into an existing AWS VPC using the Modzy AWS Terraform module. Note that the VPC must meet the requirements for an AWS EKS cluster as outlined in the AWS documentation.
The guide assumes some familiarity with Terraform by Hashicorp/use of Terraform modules to provision infrastructure in the cloud.
Architecture
With the VPC and subnets created prior, executing the steps outlined in the guide builds the following Modzy environment in the AWS Cloud:
- A highly available architecture that spans three Availability Zones.
AWS Services
AWS Services used by Modzy
- CloudWatch
- Elastic Block Store (EBS)
- Elastic Compute Cloud (EC2)
- Elastic Kubernetes Service (EKS)
- Elastic Load Balancing (ELB)
- Key Management Service (KMS)
- Relational Database Service (RDS)
- AWS Simple Storage Service (S3)
- Secrets Manager
- Simple Email Service
- Virtual Private Cloud (VPC)
Deployment Cost
The overall cost of a Modzy environment will vary based on the type, quantity and purchase method of the underlying AWS resources. Prices are subject to change; for current pricing, visit each AWS service home page.
Modzy Platform
The Modzy Platform resources are the minimum required set of resources to be able to run models but not including any of the inference nodes used to run models. The pricing list is smaller than the services used list as some of the costs below, like Simple Storage Service (S3), could grow depending on the amount of customer data in a deployment. Other services like Simple Email Service give you the first 62k emails per month for free and are unlikely to ever add a significant cost to a deployment. If the service is going to cost less than $1/month and is unlikely to grow, it’s not been included in the pricing list.
Table 1. Modzy core platform monthly component costs w/default parameters and on-demand pricing in us-east-1
Resource | Quantity | Unit Price/Month | Cost |
---|---|---|---|
Amazon Elastic Kubernetes Service (EKS) Control Plane | 1 | $73.00 | $73.00 |
Amazon Elastic Compute Cloud (EC2) - Modzy Platform | 3 x m5.2xlarge (8 vCPU + 32GiB RAM) + 100GB SSD Storage | $280.32 Platform Server + $10.00 storage volume | $870.96 |
Load Balancer | 1 | $16.42/month + bandwidth processed | $20 |
NAT Gateway | 3 | $32.85/month | $98.55 |
AWS Simple Storage Service (S3) | 100GB in an empty environment (1TB recommended) | $0.023/GB/month | $2.30 to $23 |
RDS Postgres Database (Multi-AZ H/A Deployment) | 1 Multi-AZ db.r5.xlarge (recommended) + 100GB SSD Storage | $211.70 (db.t3.large) $730.00 (db.r5.xlarge) $10 storage volume | $221.70 (small install) $740 (recommended production) |
Subtotal | $1,307.21 |
Modzy Inference Nodes
Modzy inference nodes run your AI/ML jobs. The type, size, and quantity of inference nodes are determined by you/configurable parameters for the deployment. EC2 pricing is by the second with a minimum run time of 10 minutes as the Modzy platform has a scheduling optimization to reduce result latency on a busy system by ensuring no other jobs need to be run for up to 10 minutes before terminating an inference node.
Table 2. Sample Modzy inference nodes hourly cost at on-demand pricing rates in us-east-1
Inference Node Type | GPU(s) | vCPU(s) | RAM (GiB) | Hourly Cost |
---|---|---|---|---|
Small CPU m5.large | - | 2 | 8 | $0.096 |
Medium CPU - m5.xlarge | - | 4 | 16 | $0.192 |
Large CPU - m5.2xlarge | - | 8 | 32 | $0.384 |
Deployment Cost Examples
Monthly cost of 10 concurrent processing engines running 24x7 using small CPU inference nodes
$2000/month = ($0.096/hr x 730 hrs x 10 nodes) CPU inference nodes + $1300 Modzy platform
Monthly cost of 10 concurrent processing engines running 24x7 using small GPU inference nodes
$7870/month = ($0.90 x 730 hrs x 10 nodes) GPU inference nodes + $1300 Modzy platform
These estimates assume 24x7 operation of each inference node, but the platform will scale down any unused inference nodes that are not pre-provisioned as part of the model settings after 10 minutes of inactivity. Another option to reduce costs if the workload has at least a partial steady-state is to pre-purchase EC2 reserved instances where you pay upfront for future utilization in return for a discount.
Planning the Deployment
AWS Account
If you don’t already have an AWS account, create one at https://aws.amazon.com by following the on-screen instructions. Part of the sign-up process involves receiving a phone call and entering a PIN using the phone keypad.
Technical Requirements
Before provisioning the Modzy infrastructure as outlined in the steps that follow, check your account service quota limits at https://console.aws.amazon.com/servicequotas/home and ensure you have sufficient capacity available before launching the deployment, otherwise, the deployment will fail.
Table 3. Service Quota Utilization
Resource | Default Quota | Modzy Default Configuration |
---|---|---|
VPCs | 5 per region | 1 |
VPC Elastic IPs | 5 per region | 4 |
VPC security groups | 2,500 per region | 6 |
IAM roles | 1,000 per account | 3 |
Auto Scaling Groups | 200 per region | More than 3 (3 for the platform + number of inference node groups configured) |
m5.2xlarge ec2 instances | 3 | |
Other ec2 instances | Number and types configured for inference node groups |
Table 4. Other Technical Requirements
Requirement | Description |
---|---|
Regions | Amazon EKS and Amazon EFS aren’t currently supported in all AWS Regions. For a current list of supported Regions, see Service Endpoints and Quotas in the AWS documentation. |
Key pair | Make sure that at least one Amazon EC2 key pair exists in your AWS account in the region for the deployment. Make a note of the key pair name as it’s a required parameter in the deployment launch screen. To create a key pair, follow the instructions in the AWS documentation. If the deployment is for testing or proof-of-concept purposes, we recommend that you create a new key pair instead of specifying a key pair that’s already being used by a production instance. |
IAM permissions | To deploy Modzy, you must have credentials with IAM permissions for the resources and actions the templates will deploy. The AdministratorAccess managed policy within IAM provides sufficient permissions, although your organization may choose to use a custom policy with more restrictions. |
Deployment Steps
The following steps deploy Modzy in your existing AWS infrastructure, into a VPC that must be configured prior to the deployment. This VPC must meet the minimum requirement of having 3 public and 3 private subnets provisioned and configured.
Prerequisites
Domain and SSL Certificate
- Control of an internet domain. A fully qualified domain name (FQDN) for the planned Modzy instance is required for provisioning.
- Use a service (such as Let’s Encrypt) to request a certificate for the FQDN with the domain name and wildcard (Example: modzy.mycompany.com and *.modzy.mycompany.com). The certificate and key files are required for platform configuration after installation. This certificate will be used by the API and the image registry.
- Ensure your domain has been verified in AWS Simple Email Service (SES) in the region in which you deploy
- SES is used to send user account registration emails
SAML Identity Provider Client Configuration
The following items are needed to configure the Modzy client in the SAML Identity Provider:
-
SSL URL: https://FQDN/saml/SSO
-
Audience URI (SP Entity ID): com.modzy.saml.sp
Where FQDN is the fully qualified domain name for the Modzy instance (Ex.: mymodzy.mydomain.com) and must match the FQDN input parameter the Modzy instance is to be provisioned with.
Client Tools
The following tools are required on the client machine from which provisioning of Modzy environment will be executed:
-
kubectl kubernetes command-line tool
-
kubectl kots plugin
curl https://kots.io/install | bash
Alternatively, the 1.75.0 version (or higher) of the KOTS plugin may also be installed from here
The following files are needed to configure the Modzy instance :
-
A Modzy license file
-
The metadata (file) for the SAML Identity Provider to be used
Additionally, networking information (VPC Id, Subnet Ids) for the VPC to provision the Modzy environment into must be determined prior to execution. These and other parameter values will be needed to set up variables for the Modzy terraform module.
Provisioning Modzy Environment
Provisioning a Modzy Environment is comprised of the following steps:
- Provision Infrastructure
- Setup Modzy module with input variables
- Run Terraform Init, Plan and Apply to create the infrastructure
- Deploy Modzy Platform
- Install Modzy Platform
- Upload license, configure and deploy
Provision Infrastructure
-
Setup Modzy module with input variables
-
Clone the repository as shown below for sample files. The example creates the folder modzy (this can be any valid name for a directory) and clones the contents of the repository :
git clone https://github.com/modzy/terraform-aws-modzy-sample modzy
This should create the directory with contents as shown below:
modzy │- providers.tf │- modzy.tf
-
Edit the modzy.tf template file to specify appropriate input variables for the Modzy environment to provision
Note: Each variable is described in detail in the Modzy Terraform module documentation
-
-
In a command-line/shell, from within modzy directory,
-
Set the AWS credentials profile for the command-line/shell window
export AWS_PROFILE=your-aws-profile-name
-
Execute the following in turn, reviewing the output from each :
- Init
terraform init
- Plan
terraform plan
- Apply
terraform apply
On successful completion of Terraform Apply, two local files will be created which will be used in the next section to deploy/configure the Modzy platform :
- Init
-
kubeconfig_installation-identifier
-
replicatedconfig__installation-identifier.yaml
(where installation-identifier is the value supplied for the installation-identifier variable and a postfix of randomly generated string of characters)
Note: In addition, secrets used by the installation are created in Secrets Manager in the form /modzy/{installation-identifier}/secret-type
-
Deploy Modzy Platform
-
Install Modzy platform
-
Set the AWS credentials profile for the command-line/shell window
export AWS_PROFILE=your-aws-profile-name
-
Use the kubeconfig file generated above to access the cluster context as shown below :
export KUBECONFIG=$(pwd)/kubeconfig_<installation-identifier>
-
Verify you are in the desired kubernetes cluster context by executing
kubectl config current-context
Expected output: eks_installation-identifier
-
Export install command from Terraform output with :
KOTS_CMD=$(terraform output -raw kots_install_cmd)
-
Execute the installation :
sh -c $KOTS_CMD
(Note: There may be a need to execute the above with sudo for sufficient permissions)
The following output should be displayed on completion of execution :
kubectl kots install modzy/stable \ --namespace modzy \ --shared-password "<password-value>" \ --config-values ./replicated_config_<installation-identifier>.yaml \ --port-forward=true • Deploying Admin Console • Creating namespace ✓ • Waiting for datastore to be ready ✓ • Waiting for Admin Console to be ready ✓ • Press Ctrl+C to exit • Go to http://localhost:8800 to access the Admin Console
(Note: The Admin Console Url and password-value for use in next steps. If not displayed, the password is also available in secrets manager /modzy/{installation-identifier}/app as config-password)
-
-
Log in to Admin Console and Upload license
- In a browser, navigate to the Admin Console URL as indicated at the bottom of the above output
Log in to the Admin Console with the password-value displayed in the output and upload the license file
- In a browser, navigate to the Admin Console URL as indicated at the bottom of the above output
-
Ensure the KOTS version indicated at the bottom of the page is >= 1.75.0
-
After successfully validating the license, a configuration page with various sections will be presented. Specify the indicated items in the sections listed:
- General: Choose AWS as the Installation Profile
- Security: Choose between SAML and Cognito Identity Provider
- Ingress Settings: Certificate and Private Key
- Database Settings: Choose between embedded and your own database
- Storage Settings: Leave defaults to use AWS S3
- SSO Settings: Identity Provider Metadata
- SMTP Settings: From Address (Ex. [email protected])
-
On completion, click the Save button at the bottom of the page. This brings up the preflight checks. Wait for completion of the checks and click Continue

- A status page such as the example shown below should be displayed indicating successful initiation of deployment and configuration of containers in the cluster

-
Configure DNS
After the above configuration steps, you may end port-forwarding to the admin-console.- Obtain load balancer address
While still in the kubernetes context for the cluster, execute the following to see an ingress listing for the Modzy environment
kubectl get ingress -n modzy
Note the load balancer indicated in the Address property of the resulting listed ingresses - use this value to create DNS entries for
- Fully qualified domain name (FQDN) for the Modzy Environment
- registry.[FQDN for the Modzy Environment]
- Obtain load balancer address
Deprovisioning a Modzy Environment
S3
The Modzy installation creates three S3 buckets:
- [installation-identifier]-model-assets-data
- [installation-identifier]-job-data
- [installation-identifier]-results-data
To successfully run terraform destroy to deprovision the Modzy Infrastructure, the buckets must be emptied.
Tip:
Using the AWS CLI, an S3 bucket can be emptied by executing:
aws s3 rm --recursive s3://[bucket name]
Alternatively, if there is a desire to preserve any of the buckets (and contents), you may remove the bucket from the Terraform state by executing the following
terraform state rm aws_s3_bucket.<bucket name>
RDS
The Modzy RDS Database is created with the Deletion Protection property enabled. This must be disabled in the AWS console in order for terraform to successfully remove the database if so desired.
You may also exclude the database from Terraform state management by executing the following
terraform state rm aws_db_instance.modzy
Deprovision Infrastructure
To remove the resources created by Terraform, execute the following
terraform destroy
Updated 3 months ago