Deploy MLflow Server on Amazon EC2 Instance

Author(s): ronilpatil

Originally published on Towards AI.

Hi folks! Ready to take your model deployment game to the next level? Let’s dive into setting up an MLflow Server on an EC2 instance! I’ll explain the steps to configure Amazon S3 bucket to store the artifacts, Amazon RDS (Postgres & Mysql) to store metadata, and EC2 instance to host the mlflow server. So let’s begin!

Table Of Content
— Introduction
— Create S3 Bucket
— Create EC2 Instance (Ubuntu)
— Configure PostgreSQL DB
— VPS & Security Groups
— Configure MySQL DB
— Access MLflow Server Remotely
— GitHub Repo
— Conclusion

Introduction

In this blog, we’ll delve into the nuts and bolts of deploying an MLflow server on an Amazon EC2 instance, empowering you to harness the full potential of your machine-learning projects on the cloud. By hosting MLflow on Amazon EC2 with an S3 bucket and RDS instance as the backend, we leverage the scalability and flexibility of cloud computing, ensuring seamless access to our experiments and models from anywhere in the world.
This guide will walk you through the step-by-step process of setting up our own MLflow server on Amazon EC2, S3 Bucket & RDS instance as a backend store. Get ready to supercharge your machine-learning projects and unlock new levels of productivity. Let’s dive in!

Create S3 Bucket

In my previous blog, I explained the way to create S3 Bucket. You can refer to it.

Configure DVC with Amazon S3 Bucket

A step-by-step guide to configure DVC with Amazon S3 Bucket

pub.towardsai.net

Create EC2 Instance (Ubuntu)

Step 1. Search for EC2, go to instances (left panel), and click on Launch an instance. Give any name to instance & move down.

Step 2. For sure ubuntuU+1F5A4, select it & move down.

Step 3. Click on Create new key pair, a new panel will pop up.

Step 4. Enter the key pair name and click on Create key pair.

Step 5. Look, by default it will create a new security group. Now we’ll proceed with a new security group but we can also use the existing one. Mark all checkboxes and click on Launch instance.

Step 6. CongratsU+1F389 EC2 instance created successfully. Just go to your instance and click on Connect.

Step 7. Again click on Connect, and keep the remaining options as default.

Step 8. You’ll see a command line interface. That’s it!!!

Note: Our EC2 instance is ready, now first we need to set an inbound rule for the EC2 instance for port 5000. What!!! & Why?
Look, When we send data to your MLflow server, such as experiment metrics or parameters, we do so by making HTTP requests to specific API endpoints provided by MLflow. By default MLflow server uses port 5000, so by configuring the inbound rule for port 5000, we’re allowing these HTTP requests to reach our MLflow server running on our EC2 instance.
Now external systems or users can send these HTTP requests to our EC2 instance’s public IP address on port 5000, effectively interacting with our MLflow server and enabling functionalities such as experiment logging, metric tracking, and project management. For more details check out VPS & Security Groups section.

Step 9. Go to Instances, and click on the instance ID.

Step 10. Go to Security panel, and click on security group.

Step 11. Click on Edit inbound rules.

Step 12. Click on Add rule, select Type as Custom TCP, Port as 5000, and Source as 0.0.0.0/0 (incoming traffic from any IP address is allowed), and then save the rules. That’s it.

Configure PostgreSQL Database

Step 1. Search for RDS Services, click on Create database, and select Standard create & move down.

Step 2. Select PostgreSQL and move down.

Step 3. Select Free tier and move down.

Step 4. Give any name to DB instance and Master username and move down. Keep it safe, we will need it further.

Step 5. Give any password and move down.

Step 6. You’ll see Additional configuration panel, expand it and enter the database name, move down, and click on Create database.

CongratsU+1F389, PostgreSQL DB created successfully.

Look, our MLflow server is running on an EC2 instance and it's using PostgreSQL as backend storage right? But how EC2 will communicate with this database? Let me explain it separately.

VPS & Security Groups

AWS organizes resources into virtual networks called Virtual Private Clouds (VPCs). Think of a VPC as a private space in the AWS cloud where we can place our resources, like EC2 instances, RDS, ECS (Elastic Container Service), EKS (Elastic Kubernetes Service), Redshift (Fully managed data warehouse service), and many more.

This resource uses network configuration to communicate with others. Now we need something that can make this communication controlled & secure. Here Security Groups come into the picture.

Security groups are like virtual firewalls that control the flow of network traffic to and from these resources within a VPC. They help us to manage who can access our resources and what kind of traffic is allowed.

Inbound Traffic — The traffic coming into the resources.
Outbound Traffic — The traffic going out from the resources.

We’ve an EC2 instance where we’ve hosted an MLflow server. We also have a PostgreSQL RDS instance, which serves as a backend store where MLflow stores its metadata. This includes information about experiments, parameters, metrics, and more. By configuring the inbound rule in the security group associated with our PostgreSQL RDS instance we’re allowing incoming connections from your EC2 instance (where the MLflow server is hosted) to access the PostgreSQL database using port 3306.

In summary, by configuring the inbound rule to allow traffic on port 3306 from your EC2 instance to our PostgreSQL RDS instance, we enable communication between our MLflow server and the PostgreSQL database, facilitating the storage and retrieval of machine learning experiment data.

You may think “Why not configure the outbound rule for EC2?” Very genuine doubt, even I had the same. When we create a new security group, it starts with a default rule that allows all outbound traffic but denies all inbound traffic. It can send any type of data (protocol), through any port, to any destination on the internet. But it’s not the case with inbound traffic, hence we’re configuring them individually.

It was the case with new security group, but the EC2 instance provides one default security group & it has its own story. The default security group does allow inbound traffic from other instances that are members of the same security group. This means that instances within the same security group can communicate with each other over any protocol and port unless specific rules are added to restrict this communication. By default, our RDS services are using a default security group, but we can create a separate security group for them.

In our case EC2 instance and RDS instance are associated with different security groups therefore, we still need to create an inbound rule specifically for your PostgreSQL RDS instance to allow traffic from our EC2 instance. That’s itU+1F92F

Below I explained the steps to configure MySQL DB as a backend store, the same logic will be applicable there. Let’s configure the inbound rule.

Step 7. Go to our PostgreSQL DB, and click on VPC security groups.

Step 8. Click on Security group ID.

Step 9. Click on Edit inbound rules.

Step 10. Click on Add rule, select PostgreSQL as Type, select Security Group of EC2 instance, and save the rule. That’s it.

Configure MySQL Database

Step 1. Search for RDS service, click on Create database, and choose Standard create database creation method.

Step 2. Select MySQL engine, and move down.

Step 3. Select Free tier and move down.

Step 4. Give any name to DB instance and Master username, and move down.

Step 5. Enter the Master password, and move down.

Step 6. You’ll see Additional configuration panel, expand it and enter the database name, move down, and click on Create database.

CongratsU+1F389, MySQL DB created successfully.

Step 7. Let’s set an inbound rule for MySQL DB. Go to MySQL DB, and click on VPC security groups.

Step 8. Click on Security group ID.

Step 9. Click on Edit inbound rules.

Step 10. Click on Add rule, select MySQL/Aurora as Type, select Security Group of EC2 instance, and save the rule. That’s it.

Note: We can also create AmazonRDSFullAccess & AmazonS3FullAccess IAM roles but EC2 instances can access RDS and S3 without these roles, creating IAM roles with well-defined permissions provides security, control, and flexibility for managing access to these services within your AWS environment. It’s a recommended best practice for maintaining a secure and well-managed infrastructure in AWS.

Access MLflow Server Remotely

Step 1. Just go to your instance, and click on Connect. Make sure the instance is Running.

Step 2. Keep everything default, just click on Connect.

Step 3. The CLI will open, run the below commands.

sudo apt-get update
sudo apt-get install python3-pip
pip install mlflow boto3 psycopg2-binary

The first command will update the list of packages, 2nd will install pip specifically for Python3 so that we can use it to install Python libraries and tools, and 3rd will install the required libraries to run our server.

Once everything gets installed, execute the below command to start the MLflow server.

---- for postgresql db
cmd: mlflow server -h 0.0.0.0 -p 5000 
 --backend-store-uri postgresql://<master_username>:<master_password>@<database_endpoint>:5432/<database_name> 
 --default-artifact-root s3://<bucket_name>/<folder_name(if any)>

---- for mysql db
cmd: mlflow server -h 0.0.0.0 -p 5000 
 --backend-store-uri mysql+pymysql://<master_username>:<master_password>@<database_endpoint>:3306/<database_name> 
 --default-artifact-root s3://<bucket_name>/<folder_name(if any)>

All these credentials we’ve already configured while creating the database instance. Once you execute the above command, the server will be up & running.

Step 4. Just copy the ec2 instance endpoint and add our serving port at the end with “:” That's it.
URL: http://<<ec2_instance_endpoint>>:5000

CongratsU+1F389, finally we made it.

Use URL: http://<<ec2_instance_endpoint>>:5000 as HTTP URI in mlflow. That’s it.

GitHub Repo

GitHub: Deploy MLflow Server with EC2, S3 & Amazon RDS

Contributed by ronylpatil

github.com

Note: Codebase is available at the dedicated “aws” branch.

Conclusion

Deploying an MLflow server on an EC2 instance with Amazon S3 & RDS as the backend store offers a robust solution for managing and tracking machine learning experiments. Leveraging the scalability of AWS, this setup provides a flexible and efficient platform for developers to collaborate, experiment, and deploy machine learning models seamlessly.

That’s all guy’s, If this blog has sparked your curiosity or ignited new ideas, follow me on Medium, GitHub & connect with me on LinkedIn, and let’s keep the curiosity alive.

Your questions, feedback, and perspectives are not just welcomed but celebrated. Feel free to reach out with any queries or share your thoughts.

Thank youU+1F64C &,
Keep pushing boundariesU+1F680

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI