Deploying to AWS Part II: Running a Rails app on Fargate
What's the 2018 approach to deploying a Rails app to AWS? We've partnered with DailyDrip on a series of videos to guide you through the process. We'll be covering how to Dockerize a Rails app, AWS Fargate, logging, monitoring, and CDN support.
Today's video is the next in the series of setting up our produciton app for production use in AWS ECS. In our last video, we Dockerized the app to prepare it to run on ECS / Fargate; in this video, we will work on doing just that.
Introduction to AWS ECS and Fargate
So, what are Amazon's Elastic Container Service and Fargate?
Here are some quick snippets from Amazon's website:
- ECS: "a highly scalable, high-performance container orchestration service that supports Docker containers and allows you to easily run and scale containerized applications on AWS"
- Fargate: "a technology for Amazon ECS and EKS* that allows you to run containers without having to manage servers or clusters"
What does this mean?
The Elastic Container Service is a container orchestration service much like Kubernetes or Swarm. Where the magic happens is when ECS is connected to Fargate. Fargate abstracts the idea of managing the instances, so your only concern is setting up containers, auto scaling policies, and allowing Fargate to manage the underlying hardware.
- AWS Account
- AWS CLI (install instructions)
Setting up our Repository
The first thing we're going to do is set up a Repository so that we can push our Docker image up to Amazon. In order to do that, we need to have the the AWS CLI tool configured correctly. We'll need to visit Amazon's users management page to create a user with the following security groups:
You should now have a user that looks similar to this:
Note: You don't necessarily need to add the FullAccess permissions, but we've done that to simplify this tutorial.
Next, we need to go to the "Security Credentials" tab and create an Access key. Clicking this button should pop up a modal and give us an Access Key ID and Secret Access Key. We'll need to add these to the AWS CLI.
Armed with these keys, let's switch over to the console and type
aws configure. That command should ask you for both of your keys, default region, and default output format. We input all these keys and that should be enough. You should now have something that looks similar to this (you can re-run the
aws configure command to verify).
Now, we can use the AWS CLI to create a repository:
Note: Take note of the
repositoryUri since we will be using this to tell Docker where to push the image.
Our next step is to get an authentication token to allow Docker to authenticate to our repository.
aws ecr get-login command returns a Docker login command; you can run it without the
$() eval wrapper to see the output.
Now that we have Docker authenticated, we can tag our image and push it to our repository.
If we go back to the AWS Console for Repositories, we should see our newly created
dailydrip/produciton repository and, if we click on it, we should see our image that was pushed up.
Setting up RDS
The next step we are going to take is setting up our Postgres db using RDS. To do this, we need to navigate to the RDS section of the AWS Console. Once there, we can select
Instances from the left navigation pane. On the right pane, we should see our instances listing page. On the top right of the page, click on the button
Launch DB Instance. On this page, we're going to select
Postgres and click Next.
Now, we should be on the "Choose Use Case" page. We're going to make sure the "Production" use case is selected and click Next.
We should now be on the "Specify DB Details" page where we can select the engine version, instance size, multi availability zones, and database name, user, and password. For this app, I'm going to choose the
db.t2.medium instance, set the instance identifier to
produciton, and set my username and password.
On the last page,
Configure advanced settings, we're going to use the default VPC and choose to have the DB publicly accessible (more about this later). The last change we're going to make is setting the database name to
produciton_production. After that, we can click "Launch DB Instance".
Now, if we go back to the
Instances page, we should see that our database is being provisioned.
Once our database is provisioned, we are going to run our migrations on our database from our local box. This is why we chose the publicly accessible option earlier. First thing we'll need to do is go to our RDS instance for produciton, scroll down, and click on the security group in the "details" section.
We should now be on the Security Groups page. Toward the bottom, there should be a set of tabs, the second one being "Inbound." Let's select that tab and click "Edit." Now, we're going to click "Add Rule" and for the "Source" dropdown, we will select "My IP." This will create a rule to allow only our IP to connect to the database. Our inbound rules should look like this:
Now, we should be able to connect to our database locally and run our migrations via a command similar to this:
DATABASE_URL= RAILS_ENV=production bundle exec rake db:migrate
There are a few ways to handle running migrations for our rails app. This is the easiest method, but not the best solution. For a more involved solution, we could write a script that would be our entrypoint in the dockerfile that could run migrations before it started the server. However, if the ECS container doesn't start within a certain amount of time, it could kill the container and you might end up in a cycle of failed migrations or, even worse, a busted database.
Another option that seems to be more appropriate in this case would be to create a separate task definition that is used for solely running migrations. Then, once you pushed up your Docker image changes, you could run the migrations task before restarting your application. This could be done from the AWS console or from the AWS command line via
aws ecs run-task command, which would spin up the newest image, run the migrations, and then exit (one-off task). Then, you'd just restart your application.
Setting up ECS and Fargate
Let's start by going over some terminology: We will be introducing a few different resources over the next few minutes.
- Clusters are a logical way to group resources (services and tasks).
- Services are used to run a load balance in front of a group of tasks.
- This is also where you will specify how many instances of a task should be running. The service scheduler is in charge of starting new instances in the case of an instance failing.
- Tasks are the running instances of a task definition.
- Task Definitions
- Task Definitions are where you specify the resources for a Docker container or group of containers.
- It is also where you specify the Docker image, any volumes, environment variables, and more.
Creating a Task Definition
We're going to start by creating the base resource, a task definition.
From the navigation pane on the left, let's click on "Task Definitions." From here we can create and manage the Task Definitions we’ll be using. Let's click on "Create new Task Definition," which should take us to the first step of creating a Task Definition.
At this step, we need to choose the launch type for our Task Definition. Let's choose "Fargate" and click "Next Step."
Now, we're on step 2, which is where we configure the name, roles, memory and cpu size, container definitions, and more.
Let's start by setting the "Task Definition Name" to "Web," "Task Role" to "ecsTaskExecutionRole," "Task Memory" to ".5GB," and the "Task CPU" to ".25 vCPU."
Next, we're going to click "Add Container" to setup our container. When we click "Add Container" we should see a slide out that allows us to configure our container.
In this modal, we're going to set the container name to "produciton" and our image to the URI that you used to push your image to your repository. In our case it's "154477107666.dkr.ecr.us-east-1.amazonaws.com/dailydrip/produciton:latest" (include the latest tag).
Next, in the "Advanced Container Configuration" section, we're going to make sure "Essential" is selected. Now, we're going to add a few environment variables. We're going to add these:
- RACK_ENV: production
- RAILS_ENV: production
- PORT: 80
- RAILSLOGTO_STDOUT: true
- RAILSSERVESTATIC_FILES: true
(it might be easier to open a second window to look at your RDS instance settings)
- ours will look something like:
- ours will look something like:
You should now have something that looks similar to this:
Now, we can "Add" at the bottom of the modal and "Create" at the bottom of the page. Clicking “Add” should take us to a page that gives the status of our task definition. We can click the "View Task Definition" to go back to the page and review all of our changes.
Creating a Cluster
Now, let's go to the Clusters console and click "Create a Cluster". This should take us to a page to choose a few different cluster templates. Let's choose "Networking Only" and click "Next Step."
On the next page, we can name the cluster and choose to create a VPC. Let's name it "Produciton" and leave the "Create VPC" unchecked. Click "Create."
Now, we can click "View Cluster," which should take us back to the overview of our cluster.
Creating a Service
The next step is to create a Service that will house our tasks. If we're not there, we can navigate to the Clusters section, and choose our cluster, which should take us to the overview of our cluster. From here, we can choose the "Services" tab and click "Create."
On the first page, we will choose our configuration for the service; we want to select "FARGATE" for our "Launch Type." Next, we want to select the Task Definition that we just created. In our case, we named ours Web, so if we look in the dropdown, we should see a
Web:1, which we will choose.
NOTE: The 1 in the name is the revision. As you make edits to your task-definition, you will need to make sure you update the service to point to the new version, so
Web:2 and so on.
While on this page, we're also going to set the "Service Name" and "Number of Tasks" to "produciton" and "1," respectively. Then, we can click "Next Step."
Now, we should be on the VPC and Security Groups page. Here, we're going to set the Cluster VPC to the VPC you have available, which will probably be the default one that was created. For subnets, you can choose the first one in the list, since you might not have more than one. The rest of the settings we will leave to their default settings for the time being. We're not going to worry about setting up a Load Balancer for now. Now, you can click "Next Step."
We should now be on Step 3. This is where we would configure auto scaling. We're not going to configure auto scaling for our app, but if you were to click on the "Configure Service Auto Scaling" option, you'd see a list of available configurations to set the minimum and maximum number of tasks, along with the scale in and scale out policies. For this video, we're just going to choose "Do not adjust the service's desired count" and click "Next Step."
Now, we should be on the 4th and final step. This is just to review the service settings before we create the service. Look over the configuration settings and, if you are okay with them, click "Create Service."
Now, we can click on the "View Service" button and we should see a page similar to this:
NOTE: If you don't see the task right away, you might need to click the refresh button.
Now, we've set up everything and our app should be running in a moment. Once the "Last Status" of the Task says "RUNNING," we should be able to hit the site.
Viewing the site
If we click on the task id, we should be looking at the details of the running Web task. If we scroll down a bit, we see the Network section; there, we should see an "ENI ID" that we can click on. Once we click on that, we should see a page similar to this:
Once here, we can see the "IPv4 Public IP" column and an IP address for our ENI. We can use this IP address to hit our service. Now, let's take a look.
Hmm… This is embarrassing…What happened?
Let's take a look at our logs for the running task. To do this, we’ll switch back over to our AWS Console and go back to the Produciton Cluster. Click on the "Tasks" tab and click on our Task. We should now be on the details page for our running task. Click on the "Logs" tab and let's see what's going on.
Aha! Now, we can see that it looks like our rails app can't connect to the database. First, we want to make sure that we used the correct database URL in our config. In our case, we have. So, let's make sure our service has access to our RDS instance.
Let's go back to our services page and click on our "produciton" service. We should be at the details page and see a security group. In our case it's
sg-f7d0d882. Let's switch over to our RDS instance. Now, we're going to navigate to RDS > Instances and choose our postgres instance. Go to the "Details" section and find the "Security Groups." We should see one group, probably starting with "rds-launch-wizard". Let's click on that security group, which should take us to a page like this:
Let's move over to the "Inbound" tab at the bottom of the page and click "Edit." Now, we want to click "Add Rule" and add a Custom Rule with a Custom Source which we can search for our security group from the service. Now, our rules should look similar to this:
Now, we've updated the rules which should allow our service to communicate with our database. So, let's check our application again to see if it’s working.
Voila… Now we have our app successfully running on AWS using the ECS and Fargate stack.
We've actually done quite a few things in this video. We've set up an elastic container repository (ECR) to allow us to push our Docker images up. We set up a postgres database with RDS and we've setup an ECS cluster using Fargate with a service and task definition.
Even though we have our production application running on ECS now, there are still many things left to handle. We are not hosting our assets correctly, and we have not set up logging or monitoring. Over the next few videos, we will be working toward a more "production-ready” application.