During my last year at JCore I was given the opportunity to do deep-dive in a self-chosen topic in the form of a 'Specialisation'. For this 1-year project I chose to dive deep into AWS how it works and how I, as a developer, make use of it. Some of the topics I covered during this were: DevOps, CI/CD and Security. As a demo and as use case I created a simple pubquiz application in which you can register and have a custom form for your answers. During the development of this application I faced different challenges that I had to overcome. This blog is about how I created a simple API that is exposed to the internet and how I tried to tackle the challenges of security, scalability and adaptability.

The application

The application I created consisted of a container image which hosts a simple API that is made with Java, Spring and AWS SDK. So for this backend application I created a Virtual Private Cloud(VPC) with a Fargate task in a private subnet so the application itself is not exposed to the internet. The goal was to make an API with the API Gateway that defines the API that redirects the correct call to my Fargate instance that is in a private subnet.

Here’s an overview of the infrastructure of my backend application:

aws api gateway
Figure 1. My application infrastructure

Why

So let me explain why I chose to use some specific components.

First of all the VPC with a public and private subnet where I host an ECS cluster for my Fargate instance. I want my app not to be accessible by the whole internet, so I want it to put in a private subnet. The public subnet serves for making the internet accessible via a NAT gateway to be able to pull docker images. So the app can only make outgoing calls via the NAT gateway, incoming calls are not possible. The reason I used Fargate instead of an EC2 instance is because my application is not that critical to have fixed resources and this way AWS is responsible for provisioning, configuring and managing the instance.

Second, I used the API Gateway to create a managed and safe proxy for my app. The benefits of it are that with the gateway I can easily manage access of each individual endpoint and with placing the gateway there I can redirect an endpoint to a different component (an AWS lambda for example). The API Gateway is outside my VPC and makes a direct connection to my private subnet via what is called a VPCLink. With VPCLink I’m able to publicly reach specific endpoints from outside my VPC. To be able to create a VPCLink to your Fargate instance you’ll have to create Network Load Balancer within your private subnet (see here for setting up the VPCLink).

How

Below I have described which steps I took to create API Gateway that connects towards my running container on Fargate. These steps assume you already have some basic knowledge about AWS and running Fargate tasks.

Step 1: Create a Task definition

The First step is simply creating a task definition, this task will be used by the ECS Cluster to spin up your container. The task definition will be used later by the service which we will setup at step 6.

  1. Go to AWS Elastic Container Services and create a new Task definition

  2. Choose the Fargate launch type

  3. Choose/create a task execution role (used for pulling docker images, for example)

  4. Configure your task size (memory and CPU)

  5. Add container:

    1. Give it a name

    2. Add URL to container image

    3. Configure rest of the container (had to set port mapping in my case)

  6. Click on Create

Step 2: Create an ECS Cluster:

After creating a task definition I created my cluster and with it a VPC. Especially the VPC is important because it will be used later on to configure the correct subnets, loadbalancers and gateways.

  1. Go to AWS Service ECS and click on Create Cluster

  2. Choose the Networking only cluster template

  3. Check the Create VPC checkbox

  4. Configure your subnets (default values will do)

  5. Create Cluster

Step 3: Replace one of the public subnets for a private one

In the previous step we created 2 subnets, if you followed the wizard the subnets that are created are public subnets. The difference between a public subnet and a private subnet is whether they are connected to an Internet Gateway. If it is connected to the Internet Gateway the subnet is considered public.

  1. Go to the VPC service on AWS and click on Subnets

  2. Remove one public subnet

  3. Add new subnet that is NOT connecting to an Internet Gateway (this is what makes them a private subnet)

Step 4: Create Network Load Balancer

Next were gonna configure a Network Load Balancer that will be inside our private subnet we created. The load balancer will be an internal loadbalancer and the targetgroup that will be used is the one we create in step 6. Unfortunately, creating a Network Load Balancer requires having a targetgroup but we need a Network Load Balancer if we want to create a Service (step 6). So you can create one now to delete it later on.

  1. Go to service EC2 > load balancers

  2. Create load balancer

  3. Select Network Load Balancer

  4. Choose Internal as scheme

  5. Don’t add listeners yet! (this will happen later)

  6. Choose the correct VPC (the one create earlier)

  7. Choose the earlier created private subnet

  8. Add a target group (this will be replaced later by the target group created by the service in the ECS Cluster)

  9. Create load balancer

Step 5: Create a NAT Gateway for the Fargate task to pull container images

The next challenge I wanted to tackle is pulling the latest container image from a container registry. Due to the fact that our Fargate task will be running in a private subnet we can’t access the container registry that’s running outside of it. To solve this we need to create a NAT Gateway that only allows outgoing calls from the Fargate task.

  1. Go to AWS Service VPC > NAT Gateways

  2. Create NAT Gateway

  3. Choose public subnet of VPC

  4. Click on Allocate Elastic IP

  5. Create NAT Gateway

  6. Go to AWS Service VPC > Route Tables

  7. Create/edit the route of your private subnet:

    1. Choose Add another route

    2. Enter 0.0.0.0/0 as your Destination

  8. Select newly created NAT Gateway as the Target

Step 6: Create Service in ECS Cluster

With all preparations that are done in the previous steps we can now create a service in our ECS Cluster. By creating a service our application will be up and running within our VPC. So pay some extra attention here since you will be connecting all earlier created services and debugging it when something goes wrong will be annoying.

  1. Go to your ECS Cluster created in step 2

  2. Create new Service

  3. Choose Fargate as launch type

  4. Choose the earlier created Task definition (step 1)

  5. Give service a name

  6. Choose desired amount of tasks

  7. Click next

  8. Add Private Subnet as subnet

  9. Make sure that Auto-assign public IP is unchecked/false

  10. Choose Network Load Balancer

  11. Choose the earlier created Network Load Balancer

  12. Add the Load Balancer

  13. At the listener port, choose the port that is exposed on your container

  14. Validate if the target type is IP

  15. Check if Target group name is not too long

  16. Choose desired Autoscaling group (none in my case)

  17. Review and Create

Step 7: Edit Network Load Balancer target group

Now that we have created an ECS Service we can configure the correct target group to our Network Load Balancer (remember step 4?). After you have done the steps below you can delete the old target group.

  1. Go to service EC2 > load balancers

  2. Edit created Network Load Balancer

  3. Change the Target Group to the one that is created by the newly created Service

  4. Save

Step 8: Create a VPCLink

Our container is fully set-up to connect to we can now create our private integration via VPCLink. We have to create VPCLink to our VPC.

  1. Go to AWS Service API Gateway > VPC Links

  2. Choose the Correct VPC and Click Create

  3. Wait till the VPC Link is created

Step 9: Create API Gateway

When our container is up and running and our VPCLink has been created we can create a new REST API in API Gateway. In the API Gateway you just have to create the endpoints you want and select the VPCLink as integration type.

  1. Go to AWS Service API Gateway > APIs

  2. Build REST API (not the private one)

  3. Choose Create New API

  4. In Resources click on Actions

  5. Create Resource

  6. Click on the newly created resource followed by clicking on Actions > Create method

  7. Choose the desired method and click the check

  8. Choose Integration type: VPC Link

  9. Choose Method

  10. Choose your created VPC Link

  11. Type in the Endpoint URL the url of the nat gateway api + path to desired endpoint on container

  12. Choose Actions > Deploy API

Step 10: Take a sip of your coffee (you’re done)

Now your done and your container should be reachable via the API Gateway.

shadow-left