Running AWS locally with LocalStack

More and more companies are switching over to cloud native environments. As a developer this gives me a lot of services to create awesome applications. The challenge that occurred to me right away was how to use all those new components during development, since some companies do not have a testing/development environment in their cloud provider to play with. LocalStack piqued my interest to simulate an AWS environment locally on my laptop, or when running the CI/CD pipeline.

This blog will give a brief introduction in what LocalStack can do and how you can use it for your own projects.

What is LocalStack

LocalStack is a fully functional AWS cloud stack that makes mocking/testing cloud applications simple by having everything running in your local environment. When starting up LocalStack you are able to run core features of AWS like S3, DynamoDB, SNS/SQS and many more.

A single Docker container to rule them all

LocalStack can be started within a single docker container. It has quite some possibilities to change it’s configuration. By setting the right environment variables you can configure what service you want to enable. For some of the services you can even configure behaviour for development purposes like setting a random error rate. The single container will be the host of the LocalStack application, to reach certain services you need to address a single edge service of LocalStack that is exposed on port 4566.

Example docker-compose.yml:

version: "2.1"
  services:
   localstack:
     image: localstack/localstack
     container_name: localstack
     ports:
       - "4566:4566" # port of to where localstack can be addressed to
       - "9000:9000"
     environment:
       - SERVICES=sqs,dynamodb # a list of desired services you want to use.
       - DEFAULT_REGION=eu-west-1 # This is the region where your localstack mocks to be running
       - DATA_DIR=/tmp/localstack/data
       - PORT_WEB_UI=9000
       - LAMBDA_EXECUTOR=local
       - DOCKER_HOST=unix:///var/run/docker.sock
       - START_WEB=1

After running the docker-compose command below the container will be created and started and you are good to go!

docker-compose up localstack

Running LocalStack outside of Docker

LocalStack is a standalone application and can be run outside of Docker but it doesn’t support every operating system. I will not go into details running LocalStack outside of docker, just check their documentation.

Connecting to LocalStack

To connect to your running LocalStack container from your localhost you need to expose the ports to your host machine. Once everything is set up you can connect to LocalStack like you would connect to AWS with using your localhost as AWS-endpoint.

Below are 2 examples demonstrating how to make use of LocalStack.

AWS SDK (java)

If you want to access LocalStack from your application you just need to point to the right endpoint during the call.

For this example I’ve used the LocalStack configuration that I showed above in the docker-compose.yml. Make sure it’s running when you try the example yourself. The example will connect to LocalStack, create a DynamoDB table called "MyTable" and after succeeding prints the created table name.

Listing 1. The used maven dependency

<dependency>
	<groupId>com.amazonaws</groupId>
	<artifactId>aws-java-sdk</artifactId>
	<version>1.11.756</version>
</dependency>

After we’ve added the AWS SDK dependency we need we create a program to execute everything. Within this program we will:

Create a connection to the client
Create a request object to create a DynamoDB table
Do the actual call to LocalStack
Print the name of the newly created table

To actually connect to LocalStack you need to create an AmazonDynamoDB client. All we have to do is point the client to our LocalStack, which is exposes the services via port 4566.

Listing 2. Create a client to connect to LocalStack

private static AmazonDynamoDB getAmazonDynamoDBClient() {
    //Create endpoint configuration which points to the Edge service (running on http://localhost:4566)
    AwsClientBuilder.EndpointConfiguration endpointConfig = new AwsClientBuilder.EndpointConfiguration("http://localhost:4566",
                                                                                                     Regions.EU_WEST_1.getName());

    return AmazonDynamoDBClientBuilder.standard()
			.withEndpointConfiguration(endpointConfig)
			.build();
}

Once we’ve created the connection we create a CreateTableRequest object in which we define our MyTable table.

Listing 3. Create a request for creating the DynamoDB table

private static CreateTableRequest getCreateTableRequest(){
	return new CreateTableRequest()
			.withAttributeDefinitions(new AttributeDefinition("Name", ScalarAttributeType.S))
			.withKeySchema(new KeySchemaElement("Name", KeyType.HASH))
			.withProvisionedThroughput(new ProvisionedThroughput(10L, 10L))
			.withTableName("MyTable");
}

Now we combine everything and make the actual call and print the table name.

Listing 4. The main method which executes everything

public static void main(String[] args) {
    final AmazonDynamoDB amazonDynamoDBClient = getAmazonDynamoDBClient();
    final CreateTableRequest request = getCreateTableRequest();
    try {
        CreateTableResult result = amazonDynamoDBClient.createTable(request);
        System.out.println(result.getTableDescription().getTableName());
    } catch (AmazonServiceException e) {
        System.err.println(e.getErrorMessage());
    }
}

The result of executing this program will be:

MyTable

The full class of this example is included at the bottom of this post.

Command line

Since LocalStack is built to imitate AWS locally you’re able to use the AWS CLI against the LocalStack docker container. All you need to do is make the endpoint url direct to the edge service of LocalStack (port 4566).

For example:

aws --endpoint-url=http://localhost:4566 dynamodb list-tables --profile localstack

will return the table we’ve created in the previous section about the AWS SDK:

{
    "TableNames": [
        "MyTable"
    ]
}

Normally for AWS you need to provide a valid Access key ID and Access secret but for LocalStack it does not matter, though you still need to provide dummy values for AWS CLI to work.

Conclusion

Playing with LocalStack really was fun since it gives you a free playground without any consequences. It really helped me to get the AWS Java SDK running quickly since I could easily connect it to the local cluster. If you are interested in learning more about LocalStack and its services check out their git repository.

Full AWS SDK java class

The full java class which is used for the section: AWS SDK (java)

package nl.jcore.myapplication;

import com.amazonaws.AmazonServiceException;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.client.builder.AwsClientBuilder;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDB;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClientBuilder;
import com.amazonaws.services.dynamodbv2.model.AttributeDefinition;
import com.amazonaws.services.dynamodbv2.model.CreateTableRequest;
import com.amazonaws.services.dynamodbv2.model.CreateTableResult;
import com.amazonaws.services.dynamodbv2.model.KeySchemaElement;
import com.amazonaws.services.dynamodbv2.model.KeyType;
import com.amazonaws.services.dynamodbv2.model.ProvisionedThroughput;
import com.amazonaws.services.dynamodbv2.model.ScalarAttributeType;

public class MyApplication {

	public static void main(String[] args) {
		final AmazonDynamoDB amazonDynamoDBClient = getAmazonDynamoDBClient();
		CreateTableRequest request = getCreateTableRequest();
		try {
			CreateTableResult result = amazonDynamoDBClient.createTable(request);
			System.out.println(result.getTableDescription().getTableName());
		} catch (AmazonServiceException e) {
			System.err.println(e.getErrorMessage());
		}
	}

	private static AmazonDynamoDB getAmazonDynamoDBClient() {
	    //Create endpoint configuration which points to the Edge service (running on http://localhost:4566)
        AwsClientBuilder.EndpointConfiguration endpointConfig = new AwsClientBuilder.EndpointConfiguration("http://localhost:4566",
                                                                                                     Regions.EU_WEST_1.getName());

		return AmazonDynamoDBClientBuilder.standard()
				.withEndpointConfiguration(endpointConfig)
				.build();
	}


	private static CreateTableRequest getCreateTableRequest(){
			return new CreateTableRequest()
					.withAttributeDefinitions(new AttributeDefinition("Name", ScalarAttributeType.S))
					.withKeySchema(new KeySchemaElement("Name", KeyType.HASH))
					.withProvisionedThroughput(new ProvisionedThroughput(10L, 10L))
					.withTableName("MyTable");
	}
}