Need for Speed—No, it’s not the video game, but rather a critical requirement for the success of your website in this competitive world.
Although we might think that subsecond delay is acceptable, the New York Times noted in For Impatient Web Users, an Eye Blink Is Just Too Long to Wait that humans can notice a 250-millisecond (a quarter of a second) difference between competing sites. In fact, users tend to opt out of slower websites in favor of faster ones. In the study done at Amazon, How Webpage Load Time Is Related to Visitor Loss, it’s revealed that for every 100-millisecond (one-tenth of a second) increase in load time, sales decrease 1%.
If someone wants data, you can deliver that data much faster if it’s cached. That’s true whether it’s for a webpage or a report that drives business decisions. Can your business afford to not cache your webpages so as to deliver them with the shortest latency possible?
Of course, content delivery networks like Amazon CloudFront can cache part of your website’s content, for example static objects like images, CSS files, and HTML files. However, dynamic data (for example, the product catalog of an e-commerce website) typically resides in a database. So we have to look at caching for databases as well.
Amazon ElastiCache is a fully managed, in-memory caching service supporting flexible, real-time use cases. You can use ElastiCache to accelerate application and database performance, or as a primary data store for use cases that don’t require durability like session stores, gaming leaderboards, streaming, and analytics. ElastiCache is compatible with Redis and Memcached.
In this post, we show you how to deploy Amazon ElastiCache for Redis using AWS Cloud Development Kit (AWS CDK). The AWS CDK is an open-source software development framework to define your cloud application resources using familiar programming languages like Python.
We host our web application using Amazon Elastic Compute Cloud (Amazon EC2). We load a large dataset into a MySQL database hosted on Amazon Relational Database Service (Amazon RDS). To cache queries, we use ElastiCache for Redis. The following architecture diagram shows the solution components and how they interact.
The application queries data from both the Amazon RDS for MySQL database and ElastiCache, showing you the respective runtime. The following diagram illustrates this process.
In this post, we walk you through the following steps:
Install the AWS Command Line Interface (AWS CLI) and AWS CDK v2 on your local machine.
Clone and set up the AWS CDK application.
Run the AWS CDK application.
Verify the resources created.
Connect to the web server EC2 instance.
Start the web application.
Use the web application.
So, let’s begin.
You should have the following prerequisites:
The estimated cost to complete this post is $3, assuming you leave the resources running for 8 hours. Make sure you delete the resources you create in this post to avoid ongoing charges.
Install the AWS CLI and AWS CDK on your local machine
Install the AWS CDK Toolkit globally using the following node package manager command:
Run the following command to verify the correct installation and print the version number of AWS CDK:
Clone and set up the AWS CDK application
On your local machine, clone the AWS CDK application with the following command:
Before we deploy the application, let’s review the directory structure:
The repository also contains the web application located under the subfolder web-app, which is installed on an EC2 instance at deployment.
The cdk.json file tells the AWS CDK Toolkit how to run your app.
Set up the virtual environment
This project is set up like a standard Python project. Create a Python virtual environment using the following code:
Use the following step to activate the virtual environment:
If you’re on a Windows platform, activate the virtual environment as follows:
After the virtual environment is activated, upgrade pip to the latest version:
Install the required dependencies:
Before you deploy any AWS CDK application, you need to bootstrap a space in your account and the Region you’re deploying into. To bootstrap in your default Region, issue the following command:
To deploy into a specific account and Region, issue the following command, providing your account number and Region:
For more information about this setup, visit Getting started with the AWS CDK.
You can now synthesize the AWS CloudFormation template for this AWS CDK application:
To add additional dependencies, for example other AWS CDK libraries, just add them to your setup.py file and rerun the pip install -r requirements.txt command.
Other useful AWS CDK commands
The following AWS CDK commands might also be useful:
cdk ls – List all stacks in the app
cdk synth – Emit the synthesized CloudFormation template
cdk deploy – Deploy this stack to your default AWS account or Region
cdk diff – Compare the deployed stack with the current state
cdk docs – Open the AWS CDK documentation
Run the AWS CDK application
At this point, you can deploy the AWS CDK application:
You should see a list of AWS resources that will be provisioned in the stack. Enter ‘y’ to proceed with the deployment.
You can see the progress of the deployment on the terminal. It takes around 10 to 15 minutes to deploy the stack.
Once deployment is complete, you can see the total deployment time and AWS CloudFormation Outputs on the terminal. Take note of the web server public URL.
These outputs are also available on the AWS console. Navigate to the AWS CloudFormation console and choose the ElasticacheDemoCdkAppStack stack to see the details.
Review the resources under the Outputs tab.
The web application retrieves these configurations automatically. However, take note of the web server public IP or URL. We need it to connect to the web server later.
AWS CDK application code
The main AWS CDK application is in the app stack file, elasticache_demo_cdk_app_stack.py, and the whole infrastructure is defined as the ElasticacheDemoCdkAppStack class. Read through the comments to see what each block is doing.
First, we import the necessary libraries needed to construct the stack:
Then in the class ElasticacheDemoCdkAppStack, we define the stack, starting with the virtual private network and security groups:
Then we define the data stores used in the application, that is, Amazon RDS for MySQL and ElastiCache:
Then we define the EC2 instance for the web server as well as the required AWS Identity and Access Management (IAM) role and policies for the web server to access the data stores and AWS Secrets Manager to retrieve the database credentials:
Lastly, we capture all CloudFormation stack outputs generated from the AWS CDK application stack:
As described in the preceding code, the web server userdata is stored in the user_data.sh file. The content of user_data.sh is as follows:
The web server essentially clones the same Git repository of the main AWS CDK application. The script also downloads a sample dataset from located under the subfolder sample-data under the main folder. The dataset is from Kaggle and is licensed under Creative Commons.
Verify the resources created by AWS CDK
On the Amazon EC2 console, verify if the EC2 instance was created and is running.
On the Amazon RDS console, verify if the MySQL instance was created and is available.
On the ElastiCache console, verify if the Redis cluster is available.
The Amazon RDS CDK construct also automatically creates a secret in Secrets Manager. You can view the secret name in the CloudFormation stack outputs. The secret contains the MySQL user admin, automatically generated password, database host, and database name.
The web application retrieves this information automatically from Secrets Manager, so you don’t need to note down these values.
Connect to the web server EC2 instance
We connect to the web server instance using AWS Systems Manager Session Manager through the AWS Management Console. Leaving inbound SSH ports and remote PowerShell ports open on your managed nodes greatly increases the risk of entities running unauthorized or malicious commands on the managed nodes. Session Manager helps you improve your security posture by letting you close these inbound ports, freeing you from managing SSH keys and certificates, bastion hosts, and jump boxes.
On the Amazon EC2 console, choose Instances in the navigation pane.
Select the web server and choose Connect.
On the Session Manager tab, choose Connect.
You log in as ssm-user. However, the web server’s user data script is run for ec2-user.
Switch to ec2-user using the following command:
You should land in the ec2-user home directory /home/ec2-user, which should contain the elasticache-demo-cdk-application subfolder. This the same repository that you cloned on your local machine. It contains the sample data that is inserted into the MySQL database as well as the web application. You can find the sample dataset and the web application in the sample-dataset and web-app subfolders, respectively.
Let’s navigate to the web application subfolder from the home directory and see the content:
Overview of the web application
The web application is a Python Flask application. The file webApp.py is the main application, which contains the Flask routes for each operation, for example, query MySQL database and query cache. Refer to the following webApp.py code:
The webApp.py file imports cacheLib. The file cacheLib.py contains the key procedures to perform the following actions:
Retrieve the MySQL and ElastiCache for Redis endpoints from the CloudFormation stack outputs
Load the sample data into the MySQL database
Store all configurations in config.json, which is automatically created when the application is run for the first time
Query the MySQL database
Refer to the cacheLib.py file to understand the respective functions. One of these functions is called query_mysql_and_cache, which is triggered by the web application under @app.route (“/query_cache”). This function checks if the dataset is in the cache first. If there is a cache hit, the dataset is served by ElastiCache at low latency. Otherwise, the dataset is retrieved from MySQL and then cached into ElastiCache for future queries. The following is the query_mysql_and_cache code snippet:
Start the web application
The necessary runtimes and modules were already installed at instance creation using the userdata contained in the user_data.sh file in the AWS CDK application.
Let’s start the web application with the following command:
Use the web application
From a browser, access the web server’s public IP address or URL with port 8008 (or any other ports if you changed it), for example, http://WEBSERVER-PUBLIC-IP:8008. This should take you to the landing page of the web application, as shown in the following screenshot.
If you cannot access the site using port 8008, you can be behind a VPN or firewall that is blocking that port. Try disconnecting your VPN or switching network.
Now let’s query the MySQL database. Choose Query MySQL on the navigation bar.
You can review the time it took to run the query. Now let’s try to query the cache by choosing Query Cache.
Notice that it took almost the same time to run. This is because the first time you access the Redis cache, it’s empty. So the application gets the data from the MySQL database and then caches it into the Redis cache.
Now, try querying the cache again and observe the new runtime. You should see a value in the order of milliseconds. This is very fast!
If you look at your terminal, you can see the HTTP requests made from the browser. This can be helpful if you need to troubleshoot.
To avoid unnecessary cost, clean up all the infrastructure created with the following command on your workstation:
As demonstrated in this post, you can use AWS CDK to create the infrastructure for an Amazon ElastiCache application. We showed the difference in runtime between ElastiCache and an Amazon RDS with MySQL engine. You can now build your own infrastructure and application using the caching capability of Amazon ElastiCache to accelerate performance for a better user experience.
We invite you to test the solution and contribute to the project on GitHub. Share your thoughts on this tutorial, in the comments below!
This sample code is made available under a modified MIT license. See the LICENSE file for more information.
About the Authors
Hantzley Tauckoor is an APJ Partner Solutions Architecture Leader based in Singapore. He has 20 years’ experience in the ICT industry spanning multiple functional areas, including solutions architecture, business development, sales strategy, consulting, and leadership. He leads a team of Senior Solutions Architects that enable partners to develop joint solutions, build technical capabilities, and steer them through the execution phase as customers migrate and modernize their applications to AWS. Outside work, he enjoys spending time with his family, watching movies, and hiking.
Calvin Ngo is a Senior Specialist Solutions Architect in the Developer Acceleration team at AWS. His mission is to help developers adopt cloud technologies and refactoring systems and applications using modern development techniques. He does this by taking a cross-functional, coordinated approach to developer enablement. In his spare time, he likes taking his kids for a bike ride around Singapore and also kayaking to nearby islands.
Read MoreAWS Database Blog