Point-in-time recovery (PITR) in Amazon DynamoDB is a fully managed capability that creates continuous backups of your DynamoDB table data. Continuous backups are important to ensure business continuity and compliance with regulations, and to protect against human errors, such as unintended writes or deletes. When PITR is enabled for a DynamoDB table, DynamoDB automatically backs up the table data with per-second granularity without consuming provisioned capacity or impacting the table’s performance or availability. PITR also enables you to restore table data to a new DynamoDB table for any point in the preceding 35-day period within or across AWS Regions.
Organizations often rely on infrastructure as code (IaC) frameworks, such as the AWS Cloud Development Kit (AWS CDK) or AWS CloudFormation, to provision and manage their infrastructure. Resources provisioned outside this established IaC framework, such as a new table created by restoring a PITR backup, are unknown to the IaC framework and therefore not managed. For this reason, you need a mechanism to import the table into the framework’s management.
In this post, we outline an approach for restoring a DynamoDB table from a PITR backup and importing it into an existing AWS CDK stack. In addition, we show you how to switch your application traffic from the original table to the restored table, and how to use AWS Systems Manager Parameter Store and AWS Identity and Access Management (IAM) to minimize the impact of the switch on applications that access the table.
Solution overview
As part of the architecture for this post, you set up an AWS CDK project that deploys a single CloudFormation stack. The stack contains a DynamoDB table with PITR enabled, encrypted at rest using an AWS-owned AWS Key Management Service (AWS KMS) key. To demonstrate how your applications can access the table, the stack contains two additional resources that are used by a Parameter Store entry containing the name of the DynamoDB table, and an IAM managed policy granting read access to the table.
With this architecture in place, you restore a table from a PITR backup in the following steps:
Restore the original table from its PITR backup to a new restored table using the AWS Command Line Interface (AWS CLI). Upon creation, the restored table isn’t managed by the CloudFormation stack that is created by the AWS CDK project.
Import the newly created table resource into the AWS CDK project.
Finally, promote the restored table by modifying the AWS Systems Manager parameter and IAM policy to reference it, and removing the original table from the AWS CDK stack.
In this example, the Systems Manager parameter is used to dynamically track the name of the active DynamoDB table. Similarly, the IAM policy serves as a decoupled definition of permissions to the active table. Any resource that requires access to the table, for instance an AWS Lambda function, can use these two resources to reference the active table without having to be modified or redeployed in case the table data has to be restored to a previous state.
Figure 1 that follows illustrates this architecture.
Figure 1: Solution architecture overview
Prepare the environment
For the following solution, you must have access to an AWS account, and have both the AWS CLI and the latest version of the AWS CDK Toolkit (CDK CLI) installed on your development machine.
To prepare the environment
If you haven’t done so already, bootstrap your AWS account by running cdk bootstrap. Bootstrapping refers to the process of provisioning a set of resources that are required by the AWS CDK to deploy AWS CDK applications into an AWS environment (a combination of an AWS account and Region).
With the AWS CDK CLI installed and a bootstrapped AWS environment, create a new project folder on your local machine and initialize a new AWS CDK application within the folder: mkdir restore-demo
cd restore-demo
cdk init –language python
Follow the instructions in the init command’s output to install the required project dependencies.
Set up the initial infrastructure
In this section, you set up the initial infrastructure:
A DynamoDB table with PITR enabled
A Systems Manager parameter that references the table’s name
A simple IAM managed policy that grants read access to the table
To set up the initial infrastructure
In your newly created AWS CDK project, open the app.py file with your text editor of choice. In this file, you’ll see that the project deploys a single stack called RestoreDemoStack.
Open the stack’s definition, which is found in the restore_demo/restore_demo_stack.py file, and replace the contents with the following: from aws_cdk import (
Stack,
aws_dynamodb as dynamodb,
aws_ssm as ssm,
aws_iam as iam,
)
from constructs import Construct
class RestoreDemoStack(Stack):
def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
super().__init__(scope, construct_id, **kwargs)
# CONFIG
# schema, GSI, LSI, billing mode, capacities, encryption
base_config = dict(
encryption=dynamodb.TableEncryption.AWS_MANAGED,
partition_key=dynamodb.Attribute(
name=’id’,
type=dynamodb.AttributeType.STRING
),
billing_mode=dynamodb.BillingMode.PAY_PER_REQUEST,
)
# streams, TTL, tags, policies, metrics, point-in-time-recovery
additional_config = dict(
point_in_time_recovery=True,
)
# TABLE
original_table = dynamodb.Table(self, f’original-table’,
table_name=’restore-demo-original-table’,
**base_config,
**additional_config,
)
active_table = original_table
# RESOURCES
# SSM parameter with active table name
ssm.StringParameter(self, ‘table-name-parameter’,
parameter_name=’restore-demo-table-name’,
string_value=active_table.table_name,
)
# IAM policy granting read permissions to active table
iam.ManagedPolicy(self, ‘table-read-policy’,
managed_policy_name=’restore-demo-read-policy’,
statements=[
iam.PolicyStatement(
effect=iam.Effect.ALLOW,
actions=[‘dynamodb:GetItem’],
resources=[active_table.table_arn],
)
]
)
Synthesize and inspect the stack’s template by running cdk synth from the project’s root folder, then deploy the stack to your AWS account by running cdk deploy.
When the deployment is complete, take some time to inspect the provisioned resources: aws dynamodb describe-table –table-name ‘restore-demo-original-table’
aws ssm get-parameter –name ‘restore-demo-table-name’
aws iam get-policy-version –policy-arn “arn:aws:iam::$(aws sts get-caller-identity –query ‘Account’ –output text):policy/restore-demo-read-policy” –version-id ‘v1’
As an optional step, you can also add some data to your DynamoDB table, either from the DynamoDB console or by using the following commands: aws dynamodb put-item –table-name ‘restore-demo-original-table’
–item ‘{“id”: { “S”: “id-0” } }’
aws dynamodb put-item –table-name ‘restore-demo-original-table’
–item ‘{“id”: { “S”: “id-1” } }’
# …
After enabling PITR, you can restore to any point in time between EarliestRestorableDateTime and LatestRestorableDateTime. LatestRestorableDateTime is typically 5 minutes before the current time. If you choose to add data to your table for this example, wait at least 5 minutes before continuing to the next section to ensure your changes are included in the point-in-time backup.
Restore to a new table
In this section, you restore the original table data from the most recent restore point. However, DynamoDB tables can’t be restored in-place (that is, in the existing table). Instead, the process creates a new DynamoDB table to restore the data into so you must provide a name for that table when you initiate the restore.
To restore to a new table
Run the following command to restore to a new table: aws dynamodb restore-table-to-point-in-time
–source-table-name ‘restore-demo-original-table’
–target-table-name ‘restore-demo-restored-table’
–use-latest-restorable-time
Note: You can also use –restore-date-time <unix-timestamp-in-UTC> in place of –use-latest-restorable-time to restore the table to any point between the EarliestRestorableDateTime and the LatestRestorableDateTime. For more information, see Restoring a table to a point in time.
The duration of the restore process depends on the amount of data to be restored. You can read about how restores work for more details. Use the wait command to block the command line until the newly created table is ready, followed by the describe-table command to inspect its configuration. If you added any data in the previous step, you can also query the table to confirm that the data has indeed been restored. aws dynamodb wait table-exists –table-name ‘restore-demo-restored-table’
aws dynamodb describe-table –table-name ‘restore-demo-restored-table’
aws dynamodb scan –table-name ‘restore-demo-restored-table’
When restoring to a new table, DynamoDB doesn’t copy the following settings from the source table (for more information, see RestoreTableToPointInTime):
Auto scaling policies
IAM policies
Amazon CloudWatch metrics and alarms
Tags
Stream settings
Time to Live (TTL) settings
PITR settings
To validate this, compare the PITR settings of the original and restored tables by running the following two commands and inspecting their outputs:
aws dynamodb describe-continuous-backups –table-name ‘restore-demo-original-table’
aws dynamodb describe-continuous-backups –table-name ‘restore-demo-restored-table’
Keep this in mind because it’s relevant when you import the table into the existing AWS CDK stack in the next section.
Import the restored table into AWS CDK
To import an existing resource into an AWS CDK-managed stack, you first add the resource declaration in the stack definition, then run cdk import to associate the resource with that declaration. The cdk import command is currently in preview with limitations that do not affect this post. For more information, see the AWS CDK Toolkit documentation. Apart from adding resource declarations, the template must not include changes to any other resources. This means that you must use two steps to switch over to the restored table. First, import the restored table, then modify the declarations of the Systems Manager parameter and IAM managed policy resources to reference it and remove the initial table from the AWS CDK stack.
To import the restored table in to AWS CDK
In the RestoreDemoStack file, add a new declaration for the restored table: # TABLE
original_table = dynamodb.Table(self, f’original-table’,
table_name=’restore-demo-original-table’,
**base_config,
**additional_config,
)
restored_table = dynamodb.Table(self, f’restored-table’,
table_name=’restore-demo-restored-table’,
**base_config,
# additional config items are not copied to the restored table
)
As mentioned in the previous section, some settings aren’t copied from the source table, including the PITR configuration, which isn’t enabled on the restored table. Because the resource declaration in the AWS CDK stack must exactly match the configuration of the physical resource, the declaration of the restored table only uses the base configuration. The additional settings, including PITR, are then added and deployed in a subsequent step so the configuration of the restored table matches that of the original table.
Use the AWS CDK CLI to bring the restored table into the existing AWS CDK stack: cdk import
# Output:
# RestoreDemoStack
# RestoreDemoStack/restored-table/Resource (AWS::DynamoDB::Table):
# import with TableName=restore-demo-restored-table (yes/no) [default: yes]? yes
# RestoreDemoStack: importing resources into stack…
# …
By default, the command is interactive and prompts you to confirm each resource association. Alternatively, you can supply the resource mapping as JSON using the –resource-mapping parameter.
Because the AWS CDK doesn’t validate the configuration of the imported resource against its declaration, it’s a best practice to run drift detection after each import operation. Use the following commands to obtain a list of all the resources that have drifted from their declaration: aws cloudformation detect-stack-drift –stack-name ‘RestoreDemoStack’
# wait for a few seconds for drift detection to complete
aws cloudformation describe-stack-resource-drifts
–stack-name ‘RestoreDemoStack’
–stack-resource-drift-status-filters ‘MODIFIED’
In this case, the command should return an empty list.
Promote the restored table
After importing the restored table, you must update its configuration to match that of the original table, promote it to the active table by modifying the Systems Manager parameter and IAM managed policy, and finally remove the now unused original table from the AWS CDK stack.
To promote the restored table
In the RestoreDemoStack file, remove the original table, add the additional configuration to the restored table, and point the active table variable to the restored table. # TABLE
# remove original table from the CDK stack
restored_table = dynamodb.Table(self, f’restored-table’,
table_name=’restore-demo-restored-table’,
**base_config,
**additional_config, # additional configuration w. PITR setting
)
active_table = restored_table # promote restored table
Inspect the changes by running cdk diff, then apply them by running cdk deploy.
Note that the default removal policy for DynamoDB is to retain the resource, so deleting the table’s declaration in the template removes it from the CloudFormation stack without deleting the actual table.
Validate the changes
At this stage, you’ve used PITR to restore the original table data into a new table and imported the restored table into your AWS CDK stack. You then updated the table configuration to match that of the original table and switched over the active table reference so that consumers are pointed to the restored table.
To finish, validate the setup by inspecting the state of the resources in the AWS account.
To validate the changes
Fetch the table name from the Parameter Store and validate that it references the restored table: aws ssm get-parameter –name ‘restore-demo-table-name’
# Response:
# Value references restored table name.
{
“Parameter”: {
“Name”: “restore-demo-table-name”,
“Type”: “String”,
“Value”: “restore-demo-restored-table”
“Version”: 2,
“LastModifiedDate”: “2022-10-10T10:10:10.010000+02:00”,
“ARN”: “arn:aws:ssm:<region>:<account-id>:parameter/restore-demo-table-name”,
“DataType”: “text”
}
}
Validate that the IAM managed policy now references the restored table, granting table read access with the GetItem action: aws iam get-policy-version –policy-arn “arn:aws:iam::$(aws sts get-caller-identity –query ‘Account’ –output text):policy/restore-demo-read-policy” –version-id ‘v2’
# Response:
# Resource in policy statement references restored table ARN.
{
“PolicyVersion”: {
“Document”: {
“Version”: “2012-10-17”,
“Statement”: [
{
“Action”: “dynamodb:GetItem”,
“Resource”: “arn:aws:dynamodb:<region>:<account-id>:table/restore-demo-restored-table”,
“Effect”: “Allow”
}
]
},
“VersionId”: “v2”,
“IsDefaultVersion”: true,
“CreateDate”: “2022-10-03T09:33:26+00:00”
}
}
Confirm that the restored table has PITR enabled:
aws dynamodb describe-continuous-backups –table-name ‘restore-demo-restored-table’
# Response:
# PointInTimeRecoveryStatus signals PITR is now enabled.
{
“ContinuousBackupsDescription”: {
“ContinuousBackupsStatus”: “ENABLED”,
“PointInTimeRecoveryDescription”: {
“PointInTimeRecoveryStatus”: “ENABLED”,
“EarliestRestorableDateTime”: “2022-10-03T11:55:39+02:00”,
“LatestRestorableDateTime”: “2022-10-03T16:05:40.143000+02:00”
}
}
}
Note: If you passed any additional configuration to your table, you can fully inspect the table properties by running the describe-table command.
Clean up
To avoid unnecessary charges to your AWS account, delete the resources you provisioned as part of this post. You can delete the stack either through the CloudFormation console or using the AWS CDK CLI. To use the AWS CDK to delete the resources, navigate to the root folder of your project and run cdk destroy. Keep in mind that DynamoDB tables are retained on stack deletion by default, so either run the following commands or use the DynamoDB console to permanently delete the original and restored tables from your AWS account:
aws dynamodb delete-table –table-name ‘restore-demo-original-table’
aws dynamodb delete-table –table-name ‘restore-demo-restored-table’
Conclusion
In this post, you learned how to restore a DynamoDB table from a PITR backup using the AWS CLI. You then imported the restored table into an existing AWS CDK stack so you could manage it with infrastructure as code. Finally, you switched to the new table without having to redeploy applications operating on it.
When it comes to disaster recovery, there is no one-size-fits-all approach and the design of your solution will depend on a list of factors including recovery point objective and recovery time objective requirements, your application landscape, and many more. This post provides a set of fundamental building blocks you can use as a starting point when designing a solution that matches your needs. To ensure business continuity during the restore process, be sure to consider and design a resilient mechanism capable of processing incoming application requests while your database is in the restoring state.
If you have any comments or questions, leave a comment in the comments section. You can find more DynamoDB posts in the AWS Database Blog.
About the Authors
Hrvoje Grgic is a Cloud Infrastructure Architect at AWS Professional Services based in Amsterdam. He is passionate about creating scalable and efficient architectures to help customers innovate on AWS. He specializes in the networking and DevOps domains. Outside of work, Hrvoje enjoys sports, warm weather and movies.
Patrice Marti is a Cloud Application Architect at AWS based in Switzerland, supporting customers on their journey to the cloud. He specializes in modernization, with a focus on serverless technologies. Outside of work, Patrice is a bit of a space nut and hopes to someday live and die on Mars – though hopefully not at the point of impact.
Read MoreAWS Database Blog
