How to Use AWS CDK for Cloud Infrastructure: Complete Tutorial

Instead of setting up your cloud infrastructure manually, it can be easier and safer to use code. The AWS Cloud Development Kit (CDK) is an open-source software development framework to define your cloud application resources using familiar programming languages. This complete tutorial provides a beginner guide with a step by step guide on how to use AWS CDK to define, synthesize, and deploy cloud infrastructure. We will cover CDK basics including apps, stacks, constructs, and how it all relates to CloudFormation. Then we will go through a workshop speedrun, and finally learn advanced topics such as testing and best practices.

What Is AWS CDK?

CDK stands for Cloud Development Kit. It is open source software provided by AWS that enables you to write imperative code to generate declarative CloudFormation templates. It enables you to define the how so that you can get the what. That means you can focus more on the architecture of your app instead of the nitty gritty details of IAM roles and policy statements.

CDK is available in JavaScript, TypeScript, Python, Java, C#, and it is in a developer preview for Go. CDK itself is NodeJS based. Even if you use one of the other languages, the CDK code itself will be executed via NodeJS. So NodeJS is a requirement.

CDK version two became generally available in December. Version two was a massive improvement over version one, since there is only one stable NPM library to install, instead of one for each module. In CDK v1, you had individual NPM libraries for each module like DynamoDB, Lambda, SNS, etc. In version two, you just have one, and that avoids a lot of the dependency management hell.

How CDK Relates to CloudFormation

So CDK generates CloudFormation. But what is CloudFormation? CloudFormation is an AWS service that provisions AWS resources and stacks. AWS resources are things like Lambda functions, S3 buckets, DynamoDB tables, pretty much anything that AWS offers.

A CloudFormation stack is just a collection of AWS resources. CloudFormation uses JSON or YAML files as a template that describes the intended state of all the resources you need to deploy your application. The stack implements and manages the group of resources outlined in your template and it allows the state and dependencies of those resources to be managed together.

When you update a CloudFormation template, it creates a change set. A change set is a preview of changes that will be executed by stack operations to create, update, or remove resources so that the template becomes in sync.

So CDK generates CloudFormation and CloudFormation provisions AWS resources.

CDK App and Stack Structure

What does the CDK app look like and how does it relate? The root of CDK is the CDK app. It is a construct that coordinates the lifecycle of the stacks within it. There is no CloudFormation equivalent for an app.

Within the app, you can have multiple CDK stacks, and CDK stacks can even have nested stacks. CDK stacks are one to one equivalent with CloudFormation stacks, and nested stacks are also one to one equivalent with CloudFormation stacks.

When you run cdk synth or cdk deploy, the output is the CloudFormation template for this app structure. The CDK app is a special root construct that orchestrates the lifecycle of the stacks and the resources within it.

The app lifecycle constructs the tree of the app in a top down approach, and then it executes the methods for the constructs within it. You typically do not need to directly interface with any of the prepare, validate, or synthesize methods of the constructs, but they can be overridden. The final stage of the app lifecycle is the deploy phase, where the CloudFormation artifact is uploaded to CloudFormation.

The unit of deployment in CDK is called a stack. All resources defined within the scope of a stack, either directly or indirectly, are provisioned as a single unit. CDK stacks have the same limitations as CloudFormation stacks.

Stacks are powerful in CloudFormation. In order to deploy a template to multiple environments, you need to use CloudFormation parameters. But these are only resolved during deployment. In CloudFormation, if you want to conditionally include a resource based on a parameter, you have to use the CloudFormation condition. But in CDK, you do not have to use either. You can simply use an if statement to check a value whether the resource should be defined or not. It ends up being much simpler.

You can use CloudFormation parameters and conditions in CDK. But they are discouraged since they only resolve during deployment, which is well after CDK synthesis.

What Are CDK Constructs?

We have heard a lot about constructs. But what are they? There are four levels of constructs.

Level 1 Construct Example

For a level one construct, let us look at the access analyzer module. In this case, the module does not have any level two constructs available, so there is only the level one CfnAnalyzer construct. As you can see, this is a one to one representation with the CloudFormation API. The CfnAnalyzer construct has the same properties defined as the CloudFormation API for the same. There are also new helper utility methods on the CfnAnalyzer construct.

Level 2 Construct Example

For a level two construct, let us look at the DynamoDB table construct. It does not have a Cfn prefix and includes many sensible defaults. For example, the billing mode defaults to pay per request, unless you specify replication regions. If you specify replication regions, it becomes provisioned and if it becomes provisioned, the default Read and Write capacities get set and defaulted to five. The default removal policy of the table is retained. And the table has helper methods to create global or local secondary indexes, and to create varying levels of access to the table and its streams.

Level 3 Construct Example

For level three constructs, the CDK does not offer anything specific out of the box. These tend to be created at the individual organization or community level, and provided as libraries. An example level three construct would be to create a notifying bucket. This construct creates an S3 bucket along with an SNS topic. It then adds an object created notification coming from the S3 bucket to target the SNS topic. All you have to do to use this is add a new notifying bucket to your app. And it will provision all of these resources automatically.

Synthesis, Assets, Bootstrapping, and Deploy

How do we generate the CloudFormation template? In order to do that, the app needs to be synthesized. To do this, we can run cdk synth. Since this traverses the app tree and invokes synthesize on all of the constructs, this ends up generating unique IDs for the CloudFormation resources and generates the respective YAML along with any assets that are needed.

What Are Assets?

Assets are the files bundled into CDK apps. They include things like the Lambda handler code, Docker images, layer versions, files going into an S3 bucket, etc. They can represent any artifact that the app needs. When you run cdk synth or cdk deploy, these get output to a cdk.out folder on your local machine.

What Is Bootstrapping?

What does CDK do with these assets? How do they get put into CloudFormation? Well, that is where bootstrapping comes into play. Bootstrapping creates a CDK toolkit CloudFormation stack deployed to your account. This account includes an S3 bucket and various permissions that are needed for CDK to do the deployments and upload the assets.

Bootstrapping is required when your app has assets or your CloudFormation templates become larger than 50 kilobytes. So it is pretty much always required. You are almost always going to have some sort of asset in your stack.

For CDK version two, administrator access is needed to create the roles that CDK toolkit stack needs in order to do deployments. You will not need administrator access after CDK is bootstrapped.

How Does CDK Deploy Work?

With the bootstrapping done, we can move on to deploying. When you run cdk deploy, the app is initialized or constructed into an app tree. Once the construction is done, the app goes through preparation, validation, and synthesize steps. So each construct calls its prepare method, then each construct calls its validate method. And then each construct calls its synthesize method.

Up to this point, this is what CDK synth does. From there, the app uploads the template and any other artifacts to CloudFormation, and CloudFormation deployment begins. Once this handoff is done, it is all in the hands of CloudFormation.

CDK Workshop Speedrun

Now that we have defined some of the basic pieces of CDK, let us move on to the workshop. The workshop stops at cdkworkshop.com. You should do this first and then come back. Once you come back, we will do the 30 minute speed round through the workshop.

Setting Up Cloud9 Environment

We are going to get started with this workshop by creating a Cloud9 environment. First, go to the Cloud9 console and create an environment. Name it CDK workshop. Select an instance type - the T3 small works but the M5 large would probably be better. All the other options can use the defaults.

Initializing a New CDK Project

First, create a directory to work in. Then initialize the project by running cdk init sample-app --language typescript. This creates a Git repo and NPM installs all the dependencies for a basic CDK v2 project.

Now that the CDK project is initialized, the CDK workshop will want us to run the watch command in the background to compile the TypeScript into JavaScript. Create a new terminal, change directories into the project folder, and then run the npm run watch command in the background.

Understanding the Project Structure

Let us start where CDK starts with the bin/cdk-workshop.ts file. This file instantiates the CDK app by creating stacks. That is to say, when you run cdk synth, it starts here to generate the CloudFormation templates.

CDK stacks are one to one equivalent with CloudFormation stacks. An app can have multiple stacks, and each stack would end up creating a new stack in CloudFormation. In the case of our sample app, there is only one stack, the CdkWorkshopStack.

The sample stack is pretty simple. It uses two level two constructs. One to create an SQS queue and one to create an SNS topic, then subscribes the queue to the topic. That is it.

First CDK Synth

As described before, when CDK apps are executed, they produce an AWS CloudFormation template for each stack defined in the app. Switching back to Cloud9, let us run cdk synth. I am going to output to a template.yaml file to make it a little easier to read.

Now, there are two important concepts I want to call out here. CDK is both imperative - it describes the how the app will be built. But it is also idempotent - given the same inputs, it will produce the same outputs. See here, we use the same code between the CDK workshop and my Cloud9 instance. And the CloudFormation that was generated is the same. For example, the resource hashes match between the two.

This is because we used all the same inputs that the CDK workshop did. If I went and changed the topic or queue ID, the hash would change and we would have something else. This is going to be important in unit testing later.

Bootstrapping and Deploying

Now that we have synthesized, let us deploy. In order to do that, we need to bootstrap the account. Let us open up a new console window and go to CloudFormation. In order for CDK to deploy apps, it needs to store the assets somewhere. The bootstrap creates its own CloudFormation template that we will see when we go back to CloudFormation. And that includes a number of roles and policies that enable it to store assets for deployment.

The deploy command resynthesizes the app. And if there is any IAM policy changes in the synthesis, it will display them and ask to make sure that you want to continue. Here, since we are subscribing the queue to the topic, an IAM policy needs to be in place in order to make that happen. So yes, we do want that to happen. So let us deploy it.

Creating a Lambda Function

The CDK workshop is going to take you through a couple steps at this point. We are going to clean up the stack by removing the SNS topic and the SQS queue. And then we are going to create a simple API Gateway backed by Lambda.

First, delete the sample code from our stack. Then run cdk diff to see what it shows. The cdk diff command resynthesizes the CloudFormation app and compares the YAML from one to the other. We can see that it is destroying the queue, queue policy, subscription, and the topic.

Next, deploy the changes. Now let us create the Lambda. Create a folder called lambda, and put a hello.js file in it. Inside of that, put some basic Lambda code.

Next, we need to create the Lambda in the stack. We will add the Lambda import from aws-cdk-lib/aws-lambda, then create the actual Lambda function. One of the biggest benefits of creating AWS resources using CDK is the IntelliSense. Having imported from AWS Lambda, I can create a new function using new lambda.Function, pass the stack and ID of the Lambda in, and give it some properties.

In this case, we needed to define the runtime using lambda.Runtime.NODEJS_14_X, where the code lives, which we use code.fromAsset and then lambda, which uploads the lambda folder as an asset to S3. And then the handler. In this case, our handler is hello, the hello file, and it is named handler.

Adding API Gateway

First, we will import the API Gateway module from CDK. And then we will create a Lambda REST API. Lambda REST APIs are just REST APIs with a greedy proxy that sends everything to the defined Lambda. We will pass in the stack construct and give the REST API an ID. And then we will tell it our hello function as the API handler.

Now we can run a diff. And it will show that there is actually no API Gateway already defined. So let us deploy it. What it is doing is creating a bunch of resources automatically. There is the API Gateway itself, permissions for the gateway, and for it to invoke the Lambda, etc. Now that it is done, we can see we went from three resources to fifteen in CloudFormation. And this was all handled with a couple lines of code.

Building a Custom Level 3 Construct

Let us make our own level three construct. We will make an interceptor Lambda that writes to the DynamoDB table, and then invokes any targeted downstream Lambda.

We will create a new hit-counter TypeScript file in the lib folder, and put some boilerplate construct code in here. All this does is extend the Construct and define the props.

This code uses two modules from the AWS SDK to interact with both DynamoDB and Lambda. It uses UpdateItem from DynamoDB to increment a hit counter for a path. And it uses the Lambda client to invoke the downstream Lambda, and then returns the response from that Lambda with the table and Lambda passed into this interceptor Lambda via environment variables.

Testing CDK Constructs

Testing constructs is one of the most powerful things about CDK. The CDK workshop walks you through two types of tests: assertion tests and validation tests.

Assertion Tests

For the first assertion test, we are going to create a stack and use our hit counter construct and then verify that one DynamoDB table was created. The CDK assertions library has some useful features like capture, which can intercept different properties as part of the template synthesis.

Validation Tests

Let us say as part of your construct, you want to make sure a sane number of read capacity units are being used in the DynamoDB table. We can add a read capacity property that gets passed into the construct. Use it in the table. And then in the constructor, we can validate that the value passed in is reasonable, say between five and twenty. If it is not, we will throw an error.

Now let us add a test that passes in a value out of range (e.g., 3) and verify that the error is thrown. When we run the test, it will pass because the error is being thrown. This error would be thrown as part of stack synthesis.

Advanced CDK Topics

Aspects

Aspects are a very powerful tool in CDK. There are a way to apply an operation to all constructs in a given scope. The aspect could modify the constructs such as by adding tags, or it could verify something about the state of the constructs such as ensuring that all the buckets are encrypted.

During the Prepare phase, CDK calls the visit method of the object for the construct and each of its children in a top down order. The visit method is free to change anything in the construct. In this example, the bucket version checking class implements a visit method that checks if S3 bucket versioning is enabled. Here it throws an error, but you could just as easily modify the construct to enable versioning.

Integration Testing

With CDK, you can use AWS custom resources to test that the resources are working together correctly as part of the CloudFormation deployment. If something breaks, it will automatically roll back the CloudFormation deployment.

CDK has a provider framework for interfacing with CloudFormation custom resources. These custom resources enable you to write custom provisioning logic and templates that CloudFormation runs anytime you create, update, or delete stacks. We can make use of this to do integration testing across our stack.

CDK Best Practices

For best practices, AWS breaks up their best practice recommendations into four different areas: Organization, Coding, Construct, and Application.