Creating Publicly Accessible RDS with CloudFormation

AWS CloudFormation let us create AWS resources with JSON or YAML files. With CloudFormation, you can create and update your AWS infrastructure by code.

In the previous post, we discuss how we can create publicly available RDS (How to Make RDS in Private Subnet Accessible From the Internet). In this post, let’s create a CloudFormation template for the public RDS stack.

We are going to use YAML file for the template because it is easier to manage than JSON. In fact, writing CloudFormation template in JSON is much harder as you need to worry about curly brackets and quotations.

What to create

From the previous post, we are creating the first design (diagram below). This one has more resources than the second one and would be more interesting to code.

Resources

List of resources we are creating with CloudFormation is below.

  • VPC
  • Public Subnets
  • Private Subnets
  • RDS
  • Network Load Balancer
  • Load Balancer Target Group
  • Route Tables
  • Security Groups

Limitations on Target Group RDS IP Address Mapping

The vanilla CloudFormation does not support getting IP address of the RDS instance. The GetAtt function only returns the endpoint address with Endpoint.Address. For Network Load Balancer, the target group has to be an IP address. Therefore, the mapping of RDS IP address to the target group cannot be done with just using the simple CloudFormation template (there are workarounds you can do by using SDK or custom resources). For this post, let’s keep it simple and accept this as a limitation.

Once it creates all the resources, you need to do nslookup to obtain the IP address of RDS from the endpoint, then create target with the IP and port in target group.

Using DNS

If you have your own domain name, you can create a hosted zone and map the Load Balancer as Alias as a record set. In this way, you do not need to change the client connection details every time the stack gets updated or recreated.

Parameter File

At the moment, CloudFormation does not support YAML as an external parameter file format. The parameter file is set up as below and will be referenced in the main template. I added the example range. You can update CIDR ranges for VPC and each subnet as you like.

[
    {
      "ParameterKey": "VpcCidr",
      "ParameterValue": "10.5.0.0/16"
    },
    {
      "ParameterKey": "DbPublic1ACidr",
      "ParameterValue": "10.5.4.0/24"
    },
    {
      "ParameterKey": "DbPublic1BCidr",
      "ParameterValue": "10.5.5.0/24"
    },
    {
      "ParameterKey": "DbPublic1CCidr",
      "ParameterValue": "10.5.6.0/24"
    },
    {
      "ParameterKey": "DbPrivate1ACidr",
      "ParameterValue": "10.5.1.0/24"
    },
    {
      "ParameterKey": "DbPrivate1BCidr",
      "ParameterValue": "10.5.2.0/24"
    },
    {
      "ParameterKey": "DbPrivate1CCidr",
      "ParameterValue": "10.5.3.0/24"
    }
]

Execution

Once the template is ready, we can run the command below from the folder where you keep your template and parameter files.

aws cloudformation create-stack ^
--stack-name create-public-db ^
--template-body file://main_final.yaml ^
--parameters file://parameters.json

Template

Here is the actual template. Fill the CIDR range and try running it in your AWS environment. Using CloudFormation does not cost you. But, the resources created will cost. The example only uses the resources from free-tier (the first 12 months one). Even if you ran out of the free tier credit, the cost should be minimal. Have a go!

AWSTemplateFormatVersion: '2010-09-09'
# Parameters for external parameter file reference
Parameters:
  VpcCidr:
    Description: CIDR block for the main VPC
    Type: String
  DbPublic1ACidr:
    Description: CIDR block for Public Subnet 1
    Type: String
  DbPublic1BCidr:
    Description: CIDR block for Public Subnet 2
    Type: String
  DbPublic1CCidr:
    Description: CIDR block for Public Subnet 3
    Type: String
  DbPrivate1ACidr:
    Description: CIDR block for Private Subnet 1
    Type: String
  DbPrivate1BCidr:
    Description: CIDR block for Private Subnet 2
    Type: String 
  DbPrivate1CCidr:
    Description: CIDR block for Private Subnet 3
    Type: String 

Resources:
  # (1) Define VPC
  DbPublicVpc:
    Type: AWS::EC2::VPC
    Properties: 
      CidrBlock: !Ref 'VpcCidr'
      EnableDnsSupport: true
      EnableDnsHostnames: true
      InstanceTenancy: default
      Tags:
      - Key: Name
        Value: public-db-practice

  # (3) Create Subnets
  #  public subnets
  DbPublic1A:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref DbPublicVpc
      CidrBlock: !Ref 'DbPublic1ACidr'
      AvailabilityZone: ap-southeast-2a
      Tags:
        - Key: Name
          Value: subnet1A-public-db-practice
  
  DbPublic1B:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref DbPublicVpc
      CidrBlock: !Ref 'DbPublic1BCidr'
      AvailabilityZone: ap-southeast-2b
      Tags:
        - Key: Name
          Value: subnet1B-public-db-practice
  
  DbPublic1C:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref DbPublicVpc
      CidrBlock: !Ref 'DbPublic1CCidr'
      AvailabilityZone: ap-southeast-2c
      Tags:
        - Key: Name
          Value: subnet1C-public-db-practice
  
  # (3-2) Private Subnets
  DbPrivate1A:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref DbPublicVpc
      CidrBlock: !Ref 'DbPrivate1ACidr'
      AvailabilityZone: ap-southeast-2a
      Tags:
        - Key: Name
          Value: subnet2A-public-db-practice

  DbPrivate1B:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref DbPublicVpc
      CidrBlock: !Ref 'DbPrivate1BCidr'
      AvailabilityZone: ap-southeast-2b
      Tags:
        - Key: Name
          Value: subnet2B-public-db-practice

  DbPrivate1C:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref DbPublicVpc
      CidrBlock: !Ref 'DbPrivate1CCidr'
      AvailabilityZone: ap-southeast-2c
      Tags:
        - Key: Name
          Value: subnet2V-public-db-practice
  
  # Internet Gateway
  InternetGatewayPrivateDb:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
        - Key: type
          Value: cloudformation-practice
  
  gw1:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      VpcId: !Ref DbPublicVpc
      InternetGatewayId: !Ref InternetGatewayPrivateDb
  
  # Route Table 
  PublicSubnetRoute:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref DbPublicVpc
      Tags: 
        - Key: Name
          Value: Public-Route
  
  PrivateSubnetRoute:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref DbPublicVpc
      Tags: 
        - Key: Name
          Value: Private-Route
  
  # Associate Internate Gateway to Route Table
  SubnetRoutePublic1A:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref DbPublic1A
      RouteTableId: !Ref PublicSubnetRoute
  
  SubnetRoutePublic1B:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref DbPublic1B
      RouteTableId: !Ref PublicSubnetRoute
  
  SubnetRoutePublic1C:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref DbPublic1C
      RouteTableId: !Ref PublicSubnetRoute

  SubnetRoutePrivate1A:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref DbPrivate1A
      RouteTableId: !Ref PrivateSubnetRoute
  
  SubnetRoutePrivate1B:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref DbPrivate1B
      RouteTableId: !Ref PrivateSubnetRoute
  
  SubnetRoutePrivate1C:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref DbPrivate1C
      RouteTableId: !Ref PrivateSubnetRoute
  
  # Assign routing rule to route tables (default rule does not need to be assigned)
  PublicRoute1:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      RouteTableId: !Ref PublicSubnetRoute
      GatewayId: !Ref InternetGatewayPrivateDb

  # Create DB Subnet Group
  DbSubnetGroup:
    Type: AWS::RDS::DBSubnetGroup
    Properties:
      DBSubnetGroupDescription: For Launching RDS
      DBSubnetGroupName: db-subnet-group
      SubnetIds:
      - !Ref DbPrivate1A
      - !Ref DbPrivate1B
      - !Ref DbPrivate1C
  
  # Create DB Security Group
  DbSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: For RDS Instance
      VpcId: !Ref DbPublicVpc
      Tags:
      - Key: Name
        Value: RDS-SecurityGroup

  # Attach Secuirty Group Rule
  DbIngress1:
    Type: AWS::EC2::SecurityGroupIngress
    Properties:
      GroupId: !Ref DbSecurityGroup
      IpProtocol: tcp
      FromPort: '5432'
      ToPort: '5432'
      CidrIp: !Ref 'VpcCidr'
  
  # Create Postgres Instance
  PostgresRDS:
    Type: "AWS::RDS::DBInstance"
    Properties:
      AllocatedStorage: 20
      AvailabilityZone: ap-southeast-2b
      DBInstanceClass: db.t2.micro
      DBInstanceIdentifier: Postgres-RDS
      DBName: mydatahack
      DBSubnetGroupName: !Ref DbSubnetGroup
      Engine: postgres
      MasterUsername: mydatahack
      MasterUserPassword: mydatahackrocks
      MultiAZ: false
      Port: 5432
      PubliclyAccessible: false
      Tags:
        - Key: Name
          Value: RDS-Postgres
      VPCSecurityGroups:
        - !Ref DbSecurityGroup

  # Create Load Balancer
  NetworkLoadBalancer:
    Type: "AWS::ElasticLoadBalancingV2::LoadBalancer"
    Properties:
      Scheme: internet-facing
      Subnets:
        - !Ref DbPublic1A
        - !Ref DbPublic1B
        - !Ref DbPublic1C
      Tags:
        - Key: Name
          Value: rds-load-balancer
      Type: network
      IpAddressType: ipv4
      Tags:
        - Key: Name
          Value: rds-load-balancer

  NLBListener:
    Type: AWS::ElasticLoadBalancingV2::Listener
    Properties:
      DefaultActions:
      - Type: forward
        TargetGroupArn: !Ref TargetGroup
      LoadBalancerArn: !Ref NetworkLoadBalancer
      Port: 5432
      Protocol: TCP   

  TargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      HealthCheckIntervalSeconds: 30
      HealthCheckProtocol: TCP
      HealthCheckTimeoutSeconds: 10
      HealthyThresholdCount: 3
      Name: RdsTarget
      Port: 5432
      Protocol: TCP
      TargetGroupAttributes:
        - Key: deregistration_delay.timeout_seconds
          Value: '20'
      TargetType: ip
      #Targets:
      #- Id: RDS doesn't return IP address with !GetAtt Not supported by CloudFormation
      #  Port: 5432
      UnhealthyThresholdCount: 3
      VpcId: !Ref DbPublicVpc
Git
How to specify which Node version to use in Github Actions

When you want to specify which Node version to use in your Github Actions, you can use actions/setup-node@v2. The alternative way is to use a node container. When you try to use a publicly available node container like runs-on: node:alpine-xx, the pipeline gets stuck in a queue. runs-on is not …

AWS
Using semantic-release with AWS CodePipeline and CodeBuild

Here is the usual pattern of getting the source from a git repository in AWS CodePipeline. In the pipeline, we use AWS CodeStart to connect to a repo and get the source. Then, we pass it to the other stages, like deploy or publish. For some unknown reasons, CodePipeline downloads …

DBA
mysqldump Error: Unknown table ‘COLUMN_STATISTICS’ in information_schema (1109)

mysqldump 8 enabled a new flag called columm-statistics by default. When you have MySQL client above 8 and try to run mysqldump on older MySQL versions, you will get the error below. mysqldump: Couldn’t execute ‘SELECT COLUMN_NAME, JSON_EXTRACT(HISTOGRAM ‘$”number-of-buckets-specified”‘) FROM information_schema.COLUMN_STATISTICS WHERE SCHEMA_NAME = ‘myschema’ AND TABLE_NAME = ‘craue_config_setting’;’: Unknown …