Infrastructure

How to Persist Data in Local DynamoDB Docker Container

Having DynamoDB running locally is handy for local development. There are two ways to run DynamoDB locally, (1) using a jar file or (2) using a docker image. Generally speaking, running container is better than running a jar file in your local computer because it gives you more flexibility and …

Node.js

Node Module to Retrieve Multiple Parameters from AWS Parameter Store

I created a node module to retrieve multiple parameters as the parameterName-parameterValue Json object from AWS Parameter Store. It is called aws-ssm-parameters and is available from Npm. The module is namespaced. So, you need to add the namespace @mdhnpm when you install it. npm i @mdhnpm/aws-ssm-parameters The main motivation for creating …

Front-End

Uploading File from Browser to S3 with JavaScript

Uploading a file from a browser to S3 for only authorised users is a pretty cool trick. It is even cooler if we can build it in a serverless way. Here is the entire solution for a simple serverless app to authenticate with AWS Cognito and upload a file to …

Data Engineering

Uploading and Downloading Files in S3 with Node.js

AWS S3 is probably the most utilised AWS storage services. It is affordable, highly available, convenient and easy to use. To interact with any AWS services, Node.js requires AWS SDK for JavaScript. Let’s first create a project folder called nodeS3 and install SDK. Then, create the main program file and …

AWS

Creating Publicly Accessible RDS with CloudFormation

AWS CloudFormation let us create AWS resources with JSON or YAML files. With CloudFormation, you can create and update your AWS infrastructure by code. In the previous post, we discuss how we can create publicly available RDS (How to Make RDS in Private Subnet Accessible From the Internet). In this …

AWS

How to Make RDS in Private Subnet Accessible From the Internet

When you have RDS in a private subnet, you cannot access from the Internet. Accessing RDS from the Internet is not necessary and is not a good practice if it is used for the application backend. However, your RDS is an analytics database and needs to be accessed by another …

Data Engineering

Comprehensive Guide to Download Files From S3 with Python

Using AWS SDK for Python can be confusing. First of all, there seems to be two different ones (Boto and Boto3). Even if you choose one, either one of them seems to have multiple ways to authenticate and connect to AWS services. Googling solutions can quickly become confusing as you …

AWS

How To Launch Postgres RDS With AWS Command Line Interface (CLI)

Now that we can launch a bootstrapped EC2 (How To Launch EC2 With Bootstrapping in AWS) with AWS CLI, let’s explore how to launch Postgres RDS with AWS CLI. As in the previous post, we can basically code infrastructure and resources in AWS. This is a step toward coding up …

AWS

How To Launch EC2 With Bootstrap in AWS

When you launch an AWS resource (EC2 or RDS), you will have an instance with a default configuration. Then, you have to install software and do custom configuration to bring it to a certain state. Bootstrapping enables you to script software installation and configuration and execute it while launching the …

Data Science

How To Create Your Own Personal Data Science Computing Environment In AWS

Running a training algorithm is such a time-consuming task when you are building a machine learning application. If you are developing it with your computer, you cannot do anything else for a long period of time (hours and maybe days) on that machine. Especially when we do parallel processing using …