Having DynamoDB running locally is handy for local development. There are two ways to run DynamoDB locally, (1) using a jar file or (2) using a docker image. Generally speaking, running container is better than running a jar file in your local computer because it gives you more flexibility and …

When we create a custom Python module, we can distribute it through a Git repository. By using a private repo, you can keep your module private and use it in your deployment pipeline by calling pip install from the private repo. Pretty cool trick. In this example, I created a …

Both GitHub and AWS CodeCommit are used to version-control your source code. I use GitHub to share my code with the wider community and CodeCommit for production code for AWS. You may use other source control systems like BitBucket. Whatever the choice you make, it is convenient to set up …

You can deploy lambda functions manually with hard-coded environment variables. But, that is not a good development practice. When we deploy the function to production, the deployment process eliminates the necessity of changing any input parameters or configuration. Serverless is a lambda function deployment tools that enable good development practice …

To deploy a AWS lambda function written in Python, you need to package all the modules that are used in your code before pushing it to your environment. Serverless is a great lambda deployment framework. It takes care of all the packaging and deployment. With Python, the best approach to …

Docker and Serverless do not work well with Windows. I have tried so many different ways to make them work, to no avail. My solution is to use an Ubuntu virtual machine. I primarily use those frameworks for AWS lambda developments with both Python and Node.js. Here is how I …

If you need to use the latest Python 3 version in Linux as a non-root user, you can download the latest python in your space and install it there. In this way, you will have the separate Python development environment from the one that OS relies on. Once you installed …

The scope of this post is to install Python 3 for a user and create a virtual environment in Centos, Redhat or Amazone Linux. Linux comes with Python 2.7 and the best practice is not to mess with. This is because Linux OS has some dependency and upgrading it to …

I recently installed Docker Toolbox on Windows machine (Windows 10 Home version). We all know Docker is awesome and serverless microservice architecture is hot. Docker has a better tool, called Docker for Windows. Unfortunately, this does not work with Windows 10 Home version (it works with Windows 10 Professional versions …

Apache Spark is a powerful framework to utilise cluster-computing for data procession, streaming and machine learning. Its native language is Scala. It also has multi-language support with Python, Java and R. Spark is easy to use and comparably faster than MapReduce. For example, you can write Spark on the Hadoop …