How To Install Python 3 and Create Virtual Environment in Centos, Redhat and Amazon Linux

The scope of this post is to install Python 3 for a user and create a virtual environment in Centos, Redhat or Amazone Linux. Linux comes with Python 2.7 and the best practice is not to mess with. This is because Linux OS has some dependency and upgrading it to 3 may cause problems. If you are developing with Python 3, I recommend to install it for a user. Knowing how to create virtual environments becomes handy for serverless microservice development, too.

If you want to do the same thing with Ubuntu, check out the post here

Let’s get started.

(1) Download Python Installation Package

You first need to get  the installation link (https://www.python.org/downloads/release/python-364/). Today, python-3.6.4 is the latest version. Let’s create a directory called python3 in your home directory and download it there and unzip it.

mkdir ~/python3
cd python3
wget https://www.python.org/ftp/python/3.6.4/Python-3.6.4.tar.xz
tar xf Python-3.*
cd Python-3*

(2) Install it with Pip

We first need to install Development Tools for Linux. Then, configure the installation path and include pip upgrade.

sudo yum groups install "Development Tools"
./configure --prefix=/home/ec2-user/pythonfin --with-ensurepip=upgrade

(3) Build

You first need to install openssl-devel otherwise you will get pip requires SSL/TSL error. Then you can build it.

sudo yum install zlib-devel bzip2-devel sqlite sqlite-devel openssl-devel
make

(4) Install for the user

When you don’t want to overwrite the python executable (safer, at least on some distros yum needs python to be 2.x, such as for RHEL6),  you can install python3.* as a concurrent instance to the system default with an altinstall as below

make altinstall

(5) Create Shortcut For Installed Python 3

This makes it easier for the creation of virtual environments.

export python3=/home/ec2-user/pythonfin/bin/python3.6
export pip3=/home/ec2-user/pythonfin/bin/pip3.6

(6) Install virtualenv and Create Virtual Environment

You can use the environment variables created in the previous step. In the command below, you are creating it in the pythonfin folder where Python 3 is installed.

$pip3 install virtualenv
$python3 -m venv lambda (this creates environment in the pythonfin folder)

(7) Activate Environment

Once  you activate the environment, any installation with pip will go there. The python command uses the python binary for the environment.

source /home/ec2-user/pythonfin/lambda/bin/activate

You are ready to go!

Bonus Section

To apply above knowledge, let’s bootstrap EC2 instance with Python 3 with Tensorflow and Keras installed at user level. You can save the script below as a text file and use it as user-data when you launch the instance.

To bootstrap EC2 from a text file with AWS CLI, you need to have #!/bin/bash at the start to trigger the script. Make sure to have -y for auto-installation. You do not need to add sudo as the script is executed by root.

#!/bin/bash
yum update -y
yum install zlib-devel bzip2-devel sqlite sqlite-devel openssl-devel -y
yum groups install "Development Tools" -y
mkdir /home/ec2-user/python3
cd /home/ec2-user/python3
wget https://www.python.org/ftp/python/3.6.4/Python-3.6.4.tar.xz
tar xf Python-3.*
cd /home/ec2-user/python3/Python-3.6.4
./configure --prefix=/home/ec2-user/python3 --with-ensurepip=upgrade
make
make altinstall
/home/ec2-user/python3/bin/pip3.6 install h5py tensorflow tflearn keras boto3
Git
How to specify which Node version to use in Github Actions

When you want to specify which Node version to use in your Github Actions, you can use actions/setup-node@v2. The alternative way is to use a node container. When you try to use a publicly available node container like runs-on: node:alpine-xx, the pipeline gets stuck in a queue. runs-on is not …

AWS
Using semantic-release with AWS CodePipeline and CodeBuild

Here is the usual pattern of getting the source from a git repository in AWS CodePipeline. In the pipeline, we use AWS CodeStart to connect to a repo and get the source. Then, we pass it to the other stages, like deploy or publish. For some unknown reasons, CodePipeline downloads …

DBA
mysqldump Error: Unknown table ‘COLUMN_STATISTICS’ in information_schema (1109)

mysqldump 8 enabled a new flag called columm-statistics by default. When you have MySQL client above 8 and try to run mysqldump on older MySQL versions, you will get the error below. mysqldump: Couldn’t execute ‘SELECT COLUMN_NAME, JSON_EXTRACT(HISTOGRAM ‘$”number-of-buckets-specified”‘) FROM information_schema.COLUMN_STATISTICS WHERE SCHEMA_NAME = ‘myschema’ AND TABLE_NAME = ‘craue_config_setting’;’: Unknown …