Uploading and Downloading Files in S3 with Ruby

To date, the latest Ruby AWS SDK is version 3. In this version, each resources has its own module while the version 2 had the one with everything, aws-sdk. To interact with S3 with the v3 SDK, let’s use the aws-sdk-s3 module.

Ruby has its own way to upload to S3. First of all, you need to create the s3 object and then call methods on that object. With Node.js, we first read the file and convert to the binary format and upload the data into the object (see here). With Python, we can simply pass the bucket name, key, and local file path in the upload function on the S3 object (see here).

Downloading is the same. We first need to create the S3 object that you want to retrieve and call the get() method.

The authentication is handled by configuring AWS CLI on your local machine or server. To configure AWS CLI, you can check the link here.

First of all, you need to install the module.

gem install aws-sdk-s3

Uploading File

The function takes 3 arguments, bucket name, prefix and local file path. From the File object, you can extract the file name in the path with basename(). Once the target S3 object is created, you can all the upload_file() function.

Downloading File

Downloading takes the same flow as uploading. You first need to create the object you want to retrieve with bucket name and object key. Then, call the get() method.

Data Engineering
Sending XML Payload and Converting XML Response to JSON with Python

If you need to interact with a REST endpoint that takes a XML string as a payload and returns another XML string as a response, this is the quick guide if you want to use Python. If you want to do it with Node.js, you can check out the post …

Data Engineering
Sending XML Payload and Converting XML Response to JSON with Node.js

Here is the quick Node.js example of interacting with a rest API endpoint that takes XML string as a payload and return with XML string as response. Once we get the response, we will convert it to a JSON object. For this example, we will use the old-school QAS (Quick …

Data Engineering
Downloading All Public GitHub Gist Files

I used to use plug-ins to render code blocks for this blog. Yesterday, I decided to move all the code into GitHub Gist and inject them from there. Using a WordPress plugin to render code blocks can be problematic when update happens. Plugins might not be up to date. It …