Uploading and Downloading Files in S3 with Ruby
- By : Mydatahack
- Category : Data Engineering, Data Ingestion
- Tags: aws-sdk-s3, Ruby, S3
To date, the latest Ruby AWS SDK is version 3. In this version, each resources has its own module while the version 2 had the one with everything, aws-sdk. To interact with S3 with the v3 SDK, let’s use the aws-sdk-s3 module.
Ruby has its own way to upload to S3. First of all, you need to create the s3 object and then call methods on that object. With Node.js, we first read the file and convert to the binary format and upload the data into the object (see here). With Python, we can simply pass the bucket name, key, and local file path in the upload function on the S3 object (see here).
Downloading is the same. We first need to create the S3 object that you want to retrieve and call the get() method.
The authentication is handled by configuring AWS CLI on your local machine or server. To configure AWS CLI, you can check the link here.
First of all, you need to install the module.
gem install aws-sdk-s3
Uploading File
The function takes 3 arguments, bucket name, prefix and local file path. From the File object, you can extract the file name in the path with basename(). Once the target S3 object is created, you can all the upload_file() function.
Downloading File
Downloading takes the same flow as uploading. You first need to create the object you want to retrieve with bucket name and object key. Then, call the get() method.