Data Engineering

Event-Driven S3 Data Ingestion With Node.js Lambda Function and Deploy it with Serverless

Ingesting data upon the file creating on S3 bucket enables near real-time data ingestion. For example, you may need to ingest log files from applications or API monitoring tools as soon as they land on the bucket. Just to get it started, let’s move the file from the source bucket …

Data Engineering

Uploading and Downloading Files in S3 with Ruby

To date, the latest Ruby AWS SDK is version 3. In this version, each resources has its own module while the version 2 had the one with everything, aws-sdk. To interact with S3 with the v3 SDK, let’s use the aws-sdk-s3 module. Ruby has its own way to upload to …

Data Engineering

Uploading and Downloading Files in S3 with Node.js

AWS S3 is probably the most utilised AWS storage services. It is affordable, highly available, convenient and easy to use. To interact with any AWS services, Node.js requires AWS SDK for JavaScript. Let’s first create a project folder called nodeS3 and install SDK. Then, create the main program file and …