Sending XML Payload and Converting XML Response to JSON with Python
If you need to interact with a REST endpoint that takes a XML string as a payload and returns another XML string as a response, this is the quick guide if you want to use Python. If you want to do it with Node.js, you can check out the post …
Sending XML Payload and Converting XML Response to JSON with Node.js
Here is the quick Node.js example of interacting with a rest API endpoint that takes XML string as a payload and return with XML string as response. Once we get the response, we will convert it to a JSON object. For this example, we will use the old-school QAS (Quick …
Downloading All Public GitHub Gist Files
I used to use plug-ins to render code blocks for this blog. Yesterday, I decided to move all the code into GitHub Gist and inject them from there. Using a WordPress plugin to render code blocks can be problematic when update happens. Plugins might not be up to date. It …
Loading Data Frame to Relational Database with R
Once you create a data frame with R, you may need to load it to a relational database for data persistence. You might have a data transformation batch job written in R and want to load database in a certain frequency. Here, I created a function to load data into …
Exporting LaunchDarkly Flag List into a CSV File with Python
At the moment, LaunchDarkly does not have functionality to export a list of flags as csv or excel file. This can change very near future (it may already have the functionality by the time you are reading this post). The workaround is to use API to ingest the data. Here …
Event-Driven S3 Data Ingestion With Node.js Lambda Function and Deploy it with Serverless
Ingesting data upon the file creating on S3 bucket enables near real-time data ingestion. For example, you may need to ingest log files from applications or API monitoring tools as soon as they land on the bucket. Just to get it started, let’s move the file from the source bucket …
Uploading and Downloading Files in S3 with Ruby
To date, the latest Ruby AWS SDK is version 3. In this version, each resources has its own module while the version 2 had the one with everything, aws-sdk. To interact with S3 with the v3 SDK, let’s use the aws-sdk-s3 module. Ruby has its own way to upload to …
Uploading and Downloading Files in S3 with Node.js
AWS S3 is probably the most utilised AWS storage services. It is affordable, highly available, convenient and easy to use. To interact with any AWS services, Node.js requires AWS SDK for JavaScript. Let’s first create a project folder called nodeS3 and install SDK. Then, create the main program file and …
How to Ingest Data Into MongoDB with Node.js
Now that we got data from MongoDB and loaded into Postgres with some transformation, let’s put the data back into MongoDB. This post is the continuation of the previous post about ingesting data from MongoDB. You should totally start from How to Ingest Data From MongoDB with Node.js. The aim …
How to Ingest Data From MongoDB with Node.js
Data ingestion from a NoSQL database often involves the denormalisation of their schema-less data before loading into a relational database. In this post, we will try to grab the restaurant data from MongoDB, denormalise the collection into a parent and a child table, and load them into Postgres. This exercise …