ETL

Running Jobs with Informatica Cloud REST API

Informatica Cloud offers REST API for us to interact with the platform programmatically. At this point in time, the latest official reference is found here. In this post, we are going to use Python to trigger jobs through API. Triggering a job is a 2-step process. We need to authenticate …

ETL

How To Rename Mapping Job In Informatica Cloud

Mappings are where all the magic happens in Informatica Cloud. When I started using it, it took me a while to work out how to rename a mapping job. Since then, a few people asked me the same question. So, I decided to write about it. This is probably the …

ETL

Informatica Cloud: How To Optimise Joiner Performance In Mapping Designer

Joiner is the stage to join tables in Informatica Cloud (see a quick introduction for Joiner Transformation here). If you have a large volume of data, the joiner transformation becomes very slow without performance optimisation. In this post, we will show you a few tricks that you can use to …

ETL

Informatica Cloud: How To Run More Than 2 Data Synchronization Tasks Concurrently

By default, the secure agent can run 2 data synchronisation tasks at a time. This constraint can become limiting quickly especially when multiple developers are building and testing the data synchronisation tasks at the same time. By adding a custom property on the secure agent, you can run more than …

ETL

How To Configure ODBC Connector For Informatica Cloud Secure Agent

Informatica does not have a dedicated Postgres database connector. Therefore, we need to use the ODBC connector. In this post, I will discuss how to configure Postgres ODBC in both Linux and Windows servers for the Informatica Cloud ODBC connector. Linux Server (Red Hat) There are a few instructions, but …

ETL

Informatica Cloud: How To Use Hierarchy Parser To Transform JSON File

Hierarchy Parser in the Informatica Cloud mapping designer can transform JSON or XML files into structured table (see instruction here). In this post, we will transform the JSON file obtained from Google Geocoding API. Geocoding API turn addresses (1600 Amphitheatre Prakway Mountain View CA) into geographic coordinates (latitude: 37.422, Longitude: -122.085 etc) …

ETL

Informatica Cloud: Cannot create an Unstructured Data transformation

Without properly setting up the Secure Agent server, Hierarchy Parser and Builder do not work in the Informatica Cloud mapping designer. In this post, we will discuss how to resolve the error, ‘Cannot create an Unstructured Data transformation’ when the hierarchy data transformation mapping is performed. There are a few …

ETL

Informatica Cloud: How To Resolve Unknown Host Name For Secure Agent In Linux

For some connectors (e.g. Marketing Cloud connector) to work in Informatica Cloud, we need to configure the host name in the server where the Secure Agent is installed. In this post, we will discuss how to resolve the unknown host name issue with Linux. Steps (1) Execute the command below …

ETL

Informatica Cloud: Compatibility with AWS Redshift

ETL in Redshift demands a specialised connector that optimises insert and upsert operations. Generic JDBC or ODBC ones are too slow and inefficient. When it comes to bulk loading, Amazon recommends to load data into Redshift via S3 by using a copy command (see here). The traditional insert statement is much …

ETL

Informatica Cloud: Incremental Load With Data Synchronization Task

Data Synchronization is a great tool to ingest source data into Data Lake, ODS, or Staging Area. Currently, Data Synchronization does not read database logs to do incremental loads (this is in their road map). Instead, each task automatically stores the last run time stamp ($LastRunTime) in the default task …