Informatica Cloud: How To Run More Than 2 Data Synchronization Tasks Concurrently
- By : Mydatahack
- Category : ETL, Informatica
- Tags: Data Synchroniszation, Informatica Cloud Configuration
By default, the secure agent can run 2 data synchronisation tasks at a time. This constraint can become limiting quickly especially when multiple developers are building and testing the data synchronisation tasks at the same time. By adding a custom property on the secure agent, you can run more than 2 jobs concurrently.
Steps
(1) Log in to Informatica cloud
(2) Configure -> Runtime Environment -> Pick the secure agent
(3) Go all the way down and you will find Custom Configuration tab. Select service as Data integration server. Set Type as Tomcat and Name as maxDTMProcesses. Value would be the number of jobs you want to run concurrently. For example, setting 4 means 4 jobs run concurrently.
Note
Informatica recommend to distribute the load among the available secure agents instead of running all the tasks using one secure agents. The number of secure agent you can have depends on the contract.
However, running jobs in multiple secure agent servers makes it harder to manage. If you just want to increase it to 4 or 5 jobs, it’s usually fine with updating the configuration.
The max number of concurrent jobs you can run with one secure agent depends on the volume of data as well as the size of the secure agent server. Running concurrent tasks may slow down the speed of individual tasks. The best practice is to experiment it and find the magic number where performance compromise is minimum. I recommend to monitor the server performance and capacity while you are running concurrent tasks.
Data Synchronization is probably the best tool to do one-to-one copy between relational databases in the market. It is super easy to build jobs. It is so solid and never really fails unless source database changes. Coming from the DataStage world, I found that Data Synchronisation minimises the time worrying about one-to-one copy jobs and makes me really focus on transformation, adding more value to the business.
If you are interested in incremental load with Data Synchronization tasks, check out the post here.
Enjoy!