Creating user credentials and grand privileges in Postgres can be tricky. You have to make sure each user has the right access level to the right tables and schemas. Each database works differently even they all are relational databases. Postgres has its own way of creating users and granting privileges. …

Once you create a sandbox environment in your server for WordPress and copied all the files from production, it’s time to copy the production data into the sandbox database. There are many ways to do this. I decided to use Talend Open Studio to insert production data into the sandbox …

When you play with different layouts, content delivery methods or backend logics with WordPress without affecting your public facing site, it is handy to have a sandbox environment where you can go nuts without affecting your production environment. There are many ways to create a sandbox environment. You can created …

In modern web development, we try to avoid building a website from scratch if we can. Websites are usually built on top of a platform like Sitecore, Drupal, WordPress and so on. When I first thought about creating this blog, I tried to code everything from scratch, going against the …

Joiner is the stage to join tables in Informatica Cloud (see a quick introduction for Joiner Transformation here). If you have a large volume of data, the joiner transformation becomes very slow without performance optimisation. In this post, we will show you a few tricks that you can use to …

By default, the secure agent can run 2 data synchronisation tasks at a time. This constraint can become limiting quickly especially when multiple developers are building and testing the data synchronisation tasks at the same time. By adding a custom property on the secure agent, you can run more than …

In the previous post, we used grid search to find the best hyper parameter for the neural network model with R’s caret package. Here, let’s use Python and scikit-learn package to optimise a neural network model. Just like the caret package, scikit-learn has a pre-built function for hyper parameter search. …

Once you finish training the model and are happy with it, you may need to consider saving the model. Otherwise, you will loose the model once you close the session. The model you create in R session is not persistent, only existing in the memory temporarily. Most of the time, …

Writing code to do machine learning is easy. What makes it difficult is the optimisation. By large, there are two ways to optimise your model. Feature selection & transformation Model parameter optimisation Both are hard-core topics and neither can be covered in this post. Feature selection and transformation often require …

Once you create an awesome data science application, it is time for you to deploy it. There are many ways to productionise them. The focus here is deploying Spark applications by using the AWS big data infrastructure. From my experience with the AWS stack and Spark development, I will discuss …