Blog Archives

Where is my cloud ETL?

I’ve waited several years for someone else to set up a proper cloud ETL solution. I’ve read Jamie Thomson’s excellent summary What would a cloud-based ETL tool look like? and The API Evangelist’s Bringing ETL to the Masses with APIs

Posted in Cloud ETL, Transformation

Building slowly changing dimensions from salesforce.com data

An interesting problem arose recently. A colleague was trying to build a type 2 slowly changing dimension (SCD 2) using data from the case history object in salesforce.com. The primary problem is that the case history is an audit trail for

Posted in Salesforce.com Tagged with: , , ,

Extract, Load and Transform with Salesforce

Recently I had an interesting situation where I needed to load an account hierarchy daily¬†from a single file. The tricky bit about this was that I couldn’t make any changes to the input source file as it came from a

Posted in Salesforce.com, Transformation Tagged with: , ,

Fascinating genetic atlas of human admixture history

World Ancestry for North Italy

I just heard about this on the Science in Action podcast. It is a fascinating insight into historic events.

Posted in Visualisation Tagged with: , , , ,

Salesforce have moved their documentation and Google hasn’t caught up yet.

Just a word of warning that Salesforce.com have moved all their documentation into a different location and format and Google’s indexes haven’t caught up yet. So if you do a search for something like ‘salesforce sObject methods’ pretty much all

Posted in Salesforce.com Tagged with: , , ,

Talend tHash components are great but fiddly

The tHash component in Talend is a really useful way of caching data in a flow for use in a lookup or for self-joins and loads of other useful things. However it has one fiddly draw back which is that

Posted in Transformation Tagged with: , ,

Normalising Data with Talend tMap

The other day I got a particularly tricky set of data to transform. It consisted of a list of contracts each identifying an account and a list of contacts related to the contract and by implication the account. I needed

Posted in Transformation Tagged with: , ,