Electricity and Water Utility

Customised ETL Solution for Client in Qatar

We were engaged along with a large Indian System Integrator  to replace an ETL solution which was unacceptably inefficient for their Data Warehouse on Oracle platform.

The ETL – Extract, Transform and Load – populated Data Warehouse consisting of electricity and water billing transactions and financial accounting data at summary level.

The original solution using Data Stage took about 12 hours every night to  incrementally refresh Data Warehouse. Often, the ETL job was not finished before the next day’s operations began.

We addressed this challenge and  built an ETL solution using Oracle DI tools. The new ETL solution takes about 3 hours time to refresh Data warehouse every night.

One significant challenge that we dealt with was lack of documentation; so we reverse engineered the solution:

  • We could not make any changes in Data Warehouse structure as a huge number of reports were using them
  • Huge number of  data elements  were mapped between JD Edwards ERP system and Data Warehouse
  • Hundreds of data transformation rules were decoded from the existing system and specifications were written
  • Data tally was a critical part of the  project delivery with certification of 100 % tally of about 80 selected reports.

We provided complete documentation, training and six months of support before handing over the solution to the local IT team.