'The key factor in a successful implementation of a data warehouse is modelling the business correctly. Requirements change but the business never changes. The data warehouse must be built on business functions and processes. Not basing on requirements is where 60 to 70 per cent of organisations go wrong.
Steve Hitchman, founder and managing director, Management Information Principles (MIP), has been in the data management business for 10 years. He not only forcibly expressed the above statements but also had strong opinions on what is right and wrong in the data warehouse and business intelligence world now.
"A data warehouse is best implemented in small chunks so it is manageable and an organisation can budget for it. The big bang has gone. For example, a company could start with finance then [go on to] funds dealing, making sure they are linked together. You cannot buy a data warehouse, because they evolve. Every data warehouse is completely different; as the information stored has to be pertinent to the business, it has to be a business-based data store," Hitchman said.
"The success of the data warehouse hinges on the IT department working in conjunction with its business managers, otherwise they are doomed to failure."
One of the questions that Hitchman was a little shy to answer was naming successful data warehouse implementations in Australia. "That's tough because clients do not want their competition to know they even have a data warehouse. A successful data warehouse is considered to be a strategic advantage. Nevertheless he did say that Solution 6's data warehouse still works, even though the data warehouse was implemented three owners ago. It was developed by CVSI, which was then taken over by ComputerVision and then Solution 6. The reason it still works, Hitchman said, is that the business has not changed despite changing ownership three times.
"Organisations may well set up subject-oriented data marts that are a subset of the data in the data warehouse for active analysis. On saying that, there are different styles of analysis: passive used for verification and active where value is gained from the data.
"Passive relies on Online Analytical Processing (Olap) or Relational Online Analytical Processing (Rolap), which needs a user to ask a question. Active equates to predictive and correlates to finding out things you don't actually know (knowledge discovery). It is called data mining and it can use neural networks, decision trees, clustering and segmentation.
"Data warehouses still store data on products and services, but there is a swing towards customer-centric data warehouses where there is one view of a customer, which is absolutely necessary for a successful customer relationship management (CRM) strategy. Organisations cannot do proper CRM without it, so you cannot do CRM out of a box, because it is data-based. Examples of box-based CRM solutions are Clarify, Onyx, Siebel, Valex, Great Elk and Hard Hanks."
One of MIP's biggest competitors is SAS, but Hitchman said: "MIP is the leading data mining company solution provider in Australia. The company runs three divisions: consultancy, products and risk management."
The consultancy division provides strategic advice to organisations on data warehouses, data mining and decision support. It also builds data warehouses for customers, as well as running training courses in data warehouse and data mining.
The products division promotes, amongst others, Clemetime for data mining, DataStage and MicroStrategy for Rolap. The third division consults on risk management and has developed its own risk management products.
Hitchman explained why MIP chose a Rolap product rather than an Olap/Molap (Management of Online Analytical Processing) product from companies like Brio, Cognos and Business Objects. He said: "The limitation is that these products are cube-based, and cubes are not big enough to get down to the transactional data. Whereas with Rolap there is no cube, and it provides a quick slice and dice of the data and analysis directly against the data warehouse. Rolap is the way forward. With the use of the other products (Olap/Molap) an organisation guarantees it will fail.
"A teradata data warehouse is a good data warehouse but it is very expensive and an organisation needs a good decision support tool to take advantage of the data in a teradata warehouse.
"There are a lot of data cemeteries out there. Another name for them is a data dump. Organisations have created a load of data, which does not reflect what they want. The data is unreliable; it's too slow to access it and the results cannot be trusted. MIP gets called in to do health checks on data warehouses, sometimes when up to 75 per cent of the work has already been done. This check will usually take around four to five weeks doing user interviews and looking at the data from a business perspective and a technical perspective. We remodel the data warehouse and then see how the data store fits. We look at features like extraction and loading of data, design and delivery mechanisms. After the analysis we recommend a course of action to rectify the data warehouse. Sometimes MIP does the work and other times the organisation is able to do it themselves.
"The cost of a data warehouse is substantial, but it can be done in edible chunks so an organisation can get the value from each chunk before moving on. Data warehouses cost-justify themselves now with a concerted move into the new business of publishing information from a data warehouse for use in applications like e-commerce. Banks, for example, can make a profit from [publishing information from] their data warehouses and in the process turn their data warehouses into a profit centre rather than a cost centre."