With 50,000 models and a million auto parts to keep cataloged on a constant basis, ETAI's old homegrown, stovepiped internal database system just wasn't running on all cylinders.
The French auto parts catalog and service manual vendor had nine massive databases, written in different programming languages and using dissimilar fields. That made for a system so large and unwieldy that ETAI couldn't update and consolidate the information fast enough for the 18,000 auto service shops across Europe that use its online and CD- or DVD-based manuals and parts books.
That all began to change in March when ETAI brought in open-source software vendor Talend to integrate the data and streamline the revision processes.
"Before we had Talend, we had this awful homegrown system," said Philippe Bobo, director of software and information systems at Paris-based ETAI. "It was very lengthy. We had some 15 different steps and in between we had minimal control." It took two to three weeks to update and integrate the revised databases, Bobo said. "It should have been two days of automatic, full processing."
Each of ETAI's nine databases covered specific vehicle components, including tires, vehicle glass parts and engines, chassis parts, suspension pieces and more. Making the situation worse, he said, was that existing parts catalogs and service manuals had to be updated even as 100 new releases are brought out each year to cover new vehicle models.
"Just imagine 100 of these new databases, when each takes three weeks to be built," he said. "That was really our problem."
ETAI's manuals and parts catalogs cover European-built vehicles from Peugeot, Renault, Fiat and Ford, or about 85% of the vehicles built there in the last 15 years.
Using Talend's open-source data integration application, Talend Open Studio, ETAI has so far integrated 70% of its existing vehicle parts and service databases, Bobo said. The rest will be done by the end of 2008, with the automation processes completed by the end of the first quarter next year.
ETAI tested products from Talend and three other vendors last January, settling on Talend after five weeks of trials. ETAI went with Talend because its application uses standard programming languages, offers good performance and is much faster than ETAI's legacy software, Bobo said.
Talend Open Studio also allows ETAI to automate the 15 labor-intensive steps needed to integrate the various databases, dramatically streamlining the process, Bobo said. "I'm expecting Talend to automate all the steps so I can press a button on Monday and get the finished database on Wednesday," he said.
The automation process puts the data into temporary data warehouses, checks it, outputs it, formats it and prepares it for use, he said. "All I want at the end is all 15 steps linked together in a single process."
When the data integration project is completed, ETAI expects databases for new car models, along with pricing and full product details, to be buildable in two to three days.
ETAI's vehicle parts catalogs combine parts numbers from automakers and 120 different aftermarket parts suppliers -- who often use their own data formats and applications. Adding to the data crunch are parts "interchange" catalogs; those listings alone add another 20 million to 30 million part numbers to the data that must be integrated into the ETAI catalogs.
The data integration effort is running about three months behind schedule so far, Bobo said, but that's not the vendor's fault. "It is more difficult than we expected," he said. "The support from Talend is very good. We had some times when we were blocked, when we were missing a needed component and they helped create the component. I'm very happy with it."