Etsy gets crafty with big data
Handmade marketplace Etsy has grown to 800,000 sellers and 40+ million monthly visitors. All that activity generates enormous quantities of data, which Etsy uses to drive site improvements.
Handmade marketplace Etsy has grown to 800,000 sellers and 40+ million monthly visitors. All that activity generates enormous quantities of data, which Etsy uses to drive site improvements.
"Big data" is the buzzword of the day, and learning to manage it and extract value from it is top of mind for executives across industries. A contributing factor to the surge in big data is a shift in prevailing corporate data management philosophies.
For some time Microsoft didn't offer a solution for processing big data in cloud environments. SQL Server is good for storage, but its ability to analyze terabytes of data is limited. Hadoop, which was designed for this purpose, is written in Java and was not available to .Net developers. So, Microsoft launched the Hadoop on Windows Azure service to make it possible to distribute the load and speed up big data computations.
Elasticity — the ability to ramp up or down computing resources depending on need — is one of the key benefits of cloud computing. Not being shackled to their own, in-house hardware means organisations can dial up the amount of resources they need to crunch big data sets, run their Web presence during spikes and troughs in demand and process periodic jobs without needing the internal resources necessary to cope with only peak demand.
MySpace on Tuesday will release as open source a technology called Qizmt that it developed in-house to mine and crunch massive amounts of data and generate friend recommendations in its social-networking site.
We all know what buzz is: It's noise. At InfoWorld, one of its self-appointed tasks is to extract the signal from that noise, to separate the stuff valuable to IT professionals from that which is popularly considered a big deal.