If you're staring down an ever-growing mountain of data and still working on plans for containing it, you're not alone. But climbing to this particular summit requires a lot more than just technology: rather, taking full advantage of enterprise data requires nothing short of a transformation in the way the business operates.
Core to this transformation will be the movement of businesses from long-established IT-based models of operation to a digital business model based on scale-out architectures and commodity hardware, according to Gartner research vice president Roger Cox.
Cox, who presented a five-year storage scenario for enterprises at the company's recent Infrastructure, Operations & Data Center Summit, says the digital business model will not be achievable with the status quo due to data growth, budget restrictions, increasing SLA demands, and infrastructure complexity.
The emerging digital business model will be largely software-based and driven forward by nimble innovators, Cox argues, rather than the previous monolithic hardware-based data environments of large vendors. This transition, in turn, will force CIOs into a 'bimodal IT' operating model in which inward-focused operators focused on storage safety and reliability are increasingly mixed with externally-focused staff focused on storage agility, scalability and flexibility.
By 2017, Cox believes, three-quarters of global businesses will have evolved from existing 'mode 1' operations to implement a 'mode 2' bimodal IT organisation, but half “will struggle and go through multiple attempts before reaching a working state”.
“Even people that have the skill set find it's very much a learning process,” he explains, noting that many companies will need to invest heavily in retraining to both unlearn old habits and to learn the new ones necessary to provide appropriate storage skills.
“You have to have a completely different skill set in terms of the kind of employee necessary to pursue that kind of business. And the personnel acquired to develop and manage the digital business model are more expensive than those required to support and manage traditional infrastructure.”
Particularly in such an esoteric area as data analytics, finding the right staff can be tricky: Conventional database managers know how to put data on disk and find it again, but data scientists are driving the avant-garde by designing the analytical and statistical models necessary to turn that data into actionable information.
Broad corporate support for big data, machine learning and other data-related initiatives has driven strong interest and investment in skills – Gartner, for one, recently reported that 75 per cent of respondents to one survey are investing or planning to invest in big data by 2017 – and that organisations typically go into the project targeting benefits from an enhanced customer experience (cited by 64 per cent), process efficiency (47 per cent), better targeted marketing (47 per cent), and security (23 per cent).
A number of initiatives have recently emerged to support this growth by creating new channels to feed data-scientist demand. For example, Hadoop specialist MapR recently partnered with managed-services provider Servian to launch big-data training program including courses in areas like big-data administration and data science.
Melbourne data group Data Science Melbourne, which has over 3500 industry members and hosts data-focused events such as the recent MeDaScIn 2016, recently took its own approach to boosting data-science skills with an internship program that will pair employers and data scientists keen on parlaying their university qualifications into a relevant data-based job.
“We're at a point where we want to help consumers by becoming a bit of a one-stop shop,” iSelect head of analytics and data services Yuval Marom told the MeDaScIn audience in announcing the program.
“This is all data-driven and it has been a step change to scale up our R&D capability so that we're always innovative and thinking outside the box. We always end up using the latest technology and algorithms, and we believe the internship program will help us achieve that.”
Turning data into action
Yet developing and sourcing skills are only part of the data-driven transformation: Even with the right mix of in-house skills, it's crucial that data projects be driven for the right reason and in the right way.
This, says Teradata ANZ chief data scientist Ross Farrelly, is often easier said than done. “The question is how to design processes so that the end outcome is achievable,” he explains. “Surprisingly, this is something that is often not considered at the beginning of the project.”
Engagements with customers have highlighted a broad range of maturity in expectations from big data, Farrelly says, with some organisations turning to big-data techniques because the problem is so complex or an algorithm so computationally expensive that it can't be run on existing technology.
Others turn to new data techniques to unlock value from masses of structured data – for example, security logs or XML data – or when they start to ingest new types of data that can't be easily stored within existing systems. “The noise to signal ratio can be extremely high,” says Farrelly. “We work together to simply capture and store it for longer, and then start exploring it for commercially useful value.”
Many times, this process becomes a blank-canvas exercise that requires businesses to tap into the curiosity and creative urges of their data-science specialists. A bit of brainstorming will inevitably lead to fresh ideas about productising the information, selling it to third parties or customers, or using it internally to drive the creation of new products and services.
Whatever the company's motivation for building data-centric policies, one common mistake is to bring data specialists in well after the data is collected. “Data scientists and analysts are often brought in after the fact when the collection process has already taken place,” he says. “The assumption is often that the insight is there in the data, but that's not always the case.”
Although appropriate analysis can be applied to existing data sets, Farrelly explains, companies should ideally engage their data experts at the beginning of the process rather than at the end of it – and they need to be ready to have some honest conversations with business executives who haven't done so.
“That can be a very hard conversation to have with a sponsor who is engaging you – to tell them that their data hasn't been collected in the right way and is not actually capable of answering that question,” he says. “There is a lot of education that needs to be done; this really has to be designed from the ground up so that you work with the origin and the end of the project in mind right from the beginning.”
Industry group CompTIA recently predicted that 2016 would be a landmark year for data as companies that had stumbled during initial enthusiasm over big data, found their feet and actively got their data strategies in order.
Joining the rush will, however, involve a great deal of soul-searching and some cultural transformation on the part of both technical and line-of-business managers that are often used to very process-driven problem solving.
This can lead to operational paralysis, warns Dr Eugene Dubossarsky, a partner in data-analytics consultancy Presciient, if businesses don't have mechanisms for effectively translating data insights into business action.
“In my experience there is often plenty of pretty visualisation, lots of money spent and people employed, that ultimately doesn't lead to any kind of decision-making at all,” he explains.
“The biggest problems are organisational, conceptual, and cultural. They're about who's in charge, what are their incentives, and how do they envision analytics in their organisation? When it works, the relationship is wonderful – but in a lot of places complex problem solving is not framed the right way, and a lot of people find themselves in situations that are actually untenable.”
Even having enough self-awareness to realise that an organisation has reached such a position can be difficult. But with enthusiasm growing and businesses putting real investments behind their data-driven strategies, businesses are getting serious about their data and the sky is the limit – as long as businesses keep their heads in the right place and their objectives clear from the start.
“We have to be constantly thinking about the 'why' of data science,” advises SAS Australia chief analytics officer Evan Stubbs. “It is a technical discipline and whether we're trying to fix the one company, or create a better society, we have to adapt to the changes around us. But if we can't answer the 'why', all we're doing is wasting our time and spinning our wheels.”