DevOps: Keeping your cloud transformation under control

Just when you thought it was safe to go back and visit your developers, along comes a new buzzword to empower your software processes and raise eyebrows at boardroom meetings. That buzzword is DevOps – a portmanteau of 'development' and 'operations' – and it is already revolutionising the way many IT organisations work. And if you haven't run across it yet, you soon will – if, that is, anybody can agree on exactly what it means.

Broadly, DevOps has emerged in recent years as a way of adding procedural rigour to the process of developing, testing, and deploying new software.

That process has changed dramatically in recent years as on-premises server virtualization – and, later, cloud-based platform-as-a-service (PaaS) providers such Amazon Web Services (AWS) and Microsoft Azure – empowered developers to set up new application-testing environments quickly and easily.

It wasn't too long ago that testing a new application required developers to write a business case for the purchase of a new server for thousands of dollars, wait for days or weeks until it arrived, then go through the headaches of setting it up and loading their applications.

It was headache-inducing and contributed to many development projects running slow and over time.

Virtual server and cloud environments fix this with software-based virtual machines (VMs) that can be started within minutes from a template, copied en masse and decommissioned at will.

Developers were early adopters of this technology – which has since become the basis of virtually all enterprise applications – exactly because it made their everyday activities considerably easier.

Availability of VM platforms has facilitated the broad adoption of 'Agile' project-management methodologies, in which developers work together in functional teams and build software iteratively to increase flexibility and responsiveness, and hasten ROI.

Yet for all the benefits it provides for developers, Agile requires operational discipline as well: Each of those VMs needs to be tracked, managed, and eventually shut down when it's no longer needed.

When the business is paying by the hour for each VM running on a PaaS platform, costs escalate quickly; left unchecked, Agile development environment is to VMs what ants are to your weekend picnic — turn your back for a moment, and the place will be crawling with them.

DevOps, in this scenario, is the can of Mortein that most people find out too late that they forgot to bring along.

Operational collaboration – eventually .

This, then, is one of the core conceits of DevOps: Adding control to the process by which newly empowered developers work their craft.

“For me DevOps is two sides of the same coin,” explains Hannah Browne, Victoria state manager with DevOps specialist firm Cevo.

“One is the technology piece which is easy for people to access and understand – it's about automated infrastructure and environments for provisioning – but the secondary piece is very much around people, governance, and process.”

It may seem far removed from software developers focused on writing bug-free and effective code as quickly as possible, but that secondary piece is critical because it reflects the need for DevOps to involve continuous collaboration with business organisations so that business concerns – things like customer service, efficiency and profitability – can be addressed as part of the development process.

Doug Averill, global public sector business line leader with application provider Pegasystems, sees DevOps as a key tool in the transition towards customer-focused development and service rollouts. By incorporating customer feedback and Net Provider Scores (NPS) to measure response to each new application or version, organisations can build tight feedback loops that guide Agile development and are technologically empowered through DevOps.

“DevOps is the endgame in this,” Averill explains. “Why wait for something to be negatively reviewed in the press, or by your front office workers? By taking a stepwise approach you're much more responsive to any of the operational changes that might be coming. Agencies want to understand best practices and want to be able to share; our role in this is providing world class technology and defining the art of the possible.”

Far too often, however, Cevo's Browne finds client organisations have simply failed to bridge this distance – a task that she calls her “pet” project when working on DevOps projects.

“I hate the idea that we've grown up in these organisations where we are so removed from what the end customer actually engages with our company for,” Browne says.

“I don't care whether they are in sales, marketing, operations, IT or HR; they exist to service a need for a customer. Traditional IT shops have happily siloed themselves away from the customer in the past, and I love that DevOps is changing that. By driving the realisation that we are not siloed business units – that we are all part of a value chain built around delivering something excellent for the customer – it's focusing people on what's valuable and what's important.”

Software development-tools giant CA Technologies, aided by market research firm Freeform Dynamics, has been tracking the emergence of DevOps in recent years and this year's report suggested DevOps done properly can deliver significant benefits.

‘Advanced’ DevOps users, the study found, are 2.5 times more likely to have seen improvements in customer retention; twice as likely to have seen improvements in customer acquisition; 3.4 times more likely to have seen progress on market share; and twice as likely to have seen a positive impact on revenue growth.

Yet just what constitutes 'advanced' DevOps remains up for debate. AppDynamics developer evangelist Dustin Whittle, for one, outlines just six core components that include infrastructure automation, configuration management, deployment automation, log management, performance management, and monitoring.

Yet these are also to be implemented with a contextual eye, Whittle warns: “DevOps isn’t just a set of tools,” he writes, “but a philosophical shift that needs that requires buy-in from all folks involved to truly succeed. It’s only through a high-level of collaboration that things will change for the better.”

Page Break

CA Technologies has taken a broader definition, outlining nine key areas that must be addressed for DevOps to work (these will be covered in a later instalment of this series). And while more than 70 percent of the respondents to CA's 2016 survey said they had implemented some degree of DevOps, just 20 percent of adopters were considered 'advanced DevOps adopters' who had implemented most of these nine key areas; in Australia, this figure was just 10 percent.

The 70 percent figure was “quite a revelation, to be honest,” says Ashok Vasan, APAC vice president of application delivery with CA Technologies, who has watched DevOps becoming common practice as organisations of all stripes become more actively engaged with the process of digital transformation.

“Bringing the development team and the operations team together with a common set of goals is becoming the way that people are adopting to these changes,” he explains.

“More than anything else, the big thing that people are beginning to understand is that it's not just about IT; even though the title suggests just development and operations, it's also about involving the business into this practice. That's the latent acknowledgement.”

The Agile connection .

That collaboration was absent from long-problematic 'Waterfall' development in which projects were conceived on whiteboards, deliverables specified, then thrown into the proverbial dungeon where developers worked feverishly for months or years to build something that worked as described.

The result invariably resembled what the business wanted months or years earlier – but in a world of constant change, inevitable scope creep made business delays and budget blowouts all too common.

Agile development – which breaks big projects into many small steps with regular opportunities to review progress and change direction as needed – has helped undo the damage of waterfall development.

DevOps goes even further by linking Agile's free-form development processes to the underlying infrastructure with tight management control that ensures developers have just the resources they need – no more, no less – throughout the project-delivery process.

“You really can't separate Agile and DevOps,” says CA's Vasan. “It's not just about the development teams anymore; it's about testing, operations and even product marketing.

“Some of the business functions are very critical to defining what business services and changes we want to take to market – and if you're going to make those changes in the way your organisation operates, you've got to arm teams with knowledge, processes that are across all functions. Clearly, the right tools are necessary as well.”

DevOps' focus on managing delivery of the project in operational terms means that it necessarily reaches farther and deeper across the organisation. This can be a recipe for disaster if the entire organisation isn't aligned around the transformation at stake – but here, says Cevo's Browne, past experience with Agile is a big plus.

“Those that come from a mature Agile environment certainly adapt and adopt the DevOps practice and principles far more readily,” she explains.

“A lot of people would say DevOps is really an extension of Agile, or an iteration of it. It lends itself to innovation, and prototyping, and shortening the customer feedback loop.”

Empowered by the rapid move to cloud environments where DevOps is most at home – and, as we'll cover later in this series, where cloud operators are building DevOps capabilities into their service offerings – businesses are increasingly rushing to embrace this new way of thinking.

Whereas CA Technologies' late-2014 survey found that half of Australian businesses had implemented DevOps already, in 2015 58 percent of surveyed Australian senior IT executives said they would invest in DevOps. The market for DevOps tools was expected to rise commensurately, according to Gartner, which expected it would grow from $1.9 billion in 2014 to be worth $2.3b last year – a healthy 21.2 per cent annual growth rate that was on track to push the practice into the mainstream by this year.

And why wouldn't it? With cloud also well and truly mainstream, DevOps has emerged as businesses' best chance to keep control over the cloud explosion while integrating business and development processes into one coherent procedural whole.

“It's a totally different mindset for running our organisations,” says Browne, “and it's gaining such prevalence because so much of our world is digital and technology based. It's changing how organisations work and changing how people view their roles in the world – and that is pretty exciting.”