Size (and mobility) matters. As desktop PCs lose ground to tablets and smartphones, and the cloud becomes a more mainstream means for software deployment, desktop applications are being elbowed aside by mobile apps and Web services, resulting in a significant shift in the way software is created.
Software development organisations large and small are moving quickly to adopt new tools and new paradigms, adapting existing tool sets, talent pools, and processes to make the most of new computing environments and emerging opportunities.
Gone are the days of bits being passed from one isolated team to another in service of the one true build. Modern app development requires a nimble, cross-functional approach to rapid deployment.
Here's how several leading-edge development shops are meeting the challenges of this new frontier.
1. Mobile- and service-first development: Tomorrow's wave -- or at least today's
From the outside, the one development trend fast becoming evident to everyone -- end-user, customer, and developer alike -- is the emphasis on mobile-first development, along with all the complexity that entails.
"Apps are huge now," says Matt Powers, CTO of Applico, a developer of mobile and Web apps for a variety of big clients such as Google, Asics, AT&T, National Oceanic and Atmospheric Administration, and the Mayo Clinic. "They used to run locally off the device, so the infrastructure to support them was small. Now you have people bringing their entire brand, and everything they do on the website, if they have one, to mobile."
If that infrastructure is deployed as SaaS (software as a service), it requires development practices that are orders of magnitude more rigorous and complex than those used for deploying stand-alone apps.
Intuit, the creators of QuickBooks and GoPayment, learned this lesson when the company broadened its services to meet an international market of 1.3 million users -- covering 150 countries, 143 currencies, and 46 languages.
"We needed to scale the development from small teams of 10 to 15 engineers to a team of 70-plus engineers," explained Anshu Verma, architect at Intuit for QuickBooks Online. Engineers needed to develop faster overall, and thus adopted what he calls a "safety-first design pattern," which draws inspiration from how circuit breakers work. This design allows old and new workflows to co-exist -- an important feature for a Web service. "In case of exigencies, we fall back to the old [workflow] on-the-fly and cut over to the new [workflow] when we are confident about it. It really helped us move faster without impacting customers."
Intuit's own development cycle for new features is now a mere two to four weeks, which requires them to use Scrum processes and test automation. "We use tools like Jira, Greenhopper, and Rally to facilitate the iterations across all key stakeholders -- product management, development, QA, deployment -- and created testing frameworks using modern tools like WebDriver, PhantomJS, JSUnit, DOH [Dojo Objective Harness], JUnit, and so on."
On the other hand, changing practices for the sake of going mobile doesn't make much sense unless it's backed with sound business strategy. Such is Joel Semeniuk's belief; he has been a project manager for more than 20 years and is now executive vice president of agile project management at Telerik, which creates a broad palette of software testing tools and UI components for .Net.
"Since mobile is all the rage," Semeniuk says, "it's too easy to get caught up in the mobile-first strategy and not stop to consider the real customer value provided by the mobile application." Such value includes creating the right tool for the needed job: "Some applications aren't well-suited with mobile scenarios -- for example, those that require large amounts of data entry."
2. Find the right mix of development methodologies to meet project needs
The wars over development methodologies -- agile, XP (extreme programming), waterfall, and so on -- are fast giving way to a more fluid and flexible approach to producing and refining a product. Telerik's Semeniuk is one of many in the modern development world who sees development methodologies not as dogmas to be followed to the letter, but toolkits to be raided for what's useful. Confining a development team to one methodology is becoming a thing of the past.
"We have an iteration pattern for each problem," says Semeniuk, "in which we continually adjust or 'pull in' new agile practices that solve those problems. Sometimes we 'pull in' all of Scrum, because it solves the wide range of problems that come up in that particular environment. Sometimes we pull in pieces of Scrum or XP or Combine because it makes better sense if you're in maintenance mode." Semeniuk calls this the "agile buffet table" model.
For Telerik, though, the most important motive behind using any particular development practice is why. "We like to start with the 'why' and use agile to solve a real problem we're having. The biggest reason for that is when we try to just push out practices, they don't stick; people don't identify with the reasons these practices make sense. And not all practices fit all projects," he says.
Applico's Powers says his company also uses a variety of development models -- mainly agile and iterative. In his case, the "why" is driven by client needs.
"Some clients like rigid development timelines and documentation," says Powers, "especially ones that want to bring it in house. Others like the fluidity of the agile process and the ability to be brought in the loop at all times."
Some, however, caution that agile can't simply be sprayed onto an existing development process. A former program manager who has declined to be named but has five years of experience as a Scrum master has time and again seen agile used in development, but with no corresponding changes in other facets of bringing software to market.
"There's no intermittent QA; instead, there's old-school 'toss it over the wall to QA'-style QA," he says. "Instead of regular releases, they're using agile to get a release out, then having the schedule disrupted by support." In his purview, there has been a battle between traditional software releases and agile, with a lot of people simply using agile merely to drive old-school models.
3. Go with shorter lifecycles, cross-functional teams
The "mobile first" philosophy of modern development has also changed application lifecycle management in striking ways.
"Referring to a 'shorter development cycle' is misleading for Web development," says Andrew Frankel, former VP of engineering at TopShelfClothes.com. "It's no longer necessary to actually complete a full develop-QA-release cycle for every change. Small changes, such as changing text, can skip the usual process, since they can be deployed without any user impact. That frees the QA team to focus on testing actual application changes." Mobile and desktop app developers, he adds, aren't as lucky, since every change requires a new version.
For Telerik's Semeniuk, the biggest changes to application management are in Web and mobile. For those areas, he says, "You absolutely need short release cycles, because it's very difficult to pinpoint true customer value and interaction without actually measuring it."
This means getting items into customers' hands fast via a solid automation and deployment mode. "This has triggered a new flavor of app management called devops, where the dev team and the ops team need to work closely together to make sure that, as feedback is required, they can get that software into the hands of users without a lot of pain," he says.
Semeniuk also feels that, for larger organizations, overall team composition isn't shifting as quickly as it could to react to these changes: "Teams [in smaller organizations] have been shifting from functional roles -- business analyst teams, testing teams, deployment teams, etc. -- to cross-functional teams, where all the skills to envision, build, and deploy an application are on a single team. Teams then work together as a whole to deliver that software instead of handing it off between functional teams."
Some enterprises have a hard time making this shift to cross-functional roles, but Semeniuk believes this will change when "organizations can realize that an HR structure does not need to dictate a team structure."
4. Inventive use of the standard development toolkit
Modern development teams are extending the mantel of ingenuity right down to the tools they use, employing popular development tools in new ways to spur further innovation in the development process. Consider Git, the open source revision control system -- which can be used for much more than its primary purpose. For Andrew Frankel, former VP of engineering at TopShelfClothes.com, Git was also a way to perform process automation.
"Driving deployments with Git is fantastic for release management," Frankel says. "We have a complete log of what changed, at what time, and for what reasons. Larger organizations often try to collect exactly those data points using formal change requests, which tend to be frictional in a fast-paced environment. It's much more efficient to create a process where that information is collected automatically."
At Telerik, Git was adopted by one team as an escape hatch from the usual in-house development methodology.
"We have one division that has chosen not to use our development infrastructure, which is primarily Microsoft Team Foundation-based," says Telrik's Semeniuk. "They decided to do something that better fit their culture, experience, and needs, and started off with Git. A whole different form of release management with different tools, but it fit their culture and the experience of their team members a whole lot better." The team in question might not have to choose between Microsoft's workflow and git before long, though; Microsoft recently added Git support to Visual Studio and Team Foundation Server.
Git has also been put to use to support other parts of development such as documentation. The Gitit wiki system uses Git (or another version control system) to track and preserve changes to a community-created set of documents.
It should also come as no surprise that the cloud figures into most everyone's work as a cutting-edge software development tool. But it's not just a place to host code or a site -- it's also being eyed as a testing framework. Applico, in particular, is developing a cloud-based foundation for automated testing of its apps.
"With Android especially, you have an international market with the product running on over 500 devices," Applico's Powers explains. "If you can integrate this into a system where you can simulate all these different device types, you're going to catch a lot of issues before you go to market." To that end, Applico has been looking at a few vendors to provide tools to take the company's application builds, host them in the cloud, and perform the emulation there.
This approach seems like an attempt to refute what Sebastian Holst has claimed in "The Rise of Application Analytics: A New Game Demands New Rules." There, Holst states, "You cannot simulate production," meaning "the diversity and distribution of on-premises and cloud-based services combined with the dizzying array of devices and client-runtimes makes comprehensive testing and profiling prior to production not just difficult, but impossible."
To Holst, the solution lies in application analytics: real-time harvesting of data from application behaviors as per Telerik's work. Applico's idea is to expand the way we perform and automate testing -- not to displace analytics as a test methodology, but to use the cloud as a way to reduce the burden of testing.
Two of the most widely used tools for automation, Puppet and Chef, are also being used in creative ways.
"Using Chef and cloud servers for manual testing is fantastic," says Frankel, "since those servers will only be used occasionally for a few hours. When we're done testing, we turn off the lights and avoid paying for idle capacity. It only takes a single command to re-create a new staging environment the next time we want to test." The same process is also possible with Puppet.
5. HTML5 -- a handy, albeit hyped, solution for increasing device fragmentation
Given the current focus on mobile-first development, a great deal of attention is being paid to HTML5 and what role it will play. On the one hand, developers are quickly jumping into HTML5, because not doing so would be self-defeating. On the other hand, HTML5 is clearly no cure-all.
Applico's Powers takes a dim view of HTML5 as a mobile platform.
"HTML5 will never catch up to native development," he insists. "If you think of running everything in a Web view, you're just abstracting a layer between yourself and the native code. It's always going to be a step behind, and as new versions of the OSes come out, tools like PhoneGap and Titanium have to react to those changes."
In his opinion, HTML5 is best used for enterprise apps, such as a data-submission form, not immersive-experience apps.
Powers described experiences in his work that shed further light on this. Applico competitors lured clients away from Applico, offering to build apps with HTML5 at half the cost Applico quoted. "Eight months later, those clients would come back to us and say, 'We made the wrong decision; we went with someone that promised us the world and didn't really understand the limitations of the technologies.'"
Last year, Hung LeHong and Jackie Fenn, both of Gartner, placed HTML5 at the "peak of inflated expectations" on Gartner's annual Hype Cycle Report, estimating it would be five to 10 years before the real plateau for the standard could be reached. Yet many developers are embracing HTML5 and find Gartner's analysis to be way off-base.
Kendo UI, a division of Telerik, performed its own studies and found that 82 percent of developers "find HTML5 important to their job within the next 12 months," with 31 percent planning to use it and 63 percent actively developing in it.
That said, the phrasing of these questions doesn't speak to developer preferences, only to what developers are doing -- that is, building HTML5 apps because it's part of their job description. What's more, another survey sponsored by Appcelerator and IDC for 2012 found that most of the mobile developers surveyed were "neutral to dissatisfied with HTML5" in several categories, including performance (72.4 percent of those surveyed), fragmentation (75.4 percent), and user experience (62 percent). This is striking in light of how an earlier survey by the same group asked developers, "Do you plan to integrate HTML5 as a component into the mobile apps you plan to build in 2012?" -- to which 79 percent answered yes.
Todd Anglin, vice president for HTML5 Web and mobile Tools at Telerik, questioned this conclusion, and not just because of the rapid development of HTML5 on all sides: "Developers should note that the new 'native' Facebook apps still include HTML5 in sections where Facebook wants the ability to change things more quickly," Anglin wrote, referencing the much discussed shift Facebook undertook in 2012 to native mobile apps due to shortcomings it experienced with HTML5.
In short, for now HTML5 may be best thought of as merely one ingredient in an application's overall composition, rather than the way to create an app.
Conclusions
With so much software produced now aimed at a mobile or service-oriented market, development techniques are evolving to suit. Desktop programs that went for years between major revisions are being supplanted by mobile apps that are point-revved every few months or by services that are revved continually behind the scenes.
The demands those changes make are major, but they've also spurred numerous creative new solutions, including new use cases for traditional tools and the cloud as a development and testing platform, rather than just a software delivery mechanism.
The increasing speed of development (and developer feedback) means new technologies -- witness HTML5 -- are getting field-tested and absorbed into the mix more quickly, hastening the pace of relevancy.
As always, though, application development isn't about a particular paradigm, tool, or methodology -- it's about what works, here and now.