With the long-anticipated Cobol skills shortage starting to bite, many businesses have been steadily migrating applications off the mainframe. Blue Cross Blue Shield of South Carolina has been doubling down.
The healthcare insurer processes nearly 10% of all healthcare claims in the U.S., and uses six top-of-the line IBM zEnterprise EC12 systems running millions of lines of optimized Cobol to process 19.4 billion online healthcare transactions annually. Its custom-built claims processing engine has been thoroughly modernized and kept up to date, says BCBS of SC vice president and chief technology officer Ravi Ravindra. "It was always in Cobol, and it always will be."
Cobol was designed to handle transactional workloads, and for large-scale transaction processing it still can't be beat, says Lonnie Emard, vice president of IT at BCBS of SC and president of the Consortium for Enterprise Systems Management (IT-oLogy), a nonprofit association of businesses, vendors and higher education institutions dedicated to helping build a talent pipeline for Cobol and other hard-to-fill IT-related disciplines.
"We are the low-cost provider," says Lonnie Emard, vice president of IT, Blue Cross Blue Shield of South Carolina. "We couldn't do that if we weren't on the mainframe."
"We are the low-cost provider," Emard says. "We couldn't do that if we weren't on the mainframe."
Some organizations continue to run the business using Cobol on the mainframe simply because migrating all of that code to distributed computing systems without a compelling business benefit wouldn't be worth the trouble and expense. But some of the world's largest businesses see their Cobol infrastructure not as fading legacy technology but as a state-of-the-art, competitive weapon.
For both groups, the challenge lies in finding the next generation of talent to run those systems or risk being forced off the platform. "There's been no abatement in the number of people retiring," says Adam Burden, lead for advanced technology and architecture at Accenture, and the pipeline of new Cobol programming talent has been insufficient to address the impending outflow of retiring baby boomers.
As businesses continue to struggle with the problem, Accenture has been busy adding Cobol programmers to its consulting practice to fill the skills gap for its clients, Burden says, even as IBM, IT-oLogy and others attempt to train and recruit the next generation of mainframe talent.
The mainframe's stronghold
For many businesses in healthcare and other industries with large-scale transaction processing needs, including airlines, retail, banking and government, Cobol on the mainframe remains the core transaction engine. Several mainframe vendors remain in the market, including Unisys, Groupe Bull and Fujitsu, but IBM has the lion's share of the market today, with more than 90%. And 50 years after IBM released its first mainframe, the System/360, in April 1964, the company still has 3,500 mainframe customers.
What's next for zEnterprise
IBM is working to address several shortcomings of the mainframe by focusing on three key areas, says Deon Newman, vice president of IBM System z.
Analytics. IBM wants to provide more analytic capabilities on the platform to reduce the need to continuously move transactional data off the mainframe for processing. "Our customers want to use it in real-time fraud detection, upselling and cross-selling. Some of this will be on zLinux, and some on zOS," Newman says.
Mobile enablement. Java subsystems will allow enterprise applications to be "extended out onto the Web," Newman says. And that will lower some costs of doing business on the mainframe. IBM customers pay based on the number of transactions processed, but many transactions, such as balance inquiries, don't generate revenue. Mobile has greatly expanded the problem, with more than half the workload coming in from the Internet and mobile applications. So IBM recently discounted what it charges for mobile transaction charges by 60%. "That means that software pricing won't explode in the same way that mobile transactions are," he says.
Cloud. IBM is pushing the mainframe as a consolidation platform for x86-based distributed computing in environments where server or virtual machine counts exceed 200. It sees the mainframe as a cloud appliance for hosting distributed computing workloads, and not just for folks who already have a mainframe and have excess capacity to soak up: In April IBM released the Enterprise Cloud System, a fully integrated Linux-based OpenStack-compliant system with a utility-based pricing model.
-- Robert L. Mitchell
Some 23 of the world's top 25 retailers, 92 of the top 100 banks, and the 10 largest insurers all entrust core operations to Cobol programs running on IBM mainframes, says Deon Newman, vice president, IBM System z. Since 2010, around 50 to 75 customers have left the mainframe fold, IBM says, while some 270 of IBM's 3,500 mainframe customers have come aboard as new clients since then, Newman says.
For these mainframe-centric businesses, the Cobol application suite that runs the heart of the business isn't going anywhere. "But they still need to deal with the declining Cobol workforce . . . to keep these systems viable for the next decade or two," says Dale Vecchio, research vice president at Gartner Inc.
As for the other 90% of businesses running mainframes today, Vecchio thinks the Cobol brain drain will be the catalyst for more extensive migrations off the platform, through rewrites, moves to packaged applications or recompiling and re-hosting Cobol on distributed computing platforms.
After years of foot dragging, the looming Cobol brain drain will force many organizations into making a decision -- one way or the other -- within the next three to five years. "Increasingly, I see this transition happening," Vecchio says. "Waiting isn't going to make this any cheaper, and it isn't going to reduce the risk."
You don't just walk away from one million lines of Cobol code. Jim Veglahn. program manager, United Life Insurance
United Life Insurance Co. falls into the other 90%. The midsize business migrated off its Unisys mainframe several years ago, but didn't throw out the baby with the bathwater. It kept more than 1,000 Cobol programs that run the business, recompiling those for Windows using Micro Focus Visual Cobol, says program manager Jim Veglahn. "You don't just walk away from one million lines of Cobol code," he says. And while it's harder to fill open positions than it was a few years ago, Veglahn says his strategy is to "stay on Cobol and train as needed."
But the demographic shift will, in the long term, make Cobol "almost unsalvageable," says Vecchio. "The only debate is the slope of the decline."
BCBS's Emard disagrees. "Can we keep up with the demand for Cobol talent? Absolutely. The supply needs to be increased with the knowledge that these jobs are not going away," he says.
Vecchio says the number of Gartner clients that want to talk about mainframe migrations is up sharply. "I had 200 mainframe migration inquiries last year, and I have been speaking with thousands of mainframe shops about this whole migration question," he says. Mainframes eventually will be marginalized to only the very largest organizations in the market, Vecchio adds. "They're the only ones who can invest in initiatives and create their own training programs," he says.
Streamlining mainframe applications
Even some of the biggest mainframe shops are streamlining mainframe operations for non-core functions. BNY Mellon runs nine IBM System z10s that supply a total of 54,000 MIPS of compute power and handle $1.5 trillion in transaction processing workloads each day. The Cobol codebase that powers those systems has grown from 343 million lines two years ago to 357 million lines today, adding 2,500 new programs along the way.
"I believe it gives us a competitive advantage, especially with the transaction volumes we're doing," says David Brown, managing director and chief application architect.
But the growth of BNY Mellon's Cobol codebase reflects enhancements to the bank's core transaction processing systems. Few organizations, BNY included, are building entirely new applications in Cobol anymore.
In other areas, BNY Mellon has also been steadily peeling away mainframe applications that aren't strategic to its banking operations. It has moved to packaged applications in some cases, recast some applications in need of a heavy rewrite onto distributed computing platforms and pushed ad-hoc reporting capabilities off the mainframe as well.
I believe [Cobol] gives us a competitive advantage, especially with the transaction volumes we're doing. David Brown, managing director and chief application architect, BNY Mellon
"If you're doing sequential processing and you have a mainframe footprint, that's where that functionality belongs," says Brown. "But on things you're doing ad-hoc -- mobile, big data -- all of those run better on distributed platforms."
When applications that used to be hosted on the mainframe are re-hosted or rewritten to run on distributed computing platforms, however, that makes things more complex when it comes to accessing data that still resides back on the mainframe, Brown says.
To avoid that, he has been looking at hosting rewritten applications on a zLinux partition on the mainframe. "I can rewrite a piece of Cobol but closely interface with the mainframe program and database without jumping between a mainframe box and a distributed box," he says.
As for BNY Mellon's core systems, Brown says, "I see Cobol on the mainframe continuing on in perpetuity. Over half of our major applications are still on the mainframe. It offers the best scalability, reliability and availability for doing sequential processing."
Rebuilding the talent pipeline
The idea that Cobol and the mainframe are legacy technology is a perception issue, BCBS of SC's Ravindra argues. It's a mindset that has pushed students away from taking Cobol and other mainframe technology courses, and convinced schools that they shouldn't offer mainframe-related technologies in computer science curricula.
Despite efforts such as the 8-year-old IBM Academic Initiative, the talent pipeline hasn't kept up as the baby boomers who dominate today's mainframe shops begin to retire in larger numbers. "The market, universities and junior colleges aren't generating enough developers to replace those who are retiring," Gartner's Vecchio says -- a sentiment shared by Emard.
"There was no pipeline for Cobol talent, and we realized we couldn't do it by ourselves. It's costly to go it alone," says Emard. So BCBS of SC helped form IT-oLogy. Emard says the goal is to make sure that lack of talent doesn't force companies to move off the mainframe.
The initiative promotes awareness of job opportunities in the discipline, college-level training, internships and cross-training of existing IT professionals. IT-oLogy has doubled the number of schools offering its Mainframe Academic Initiative, and the programs have reduced BCBS of SC's onboarding costs by 50%, Emard says.
How IT-oLogy aims to close the skills gap
While programs like the IBM Academic Initiative promote curricula to higher education institutions for Cobol and other mainframe disciplines, universities weren't always receptive to the idea coming from a vendor, says Lonnie Emard, president of the Consortium for Enterprise Systems Management (IT-oLogy). IT-oLogy conveys the message that its members, including Blue Cross Blue Shield of South Carolina, Time Warner Cable, Fidelity Investments and other firms, are ready and willing to hire graduates with training in Cobol and other hard-to-fill IT disciplines. "We're all about skills in short supply," Emard says.
The program has three pillars:
- Promote IT offers outreach to students in grades K-12 to advocate for IT careers and talk about what IT skills will be in demand in the future.
- Teach IT is a collaboration between industry and academia to ensure that curricula include subjects that meet the needs of businesses hiring in the IT profession, as well as to provide internship opportunities. Emard sees the program as a "minor league farm system" to develop future IT professionals.
- Grow IT focuses on professional development and cross-training for existing IT professionals.
-- Robert L. Mitchell
"Today kids have no bias against the mainframe. They're just glad to hear that they can make $60,000 or $70,000 a year. If someone is paying that kind of money it's a nonissue," he says.
Better development tools are also making training new talent easier.
Newer versions of Cobol that work within the Eclipse and .Net integrated development environments, such as Micro Focus Visual Cobol, have made it easier to pull over existing developers from the object-oriented side of the house, although the transition from object-oriented programming to the procedural Cobol language is still jarring, says BNY Mellon's Brown. "You can use the same tools with a common structure, but the paradigm is different," he says.
With modern development tools there's much less actual coding involved. "At BCBS of SC we're not asking people to write lines of pure Cobol code anymore," Emard says. Instead, the business uses an application generator from Micro Focus. "The key is, how do you optimize that with all of the other stuff that comes into play, such as the website and mobile apps. But we still need folks who understand the development environment that has Cobol at its core," he says.
The offshoring option
Offshoring can also help with the Cobol talent shortage. "Accenture has seen a material increase in the number of resources we have that are Cobol knowledgeable," in response to an increase in demand, says Accenture's Burden. "Our clients are looking to third parties to maintain these Cobol applications as it becomes more difficult due to the skills shortage."
But not all businesses want to entrust to outsiders the institutional knowledge of all of the business rules encoded in that Cobol code. What's more, says Vecchio, "A lot of the Indian service providers don't want to do Cobol and be the garbage dump for legacy apps. They want to move upstream and be strategically involved."
U.K. retailer Tesco hired hundreds of local Cobol programmers in India. "Our Cobol development community there is fairly young," says Tom Kadlec, director of information services.
U.K.-based retailer Tesco runs millions of lines of Cobol application code on a 70,000-MIPS zEnterprise EC12 mainframe infrastructure to power its store replenishment, credit card authorization and settlement, and payroll systems. Tesco avoided the institutional-knowledge issue by opening an office in India and directly hiring hundreds of local Cobol programmers as employees. And in India, Tesco doesn't have to worry about the impending retirement of baby-boomer employees. "Our Cobol development community there is fairly young," says Tom Kadlec, director of information services.
Like Ravindra, Kadlec has no intention of migrating the core systems that run the business over to a distributed computing platform. "The combination of our Cobol applications and the mainframe platform is what gives us a competitive advantage," he says.
Preparing for the inevitable
At BNY Mellon, the talent crunch is still about five years away, Brown estimates. Many of the bank's 300 to 400 Cobol programmers were hired right out of college in the 1980s and range in age from 45 to 55, Brown says. "We haven't had the big exodus yet," but as the economy picks up he says he worries that the impending wave of retirements could come sooner rather than later.
The company is working with universities and offshore resources to bring in "younger blood" and get the talent pipeline flowing. But so far, he says, it's been a slow process. Brown has been able to fill vacancies as they crop up, but in the future, retention bonuses and pay premiums may be required to acquire those scarce Cobol skills, he says.
While the types of companies using mainframes and the types of workloads on the mainframe may be evolving, Cobol and related mainframe skill sets will continue to provide jobs for the foreseeable future, says Burden. "On a macro basis I haven't seen a material decline in the amount of Cobol code that's out there." And for every program that does a migration there are still a dozen running -- and those continue to grow and evolve, he says. " I don't see that changing for a decade -- or more," he says.
This article, Meet Cobol's hard core, was originally published at Computerworld.com.
Robert L. Mitchell is a national correspondent for Computerworld. Follow him on Twitter at twitter.com/rmitch, or email him at rmitchell@computerworld.com.
Read more about applications in Computerworld's Applications Topic Center.