A prescription for lower costs

McKesson is a multifaceted healthcare company, a large distributor of pharmaceuticals and a thriving developer of healthcare-related IT systems. Its software and hardware are installed in more than 70 percent of US hospitals with more than 200 beds, and handle everything from billing and scheduling to capturing MRI-machine images and preventing dangerous drug interactions. For the last five years, the company has used open source technology to deliver products at lower cost and greater speed, says Randall Spratt, executive vice president and CIO. After seeing open source, Spratt considers it an essential part of McKesson's product development strategy.

What role is open source playing in your strategy?

In our technology division, our flagship line of software products is called the Horizon suite. The reference architecture for that suite is dependent upon open source components and tools to create and develop them. We don't talk about product names, but we employ open source operating systems, an open source object-model interface, a number of different open source user-interface widgets and libraries, open source middleware and Web servers, and a variety of open source tools that not only provide low-level program libraries but also support the programming process in general.

What are the key benefits of open source?

The benefits for us came from the requirements of the markets we serve. Healthcare is an extremely low-margin business with constant cost pressures. Frankly, our customers were not able to consume the solutions they needed at the pace they needed because of cost constraints. So, we went to open source primarily as a strategy to reduce the extent of third-party costs -- primarily hardware and operating system costs -- that were in the solutions we sold to customers. We saw those benefits emerge dramatically -- an order-of-magnitude reduction in the expense around hardware, for example -- but we also got unexpected benefits in speeding some aspects of development and higher levels of performance.

What were the development benefits?

We got access to libraries of capabilities that we would have had to develop on our own -- the ability to take in everything from user-interface widgets to libraries of software routines and schedulers, for example.

And how does open source reduce hardware expenses?

In two ways. The operating systems make more efficient use of lower-cost hardware than many commercial operating systems, and we architected an environment where the application runs on any number of blades that sit on top of one or more database servers and the load is then automatically distributed. Hospitals can start out with a relatively modest investment and as they add users or applications, scale by adding low-cost blades rather than forklifting out an expensive Unix server and replacing it with a larger server. So, not only do we get the efficiency benefits in the first place, we get a much more scalable environment, where each step in the scale is a modest step upward.

Page Break

We see open source as typically more of a suboption under "build." We’re going to buy a capability or we're going to build it; and when we build it we typically look at open source as a partial solution to the build. "Build" means it's our product, our intellectual property, we take it out to the market and we sell it. "Buy" means we buy it from somebody else and resell it. We would see open source as kind of a blend of those strategies, but very rarely would we take a pure "buy" option.

When you do use open source, is it pure open source or a supported version?

It's a mix. For the tools we deliver to our customer base -- operating systems, for example -- it would be a fully supported operating system. For the open source libraries and such, it would run the gamut all the way out to true open source.

Who supports the machines you put in hospitals?

Ultimately, we do. The first line of support typically is the hospital IT department, but they contract with us for Tier 2 and Tier 3 support, and tiers 1, 2 and 3 support of the application.

What's your experience been with supporting open source operating systems?

Like everyone else, it's been a journey. Initially we ran into issues and a number of problems with scalability, but I think today we would say it's a very good experience.

How long have you had open source-based products installed in customer locations?

Three to three and a half years.

Can you talk a bit more about initial problems?

We architected a load-balanced solution, and we had some difficulty with lost connections and issues with performance of some components. Operating-system support was generally pretty good; but in a healthcare environment, if you run into a problem, it can literally be a matter of life and death. If we had a downed system and couldn't figure it out ourselves pretty rapidly, we needed Tier 3 support right away. I think we've worked those relationships out. It's just a matter of the companies behind the products maturing in their business models, more than the technology itself.

Page Break

I don't think it was real important. It's turned out to be beneficial, especially in the widget libraries and low-level code libraries to have that kind of access.

What about in helping you troubleshoot issues, such as the ones you mentioned?

Probably not so much. We've got some talented developers of our own. I'm sure there was help offered from the community, and I'm sure we took advantage of that; but I can't say it's a major part of our strategy or a major part of the benefit to us. You have to recognize we're application developers, so we're not cutting-edge with respect to the technologies we deploy. We're consumers of technologies others have built and successfully defended.

Are there any projects you've undertaken using open source technologies that couldn't have been implemented any other way -- from either a technical or practical perspective?

I don't think there's anything we couldn't have implemented from a technical perspective. But practically -- meaning within the same time or cost of delivery windows -- certainly. Open source has definitely improved our time to market in several key areas, and it's improved the cost of delivery in several key areas. So -- practically, yes; technically, no.

Do you have any metrics around what those improvements have been?

I've got three case studies we did of hospitals that either adopted some of our open source-based applications or replaced existing applications with the open source versions. There is a fairly good-sized hospital in New Hampshire that reduced its hardware and related licensing capital costs by two-thirds. Overall cost of ownership was reduced 60 percent in five years for apples-to-apples applications that it replaced with ours. The hospital also cited much easier expansion and upgrade capabilities.

A large hospital of almost 700 beds in Houston went with our solution on bladed open source servers running over Oracle RAC [Real Application Clusters] to deploy a physician portal. It reduced its hardware costs by one-third in a 300-concurrent-user environment -- where users are physicians -- with response times well under one second. A small hospital in Colorado got a 25 percent reduction in cost, improved response time with reports running twice as fast, and reported an increase in scalability.

Page Break

One of the challenges was the business model of some of the companies that we depended on. We're in a tightly controlled and carefully licensed world. Not all suppliers of open source software appreciate the license strictures we have, and in some cases don't appreciate our profit motive. Some were more interested in producing interesting code than building a survivable business. So, it took a little while for us to work through the licensing and legal issues in how we would use, package and protect ourselves from potential downstream claims on our intellectual property. And it took a while to reach good working relationships with our key suppliers.

Did you have to change any suppliers because you couldn't work through those kinds of issues?

Early on we did. But I think the industry in general faced that set of issues more or less simultaneously. Everybody was working through them at about the same time.

Have you encountered any dangers in using open source?

Like any software, you have to subject it to diligent testing and add your own quality measures on top. I can't say that all the open source software we've attempted to use has been high quality. But I don't think it's presented a danger, more of a challenge.

To what extent are you using open source internally in your IT organization?

Internally we have probably 15 percent or so of our server environment under Linux. And we're constantly standing up new Linux servers for R&D. But in terms of production environments the company depends on, it's modestly penetrated.

Is it growing?

Yes, it's definitely growing. We were going down a Linux strategy pretty heavily three years ago when we ran into a pretty good speed bump in trying to scale one of our critical applications, and had to retrench and go back to a proprietary operating system to get the performance we needed. That made the businesses very cautious about pursuing it aggressively again. All of our experience tells us that the maturation has been significant since then, but we're slow-rolling it. We tend to put newer, lower-end applications up on Linux and leave the existing apps that are already running on other systems on those systems rather than undertake the risk of change.

Page Break

My first piece of advice is it's a very good strategy. But it does not relieve you of the responsibility to manage your development shop effectively. You can't expect to defer portions of your development lifecycle -- your [quality assurance], your test plans, your quality measures and such -- to someone else. Using the more mature platforms, such as the operating systems that are fully supported, it's been an extremely cost-beneficial strategy for us. My advice would be to not shy away from tackling it.

What are your plans for open source going forward?

We continue to investigate other open source offerings. We’re testing out some open source database capabilities that have the potential to replace proprietary database offerings. We continue to extend open source platforms in our infrastructure. As open source applications mature, we’re keeping an eye on and evaluating the replacement of some proprietary applications, like [Microsoft] Office for example.

How is that evaluation going?

We don't think [open source office-productivity products are] quite ready for an organization of our size, but I think they're getting close. The replacements for Word and Excel and PowerPoint are probably closer than replacements for Exchange. I think the e-mail and calendaring have a ways to go, but the document-based solutions, for at least a segment of our user base, are getting close. I don't want that to be portrayed as we're ready to switch, but we're certainly seeing a closure in the gaps.

What do you think the future holds for the role of open source in the industry in general?

I think it plays a strong role in the future. There are some uncertainties about the revenue model. In the end, everything depends on survivable solutions, so if you look at some of these little, tiny companies that are trying to pin a financial future to small code-libraries and such, I think they're going to find it difficult to stay alive. But in general, I think it will mature and the open source industry probably will undergo some reorganization itself through market forces not only around consolidation but probably more converged access and more converged common connections or interfaces between the software.

Expand on that -- converged access and common connections.

If you look at the open source world, it's a collection of everything from useful little applications to little parts and pieces used to build applications and operating systems to run those applications on. But when you get out to the end of the application world, you have fairly large-scale applications that are supporting significant business processes; and those processes need to connect with related business processes. For example, if I'm developing a contract for a customer, that contract has to connect to our quoting system, our contract-management system, our invoicing system and the like. And the open source industry isn't yet to a point where it's offering these large-scale applications. So, for open source to mature beyond what is essentially a tools environment, it’s going to need to penetrate the space that's presently occupied by proprietary application vendors. And that means converging some of these tools into larger-scale applications and having those applications share everything from common data definitions to common information-interchange protocols. Think of an open source SAP or an open source Oracle Financials. How do we get there? Because that’s really where the business value is. Bottom line is I think open source has a strong and bright future, but it's really hard to predict the direction it's going to go because it's so facile.