The riddle in the middle

Are you confused about middleware? Welcome to the very large club. It's a term that expands and contracts in meaning over time and is used differently by a zillion different people with as many different agendas.

Unfortunately, discussions of middleware, even allegedly rudimentary ones, often start simply but spiral quickly into asynchronous queuing, object monitors and variations of the acronym COM. Here we endeavour to explain it all from the ground up.

What is middleware?

Essentially, middleware is software that connects applications, allowing them to exchange data. It offers several key advantages over hardwiring applications together, which typically entails adding code to all of the applications involved, and instructing them on the particulars of talking to each other. Middleware adds an independent third party to that transaction - a translator.

Why are enterprises using it?

From a business standpoint, connectivity among applications is a given today. Shop floor, inventory, accounts receivable and advanced planning applications need to communicate so that companies can make accurate promises to customers and executives can make educated decisions more quickly.

E-business in particular demands better integration by an order of magnitude. That's because Web customers commonly want to see several bits of up-to-the-minute information at the same time - product specifications, availability, shipping times and account status. Call centres have reps logged on to multiple applications (to answer those kinds of questions). That's OK when you're paying them to do it, but when you start to let customers access that information themselves, it's too complicated. Enter middleware to tie together all those apps and connect them to a Web front end, hiding the complexity from the customer.

From a technical standpoint, middleware offers several benefits, depending on the type you choose:l Simplicity In today's corporate, government and education computing environments, many applications have to share data. Putting middleware in the middle can mean each application needs only one interface - to the middleware - instead of a separate interface to each application it needs to talk to. (However, if you're connecting just two applications to one another, it might actually be more complicated to introduce middleware than simply coding the two apps to talk to each other.)l Persistence Middleware can capture data and hold on to it until it has been recorded appropriately by all the applications or databases that need the information. In technical vernacular, this ability is referred to as "persistence".l Services If your data needs to be checked for integrity, printed out, reconciled with data from other applications, merged, split or reformatted, various kinds of middleware can handle those tasks efficiently. This means you don't have to rewrite those services again and again for each application that uses them. As middleware products evolve, the breadth of services they can provide grows. Incidentally, vendors that sell such robust products often try to disassociate themselves from the term middleware.

There are three reasons why this is confusing. Firstly, this whole technology space is fraught with incomprehensible jargon because there is a lot of legitimate technical detail (this article doesn't aspire to cover everything, just the fundamentals). Secondly, the vendors regularly change their terminology and product names. Thirdly, the products grow in functionality, which makes it harder to delineate categories.

Types of middleware

It isn't uncommon for companies to have several types of middleware at work in a single enterprise, with different kinds proving more appropriate for different integration chores. Bigger companies, which usually have more complex integration requirements, tend to gravitate toward more sophisticated middleware products such as EAI.

1) RPCs and Database Gateways

Perhaps not truly middleware but included for context. Both are older ways of handling application connectivity, particularly in distributed environments.

RPC stands for remote procedure call, which means a slice of code in a client application that invokes a procedure on the server application. RPCs aren't middleware proper by today's definition. Although still in use, they're the method that modern middleware often replaces because they require programmers to rewrite them over and over when wiring a bunch of applications together. Other middleware approaches are often more efficient as the numbers of applications grow.

Database gateways fit the definition of middleware better than RPCs, since they are a third-party facilitating access to data. Database gateways connect applications to a particular kind of database platform. For example, consider a company with legacy applications and data residing on an HP3000 server. Many new shrink-wrapped applications are designed for accessing common databases like Oracle or Sybase, but they may need a little help connecting to the 3000's older TurboImage DBMS. That's a typical use of a database gateway.

2) Message-Oriented Middleware (MOM)

Examples: MQSeries, IBM MSMQ, Microsoft SmartSockets, Talarian.

Message-oriented middleware takes charge of relaying data from one application to another by putting that data in a message format, akin to the way e-mail works. In fact, before off-the-shelf MOM products became prevalent, Cargill Corporation, for example, built its own MOM integration tools by using the Hewlett-Packard OpenMail e-mail engine to move data.

If you understand e-mail, you understand the principle behind MOM. And as with e-mail, a key advantage of MOM is that the data from application A goes into a queue (a waiting line or holding pen, if you will) and can be retrieved later, if necessary, by app B. That protects the integrity of the data if, for example, app B happens to be down for a reboot at the moment app A is trying to pass along information. With this asynchronous approach, the middleware server waits for app B to get up and running again, then hands over the data in the queue in the right sequence (this retention of the data-bearing messages is an example of persistence).

MOM is commonly used for simple, one-way exchanges of data where relatively few operations are being performed on the data and where the timing of the data exchange is not critical. For example, imagine a checking account that is linked to a savings account, which is tracked in a different database. When the customer updates his address for the checking account, the system is designed to update the address for all of the customer's accounts, but it doesn't matter much if the update takes three seconds or 10 minutes.

3) Transaction Processing (TP) monitors

Examples: CICS, OpenCICS, IBM BEA Tuxedo, BEA Systems.

Transaction processing monitors have been around for some time (longer than MOM), having been developed originally for mainframe environments. TP monitors sit between front-end applications and back-end databases to manage the writing and reading of transactional data.

TP monitors are much more application-intrusive than message-oriented middleware. That means they demand more modification of the applications themselves in order to take advantage of the monitor's specific services. They also provide extra security and data integrity protection.

TP monitors are intended for heavy transaction load environments. Again from banking: all transactions recorded at ATM machines have to be managed carefully to make sure the institution keeps accurate tabs on customers' account balances - TP to the rescue. In particular, a TP monitor is useful when multiple applications need some of the same basic functionality such as security and directory services.

Users tend to see the benefit of, as you write new applications, you don't have to keep rewriting these services. Applications also talk to each other through this kind of middleware, but it goes beyond that. The general opinion is that TP monitors would be "overkill" for simpler, one-to-one application connectivity tasks.

4) Object monitors

Examples: IBM Component Broker VisiBroker Integrated Transaction Server, Inprise Microsoft Transaction Server.

Object monitors, also called object TP monitors, are advanced versions of the TP monitors described above. An emerging product category, object monitors provide TP monitor functionality but are built according to object-oriented specifications like the object request broker models described immediately below. This increases businesses' ability to modify the services provided by the TP monitor, for example, without requiring changes to the applications.

Electronic commerce applications such as online shopping carts and associated ordering apps tend to make use of object monitors. That's because e-commerce apps quite often need to draw data from many different sources and may also need to be modified frequently as the demand for ever-greater functionality grows. This need for flexibility makes a component-based architecture appealling because it lets you make changes in one part of the system without affecting the others. Web commerce also demands great data integrity and the ability to manage very high transaction volume.

5) Object Request Brokers (ORBS)

Examples: Common Object Request Broker Architecture (Corba), Object Management Group Enterprise JavaBeans (EJB), Sun Microsystems Component Object Model (COM+).

ORBs mediate between applications and network services (security, performance monitoring and printing, for example). They are a key part of broader architectural standards for building services and interoperable applications. The three standards efforts listed above are the key players. An example of an actual middleware product suite that implements one of these architectural plans is Iona Technologies' Orbix 2000, which is based on the latest Corba standard, release 3.0.

The premise behind ORBs is this: many applications need access to the same set of services as well as to other applications. The reality of computing today is heterogeneous computing hardware and operating system platforms, plus applications built with a wide variety of development tools and languages. To get all of those components talking to each other, the enterprise needs a consistent, object-oriented architecture.

And with the increasing need for applications in one company to share data with business partners or suppliers, this kind of standardised approach can greatly simplify the integration chore.

Corba is historically focused on Unix environments, while Enterprise JavaBeans is an architecture best suited for companies developing in the Java programming language. The good news is that those two standards' efforts are meshing nicely. Corba adherents should be able to integrate Java development projects with a minimum of fuss.

However, Boucher notes that server objects designed under the Corba model don't currently communicate well with objects on a server that follows the COM+ model, and vice versa. This means large organisations are better off choosing one model or the other, not trying to mix and match. Users with Windows NT/2000 servers are likely to be best served by COM+, while those with Unix servers (even accessed by Microsoft desktops) are probable candidates for Corba.

Microsoft's middleware architecture and product plans have evolved, and so has its terminology, which is a source of some confusion for anyone who hasn't kept up carefully. The original designation was simply COM. Next came DCOM, for distributed common object model. According to Boucher, Microsoft has now done away with DCOM and moved on to the current name, COM+. COM+ is also part of a greater Microsoft architectural plan called DNA, for distributed network architecture. DNA includes products and services well beyond the scope of middleware.

6) Enterprise Application Integration (EAI)Examples: ActiveEnterprise, Tibco Software NEON Impact, New Era of Networks e-Gate, Software Technologies (STC) BusinessWare, Vitria Technology Geneva Enterprise Integrator (EI), Level 8 Systems.

Many vendors are likely to spit nails over calling EAI products middleware -a fuddy-duddy term compared with EAI, which incorporates the sexy word enterprise. But the basic idea of EAI is integration. Unfortunately the term EAI is also a hot one for marketing purposes, so tons of vendors are latching onto it even though their products provide widely differing functionalities.

EAI is middleware plus. Like ORBs, EAI tools typically use a message broker as their underlying mechanism for data transport. In addition, they can parse, duplicate or transform the data to present it in a palatable format for each idiosyncratic application that needs to receive the data. In a big company, a sufficiently robust EAI product could conceivably displace the use of lower-level integration tools like MOM, although the two can certainly coexist.

The next level of functionality for EAI tools, and probably the functionality that best distinguishes EAI from other forms of middleware, is the support for business process rules. EAI allows the user to define proper business processes and make data integration subject to those rules; for example, the data moves automatically from the purchasing application to the accounts receivable app, but not until it has been signed off by the appropriate authority.

According to analyst the Hurwitz Group, Vitria's product was the first to incorporate this kind of business process support. Now others are adding the function both inside the corporation and across corporate lines in the supply chain.

Extensible markup language (XML) support is a key addition to EAI products, as is support for standards like BizTalk (a Microsoft-sponsored set of business-to-business communication protocols) and RosettaNet, a consortium working to establish electronics industry communication protocols.

With most EAI products, users purchase a central module and the specific interfaces they need. For instance, STC's e-Way product line includes separate "adapters" for SAP, Siebel, Lotus Notes and an array of back-end databases. Most EAI vendors also have a service branch to do custom programming if you need a connection to a homegrown application. "They seem to follow the 80-20 rule, meaning 80 per cent of your interfaces will be prebuilt, and then they come in to do custom work on the other 20 per cent," says John Campanale, vice president of ARC, a US-based consultancy.

EAI is suitable for big companies that need to integrate lots of applications. Cargill, for example, uses BEA Systems' eLink EAI product suite to interconnect a broad portfolio of apps including ERP, maintenance management, inventory and cost accounting systems, among others.

WHEW! Which middleware is right for your customers? There's no single answer. Different kinds of applications and integration needs are best served by different kinds of middleware. On the other hand, using a single approach can offer great economies of scale and development effort. At any rate, weighing the options is a lot easier when you know what the terms mean.