10 Microsoft research projects

A sneak peek at 10 technologies developed in Microsoft's R&D labs
Surface: It looks like an ordinary table, except for the object recognition and sophisticated software that allows multiple people to "control" their part of the interactive screen via touch.

Surface: It looks like an ordinary table, except for the object recognition and sophisticated software that allows multiple people to "control" their part of the interactive screen via touch.

Innovation is not just about cool new products. In technology, the best ideas require a) really smart people and b) lots of funding. For the past 33 years, Microsoft has had both in spades.

Yet, the precursor to any product release is the research and innovation that occurs before the first shipment to customers, that "pre-pre-alpha" stage where ideas are born.

In a recent two-day visit to Microsoft corporate headquarters, writer John Brandon met with several researchers working on new projects.

A few of these projects have already resulted in shipping products; others may never see the light of day -- they are meant as a proof of concept. Some could change how we do computing altogether. Yet, all of them are driven by bright thinkers who are working to solve real-world technical problems.

Codename: Eagle 1

In a major disaster, the fire and police departments, FBI, local government, and other public safety officials coordinate a search and rescue mission for survivors. Some of the techniques they use, such as closing roads and creating a chain of command, are tried-and-true measures.

However, one of the challenges is in coordinating the IT infrastructure needed for collaboration and communication. In most cases, even as recently as Hurricane Ike, this has been a major challenge, especially when any existing infrastructure has been damaged or is now inoperable. Barry is championing a new project called Eagle 1, which is a data visualization and mapping tool.

"After any major disaster, during the debrief, the first thing that always comes up is the communication and collaboration," he says. "Trying to get real-time information from all of the agencies involved to make life-saving decisions, the more quality information you can have, the better. There's a difference between information you can use and just raw data. The physical side is done to death, the hands-on side is mastered. The technology has been too slick and high-tech, but it has not been able to present the information in a form you can react to."

Eagle 1 pulls information from multiple databases and uses geospatial mapping technology to create an interactive map that would show, for example, all the schools, military bases, hospitals in the affected area. Plus it would show how many people are in the hospitals, current evacuation models and casualties or danger zones, even a plume model that could show where a gas leak is heading.

All the data is shown in a real-time interactive map using Virtual Earth, but the key is how Eagle 1 pulls data from many different sources (such as from both Oracle and SAP databases) and presents the results on one screen that can even run on a Microsoft Surface table. Barry said the process of configuring the data extractions will likely involve a team of Microsoft disaster specialists.

Page Break

Codename: Surface

Microsoft Surface is now becoming a viable product that is extensible with customized software such as photo viewers and extra games. The device, which is about the size of a card table, is not sold directly to consumers. Instead, it's sold to companies such as AT&T and Sheraton Hotels who use it in their lobby or as an attention getting conversation starter.

These "partners" can request custom interfaces and programs, or develop their own. Surface is made of a hard acrylic material that can withstand a lot of abuse. At a Harrah's iBar in Las Vegas, for example, people spill drinks and food on it all evening. Up to 52 people could crowd around a Surface table and control their corner of the interface, although it's usually a two- or four-user experience.

"It was interesting to start with a project in an incubation phase and scale it up and out as a viable product," Champagne says. "The application launcher comes with choices, content the customer wants to load. Collaboration is a big part, it has object recognition -- it is meant to interact with physical objects. There are infrared cameras that look at the surface. We have an optical tagging technology where you can tag items."

He mentioned how, at AT&T, you can place a phone on the table and the features of the phone will appear -- more information than the cellular provider could ever list on an in-store sign. At Sheraton, you can call up a virtual concierge and see maps of the area with theater or restaurant suggestions. Microsoft has also targeted retailers such as Best Buy for Surface tables. The basic Surface product retails for US$12,500, but there's a volume discount for mass deployment.

Code name: Pictionaire

Andy Wilson's lab, located in Building 99 on the Redmond campus, is low-lit and spacious with several glowing monitors scattered about the room. There's an early prototype of Microsoft Surface in one corner, an LCD monitor set at a 30-degree angle in another. Near the back of the room, an orb sits idly on a podium, a precursor to the Microsoft Sphere project. Yet, the most striking device is a large glowing 4x6-foot table.

"I can start typing on this keyboard," says Wilson as he drops a keyboard into the table surface. Icons suddenly appear next to the keyboard like something from a sci-fi movie.

He drops another keyboard and a mouse onto the surface, and icons appear for those devices. Next, Wilson grabs a small whiteboard and starts making notations. A camera records his sketches, and the image appears on the table surface -- which he can further manipulate. Like the Microsoft Surface project, this new table -- which runs software called "Pictionaire" -- allows Wilson to type e-mails, play video and music. "I can even capture the entire table" as a screenshot.

What's most striking about Pictionaire is it allows unprecedented collaboration. In a meeting, an entire group of employees could gather around the table, each with their own keyboard and mouse, and engage in a project with images, text, video and other objects. While the Surface table allows up to 52 people to participate simultaneously, that's not a practical limit -- Wilson says Pictionaire could support at least that many concurrently. It uses a multi-touch interface as well, so you can zoom in on objects and move them around. One can imagine several teams using Pictionaire surfaces at different locations as well, with video collaboration. The Pictionaire demo used Windows Vista, but it's not hard to imagine running Linux in a virtual window.

Page Break

Codename: Touch Wall

Like Wilson's Pictionaire project, the Microsoft Touch Wall -- which Bill Gates demoed during his keynote at CES 2008 -- is a new software-driven computing paradigm. It's partly a multi-touch hardware interface (for example a wall-sized version of the iPhone) but primarily a new software interface that works remarkably similar to the scene in Minority Report where Tom Cruise controls a computer with his hands. Sands started out working on interactive television systems 13 years ago and developed early prototypes in the mid-1990s. He also worked on the MS-NBC and Slate Magazine launches, working on interactive media.

"We needed to come up with something beyond the typical PowerPoint and whiteboard idea -- things like video and interactive media to demonstrate rather than tell what the future might be like," Sands says. "We want to put out plausible scenarios that leverage today's technology."

The Touch Wall is more like an operating system than software. There is a large white background with several objects on the screen -- documents, video, music, slideshows. You can zoom in on the interface by flicking out with two hands. You can play a video or slideshow by clicking with a finger. Most impressively, you can mix and match media on the same large-screen display, playing a video in one corner and holding a video chat with someone in another corner. Like Microsoft Surface, multiple people can use the Touch Wall at the same time and interact with other people using a Touch Wall somewhere else.

Touch Wall is primarily a user interface and not an operating system -- it runs on a standard PC in Windows Vista using an LCD rear projector and a two-way glass panel. Sensors attached to the side of the Touch Wall read movements and feed them to the interface, which is called Plex. However, like the Surface table, it's possible the Touch Wall will be developed into a standalone product that could be used for meetings and sales presentations, or one day in homes as a natural interface.

Codename: Paris/Social Streams

Some ideas are born out of necessity. The Paris project -- the formal product name will be Political Streams when it becomes available -- provides a big picture view of political news and blog chatter. It's essentially a trend aggregator similar to Google Trends or Yahoo Buzz, except that it crawls the Web for actual content, rather than just aggregating search terms. Alex Daley -- the group product manager at Microsoft Live Labs -- showed a demo where Sarah Palin news reports and blog posts appeared on a graph in comparison to reports on Barack Obama.

"This is all in real-time, and we can effectively filter across various industries -- we are starting with politics," Daley says. "We can see the relationship between political reports. John McCain has had much more media interest than Barack Obama ever since the Sarah Palin announcement. We use a technique called entity extraction, a machine learning technique for classifying documents and text, such as this is a name, this is a place, or a recipe, or a review or product manual. We extract the core data and drawing relationships."

(Note: Live Labs is a seed farm at Microsoft, consisting of small five-to-eight person teams who develop innovative services and Web sites such as PhotoSynth. The small team size is intentional because Live Labs is intended to germinate ideas, some of which may not become actual products. In fact, the PhotoSynth project itself -- which is a way to see 360-degree views of a real-world location -- was not a raging success at first because Live Labs found that people would take the same photos of buildings and sites. Today, it has become more of a social networking site -- people decide together to "stitch" a scene more intentionally.)

Page Break

Codename: LucidTouch

Ask anyone with big hands whether they like the Apple iPhone and you will likely get a resounding "No!" in response. The reason? The 2x3 inch screen requires fairly small fingers to control the interface. If your fingers are too big, you'll likely make frequent errors.

The Microsoft LucidTouch V2 technology seeks to solve this problem. It's one of those early research projects that seems like a head-scratcher at first: A device with a small 2x2 screen that's about the size of a thick credit card that allows you to reach behind the screen to make selections. A representation of your fingers shows up on screen. Your fingers can be shown smaller, or with a red dot that shows your fingertips.

"A touchscreen device is governed by the size of your fingers," says Baudisch, who studied human interfaces in Germany before coming to Microsoft. "If you look at home automation systems, they are targeted to a bigger screen size. We're asking: what happens in a few years when a touchscreen is embedded into a watch? It turns out that touchscreens don't do well at these sizes. Since it's difficult to make your fingers transparent, why not make the device transparent?"

The project reminded me of several products Nokia tried a few years ago in which a very small interface was embedded into a locket or other jewelry, but they were still difficult to use. LucidTouch could be used to power very small gaming devices or cell phones.

Codename: OSLO

In the early days of computing, a model -- such as a project organizational chart or the development plan for enterprise software -- was a static document built in Microsoft Excel. The problem: in modern software development, models need to become living documents that many people can access, including business analysts, executives, quality assurance testers and project managers.

The OSLO project, named after the city in Norway, is a framework that helps all contributors -- both technical and non-technical -- access data models in a repository.

"Modeling is something that Bill Gates has talked about as a future trend -- it plays an important role in the application life cycle," Kawasaki says.

One project that is part of that effort is OSLO. It contributes three things. One is a repository where you can change definitions of models from developers and architects all the way to data center mapping. Secondly, you need a way to describe the models, so OSLO has a new declarative language. Third, there are visual tools, especially for the non-technical user.

OSLO is like the SharePoint of application development modeling. It breaks out of the traditional app development process where models are used only during development workflows and helps any contributor see data models as they change and evolve. It also addresses the siloed approach so common in the modern development process.

Page Break

Codename: Visual Studio 2010

Visual Studio is the staple of most Microsoft-centric development shops, and the next release (which will likely ship next year) will focus on new collaborative tools, direct access to the OSLO repository for data modeling, and an update of .Net to Version 4.

One of the most compelling new features is the brand new Architecture Explorer, which allows development teams to see a model of the existing development framework and find any existing code assets that are not well categorized.

Other new features include unified modeling language support and a new debugging tool that is especially useful for find non-reproducible bugs by automatically creating data sets.

Codename: BlueTrack

Laser technology is advancing quickly, and one product proves this point dramatically.

The Microsoft Explorer 1362 mouse (and Explorer Mini 1363) uses a newly invented BlueTrack technology that works on a variety of rough surfaces.

I tested the mouse on metal, wood and tile -- it worked perfectly, while a "last gen" laser mouse from Microsoft failed to even move the cursor.

BlueTrack captures 8,000 images per second, casts a much wider and brighter beam, and -- most importantly -- reads data using a high-contrast sensor.

For those who need mouse precision anywhere, the 1362 model captures data at 4000DPI.

Codename: Robotic Receptionist

Here's a project I can relate to on a personal level, having found myself lost and running late for meetings during my visit to Microsoft in mid-September. The campus consists of more than 100 buildings spread across a wide swatch of Redmond and the surrounding area.

Although it's so new that the only information available on this project comes from a speech by Craig Mundie at the EmTech08 Emerging Technologies Conference held recently, the robotic receptionist project is clearly a sign of how computer technology is evolving. Mundie said in his EmTech keynote that natural interfaces equipped with voice and facial recognition features will become part of our daily lives over the next 10 years and will not require any hands-on input from the user.

The robotic receptionist -- which will be used at Microsoft headquarters, likely next year -- will help Microsoft visitors find shuttles to get around campus. The receptionist can even identify visitors based on what they are wearing and provide information on shuttle routes using GPS tracking data.