VOIP (voice over Internet Protocol) phone systems still lag significantly behind the traditional phone network for reliability and quality, and there are major differences in quality among brands of VOIP service, according to a company that evaluated six providers in the US.
Keynote Systems tested the services for reliability and quality by making domestic long-distance and local calls from VOIP services to the PSTN (public switched telephone network) in May and June this year. Call quality was evaluated by software and used an industry standard for listening quality, according to Dharmesh Thakker, senior product manager at Keynote. The study involved more than 150,000 automated VOIP-to-PSTN calls. Keynote evaluated these kinds of calls because they are the most common type made by VOIP users, he said. The company tests Web application performance and is branching out to VOIP.
The survey evaluated VOIP services accessed via DSL (digital subscriber line) and cable modem, which are geared mostly to consumers. It gives valuable new data on the state of such services, which are exploding in popularity, said IDC analyst Will Stofega, who is familiar with the results. IDC expects 3 million US consumers to be using VOIP by the end of this year and 28 million by 2009, he said.
Vonage Holdings' service came in first place for reliability, a measure that included how often the service was available, the length of any outages, the average number of call attempts before a call went through, and the number of dropped calls, Keynote said. Vonage scored 80 points out of a possible 100. There was a noticeable gap in reliability between Vonage and the other providers, which included AT&T's CallVantage, 8x8 's Packet8 Broadband Phone Service, Primus Telecommunications' Lingo, Skype Technologies' SkypeOut and Verizon Communications' Verizon VoiceWing, Keynote said.
AT&T CallVantage ranked first for audio clarity, which included audio delay, listening quality, and the consistency of both of those factors over call time and distance. It earned 78 points out of a possible 100, and CallVantage's quality score was noticeably better than those of other providers. The listening quality rating was based on the ITU-T P.862 standard of the International Telecommunication Union and was determined using software instead of subjective human evaluation.
Though they outscored other services, even Vonage and CallVantage have some room to improve, according to Keynote.
On average, the VOIP calls lagged behind both PSTN and cell phones using the ITU's established standards for listening quality, which includes elements such as static, hum and hiss, Thakker said. The average VOIP call scored 3.5 out of 5, whereas cell phone calls average 3.6, he said.
Some factors affecting the quality of VOIP services are out of the service provider's hands, the company pointed out. The survey revealed that service quality and reliability can vary from place to place, sometimes because of differences between local DSL or cable networks. The quality of long-distance backbone networks can affect service, too. Keynote tested calls carried over the AT&T, Sprint and MCI. UUNet backbones as well as over DSL connections by SBC Communications and Verizon and cable lines from Comcast and Time Warner Cable. Technical cooperation can play a key role in boosting quality, Thakker said.
"The Vonages of the world really need to work with network carriers to get more effective service level agreements in place," he said.
VOIP service providers are making gradual technical changes that should improve quality over time, though those changes are generally under the covers as far as consumers are concerned, IDC's Stofega said. He doesn't see one big breakthrough that will address the woes of consumers who have been disappointed with VOIP.
"What has colored people's experience has been some issues related to quality of service," Stofega said. "You've got to try it again to see if it meets up with your standards."