When I first met with startup LinkGuard Ltd. at NetWorld+Interop last May in Las Vegas (www.nwfusion.com, DocFinder: 1225) the company was setting out to rid the world of broken Web links.
I caught up with LinkGuard again at Interop in Atlanta a few weeks ago and, while the company is basically on track, it turns out the problem is even bigger than the founder first thought.
To refresh your memory, LinkGuard wants to offer a link integrity service based on a database of public Web links. Unlike products used to find broken links on a site, LinkGuard identifies broken inbound links.
That's important because inbound links can be a good source of traffic. If, for example, Cisco published a link to a Network World story and we subsequently moved that story, we would be shutting the door on potential readers.
If we were a LinkGuard customer, instead of returning a 404 error message to the cisco.com surfer, we would redirect the request to LinkGuard that would look up the proper location in its LinkMap database of public Internet links. Then LinkGuard would notify Cisco about the dead link or, if Cisco was a LinkGuard customer, go into cisco.com and fix the link.
Obviously the key to this is the LinkMap database. Company Founder and CEO Franck Jeannin started off thinking that the roughly two billion Web pages on the Internet probably had about 10 links per page. That meant he had to map 20 billion links, something he could achieve with an 8-terabyte database.
It turns out there are about 52 links per page, meaning he has to map about 100 billion links, a job that will require a 40-terabyte database.
While that sounds daunting, Jeannin says building the database will take crawlers about 10 days. What's taking the time is selecting vendor partners (Dell and Microsoft), hiring people, raising money and finalizing the software.
One interim advancement since last we talked: the ability to scan for bad JavaScript links. LinkGuard can now run the scripts like a browser and check the links generated.
LinkGuard has 500 companies using the software to test their own sites for broken links but won't make the inbound testing available until the database is completed in March.
An end to bad links may be on the horizon.
- John Dix
Editor in chief
jdix@nww.com