WASHINGTON DC -- Websites get looked at by two different kinds of visitors: the human ones who peer around, look at the graphics, think about the links and click slowly; and the spiders, those automated scanners that come in from search engines like Google, or, more ominously, from malicious attackers, competing businesses and spammers looking for e-mail addresses.

Fortunately, it has always been pretty easy to tell the difference between the two in server logs, and block unwanted or anti-social crawlers. But research presented at the Shmoo Con hacker conference here Friday may change that.

Billy Hoffman, an engineer at Atlanta company SPI Dynamics unveiled a new, smarter web-crawling application that behaves like a person using a browser, rather than a computer program. "Basically this nullifies any traditional form of forensics," says Hoffman.

Look, Ma, it's Acidus on Wired!

-janelane, fiancee extraordinaire

Billy Hoffman, an engineer at Atlanta company SPI Dynamics unveiled a new, smarter web-crawling application that behaves like a person using a browser, rather than a computer program. "Basically this nullifies any traditional form of forensics," says Hoffman.

Tim Ball, director of systems and development for the U.S. Senate's Democratic Policy Committee knows what it's like to be under constant spider attack. The Senate website relies extensively on server logs for forensics, but Ball is no longer confident that approach will be helpful in the long run.

Ball says the research will make it easier for attackers to automatically and discreetly spot flaws on websites they previously had to root out by hand. "What Billy's done is massively simplified the process and make it faster," says Ball.

Hoffman hopes the street will find its own positive uses for his work as well. "One of the really cool things I have had to do was to score how interesting a link would be," he says. His technique is similar to applications like Google's page scoring system, but is publicly available in open-source Java code anyone can use.

Much fun was had and much work was done by the entire crew in DC this week.

In regard to hackers like Billy Hoffman and Mike Lynn, it must be understood that while their work may appear on its face to only help evil doers, it couldn't possibly be farther from the truth.