Billy Hoffman, an engineer at Atlanta company SPI Dynamics unveiled a new, smarter web-crawling application that behaves like a person using a browser, rather than a computer program. "Basically this nullifies any traditional form of forensics," says Hoffman.

Tim Ball, director of systems and development for the U.S. Senate's Democratic Policy Committee knows what it's like to be under constant spider attack. The Senate website relies extensively on server logs for forensics, but Ball is no longer confident that approach will be helpful in the long run.

Ball says the research will make it easier for attackers to automatically and discreetly spot flaws on websites they previously had to root out by hand. "What Billy's done is massively simplified the process and make it faster," says Ball.

Hoffman hopes the street will find its own positive uses for his work as well. "One of the really cool things I have had to do was to score how interesting a link would be," he says. His technique is similar to applications like Google's page scoring system, but is publicly available in open-source Java code anyone can use.

Much fun was had and much work was done by the entire crew in DC this week.

In regard to hackers like Billy Hoffman and Mike Lynn, it must be understood that while their work may appear on its face to only help evil doers, it couldn't possibly be farther from the truth.

Wired News: Covert Crawler Descends on Web

We are scholars of constitutional law and former government officials. We write in our individual capacities as citizens concerned by the Bush administration's National Security Agency domestic spying program, as reported in The New York Times, and in particular to respond to the Justice Department's December 22, 2005, letter to the majority and minority leaders of the House and Senate Intelligence Committees setting forth the administration's defense of the program.

This letter is a bit repetitive due to its structure, but the legal explanation offered here is relatively clear and concise.

On [Domestic] NSA Spying: A Letter To Congress

noteworthy wrote:

Cell Phone Call Record $110

Give us the cell phone number and we will send you the calls made from the cell phone number.

I like this part:

This report is for informational purposes only. This is not for use in court. If you need the records for court, you will need to subpoena the records directly from the carrier.

The whois records are pretty opaque:

Registrant:
Ist Source Information
ATTN: LOCATECELL.COM
c/o Network Solutions
P.O. Box 447
Herndon, VA 20172-0447

The site appears to be fairly new; the record was created on September 26 of last year.

UPDATE: You can read a recent Chicago Sun-Times article, "Your phone records are for sale", about Locatecell. This article was posted to the cryptography mailing list, which is probably what prompted the MemeStreams thread.

This was covered in the Washington Post more than six months ago, "Online Data Gets Personal: Cell Phone Records for Sale."

"This is a person's associations," said Daniel J. Solove, a George Washington University Law School professor who specializes in privacy issues. "... It's a real wealth of data to find out the people that a person interacts with."

The company that operates Locatecell is Data Find Solutions, and they are located in Knoxville, TN.

I like this part of the Locatecell order form:

Phone searches are provided by third party, independent search experts. These experts are independent researchers and Data Find Solutions Inc does not know how they do the research or what databases they access.

As the news articles explain, EPIC has asked the FCC to investigate. EPIC offers a compendium of 40 Websites Offering Telephone Calling Records and Other Confidential Information (PDF).

Looking for startup capital -- or a business model? MemeStreams could put the social network information behind a walled garden. But would anyone want in?

Well folks, it looks like you don't need unchecked presidential executive powers to get phone records without a warrant...

Cell Phone Number Research

The 38 million subscribers to MySpace, which News Corp bought for $629m (355m) last July, discovered that when they wrote to each other about rival video-swapping site YouTube, the words were automatically deleted, and attempts to download video images from YouTube led to blank screens.

However, MySpace managers promptly shut down the blog forum on which members had complained about the interference. An online notice said the problem was the result of "a simple misunderstanding".

The explanation did not, however, calm the bloggers. "There was an outcry by some members after MySpace's acquisition by News Corp. People were afraid they might start monitoring or censoring MySpace," Ellis Yu wrote to the Blog Herald. "At the time, their CEO said nothing like that would happen. Well, now it has. MySpace was built on an open community and now they're trying to censor us, putting business interests above its members!"

I'm pretty sure I'm on the record, somewhere, predicting News Corp would do just this kind of thing. MySpace is, and will continue to be, a journey in learning about the necessary dynamics of Social Networking sites, and not the final solution to the online community.

Mr Murdoch, 74, last week appointed 33-year-old Jeremy Philips to run News Corp's internet strategy and armed him with a $1bn fund to buy more sites.

Let me give my complete assurance that Industrial Memetics will never entertain a buyout offer from News Corp.

Myspace stumbles

The 2005 Year in Graphs is being held up due to my broken laptop. We have never released it on time, so this should not shock anyone. This year I get to blame it on Apple.. Which is so much better than just saying "my code is broken" or "this database is insane", which is usually (truthfully) the case. I should be back to those default explanations in about a week, depending on how fast Apple support repairs my machine. At worst, when I'm no longer traveling I'll be able to get it done on my backup machine.

I'm looking forward to seeing it myself. It was an interesting year. Interesting in the "may you live in interesting times" kinda way. I hope that comes through in the end result.

2005 was a crappy year. I am very happy that it is now 2006. The first half of the uh-oh's is over. Is this the point where the decade starts to turn around? I know things don't necessarly work that way, but hope and delusion are closely related.

The pace and scope of attrition in the ranks of the FBI's analysts suggest root causes that are more serious in nature and more systemic in effect than the inspector general and the bureau realize. It wasn't the photocopying or the lack of promotion potential that compelled me to leave my job as an FBI analyst this year -- it was the frustration of working in a system that does not yet recognize analysis as a full partner in the FBI's national security mission.

For example, not all the people carrying the title "All Source Analyst" in the division for which I worked even had desktop access to the Internet or to intelligence community e-mail and intranet servers.

The analyst's access to investigative data becomes almost entirely a function of personal relationships cultivated with agents in the field -- a difficult task for those whose work it is to assess threats emerging across the nation and overseas.

The FBI needs MemeStreams, period.

The FBI's 2nd-Class Citizens

This is horrible! To the best of my knowledge, the Nun-Bun is Nashville's only relic. Not to mention, BongoJava is my primary Nashville hangout. I take this as a personal affront.

We must catch the culprit(s). For great justice!

"My gut feeling is that it's destroyed," said Bob Bernstein, the owner of Bongo Java coffee shop, where the bun had been on display for nearly 10 years. "Someone took it, destroyed it and it's the last we'll hear of it."

Someone broke into the Belmont Boulevard coffeehouse yesterday morning, apparently with the sole purpose of stealing the pious pastry. Bills and loose change in charity-donation containers near the bun's glass display case were untouched, Bernstein said.

"They went right for the bun," he said. "What the heck they are going to do with it, I can't imagine. It's sure not something anyone would eat. I hope they do eat it. It will teach them a lesson."

"It's weird," Bernstein said. "You laugh about it a little bit, but it's an empty feeling. It's like the end of an era."

I wonder if Bob had any type of insurance on it...

BongoJava Nun Bun stolen!

I hope everyone is having a good holiday season this year. I'm on the road, so my blogging will be very light for the next two weeks. My new laptop apparently has a bad stick of ram causing it to crash on me, making matters worse.

I'm going to pretend its a good thing. I could use a bit of a vacation.

It turns out that looking at the aggregation of billions of search queries people type into Google reveals something about our curiosity, our thirst for news, and perhaps even our desires. Considering all that has occurred in 2005, we thought it would be interesting to study just a few of the significant events, and names that make this a memorable year. (We’ll leave it to the historians to determine which ones are lasting and which ephemeral.) We hope you enjoy this selective view of our collective year.

Google's Zeitgeist reports are always interesting, but I also always feel like they could do more. Google is sitting on top of one of the most amazing collections of information mankind has ever assembled, and has all the metrics on people's usage of it. If all the various TV outlets ranging from CNN to VH1 can assemble "year in review" programs every year that go into the year's events in such depth, Google can do better than this.

Google Press Center: Zeitgeist

Was the secret NSA surveillance program legal? Was it constitutional? Did it violate federal statutory law? It turns out these are hard questions, but I wanted to try my best to answer them. My answer is pretty tentative, but here it goes: Although it hinges somewhat on technical details we don't know, it seems that the program was probably constitutional but probably violated the federal law known as the Foreign Intelligence Surveillance Act.

This is a detailed analysis. The Administration's Position is also available.

Legal Analysis of the NSA surveillance program