The Department of Homeland Security is at work on its own version of a massive, anti-terrorism database-sifting application -- not dissimilar to the Congressionally-halted Total Information Awareness program -- but it may have already run afoul of Congressional auditors by testing the system on data about American citizens that wasn't thoroughly anonymized, according to a story in the Washington Post last week.
Congress's investigative arm, the Government Accountability Office, will soon be issuing a report saying that the system, known as Analysis, Dissemination, Visualization, Insight and Semantic Enhancement (ADVISE), violated basic data privacy practices by re-using citizen data without notice or approval, according to the story. That program got $50 million in funding in 2006, and one employee says it has already helped foil terrorist plots.
What does the system do?
That's hard to say. From what little I've read it sounds like it's supposed to discover terrorists plots in real time and create social network graphics to find leads for investigators by translating news and blog stories into structured information in real time and by monitoring who is communicating with whom in real time.
According to this DHS Workshop paper (.pdf), the system is supposed to be able to handle one billion structured and one million unstructured text messages per hour.
The only data I can think of that would provide one billion structured pieces of information per hour would be the world's phone and internet traffic logs.
That same document also wonders if the system could help solve the following questions:
* Can patterns in the financial transactions of terrorists be detected and exploited?
* What is the structure of power in a group of terrorists?
* What were the main topics of intercepted terrorist messages over the past five years?
* Can a group that is purposely trying to deceive by swapping cell phones with innocents be tracked? In other words, can such changes be tracked over time?
The diagram above comes from this paper (.pdf), which is also related to the ADVISE system.
