The TIDE database

Databases are wonderful things, when the data is consistent.  I’ve talked about databases before, but in the weekend Washington Post, a fine article by Karen DeYoung highlights the problems the US Department of Homeland Paranoia is having in keeping their waste products linear.

The Terrorist Identities Datamart Environment (TIDE) is the storehouse of individuals that the various American intelligence agencies have considered to be potential risks.  Data comes from several hundred sources of varying degrees of accuracy.  The database in 2003 was about 100,000 files:  Now it is 435,000 files.  It isn’t the number of files that is the problem, as things like SQL can handle databases much larger than that. 

The problem is the veracity of the data.  Remember the Old Skool Computer Acronym GIGO? Garbage In, Garbage Out.  The TIDE database gets some of the data from the FBI, some from the TSA, some from Consular Lookout and Support System and some from state and local cops.  Parts come from the National Crime Information Center database, other parts are Secret.  It all piles into one big monkey vat of data lovin’. 

Knowing that, you must also understand how a database works.  Databases are nothing more than a big collection of electronic files, put away by keywords. so you can find them again. 

In the distant past, if you wanted to find something in "The Joy of Cooking" and all you had in the kitchen was a dead possum, flour and garlic, you could turn to the index, look up opossum and it would have a list for recipes that featured possum.  The index would list page 454 as the page that has the specific recipes for possum. Eventually you would narrow your search down to that one recipe that has possum, flour and garlic as ingredients.

That is the simplest and most common type of database search there is.  Computers can do that searching much faster than humans, searching on several dozen terms at the same time, as well as the relationships between keywords, as long as the data is consistent.  Please underscore that last line. 

For example, from the WaPo article, let us consider Cat Stevens.  Cat Stevens is the former name of the singer now known as Yusuf Islam.  Yusuf Islam is on the TSA No-Fly list for "secret" reasons. 

I’m guessing here, but odds are some CIA pud found a piece of paper in London with the name Yusuf Islam written on it, in the same time zone as a rocket propelled grenade.  Therefore anyone named Yusuf Islam has been near or associated with explosives, is obviously a Jihazi, dangerous, crazy and shouldn’t be allowed into the US. 

This is the same logic as Googling David Smith and assuming I am a mathematician who died in 1944 or I work at NASA Ames Research.  I am not dead and I don’t work at NASA Ames, let me be clear on that.

However, if your name is Catherine Stevens, you might be stopped at the airline, or the border, as Cat is a common enough contraction of Catherine and the last name matches, so you must be Yusuf Islam, a "bad" guy and writer/performer of "Peace Train"  Or, you could be the wife of US Senator Ted Stevens (R-Alaska), trying to get to Washington. 

Again, obviously a bad guy intent on transvestite sex with a Federal politician and destroying the Wal-Mart in Cookville, Tennessee with explosives hidden in a can of Alaskan crab legs.  I am kidding; it is a waste of good crab.   

Perhaps the worst part of TIDE is that it is a data warehouse without anything beyond rudimentary vetting based on the assessments of 80 data analysts.  I suspect that Chester Field, Heywood J. Ablome and Suk Mi Ohf are also somewhere in the TIDE database, as that seems to be the level of sophistication and accuracy that the various intelligence agencies can muster on a good day.

The most telling sentence however, is this one, "(Rick)Kopel (TIDE Acting Director) insisted that private information on Americans, such as credit-card records, never makes it into the screening center database and that "we rely 100 percent on government-owned information."  

DCS 1000 (formerly Carnivore), the data, email, phone call and chat domestic spying tool is run by the FBI.  The FBI is government-owned.  Therefore, Rick Kopel is telling the truth, but overlooking the pervasive no-warrant/no-oversight/no-review powers that DCS 1000 and Echelon have under the Patriot Act. 

The Department of Homeland Paranoia, renown for their transparent and forthright communications with Congress, Justice and the citizens of the United States of America, is the owner and operator of the Patriot Act.  Which also explains why there is no mechanism to get yourself off the watch list.

To put the fine point on it, TIDE has access to and from all the data you could possibly want.  The value of the data, based on the output that crops up in the TSA, is garbage.  It isn’t even funny garbage.  

Even more frightening, is that there will be a file somewhere in TIDE database tomorrow that links the keyword term Heywood J. Ablome to David Smith to the keyword Jihazi to the keyword TSA.  What kind of assumption could you make of that linkage?  The wrong one, to be sure.