The database war

Ever since Bob Woodward hinted at the existence of a ‘Manhattan project’ type secret weapon which was largely responsible for an increase in US military effectiveness in Iraq, the press has been casting around eagerly for clues as to its nature. Woodward said, “If you were an al Qaeda leader or part of the insurgency in Iraq, or one of these renegade militias, and you knew about what they were able to do, you’d get your ass outta town.”  The LA Times is focusing attention on a kind of surveillance device that allows UAVs to identify and track suspects from a distance, in a manner not unlike a giant supermarket barcode scanner looking down from the sky at objects parading miles beneath.

American officials requested that details of the new technology not be disclosed out of concern that doing so might enable militants to evade U.S. detection. But officials said the previously unacknowledged devices have become a powerful part of the American arsenal, allowing the tracking of human targets even when they are inside buildings or otherwise hidden from Predator surveillance cameras.

Equally important, officials said, the systems have significantly speeded up decisions on when to strike. The technology gives remote pilots a means beyond images from the Predator’s lens of confirming a target’s identity and precise location.

Leaving aside the question of whether this remote scanning capability is the ‘secret weapon’ referred to by Bob Woodward, two conjectures about it can be safely made.  One is that the secret weapon — whatever it is — relies on a database which stores at least two fields, one containing a physical identifier against which the person can be identified and the second containing an his wanted status. For example, a hypothetical record for “Osama bin Laden” may contain a value for his physical identifier (more on this later) and another field will contain the entry “Wanted, Dead or Alive”. Thus, if Osama bin Laden comes within range of the ‘secret weapon’, a database lookup will show him to be within the field of view.

The second conjecture is that there exists some means of collecting information that goes into the database.  Since the properties of the matching system of the ‘secret weapon’ are unknown, it is impossible (and undesirable) to identify which data collection effort underpins it. But we do know that a lot of information is being collected in Iraq. Indeed, collecting information can now rightly be called a major operation of war. The Strategy Page writes:

For the last three years, U.S. troops have been fingerprinting every suspicious character they have come across. The guy they turned lose three years ago for lack of evidence, may be on a wanted list today because his prints were found all over some warm weapons or bomb making materials. Prints can even be lifted off some fragments of exploded bombs.

The army and marines have been doing the same thing police forces and corporations have been doing for over a decade; taking data from many different sources and quickly sorting out what all the pieces mean. It’s called fusion and data mining, and it’s a weapon that is having a dramatic impact on what many thought was an unwinnable war.

Beyond these two known points — that the ‘secret weapon’ operates on a database and that the database information is collected somehow — we know nothing in the public domain. The suggestion that some device mounted on a UAV can “see” suspects through walls suggests something about the physical signals that particular device works on, whether or not it is the ‘secret weapon’ Bob Woodward describes. But keeping to the knowns, or at least to the highly probables, what else can we say about the database? One: that it is a very dangerous and highly valuable piece of information. Second, that the “wanted status” field is by far the most important column in the system.

Harper’s Magazine published a little noticed article in August 2007 describing the concerns of Marc Rotenberg, Executive Director of the Electronic Privacy Information Center (EPIC) about the security of identification databases supporting warfighting. Specifically, Rotenberg was worried that database information could be used by Iraqi sectarians to rub out ethnic rivals or political opponents; that this marvelous datastore could become, in the wrong hands, a computerized death warrant for thousands instead of a tool for fighting bombers and killers.  The key design issue in any such warfighting database is who gets to fill in the “wanted status” field; who gets to write “innocent” or “Wanted, Dead or Alive” in that vital column.  At the moment there is presumably a classified process of review controlling how that field is populated.  But if the database fell into the wrong hands …

The routine and depressing report of database thefts provide little assurance that any such database would be safe forever. But even its loss would be mitigated by two things. Unless the thief also had updates available from the data collection system it would soon fall out of sync and the thieves would still not have the physical ‘secret weapon’ which the database supports. It is one thing to have that database and another to possess all the physical components needed to collect, process and act on the information. Truly effective ‘secret weapons’ are defended not simply by confidentiality but by complexity and the engineering challenges required to overcome them. The Atomic Bomb has remained largely out of reach to terrorists for so many decades not because its design is ‘secret’ but because it is relatively difficult to physically manufacture. Weapons based on “fusion” and “synergy” effects, marrying databases to advanced weapons and sensor platforms require that any copycat replicate not just one aspect of the system, but all of it.  Those difficulties mean that for the present, the US will enjoy a temporary monopoly in the use of whatever this thingamajig is. But all monopolies are temporary and that’s a sobering thought.

Tip Jar.