NewAssignment.Net

User login

Join NewAssignment.Net’s Facebook Group.

WHERE WE ARE

BeatBlogging.Org

13 beat reporters build social networks into their beats.

OffTheBus.Net

Help us cover the presidential elections at OffTheBus.net

Broowaha.com

A citizen journalism network to experiment with distributed reporting.

Readable Laws

Explaining Congressional legislation in plain English.

Assignment Zero

Published in Wired News.


Want To Learn More About NAN?

Check out this 7-minute interview with Jay Rosen. Or watch the full presentation at the Berkman Center, also available in MP3, or this five part nicely edited
series
.


Browse archives

« October 2008  
Su Mo Tu We Th Fr Sa
      2 3 4
5 6 7 8 9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28 29 30 31  

Spying Goes Open Source

by Kevin Friedl on December 4, 2006 - 7:36am.

Proponents of user-generated media make a lot of lofty claims about the potential of open source. Under the right conditions, it can be a great way of gathering and sorting information. So could it also be used by some of the biggest information sifters on the planet—U.S. intelligence agencies?

This week’s New York Times Magazine cover story, “Open-Source Spying,” written by Clive Thompson, reports on efforts by the intelligence community to replace its current clutter of outdated technologies with more a responsive and bottom-up “Spying 2.0.”

“Spies are beginning to wonder why their technology has fallen so far behind,” writes Thompson. “The answer may lie in the interactive tools the world’s teenagers are using to pass around YouTube videos and bicker online about their favorite bands. Billions of dollars’ worth of ultrasecret data networks couldn’t help spies piece together the clues to the worst terrorist plot ever. So perhaps, they argue, it’s time to try something radically different. Could blogs and wikis prevent the next 9/11?”

Thompson began pursuing the story as a follow-up on how American spy agencies have adapted since the intelligence failures that led to the terror attacks. Even as he learned more about the vast technical challenge of combining incompatible databases from different agencies, it became obvious that this was only part of the story.

Intelligence analysts, Thomspon said, “are very interested in data sorting and how the power of mobs could be applied to intelligence. The more I realized they were taking these ideas seriously, the more interested I became.”

“You wouldn’t think that these guys, several steps from the president in some cases, are thinking about this stuff, but they are. Intelligence is really about connecting dots, which is exactly what this sort of Web 2.0 social network software is all about.”

Applying these new technologies to sort intelligence data will require overcoming the usual bureaucratic inertia, plus the spying community’s internecine agency rivalries and the cult of secrecy that still lingers like a hangover from the Cold War. But the potential reward is a self-organizing network capable of tracking small and resilient terrorist cells by quickly and openly sharing information between agents through wikis, blogs, linked documents and online discussions—much the same tools available to citizen journalists.

The analogy between spying and journalism is an old one—after all, there needs to be a bit of spy in every investigative journalist and a reporter in every spook—so maybe it’s no surprise that the two are applying the lessons of open-source in similar ways.

“I write for The New York Times. I love The New York Times,” Thompson said. “But they’re a huge corporation. You can see them trying to adapt to this stuff. ‘Should we have blogs on our web sites? What the hell do we do?’ … The debate is the same one you see in large manufacturing companies, spy agencies, big media companies and entertainment companies.

Thompson also noted that, although their ultimate ends are different, the distinction between how spies and journalists operate could break down even further as top-secret information becomes “less useful than it used to be.”

“Beat cops in Indiana might be as likely to uncover evidence of a terror plot as undercover C.I.A. agents in Pakistan. … The most valuable spy system is one that can quickly assemble disparate pieces that are already lying around.”

The challenge, then, for anyone trying to gather information into a coherent picture of the world, as both spies and journalists do, has become dealing not with a drought but with a deluge of data. As the ocean of information available online continues to rise, it could sink those who haven’t learned how to navigate it.

Kevin Friedl is a writer living in New York. He has worked at The Atlantic Monthly, Columbia Journalism Review, and Seed magazine. He is currently an assistant news editor at Forbes.com.