In an anonymous industrial park in Virginia, CIA analysts who jokingly call themselves the "ninja librarians" are mining the mass of information people publish about themselves overseas, tracking everything from common public opinion to revolutions.
The group's effort gives the White House a daily snapshot of the world built from tweets, newspaper articles and Facebook updates.
The agency's Open Source Center sometimes looks at 5 million tweets a day. The analysts are also checking out TV news channels, local radio stations, Internet chat rooms - anything overseas that people can access and contribute to openly.
The Associated Press got an apparently unprecedented view of the center's operations, including a tour of the main facility. The AP agreed not to reveal its exact location and to withhold the identities of some who work there because much of the center's work is secret.
From Arabic to Mandarin, from an angry tweet to a thoughtful blog, the analysts gather the information, often in a native tongue. They cross-reference it with a local newspaper or a clandestinely intercepted phone conversation. From there, they build a picture sought by the highest levels at the White House. There might be a real-time peek, for example, at the mood of a region after the Navy SEAL raid that killed Osama bin Laden, or perhaps a prediction of which Mideast nation seems ripe for revolt.
Yes, they saw the uprising in Egypt coming; they just didn't know exactly when revolution might hit, says the center's director, Doug Naquin.
The center already had "predicted that social media in places like Egypt could be a game-changer and a threat to the regime," he said in an interview.
The CIA facility was set up in response to a recommendation by the 9/11 Commission, its first priority to focus on counterterrorism and counterproliferation. Its predecessor organization had its staff heavily cut in the 1990s - something the CIA's management has vowed to keep from happening again, with new budget reductions looming across the national security spectrum.
The center's several hundred analysts - the actual number is classified - track a broad range of subjects, including Chinese Internet access and the mood on the street in Pakistan.
While most analysts are based in Virginia, they also are scattered throughout U.S. embassies worldwide to get a step closer to their subjects.
The center's analysis ends up in President Barack Obama's daily intelligence briefing in one form or another almost every day. The material is often used to answer questions Obama poses to his inner circle of intelligence advisers when they give him the morning rundown of threats and trouble spots.
"The OSC's focus is overseas, collecting against foreign intelligence issues," said CIA spokeswoman Jennifer Youngblood. "Looking at social media outlets overseas is just a small part of what this skilled organization does," she said. "There is no effort to collect on Americans."
The most successful open source analysts, Naquin said, are something like the heroine of the crime novel "The Girl With the Dragon Tattoo," a quirky, irreverent computer hacker who "knows how to find stuff other people don't know exists."
An analyst with a master's degree in library science and multiple languages, especially one who grew up speaking another language, makes "a powerful open source officer," Naquin said.
The center had started focusing on social media after watching the Twitter-sphere rock the Iranian regime during the Green Revolution of 2009, when thousands protested the results of the elections that kept Iranian President Mahmoud Ahmadinejad in power. "Farsi was the third largest presence in social media blogs at the time on the Web," Naquin said.
After bin Laden was killed in Pakistan in May, the CIA followed Twitter to give the White House a snapshot of world public opinion.
Since tweets can't necessarily be pegged to a geographic location, the analysts broke down reaction by language. The result: The majority of Urdu tweets, the language of Pakistan, and Chinese tweets, were negative. China is a close ally of Pakistan's. Officials in Pakistan protested the raid as an affront to their nation's sovereignty, a sore point that continues to complicate U.S.-Pakistani relations.
When President Obama gave his speech addressing Mideast issues a few weeks after the raid, the tweet response over the next 24 hours came in negative from Turkey, Egypt, Yemen, Algeria, the Persian Gulf and Israel, too. Tweets from speakers of Arabic and Turkic contended that Obama favored Israel, while Hebrew tweets denounced the speech as pro-Arab.
In the following days, major news media came to the same conclusion, as did analysis by the covert side of U.S. intelligence based on intercepts and human intelligence gathered in the region.
The center is also in the process of comparing its social media results with the track record of polling organizations, trying to see which produces more accurate results, Naquin said.
"We do what we can to caveat that we may be getting an overrepresentation of the urban elite," said Naquin, acknowledging that only a small slice of the population in many areas being monitored has access to computers and Internet. But he points out that access to social media sites via cellphones is growing in such areas as Africa, meaning a "wider portion of the population than you might expect is sounding off and holding forth than it might appear if you count the Internet hookups in a given country."
Sites such as Facebook and Twitter have become a key resource for following a fast-moving crisis such as the riots that raged across Bangkok in April and May of last year, the center's deputy director said. The AP agreed not to identify him because he sometimes still works undercover in foreign countries.
As director, Naquin is identified publicly by the agency although the location of the center is kept secret to deter attacks, whether physical or electronic.
Naquin says the next generation of social media will probably be closed-loop, subscriber-only cellphone networks, like the ones the Taliban uses to send messages among hundreds of followers at a time in Afghanistan and Pakistan. Those networks can be penetrated only by technical eavesdropping by branches of U.S. intelligence, such as the National Security Agency - but Naquin predicts his covert colleagues will find a way to adapt, as the enemy does.