1.2 Million “osama” & “bin laden” Tweets…and Counting

 Posted on May 4, 2011 by Stuart Shulman

On the May 1, 2011 evening it was announced that Osama bin Laden had been killed, we started running repeated fetches against the Twitter API for the terms “osama” and “bin laden”. On May 3, we posted more than 1.2 million tweets in XML format. Since then, the live feed collection on DiscoverText keeps rolling along.

The Twitter API serves a maximum of 1500 items per fetch. The DiscoverText live feed scheduler can fetch as often as every five minutes. During the peak of the Tweet storm, running a single repeated fetch could not get 100% of the Tweets. The work around that produced these two large collections was to set up several repeating fetches in DiscoverText that all fed the same archive. The results are frankly more Tweets than anyone might ever need to understand this slice of the the micro-blogging public sphere during a critical juncture in world history.

1:46 PM UPDATE: Approaching 1 million tweets per archive

Post to Twitter Tweet This Post

About Stuart Shulman

Stuart Shulman is a political science professor, software inventor, entrepreneur, and garlic growing enthusiast who coaches U13 boys club soccer and in the Olympic Development Program with a national D-license. He is Founder & CEO of Texifter, LLC, Director of QDAP-UMass, and Editor Emeritus of the Journal of Information Technology & Politics. Stu is the proud owner of a Bernese/Shepherd named "Colbert" who is much better known as 'Bert. You can follow his exploits @stuartwshulman.
This entry was posted in general and tagged , , , , , , , . Bookmark the permalink.