It is known that Twitter users are not a fair sample of the whole population in any given city, but a sample with socio-economical and self-selection biases (young, affluent, tech-savvy, etc.) It is also known that a significant population do not use or have no access to social media channels like Twitter.

In addition to the above biases, we have two additional levels of bias introduced while collecting the data:

1. Twitter’s public API does not provide all tweets. It only provides access to at most 1% of all tweets via its public streaming API [1].
2. To collect all tweets from a spatial region using the public API, we define a geographic boundary (e.g. the city of Nairobi). Therefore, by design we only collect geo-tagged tweets, which are a sample of all tweets sent from a the specified region.

We are aware of these biases, and other researchers have started investigating them. For instance, the authors in [2] performed a statistical comparison of data obtained via the streaming API with the entire data stream of Twitter. This sort of analysis is restricted to those who have access to the entire Twitter data feed (e.g. by working with Twitter or paying for the data). So for the moment we are limited to what Twitter offers for free.

This said, we believe that the data still offers valuable information, and we are exploring this in the context of African cities, starting with Nairobi. The visualization itself is useful to create a dialogue (of which this post is an example) and we hope others find it interesting to browse the visualization as we add features in the future. Besides this community use, our current work includes characterizing active contributors of geo-tagged tweets, popular urban areas among these users, and urban “gaps” related to the socio-economic factors mentioned above, through a combination of methods.

Finally, to give you some context on the data we have today, as of April 2014, we have collected 680,000 geo-tagged tweets from 20,000 users. Let me remind you that the visualization was created with only 3 months worth of data, involving roughly 200,000 geo-tagged tweets.

[1] https://dev.twitter.com/docs/faq#6861
[2] Is the Sample Good Enough? Comparing Data from Twitter Streaming API with Twitter Firehose, ICWSM 2013 — http://www.public.asu.edu/~fmorstat/paperpdfs/icwsm2013.pdf