Making Ushahidi

[Below is my Tech4Africa talk, given today in Johannesburg, South Africa, titled “How we built Ushahidi, w]

I’m used to talking about Ushahidi, and as all of you guys who frequently talk about your product or company know: it gets old spouting off the same old stuff over and over again. That’s why I’m excited about today and for being invited to this excellent conference, since I’ll be telling the backstory, the quirks and funny bits that got us to this point and made our Ushahidi culture what it is today.

This is my story of Ushahidi – Of a small organization that dislikes hierarchy and being told what we can’t do. One that questions everything, embraces innovative thinking, takes risks boldly, and sometimes learns the hard way that we’re human after all.

In January 2008 I spent a week watching news reports roll in from Kenya, frustrated. Frustrated because I had said for years that “technology helps us overcome inefficiencies”. Wasn’t the madness of Kenya, in it’s post-election violence throws, it’s lack of media coverage, and lack of real information just this? Why was I unable to do anything?

It turned out that I needed an idea, and for once I couldn’t come up with one on my own. That seed of an idea that grew into what you see today came from a simple bullet point by my friend and fellow blogger Ory Okolloh, asking if we could map reports of violence around the country. Thus Ushahidi was born.

I’m going to walk you through three defining moments for our organization, and our platform, not all of them pretty, but which make us who we are.

1. Let’s look at the ad hoc cast that got it started:

The Ushahidi Team - circa Jan 2008

Ory Okolloh – lawyer, blogger and Kenyan political pundit
Juliana Rotich – renewable tech geek, blogger and database admin
David Kobia – developer and top Kenya forum webmaster
Daudi Were – blogger and web guy
Erik Hersman – Africa tech blogger, web guy
Others – a various cast of tech and non-tech people swarmed around the first Ushahidi deployment in Kenya, helping with small tasks and then disappearing.

Key points:

  • You’ll notice that there was not a single one of us who had any humanitarian experience
  • None of us had taken part in any open source project. (v1 was built in .NET)
  • Most of us were self-employed, running our own businesses or consulting, and didn’t like working for big companies.
  • The only common denominators that we shared was our love of our home; Kenya, and the ability to blog.

Thus, we felt we were the best placed to create an African open source platform for crowdsourcing information, our tech gift to the rest of the world.

We didn’t think of that at all actually. Instead we were madly Skyping, emailing, wireframing and coding over a 3 day period to get something up as quickly as possible.

We were brutal about every decision:

  • If it wasn’t absolutely necessary, throw it out.
  • Pick a name, any name, we don’t care if non-Kenyans can’t say it, just get a domain up asap
  • Launch this app, it’s functional, we’ll fix bugs and features on the fly
  • No one has a short code for us yet? Screw it, it’s not worth waiting, we’ll get one eventually.
  • Money, what’s that for? Media budgets are overrated, we’ll blog it.
  • We don’t have a logo. Oh well… Launch already!

How our team came together, the way we made those initial decisions and how we interacted and leaned on what would become our community was defining. It still colors how we operate, our organizational communications and our community focus.

Lessons learned:

  • This taught us to keep a shallow and wide decision-making structure so that everyone had access to all the information about ops or platform that they desired. Anyone was empowered to make decisions, since thy understood the macro-game.
  • Release code early, it’s better to have it out and being tested and worked on in the real world, than hidden away in a sandbox somewhere.
  • If you want it done, build it yourself, don’t put it off onto another team member.
  • Community = success
  • No money, no worries. Build good stuff and good stuff happens, money follows.

2. Technology is only a tool

allocation

No background in open source projects meant that we had little experience in how to engage programmers, designers and the help needed to get things moved from that initial .NET build into an open source language. David and I were trying to decide what language to write this in, and we ended up picking PHP over Python since we thought more African programmers would be proficient in it.

David wasn’t a PHP guy (yet), so the early helpers, the volunteers like Jason Mule, Henry Addo and Chris Blow were a huge help in making the decision to go with the Kohana framework and a myriad of other decisions.

3 months later we announced v0.1 of “THE NEW AND REBUILT USHAHIDI PLATFORM!”

We were very excited, after all, wasn’t this the platform that would save the world? And we were ready to show the world just how it could be done. Gamely mounting our white steeds we charged into a deployment of Ushahidi in the troubled North Kivu region of the DR Congo.

Echoes of that failure splatting against the ground remind us still, today, of the complexities of the space we build software in. We learned from those lessons though, and Ory wrote a good blog post making sure that it was shared within and without.

Lessons learned:

  • Technology is only 10% of the solution needed. The rest is administration and messaging.
  • Stick to what you do well. Our team is built to build software, not be a deploying organization
  • (caveat! We do help in deploying rarely, like Haiti and Kenya, but we now pass those off, or partner)
  • Own your failures publicly, learn from them.

3. Enter the failephant!

The Ushahidi Failephant

Only a few months later, after the DRC debacle, we were rested and ready to fail again.

Al Jazeera had used the alpha version of the Ushahidi platform in Gaza, a group of organizations and individuals were deploying it to monitor the worlds biggest elections in Indian, and we had a number of groups in East Africa testing it out.

Our model was that we had a small team at Ushahidi whose job was to come up with and guide the core architecture of the platform. Volunteers also worked on core, but were also encouraged to extend the platform in their own ways. It was working very well, and still does.

We were ready to release the code publicly.

Before I say anything, let’s revisit that point earlier about none of us having eroded on an open source project before…

Preperations were made, blog posts were written, tweets were tweeted – and we got lambasted by one of the guys we respect a great deal in the open source community. Rabble called us out on all the things we did wong.

– The code repository was behind a user/password wall
– We weren’t available in the normal programmer channels like IRC
– Hard to plug into the rest of the dev community

Our team went to work, madly working over the next 12 hours to get our stuff straightened out. Finally I wrote another blog post, introducing our failephant mascot and apologizing for our ignorance and missteps.

Lessons Learned:

  • Listen and apply that listening to real changes
  • Again, own your failures. Fix things that are wrong.
  • It’s okay to think different in how you execute on a project as long as you don’t stray from the spirit of your community and self
  • .

Finally, I’ll end with this.

We’ve learned that technology does overcome inefficiencies, but that it still takes people to make it happen.

We’ve learned that more people need to buck the status quo, that questioning everything makes us better.

We’ve learned that Africans can build world-class software, and to expect nothing less.

Ushahidi Comes Full Circle in Kenya

It’s been hectic lately… In the course of one week I’m going from the madness that is running any situation room for a major Ushahidi deployment (Uchaguzi), to what is looking to be one of Africa’s best tech conferences (Tech4Africa).


(video by Jon Shuler)

Uchaguzi: Monitoring Kenya’s Referendum Vote

Uchaguzi is a deployment of the Ushahidi platform that marries up traditional election monitoring groups and practices with voices from the crowd. It was an experiment in a more holistic approach to monitoring an election.

Our goal is to make this an election monitoring platform that can be used by anyone (at least in E. Africa), as a mixture of the core Ushahidi platform, with a package of customized plugins that do things such as:

  • Map known election monitor phone numbers to specific locations
  • Content-map the election monitoring number codes into an automated full report
  • Use shape files to get make reports not just point-based, but heatmapped
  • Ticketing system for escalated items
  • Ability to mark items as “actionable” and/or “action taken”

We started Ushahidi 2.5 years ago here in Kenya to crowdsource and visualize some of the stories coming from ordinary people in the midst of Kenya’s post election violence. Last Wednesday the whole country went to the polls again, this time to vote “yes” or “no” on a referendum for a new constitution for the country – arguably something even more important than a politician who will only be in office for 5 years.

Being Ushahidi, and this being Kenya, we were ready to do our part. This came in the form of Uchaguzi, a deployment where we partnered with local groups like SODNET, Twaweza, CRECO and HIVOS. Ordinary Kenyans and election monitors alike could send in text messages to a local shortcode, which was widely advertised before the date. (read more here)

IMG_1589

Over 50% of all incoming reports were verified in real-time, and an overwhelming 60+% were reports that things were going well. A win for both the deployment and the country!

A Thank You

Through a combination of great partners and a huge volunteer outpouring of time at the iHub, we were able to manage the inflow of information, mapping and verification.

The Uchaguzi project brought more than 70 volunteers to the iHub August 3rd and 4th (with at least 12 others joining remotely). Volunteers helped map and process over 1400 messages as well as assisted our team of Ushahidi developers fix bugs that popped up during the Uchaguzi deployment. The volunteers met the challenge with incredible enthusiasm, focus, patience, and a spirit of fun! We couldn’t be prouder to have such a wonderful Ushahidi community!

“We” isn’t just the Ushahidi team. Yes, deployments like this do take some time to customize and we did build some new functionality in (than everyone now has access to use), but it’s largely not the technology, it’s the people. The 80+ volunteers, tech and non-tech alike, were amazing and came through in a big way. Not enough can be said about Jessica Heinzelman, Ushahidi intern for this summer, who wrangled all of the volunteers and operations for the situation room.

Media Hits

Fast Company
Christian Science Monitor
Business Daily Africa
UN Dispatch
CNN iReport
All Africa
Reuters
Internews

SwiftRiver 101 at the iHub

Jon Gosier 3

Jon Gosier is the founder of Appfrica Labs in Kampala, Uganda, he’s also a Senior TED Fellow, a great African tech blogger and a good friend. I’m fortunate enough to work with him at Ushahidi too, where Jon heads up the SwiftRiver initiative with his team in Uganda.

Screen shot 2010-06-15 at 1.32.30 AM

“SwiftRiver is a free and open source software platform that uses algorithms and crowdsourcing to validate and filter news.”

SwiftRiver v0.2.0 (Batuque) is out. There’s a new plugin structure called “turbines” already sporting 3 new ones for natural language processing, Google’s Language Services and TagTheNet. (see video below for more)

Jon is in Nairobi, he’s here to lead two SwiftRiver 101 classes at the iHub tomorrow. The morning session is for non-techies, or anyone who just wants a primer on the platform. The afternoon session is for programmers who want to see how they can get involved.

Join us!

Getting Started with SwiftRiver – Batuque from Ushahidi on Vimeo.

The People You Work With

There’s no greater joy in (work) life than doing what you love with people that continually amaze you and with whom work isn’t considered work.

Two years ago none of us would have realized that an ad hoc group of blogging friends and techies would grow and become an organization of our own. I don’t work at Ushahidi due to the tech or the challenges, though both are great perks. I stay here because of the people I get to work with every day (virtually).

This is a picture of the Ushahidi core team (minus myself). It’s been a pleasure to work with each of them, even through the hard stuff.

Working on Ushahidi’s Haiti Response

Last Wednesday started out pretty normal for me. Then it stopped. The US-based members of the Ushahidi team informed me of the earthquake in Haiti, and then the madness began… 6 days later, what’s happened?

Ushahidi is heavily involved in mapping and integrating crowdsourced information from Haiti into an aggregated map that is being used by both people on the ground who need help and those who can provide relief. Teams of volunteers in Kenya, Uganda and the US have been working to solidify the platform and make this effort work. Keep up-to-date in our Situation Room and our blog.

Though it’s not a completely accurate description of what we’re doing, it’s close: We’re running what’s basically the 911 system for Haiti through a local shortcode on the Digicel network 4636. More on the 4636 number and campaign.

How you can help

Pass this message on, try to get it to people, media and organizations IN Haiti:

“In Haiti? Text 4636 (International:447624802524) on Digicel with your location and need. Report emergencies and missing persons.”

Help with open mapping of Haiti campaign through OpenStreetMap, CrisisMapping Network and CrisisCommons via the “Drawing Together” campaign.

Other links you should know about

Missing persons index
In-Haiti relief organization registry
Twitter Tracker/Filter
Crowdsourced facial recognition
OpenStreetMap Haiti
Crisis Commons Haiti
ICT4Peace – useful links wiki

It turns out this little experiment that started two years ago to crowdsource information from the public in Kenya during the post-election violence might have a future after all… :)

Thanks for your support, and for your help.

Heading Home

You might have noticed a lack of posts, or at least an erratic nature to my blogging over the last 6 weeks. This is due to the fact that I’ve been packing up and getting ready to move my family back to Kenya after living the last few years in the US. I won’t lie, it’s been pure madness and I apologize to all the people who I didn’t answer emails from, return calls or tweets…

Nairobi Skyline

I’m very excited though, as this is where it’s happening. Nairobi, where I’ll be living, is one of the four main technology hubs in Africa (Jo’burg, Accra and Cairo being the others). Nairobi is also home, that’s where I grew up and where I know the most people. It’s where I can relax and eat nyama choma (which I will do in abundance). :)

Why?

Checking out the latest Ushahidi build

Nairobi is also where Ushahidi started almost 2 years ago (wow, time flies!), which is providing the means and the reason for this move. There will be two main activities that I’ll be involved with:

  1. I’ll be working with the Nairobi programmers, designers, end users and members of the Ushahidi community in person. (These guys and gals are already rockin’ it, wait until you see the “Mogadishu” release of the code next week!)
  2. My other main focus is opening up an innovation hub, a physical nexus point for the tech community in the city. This hub will also be a place for us at Ushahidi to reach out and better engage with our own user and dev community.

There is a slew of big announcements coming out on the Ushahidi-front over the next week. I’ll be in the air for the biggest of them, but will link to it when I land. Keep an eye on our blog and twitter feed for more.

Leaving

Alex and Me

Orlando has been a good home base for us. While it might not be the tech capital of…well, anywhere, it’s still home to some amazing people and we’ve loved being a part of it. A big thanks for the friendships and a shoutout to those techies who have made Central Florida home:

Chris Scott, Alex Rudloff, Josh Hallett, Ted Murphy, Scott Allen, Dawn Hatton, Gregg Pollack, MindComet, Paul Lewis, Cory Collier, Bill Ferrante, Celly, Bill Dean, Etan Horowitz, Ryan Price, Eric Marden, Jason Seifer, John Rife, Ochie, Alex Spoerer, Doug White, Robert Jordan, Jim Hathaway, Robert Shade, Scott Toncray, Damian Scott, Chris Droessler, Allison Jordan, Gavin Hall, Gabriel Chapman, Jermaine Pulliam, Josh Lindsey, Marcelle Turner, Jon Shuler, (and many others I’m forgetting)…

I think of all the great BarCamp Orlando’s, BlogOrlando’s, Likemind and Florida Creative meetings… For a small tech community, it does throw an awfully big punch.

So, a big thank you to all the friends and family that have made this next chapter in our lives possible. We’re looking forward to it. All will be the same, just from the Kenya primarily.

Tags, Time and Location

On Friday I had a long conversation with Noam Cohen from the New York Times about Ushahidi and Twitter. He was doing some homework for an article he was writing on the increased value that geolocation data can add to the massive streams of data coming out of tools like Twitter, called “Refining the Twitter Explosion with GPS“.

A lot of our discussion was centered around location, especially since he was thinking of the Ft. Hood shootings and the value of location in determining useful information from the Twitter stream during that crisis. This is what we’ve built Ushahidi around of course, the idea that location and even small bits of information give us a better understanding of an unfolding crisis. This is just as true of mundane information, or trending topics in a locale, which is why Twitter is building a new geo infrastructure. It couldn’t be in better hands either, with both Ryan Sarver and Schuyler Erle on the team, what Twitter puts out will be top notch.

What was more interesting than just geographical references for information was the combination of two other big ways to parse this data: Time an Tags. We’ve started to see a lot more apps mixing time and location in the past year or two, and we’ll see more as the visualizations for it improve. Categorizing information, pictures and video by keywords (tags) have been around even longer.

TwitterThoughts

We need to see more combinations of tags, time and location in visualizations and platforms. I can’t think of anyone who does all three really well (if you can, please leave the link), though there are a number who do two of them incredibly well – including Flickr’s geocoding of images (tag + location), TwitterThoughts (tag + time) and TwitterVision (time + location), etc.

We have a widening stream of information. The lowered barriers for entry globally, and the encouragement by social tools, means we’re seeing exponential growth rates. Twitter alone saw an increase from 2.4 to 26 million tweets per day in just the last 8 months. We need some way to make sense of this information. Our ability to create information has far surpassed our ability to understand it in a timely manner.

Chris Blow outlined this best with a visual for Swift River for use in a presentation I did at TED this year:

information produced vs information processed

It’s a serious problem and one that only gets deeper with every month that passes. In most areas, it’s not a big deal, but when a crisis, emergency or disaster hits the misinformation and lack of understanding has very real consequences.

I’d love to see more work being done with all three: Time, Tags and Location.

Quick Hits around African Tech

Understanding what drives Mpesa agents
Growing the agent network is one of the most challenging parts of a mobile payment system.

“The number one cost for most agents was liquidity management – moving cash. Agents report a host of expenses, including bank charges, transport costs, and fees to aggregators who advance commissions and provide easy float/cash swaps for agents. On average, liquidity management consumed 30% of total expenses.”

Asynchronous Info, Disjointed Data and Crisis Reporting
Jon Gosier talks about Uganda’s riots and what he’s learned in the process.

Africa’s diaspora and the cloud
Teddy Ruge writes a great essay on the web and Africa’s diaspora.

“There’s a cloud gathering over Africa; a storm of connected thoughts and ideas that are pushing African countries violently forward. The Diaspora is using emerging web technologies in increasing numbers, frequency, and variety to stay connect with Africa, simultaneously charting a new digital course for it’s economic independence on the world stage.”

New Africa broadband ‘ready’
The BBC Digital Planet team is in Kenya and doing a knock-up job of interviewing people about what’s going on around the tech space there.

Emmanuel Kala in Nairobi
(Note: all the people in the BBC “in pictures” for this day are part of the Ushahidi extended dev team in Kenya)

Mobiles offer lifelines in Africa
Ken Banks writes about mobile phone growth and development in Africa, stating “Africans are not the passive recipients of technology many people seem to think they are.”

Hanging out with Ghanaian Devs

Though I’ve been busy running around getting ready for this Maker Faire Africa event, I’ve also had some nice chats with the a few of the programmers and designers in Accra. I know there are still a lot that I haven’t met, but this has been a good start.

I’ll start out by saying this: one of the most disappointing things that I’ve come across here is the lack of community within the tech scene. There is no mailing list, forum, or other digital touch-point for the devs to keep in touch with each other and discuss pertinent local issues. I know just how valuable this is due to my involvement with the Skunkworks tech community in Nairobi.

Offsetting that is the fact that they do have places like Busy Internet and AITI, two facilities that are well-known for supporting the tech community, that act as nexus points for tech meetups and user groups. There are also a number of good tech outfits with quality programmers.

First impressions leave me excited about the talent, but surprised at the lack of connectivity within the community.

Ushahidi meetup

Tonight we had an Ushahidi meetup night in Accra over at the Suuch Solutions office. It’s a great location, butting up to a hotel with nice seating for a get together like ours. We had a couple devs from BusyLab (which I’ll write a full post on soon), a couple from Succh, as well as Henry Addo from our team.

Ushahidi Ghana Meetup

It looks like we have a couple new guys to help out on some interesting parts of the platform. George is an HTML markup guy who is itching to get his hands into some design work, and Chinedu is going to dig into the API with Henry.

The BugLabs Device

We also had a chance to break out the Bug kit from BugLabs, a completely modular hardware device that can be programmed using Java. I’m not sure who will be having the first crack at it, but everyone was amazed with it. What geek doesn’t like this type of stuff? I mean, accelerometer, GPS, camera, LCD, WiFi and control unit. It’s just such a crazy-cool device.

Buglabs kit in Ghana

Blogging this week

This is a courtesy post so that you know most of my blogging this week is taking place at AfriGadget due to being one of the organizers for Maker Faire Africa coming up this weekend.

I’m also doing some work on the “FLAP Bag Project“, testing out modular, solar and light-equipped bags in Ghana, Kenya and Uganda with Timbuk2, Portable Light and Pop!Tech.

We’ve got a big release of Ushahidi coming up this week too, so keep an eye on the Ushahidi blog where I have another write-up coming.