Comments on: Never Good Enough: Speed (pt 1/3)

By: Ben Colmery

Ben Colmery — Wed, 30 Sep 2009 22:54:23 +0000

I'm completely with you on this issue. From the standpoint of trying to get grants from online funding organizations for development in low bandwidth areas of the world, I have gone bonkers over some of this stuff. To the point where it prompted me to write this post as a guide to thinking about the issue, from the perspective of funding organizations. Web Design for Donor Organizations for Low Bandwidth

By: Louis

Louis — Thu, 17 Sep 2009 06:16:58 +0000

i say we should all just move to HTML5 i know this does not solve the latency issues but gives developers more flexibility in dealing with the bandwidth and latency issues and probably improves the user experience too.

By: Jeff Rafter

Jeff Rafter — Wed, 16 Sep 2009 14:50:07 +0000

@Alex Low earth orbit satellites and WiMax might have an impact sooner, at least in Malawi. Last year there was a lot of talk about the possibility of tracking satellites to reduce latency… but like you said, equipment purchase is expensive. I thought I should bring up one more point which is the fact that HTTP operates over TCP/IP which is a lossless protocol and therefore requires acknowledgment of packet receipt. To achieve this TCP/IP requires a packet buffer to be kept to hold onto packets until they are acknowledged (which we know takes a long time). The bad part: the default packet buffer is only 4K. Doing the math this means that you can only achieve ~64Kbps max when using TCP/IP over Sat. Fwiw, Skype generally uses UDP sockets (and you can tell when you get a direct TCP/IP connection because the call jumps). Some of this is solvable but it won’t be solved for a while.

More reading here:
http://www.isoc.org/inet96/proceedings/g1/g1_3.htm

This means that my solution does have serious limits and caching will win in repeat cases, hands down. @Christian’s .htaccess is great for that and also enables HTTP compression which is even better. Just make sure you have mod_deflate enabled so that his .htaccess can take advantage of it (which based on your speedup for initial visits, I am guessing you do).

By: Alex

Alex — Wed, 16 Sep 2009 08:54:47 +0000

@Christian – thank you for the .htaccess (I put it in my httpd.conf to speed things up a bit more). For my extranet services, it cut initial requests down from 37 to 25; requests on repeat visits are down to 5 (the site is mostly static css and js). Page load times here in Tanzania are down to about 6-7 seconds on initial visits, and 2-3 seconds for repeat visits. Thanks!

Due to factors explained by Jeff Rafter, reducing the number of requests has a dramatic effect in Malawi and Tanzania, where we share bandwidth with much of East Africa on a single satellite that hovers over the Atlantic (I’ve seen working satellite dishes pointed almost parallel to the ground here in Tanzania). There’s also (supposedly) some connectivity through another satellite over the Indian Ocean, but the latency issues remain. Sometimes round-trip pings here are up to 2000ms.

@David K: The SEACOM fiber optic line landed in Tanzania about two months ago, but most users are not seeing a difference yet. Startup costs are difficult for local ISPs to manage, i.e., upgrading and physically moving equipment to the cable landing site, and, it is rumored, the fact that ISPs need to pay up front for a year of access to bandwidth – many just don’t have the cash reserves to afford that. I expect that the EaSSY and TEAMS fiber lines scheduled to arrive next year (2010) will face similar startup issues.

I think it will probably be up to another year or so before many users in Dar es Salaam start seeing any difference (ISPs need to upgrade infrastructure, build up cash reserves, contracts with sat. providers need to expire, new fiber competitors may arrive). With the possible exceptions of Uganda, Rwanda, and portions of Kenya (Nairobi) and SA (Jo’burg), it will likely be years, if not decades, before inland East African users see any impact. I’m not aware of any new fiber coming to West Africa anytime soon. Sadly, it seems that in terms of lit fiber, Africa will be a “Dark Continent” for the next several years (see also http://bit.ly/17rZCz ).

By: Wayan @ Inveneo

Wayan @ Inveneo — Tue, 15 Sep 2009 16:07:29 +0000

Here are my bandwidth observations at various places in Africa: http://www.flickr.com/photos/dcmetroblogger/tags/bandwidth/

My favorite, which probably approaches Liberia, is this 7.47Kbps download speed in Nigeria: http://www.flickr.com/photos/dcmetroblogger/3799347808/

The speedtest I use is http://www.internetfrog.com/mypc/speedtest/

By: Miquel

Miquel — Tue, 15 Sep 2009 15:32:02 +0000

Yeah, I'm aware of this just a little bit. Same situation in all of DRC (a country of 60 million) which is why we started doing what we're doing with Maneno. In fact, dealing with this kind of high latency, narrow bandwidth is one of the cornerstones to the project.

By: Qurbajoog

Qurbajoog — Tue, 15 Sep 2009 04:31:18 +0000

@Hash you may want to talk Prof. Vivek Pai and his grad students who developed HashCache http://www.technologyreview.com/web/22119/ & http://www.cs.princeton.edu/~vivek/

By: Adam Hooper

Adam Hooper — Mon, 14 Sep 2009 23:04:20 +0000

Many site optimizations are extremely “easy wins”, e.g. (on http://drc.ushahidi.com):

1. Setting “Expires” headers on static assets (1-2 lines of web server config)
2. Using a Content Delivery Network (particularly one which works in Africa, if there is one–if not, there’s a market here!). They’re very easy to set up, though they cost money. As a last resort “poor man’s” CDN, use multiple asset hostnames (e.g., “assets0.ushahidi.com”, “assets1.ushahidi.com”, etc., all pointing to the same web server) so the browser can parallelize downloads. Downloading eight files at a time is much faster than two, especially in high-latency environments (such as Africa).
3. Combine common JS into a single file. For instance, putting OpenLayers.js and jquery.js into the same file, if they’re used on every page, can probably win you half a second off the initial page load in Africa. Ditto for CSS.
4. Serve images, CSS, and JavaScript from a different domain than the website’s domain: if the website uses cookies and serves all files from the same domain, the browser has to send the cookie with each request (which costs the browser upstream bandwidth, another bottleneck). If your cookies are tied to “*.ushahidi.com”, then register another domain name, e.g. “ushahidi-assets.com,” for this big win.
5. Skip images or make them into CSS sprites. The navbar on drc.ushahidi.com is a perfect place to use a sprite; the language flags at the top of the page could be made into a sprite, also. These aren’t enormous wins, but they’re not difficult and the improvements would be easily measurable.

These basic tips aren’t exactly difficult to implement: they can probably all be finished with cursory testing within a day with one developer’s time. The effort would make initial page load several seconds faster. And #1 alone would be subsequent page loads many times faster.

I’m not the cleverest guy: I’m just parroting what Google’s “Page Speed” Firefox add-on says. It’s a fantastic tool: get it at http://code.google.com/speed/page-speed/.

After that you can work on the appearance of speed; there are some tips for that at http://developer.yahoo.com/performance/rules.html.

But every change is meaningless without metrics, right? Test your site’s actual performance at http://www.webpagetest.org. It may have a stupid name; it may be near-impossible to find on Google (bookmark it!); but I guarantee it will help you make your website faster.

Don’t worry too much about database calls–at least, not until you get these big fish out of the way. Even if the page takes 3 seconds to render on the server side, that’s nothing compared to the 97 subsequent seconds it takes to load all the assets and run all the scripts.

And as for, “do we need to redesign this from the ground up?” The answer is no. Every site–and every redesign–experiences this kind of growing pains. A software engineering mantra: “features first; optimizations after.”

I would expect the above list (and tools’ recommendations) to make ushahidi.com twice as fast or even faster, with moderately low risk and no negative side-effects.

Good luck!

By: Christian Siegert

Christian Siegert — Mon, 14 Sep 2009 22:34:22 +0000

Here is my .htaccess http://pastie.org/616785 . I hope it helps someone.

If you want to see what of your website needs optimization, install Google’s Firefox add-on “Page Speed” websites http://code.google.com/speed/page-speed/

By: David K

David K — Mon, 14 Sep 2009 22:09:53 +0000

@christian, I’ve played around with the 304 not-modified status header and caching on the client side – definitely helps. CSS Sprites have also been a major help because then only one call for images is called – since all the images are in a single css sprite (master image) – http://css-tricks.com/css-sprites/.

I think my only issue at this point is that the rest of the world is quickly forgetting about designing for slow speeds as everyone else gets on the broadband bandwagon. Can anyone give an estimate for how much longer African countries have to endure this??

@Ssembonge if hypothetically one spent 5 hours extra on the net every week waiting for something to load, that’s approximately 10 days out of the year that one could have spent doing other productive things. Our continent can’t afford to waste anything at this point, especially time since we’re so far behind.