Simulating low bandwidths: how to make sure your apps work in the field

I’m going to write about four ways to simulate a slow internet connection and a bit of background about why you’d want to do it. Simulation is great but I’ll say this now: there’s no substitute for testing stuff in the field. However, before you release to the team on the ground or grab your bag and hop on a plane, read this.

Why simulate low bandwidths?

Aptivate builds online software for people in the international development sector. Our users are in places where internet connections are slow and unreliable – we need to make sure our stuff works for them.

More people are accessing the web using mobile handsets or mobile internet connections (3G dongles and tethered phones). The bandwidth, latency and stability characteristics of these links are very different to the “always on broadband” that most developers target.

Finally, we’re involved with CrisisCamp London over the next few weeks – part of an international effort by dedicated volunteers to provide remote technical support and build tools for individuals and organisations working in Haiti. What they build needs to work in that environment – hopefully this post is going to be useful.

So, if you think your technology is going to be used in the scenarios above, read on. If you’re just awesome and like to know about this stuff, read on too…

Four ways to simulate a slow connection

There are probably a few more but I’m on a plane from Rome to London to go the the first CrisisCamp and can’t think properly!

1. A profiler + simulate it in your brain.

This is particularly useful if you’re developing for the web.

There are a few online and in-browser tools that give you a breakdown of the resources your website is using.

• Combination of Yahoo’s Yslow + Firebug for Firefox
• Safari or Google Chrome’s built-in (webkit based?) inspectors
• Pingdom tools online

Three important things to get from these tools is:

The overall bandwidth is a feasibility test. We say aim for 25k per page, but use your own judgement – how fast is your user’s connection, how long will it take for them to get to something useful (hint – if it’s longer than 5-10 seconds: #FAIL)

The number of requests also gives you an indication about performance over high latency or intermittent connections – in short, use fewer objects and cache them when you can.

Finally, if you’ve got a network usage meter (I have a noddy one running that comes with iStatMenus on the Mac ) you can get a rough idea of how much bandwidth an app is consuming (should work fine even if you’re developing an app in a mobile simulator). I’ve seen stuff for Windows I can’t remember, on Linux you could use BWM or get fancy with logging modes in IPTables – Google for more.

That’s it.

2. Use Aptivate’s online Low Bandwidth Simulator

This technique is only useful if your site is accessible from a public URL. It only simulates bandwidth, not latency or packet loss.

We make Loband, and online service that strips the junk out of webpages and gives you a compressed, simplified version that works better on slow links.

As part of the Loband code, there’s a simulator which you can access here.

You plug in the URL of your site, select the bandwidth you want to simulate and hit go. I haven’t tested it recently with any serious AJAX/HTML5/Flex/Flash stuff so your mileage may vary if you make heavy use of these tools.

Do what a user would do with your app and see if it’s usable.

That’s it.

3. Use the Sloppy desktop Java app

This technique is great if your site is running on a local dev box or even if it’s online. It only simulates bandwidth, not latency or packet loss.

Make sure Java lives on your machine. Download Sloppy. Run it, start it, point it at your app.

Do what you would have done with 2.

That’s it.

4. Get a machine, (maybe two network cards) do IP traffic shaping.

This technique is the best of the bunch: you can simulate bandwidth, latency and packet loss and do so for anything running on your machine or LAN. That’s anything: browser apps, mail clients, Skype, mobile simulators etc. It’s not hard but is a little fiddly. There are two broad ways you could do this: for yourself on a single machine or, for a bunch of people on a LAN.

Quickly,  to do it for yourself, on your own machine to do app testing: if you’re running FreeBSD / MacOSX,, follow Bjørn Hansen’s tutorial.

It gets a bit trickier if you want to do it for several machines at once.

What we’re trying to do is turn a machine with two network interfaces (NICs) into a “router”. Traffic goes in/out of the first interface at normal speeds, but the traffic goes in/out of the second interface at user-selected levels of crapness (bandwidth, latency, packet loss)

Relatively speaking: this is easy on a Mac / BSD box, trickier on Linux and hard on Windows. While most laptops actually have 2 network interfaces (wifi + ethernet) – I normally do this with a desktop that’s got 2 NICs  or a laptop + a USB / CardBus/PCMCIA NIC.

On a Mac/BSD you’re going to be using ipfw to control the dummnynet traffic shaper. Man up to find out more. In short: ipfw’s a firewall that classifies packets (e.g. by which port or IP they’re going to) into “flows”. Dummynet takes a flow and sticks it in a “pipe”. A pipe emulates a link with given bandwidth, propagation delay, queue size and packet loss rate.

….how on earth do we get this working?

There are better tutorials than I can write quickly here and  here. But in brief:

In reality, this always takes me half an hour to get right – I’ve never had this go smoothly first time.

First things I check if it’s not working:

• Is OSX / BSD doing some daft routing / automatic internet connection sharing that messing with your ipfw settings?
• Are you routing using the right interfaces? I’ve actually got 7 network interfaces that show up in ifconfig to choose from (firewire, bt, vm, wifi, ethernet etc.)
• bit/s and Byte/s are quite different…
• Don’t despair, it will work, there’s pictures of me doing it here.

Typical bandwidth / latency / loss scenarios

The key commands you’ll be running to set parameters will look like:

ipfw pipe 1 config bw 50Kbit
ipfw pipe 1 config delay 200ms
ipfw pipe 1 config plr 0.2

The three variables you have to play with are bw (bandwidth) plr (random packet loss rate) and delay (latency). Here’s a super-rough guestimate for some typical scenarios, please advise if I’m way out or there are other common scenarios:

Scenario	Bw (Kbit)	delay (ms)	pr (ratio)
2.5G mobile (GPRS)	50	200	0.2
3G mobile	1000	200	0.2
VSAT	5000	500	0.2
Busy LAN on VSAT	300	500	0.4

What about windows and linux?!

I promise to update this bit with more info when I’ve got Linux, Windows boxes and Chris to hand.

In short though: with linux it’s the same idea, machine with 2 NICS, get them routing, use IPTables and the linux traffic shaper, tc. It’s not as good as dummynet (no packet loss IIRC) but gets the job done. For Windows, I’d honestly have to do some more research, last time I tried it, I just pulled out my Macbook.

Please add any tips and corrections of  below!

Tariq

Tags: bandwidth simulations, loband, low bandwidth, Sloppy, YSlow

This entry was posted by tariq on Saturday, January 23rd, 2010 at 16:41 and is filed under Appropriate Technology, bandwidth. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.