Chris Wilson
Aptivate Ltd, UK
My own opinions, not my employer’s
A presentation at AfNOG 2012

Download here:
Improving Network Management for Universities and SMEs

A local TV company has reported about us on the local news, and you can watch the video. (Thanks to Nadejda Loumbeva for finding it on YouTube!)

These are my notes from our setup for 2012, building on the virtualisation setup from last year.

We are hosting on CentOS 6 this year. I had a lot of problems installing the VirtualBox RPM, with this error message:

error: unpacking of archive failed: cpio: read failed - No such file or directory

That almost had me abandon CentOS, but it turned out to be a corrupt download file.

CentOS 6 seems to install headless, which saves us memory, but makes the configuration of VirtualBox more challenging. Luckily we had most of the VBoxManage commands from last year.

Remote headless installation

VirtualBox for CentOS doesn’t appear to have VNC support at all. You need to install the extension pack, to enable RDP support, to get nice remote access to the machines to do the installation:

sudo VBoxManage extpack install Oracle_VM_VirtualBox_Extension_Pack-4.1.14-77440.vbox-extpack

Configure the Master virtual machine, ensuring that its first network interface is bridged to the host’s physical Ethernet interface and promiscuous mode is enabled:

VBoxManage modifyvm Master \
	--memory 256 \
	--nic1 bridged --bridgeadapter1 p4p1 \
	--nicpromisc1 allow-all --vram 4 \
	--pae off --audio none --usb on --uart1 0x3f8 4 \
	--uartmode1 server /home/inst/.VirtualBox/Master-console.pipe

To attach a 20GB virtual hard disk image to the Master virtual machine for installation:

VBoxManage createhd --filename VirtualBox\ VMs/Master/FreeBSD.vdi \
	--size 20480 --format VDI
VBoxManage storageattach Master --storagectl "IDE Controller" --port 0 \
	--device 0 --type hdd --medium VirtualBox\ VMs/Master/FreeBSD.vdi

To attach a FreeBSD ISO image to the Master virtual machine for installation:

VBoxManage storagectl Master --name "IDE Controller" --add ide
VBoxManage storageattach Master --type dvddrive --storagectl "IDE Controller" \
	--port 1 --device 0 --medium FreeBSD-8.3-RELEASE-i386-dvd1.iso

You can remove the virtual disk from the virtual CD-ROM drive like this:

VBoxManage storageattach Master --type dvddrive --storagectl "IDE Controller" \
	--port 1 --device 0 --medium emptydrive

This seems to trigger a serious bug in FreeBSD that makes the system unbootable. So instead, you can leave it connected but change the boot order so that the VM boots from disk instead of CD:

VBoxManage modifyvm Master --boot1 disk

To enable an RDP remote display on port 5900, for headless operation:

VBoxManage modifyvm Master --vrde=on --vrdeport=3380

To start the Master virtual machine:

VBoxManage startvm Master --type headless

If your virtual machine refuses to start, with an error like this:

[inst@host1 ~]$ VBoxManage startvm Master
Waiting for VM "Master" to power on...
VBoxManage: error: The virtual machine 'Master' has terminated unexpectedly during startup with exit code 0
VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005), component Machine, interface IMachine, callee

It probably means that you’re running the command remotely without a GUI, and you forgot to add --type headless, so the graphical display can’t start. Thanks VirtualBox!

Install a caching nameserver such as BIND on the host, and when configuring the Master machine, set the DNS server to 10.0.3.2 to use the host’s DNS. This avoids problems using a DNS server on the network while all the clones are using the same IP address, resulting in long delays for SSH logins.

Remote virtual serial console

ConMan does indeed work for host to guest communication over the serial pipe, as an alternative to RDP. The following line enables the UNIX socket for the serial console, if you haven’t done this already (requires the virtual machine to be stopped):

VBoxManage modifyvm Master --uartmode1 server /home/inst/.VirtualBox/Master-console.pipe

The following line in /etc/conman.conf on the host enables the conman server:

console name="Master" dev="unix:/home/inst/.VirtualBox/Master-console.pipe"

To enable a serial terminal in a FreeBSD guest, add the following line to /etc/ttys:

ttyu0   "/usr/libexec/getty std.9600"   vt100   on secure

Then, after you start (or restart) conmand, you should be able to connect to the Master’s serial console like this:

conman Master

The escape sequence to terminate the conman session is &. But this still isn’t a good way to mass-control multiple machines because conman doesn’t like commands being piped into it.

Remote port-forwarded SSH management

You can set up NAT port forwarding to a second virtual network interface in the guest, configured by DHCP, like this:

VBoxManage modifyvm Master --nic2 nat --natpf2 ssh,tcp,,4400,,22

This is really helpful when all your virtual machines start off with the same IP address. As long as you have SSH keys, you can iterate over all of them, using the host IP addresses, and update their network configurations to give them unique, preassigned IP addresses.

FreeBSD proxy server configuration

To save bandwidth and speed up package installation, I added the following to /etc/profile:

export FTP_PROXY=http://196.200.223.1:3128
export HTTP_PROXY=http://196.200.223.1:3128
export http_proxy=http://196.200.223.1:3128

Bulk virtual machine management

This is a shorter and more efficient script to create the guest virtual machines by snapshotting and cloning the master image, resulting in less disk usage (faster host and guest cloning) and more efficient disk cache usage (faster guest operation):

create-vms.sh

#!/bin/bash

set -e
set -x

master=Master
snapshot="Clone Source `date --rfc-3339=seconds`"
VBoxManage snapshot "$master" take "$snapshot" --pause

cd ~/"VirtualBox VMs"
for i in {1..7}; do
	# echo $i
	vmname=VM0$i
	if [ "$1" = "-f" ]; then
		VBoxManage unregistervm "$vmname" --delete || true
	fi
	VBoxManage clonevm "$master" --name "$vmname" --options link \
		--options keepdisknames --snapshot "$snapshot" --register
	VBoxManage modifyvm "$vmname" --memory 256 \
		--natpf2 "ssh-$vmname",tcp,,$[4400+$i],,22 \
		--uartmode1 server /home/inst/.VirtualBox/"$vmname"-console.pipe \
		--vrde=on --vrdeport=$[3380+$i]
done

Note that you have to use a unique snapshot name for each new snapshot of the master image. VirtualBox doesn’t stop you from reusing an old name, but when you clone the master, you get a copy of the oldest snapshot with that name, not the latest one.

This script logs into each VM using the NAT port forwarding from the host, which works even when their IP address is not configured yet, sets their hostname and IP address and reboots them:

configure-vms.sh

#!/bin/bash

# set -e
set -x

guests_per_host=7

for host in 1; do
	hostname="196.200.219.$[200+$host]"
	for guest in `seq 1 $guests_per_host`; do
		nat_ssh_port=$[4400+$guest]
		guest_ip_suffix="$[($host-1)*$guests_per_host + $guest]"
		guest_ip="196.200.219.$guest_ip_suffix"
		guest_name="vm0$guest-host$host-ip$guest_ip_suffix.sse.ws.afnog.org"
		ssh root@$hostname -p $nat_ssh_port \
			-o "StrictHostKeyChecking no" << EOF
set -x
sed 	-e 's/hostname=.*/hostname="$guest_name"/' \
	-e 's/ifconfig_em0=.*/ifconfig_em0="inet $guest_ip netmask 255.255.255.0"/' \
	-i '' /etc/rc.conf
reboot
EOF
	done
done

Starting virtual machines automatically

We want all the virtual machines to start at boot time. My usual sudo trick doesn’t work in /etc/rc.local, but thanks to this blog post I found a way:

/etc/rc.local

su --session-command "VBoxManage startvm VM01 --type headless" inst
su --session-command "VBoxManage startvm VM02 --type headless" inst
su --session-command "VBoxManage startvm VM03 --type headless" inst
su --session-command "VBoxManage startvm VM04 --type headless" inst
su --session-command "VBoxManage startvm VM05 --type headless" inst
su --session-command "VBoxManage startvm VM06 --type headless" inst
su --session-command "VBoxManage startvm VM07 --type headless" inst

We also want the host to shut them down cleanly:

/etc/default/virtualbox-ose

SHUTDOWN_USERS=inst
SHUTDOWN=acpibutton

Assign unique MAC addresses

After creating the remaining virtual machine hosts, by cloning the first one, and if you update and copy out the virtual machine disk images again, it’s important to randomise all the MAC addresses of all the hosts. If two virtual machines have the same MAC address, routers will have problems learning their addresses. You can do that in bulk with this command:

for i in `seq 201 208`; do
ssh -t inst@196.200.219.$i '
./manage-vms.sh controlvm acpipowerbutton
./manage-vms.sh modifyvm --macaddress1 auto
./manage-vms.sh startvm --type headless'
done

Automatic Ubuntu desktop installation

We received 12 desktops with Windows in various states of repair, and decided to reformat them all and install Ubuntu. Since there were several hardware variations, I did this using Debian/Ubuntu installation preseeding instead of system imaging.

Preseed options which need to be decided before networking is initialised cannot be stored in the preseed file, so they must be
passed to the kernel on the command line:

• debian-installer/locale=en_US
• netcfg/get_hostname=desktop.sse.ws.afnog.org
• console-setup/ask_detect=false
• console-setup/layoutcode=us

This is the pxelinux configuration that we used:

/var/lib/tftpboot/pxelinux.cfg/default

label ubuntu
  menu label Install ^Ubuntu (10.04.4 alternate i386)
  kernel ubuntu-10.04.4-alternate-i386/linux
  append initrd=ubuntu-10.04.4-alternate-i386/initrd.gz url=http://196.200.219.200/tftp/ubuntu-10.04.4-alternate-i386/preseed.txt netcfg/get_hostname=desktop.sse.ws.afnog.org debian-installer/locale=en_US console-setup/ask_detect=false console-setup/layoutcode=us
  menu default

Using preseeding, we were able to automate almost all of the installation process, providing a nearly unattended install. I wasn’t able to preseed the partition selection for some reason; the instructions did not appear to work.

This is the preseed configuration file we used:

preseed.txt

d-i netcfg/wireless_wep string
d-i mirror/country string enter information manually
d-i mirror/http/hostname string 196.200.219.200
d-i mirror/http/directory string /ubuntu-10.04.4-alternate-i386/ubuntu/
d-i mirror/http/proxy string http://196.200.219.200:3128/
d-i partman-auto/init_automatically_partition select Guided - use entire disk
d-i partman-auto/disk string /dev/discs/disc0/disc
d-i partman-auto/method string regular
d-i partman-auto/purge_lvm_from_device boolean true
d-i partman-lvm/confirm boolean true
d-i partman-auto/choose_recipe \
       select All files in one partition (recommended for new users)
d-i partman/confirm_write_new_label boolean true
d-i partman/choose_partition \
       select Finish partitioning and write changes to disk
d-i partman/confirm boolean true
d-i clock-setup/utc boolean true
d-i clock-setup/ntp boolean true
d-i time/zone string Africa/Banjul
d-i passwd/user-fullname string AfNOG User
d-i passwd/username string afnog
d-i passwd/user-password password afnog
d-i passwd/user-password-again password afnog
d-i user-setup/allow-password-weak boolean true
d-i user-setup/encrypt-home boolean false
d-i grub-installer/only_debian boolean true
d-i grub-installer/with_other_os boolean true
tasksel tasksel/first multiselect standard, ubuntu-desktop
popularity-contest popularity-contest/participate boolean true
d-i pkgsel/update-policy select unattended-upgrades
d-i finish-install/reboot_in_progress note
xserver-xorg xserver-xorg/autodetect_monitor boolean true
xserver-xorg xserver-xorg/config/monitor/lcd boolean true

I ran into this issue while preseeding the Lucid desktop install: the installer reports an error and refuses to proceed:

http://server/ubuntu/lucid/dists/lucid/restricted/binary-i386/Packages was corrupt

The solution presented there basically worked. The steps I used were:

sudo cp -a ubuntu-10.04.4-alternate-i386/dists/lucid/restricted/binary-i386/ lucid-restricted-binary-i386.bugfix
cd lucid-restricted-binary-i386.bugfix/
sudo gunzip Packages.gz
cd ..
sudo mount --bind lucid-restricted-binary-i386.bugfix ubuntu-10.04.4-alternate-i386/dists/lucid/restricted/binary-i386/

If you use a Squid cache in front of your Ubuntu repository web server, for caching security updates, then it will cache the corrupted Packages file that you deleted in the above step, and the error will persist. I recommend you exclude the local web server from your Squid cache like this:

/etc/squid/squid.conf

acl localweb dstdomain 196.200.219.200
cache deny localweb

We haven’t done many deployments of Django applications on Windows, and it always seems to be a challenge. Integration with Microsoft IIS seems to be a non-starter. Apache works, but getting the right combination of Python, MySQL, Apache and mod_wsgi installed is very challenging. I thought our worries were over when we discovered BitNami djangoStack, which installs a well-tested combination of all the above, and we successfully deployed a project on this stack.

So I didn’t worry much about how to get our newest application running on Windows. How wrong I was. This little jolly turned into a two-day exercise in frustration that pushed all my Python, C, Java, Windows and SQL skills to the limit.

Learning 1

Never assume that deployment on a new platform will be easy. Especially on Windows. Software deployment is hard.

I had some problems with our deployment script, which is supposed to make deployment easier by scripting the steps necessary to install and configure the applications’ dependencies. This is very useful for allowing users without Django expertise to deploy applications, by reducing the number of steps required, but in many places they assume a Linux environment. I think I’ve fixed all of these now.

One of the tasks of this script is to create a virtualenv which allows an application’s Python dependencies (Python extension libraries) to be installed in a subdirectory, where they can easily be removed and reinstalled. This even works for libraries that require compilation, if you have a compiler installed. All our Linux servers have gcc installed by default, so this hasn’t been a problem for us, except for the above-mentioned first Windows deployment. At that time, we discovered some binary packages that could be installed on the Windows host system, outside the virtualenv, to provide these features. But they didn’t include everything that our Intranet needs, so I had to add PyWin32 and PIL to that list.

Then I discovered that our migrations wouldn’t run. There are several bugs with migrations on MySQL:

• Renaming a column which is a foreign key: “MySQL doesn’t bother to re-point the foreign key constraints when you change a column name.”
• Making a foreign key nullable after creation: MySQL 5.5 broke backwards compatibility, the fix is in South 0.7.4.
• You also can’t rename a column that has a UNIQUE key on it, and apparently can’t remove the UNIQUE key either, because South complains that it doesn’t exist.

I ended up having to completely abandon the migrations in one of the apps that makes up the Intranet. Luckily nobody’s using it yet, so it won’t cause problems.

The second issue was with the Apache Tika integration. I already spent several days working out how to integrate a JVM into the Python server process directly. I decided to abandon that approach for Windows, because I didn’t want to deal with the issues of compiling custom DLLs and embedding JVMs on a platform that I know less about, including potential threading conflicts between Apache, Python and Java in the same process.

Tika has a standalone server mode with a web service API which turned out to be fairly easy to call from Python:

    from httplib import HTTPConnection
    conn = HTTPConnection('localhost', 9998, True, 30)
    buffer = open(path) # wrong! see below
    conn.request('PUT', '/tika', buffer)
    response = conn.getresponse()
    if response.status != 200:
        raise Exception("Unknown response from TIKA server: %s: %s" %
            (response.status, response.reason))
    return response.read()

This server runs as a stand-alone application, and unlike Unix, Windows makes it fairly hard to set up such an application to be started at boot time, with no users logged in to the server. I had assumed that for a Java application, this was a solved problem, but I was wrong. As usual, Stack Overflow was very helpful, turning up a bunch of suggestions, several of which I tried and failed to implement:

• srvany.exe abandons the child process, which means that you need to use Task Manager to kill it if you want to restart it. It remains an option, but not ideal.
• sc.exe is bundled with Windows XP, but also abandons the child process.
• Java Service Wrapper is not free for 64-bit Windows, and I didn’t want to become dependent on a 32-bit JVM because that might not be possible in a future deployment, but it remains an option, if the 32-bit version runs on 64-bit Windows (not tested). Also, “the configuration file can get a little crazy”!
• YAJSW aims to clone JSW without the license restriction, but I couldn’t understand how to use it from the documentation.
• Apache Commons Daemon requires launching a separate process to tell the server process to shut down. Tomcat supports this with its control port listener, and so does plain Jetty, but Tika’s embedded Jetty doesn’t provide such a listener, and I didn’t fancy writing one.
• WinRun4J looked ideal, and ran the application absolutely fine on the command line.
• JavaService, apart from its obnoxious “free registration required” to download, was really difficult to work out how to use with the limited and broken documentation. When the command-line tool failed to accept any of the (documented) arguments I tried to pass to it, I gave up.
• I didn’t like the idea of writing a service wrapper in Perl or .NET and adding another dependency.
• WinSW is written in .NET, I didn’t investigate further.
• FireDaemon would probably work, it’s not free.
• JSL (Java Service Launcher) would probably work too. I couldn’t find the documentation when I first looked at it.
• JSmooth might also work, I missed it the first time.

At this point I was left with only WinRun4J as an option. I discovered that you have to write a service class, using their JAR, to run your application as a service, so I wrote one. I had to bodge the classpath to compile it, since Tika uses Maven and WinRun4J isn’t available in Maven repositories.

Then I wasted an hour with an undocumented problem when I added the service.class to my winrun4j.ini without removing the main.class entry, causing this misleading error message:

[err] Could not find service class
java.lang.NoClassDefFoundError: org.apache.tika.server.TikaServiceWinRun4J

After solving that one, I came to a problem that I couldn’t solve:

SEVERE: Can't start
java.lang.NullPointerException
        at com.sun.jersey.core.spi.scanning.PackageNamesScanner$ResourcesProvider$1.getResources(PackageNamesScanner.java:170)
        at com.sun.jersey.core.spi.scanning.PackageNamesScanner.scan(PackageNamesScanner.java:135)
        at com.sun.jersey.api.core.ScanningResourceConfig.(ScanningResourceConfig.java:80)
        at com.sun.jersey.api.core.PackagesResourceConfig.(PackagesResourceConfig.java:104)
        at com.sun.jersey.api.core.PackagesResourceConfig.(PackagesResourceConfig.java:78)
        at TikaServiceWinRun4J.serviceMain(TikaServiceWinRun4J.java:83)
[info] Service method completed...

I spent much time wondering how to debug this, whether I could attach a debugger to a WinRun4J process while it was still starting up, etc. I gave up at that point and went home, still turning the problem over in my mind. Then it occured to me that the Tika web service is implemented using what appears to be a standard Java servlet, and if I could install that servlet into a servlet container like Tomcat, which has its own service integration with Windows, that would solve the startup problem. And it did, with the help of the Maven WAR plugin which made it trivial. (At times like this I love Maven; at other times I hate its complexity.) I submitted a patch to the Tika developers in the hope that this will be adopted as a standard deployment method for Tika server.

Along the way, I discovered that the standalone Tika Server (command-line JAR) is pretty badly broken: it appears to work, but it can only extract content from Ogg Vorbis files and nothing else! I didn’t manage to fix this properly, but I did come up with a workaround, described in that ticket, which disables the Ogg Vorbis support and re-enables support for other formats that are more important to us, like Microsoft Word and Excel.

So finally, I was able to run the Django application under Apache on Windows, and log in and upload documents. But wait, what’s this? Error 500 from Tika? WHY?!

Tika wasn’t logging anything useful, so I had to fire up Wireshark to inspect the conversation between the client (the Intranet application) and the server (Tika). And thanks to Microsoft’s infinite wisdom in removing the loopback interface when they ripped thir TCP/IP network code out of BSD, it’s not possible to listen for loopback traffic on Windows, so I had to point the Intranet on Windows at a Tika server on another machine to capture the traffic.

Then I discovered that httplib was telling the server that it was going to send 7168 bytes, but only sending 6. Tomcat waited 20 seconds for the rest, and then timed out the request, returning an Error 500 as I saw earlier, and helpfully not logging anything. So why was httplib not sending all the data? I stepped into it with a debugger and discovered a problem with one of the lines I wrote above:

    buffer = open(path)

On Unix this works fine, but on Windows it opens the file in ASCII mode by default, which stops at the first ^Z (hex 1A) character in the file. And guess what’s at byte 7 of every Excel XLS file?

And so onto the fifth bug. Although the intranet appeared to work fine, it would sometimes pop up a scary error message on the server:

Program: C:\PROGRA~1\BITNAM~1\apache2\bin\httpd.exe
R6034
An application has made an attempt to load the C runtime library incorrectly.

Past experience told me this was probably due to one of the Python extensions missing an embedded manifest, which would be required to tell Windows which version of the Microsoft C Runtime it’s supposed to use. However I had great difficulty tracking down the culprit. A bug in the PIL installer turned out to be a red herring, and while we’ve seen this problem in LXML in the past, it wasn’t there this time.

Eventually, after much soul-searching, I downloaded Process Monitor and configured it to monitor all file activity from httpd.exe processes. I found out that Apache was loading msvcr90.dll from C:\Program Files\BitNami DjangoStack\Apache2\Bin, which is a strange place to find it, because normally it would be installed in C:\Windows\WinSxS. I discovered manifest files in that directory that might confuse Windows into loading this copy instead. That can potentially end up with multiple copies of the DLL loaded, which can result in random crashes, but in this case I think it was detected early.

Removing the msvc*90.dll and Microsoft.VC90*.manifest files from that directory didn’t solve the problem completely, because there was another copy of them in C:\Program Files\BitNami DjangoStack\Python. But with those second copies removed as well…

The intranet now runs on Windows!

I spent some time coding template support for it, and then when I tried to make it reusable, discovered that others had had the same idea.

Learning 1:

django is conservative. Lots of useful things never make it into the core. Search djangopackages.com for what you want before writing it. It has a LOT of packages.

Learning 2:

there’s an existing document management package called Mayan EDMS, that does most of what ours does, which we could look at replacing or merging ours with.

I found two email-sending packages that support templates:

• django-mail-templated has an object-oriented interface but no tests;
• django-templated-email has tests but no object-oriented interface.

I decided to check out the object-oriented one, on a pretty much random basis, and add tests to it. You can find my fork of it here.

Learning 3:

If you want to send emails from a Django project, use our email app.

Learning 4:

if you want to send notifications in some kind of generic framework, not just email, have a look at the django-notification app instead.

Learning 5 (meta-learning):

it’s useful to email round things that we discovered, like this one, as someone might have discovered this before and not told anyone, meaning that I had to spend time discovering it again; and also we can comment on each others’ discoveries and add to them.

• PDF files
• Microsoft Office DOC, XLS and PPT files
• and the new XML equivalents, DOCX, XLSX and PPTX.

I found various tools online to help extract this text, largely thanks to Stack Overflow here and here. This ended up with a hodgepodge of tools:

• PDF Miner for PDF files
• python-docx for DOCX files
• DocToText for PPTX, XLSX and PPT files
• antiword for DOC files

There were a number of problems with this hodgepodge:

• I was unable to find any Python or command-line solution for old Excel (XLS) files;
• These solutions did not extract metadata, only document text;
• The choice of which tool to use depends on the MIME type returned by the file(1) command, which varies depending on the OS (Debian/Ubuntu or CentOS) and which version of the library is installed

Another Stack Overflow post recommended Apache Tika for metadata extraction. It appears to support all the document formats that we need, and to have auto-detection of the document format, which solves all the MIME type problems as well. However, it introduces a new problem: it’s written in Java, which is hard to access from Python.

Luckily I found some instructions for building a Python wrapper around Tika, using some tools that I’d never heard of, and this seemed like a good approach. Unfortunately the installation process is very non-standard, which would not fit in with our fabric-based automated deployment process, and would make it harder for users to install the Intranet themselves.

The instructions are somewhat outdated at the time of writing, as they refer to Tika version 0.7, while 1.0 has been released. I was unable to register for an account to update that page, so I wrote to the author with the details that I discovered, and will also document here that the following command works for me:

python ../jcc/jcc/__main__.py \
        --include /usr/share/java/org.eclipse.osgi.jar \
        --jar tika-parsers-1.0.jar \
        --jar tika-core-1.0.jar \
        java.io.File java.io.FileInputStream \
        java.io.StringBufferInputStream \
        --package org.xml.sax \
        --include tika-app-1.0.jar \
        --python tika --version 1.0 --reserved asm

I was able to go further than this, and package Tika in a way that makes it easy to install with Pip, and thus integrate with our deployment process.

The wrapper is written using JCC, which works by generating and compiling C++ code that links to the Java classes, and then a Python wrapper around that C++. This means that it needs to be recompiled for each platform, so I couldn’t just distribute a binary blob with the Intranet (I had the same problem with DocToText above).

The version of setuptools on our servers doesn’t support JCC’s shared library mode. JCC dies with an error if it’s not explicitly disabled or the patch applies. I couldn’t do either of these as part of our standard deployment process. So I patched JCC to disabled shared mode, since we don’t need it anyway. I also added some patches to allow various setup.py commands used by pip to be forwarded through JCC to the setup function call.

This seems to be enough to allow you to install JCC like this:

pip install git+git://github.com/aptivate/jcc.git

I also wrote a setup.py file that handles pip’s command line invokations and passes the necessary options to JCC, and JCC’s invocation of the setup function. This seems to be enough to install the package using pip:

pip install git+git://github.com/aptivate/python-tika.git

and you can use the last parameter as a package specification in pip_packages.txt, or whatever you pass to pip -r.

You can find the pip-installable Tika package, complete with Tika 1.0 JAR files, in our python-tika repository on Github. This will save you the work of downloading and compiling Tika and all of its dependencies. I have started a discussion with the JCC developers about merging these changes into the upstream project.

• A User Guide for the system that we developed for UNICEF, IDS and RuralNet Zambia
• A Developers’ Guide for anyone wishing to build something similar
• Notes on lessons learned that may assist future implementers

Project goals

Automate the data entry part of a long paper-based survey, by replacing the paper forms with electronic devices.

Hardware and application selection

The survey has several long and complex questions, and long sets of multiple-choice answers. The data collection needs to be done in dusty rural Zambia, and the devices might need to be used for a full day without power. Collected data should be sent wirelessly to a secure data repository at some time after collection.

Text entry is required for many fields. That means either a real keyboard with keys, or a sufficiently large touch screen to type comfortably on. Use of the device camera, and presentation of reports and graphs on the same device, might be required in future.

Two possible hardware platforms were identified:

• Tablet laptops with touch screens
• Tablet mobile devices (iPad or Android tablet)

We selected the latter for this project due to lower cost, lighter weight, better usability and longer battery life.

The available software options that we identified were:

• EpiSurveyor (Java J2ME, partly closed source, we have used before and fixed bugs)
• OpenXdata (Java J2ME, open source, developed and supported by an Aptivate alumnus among others)
• Open Data Kit (ODK) (Android, open source, active community)
• Bespoke online/offline survey in HTML5

Of these, we eliminated EpiSurveyor and OpenXdata due to lack of compatibility with the hardware platform(s) we had chosen.

We chose ODK over a bespoke system due to limited time available for development, and ability to easily take photos and record GPS coordinates using the device’s hardware.

Of the available Android tablet devices, we chose the Samsung Galaxy Tab for the pilot project, due to its high quality construction. For future projects we would probably use a lower cost device; see the lessons learned for details.

Form creation

Since the survey is quite long (about 230 questions) we wanted an easy way to enter the questions. The ODK application requires the form to be in XForms format. We identified the following tools for creating XForms:

• ODK Build
• PurcForms
• Kobo Form Builder
• XLS2XForm (actually the pyxform fork)

We decided to use XLS2XForm, which enabled us to enter the large number of questions easily in Excel. The others all have graphical builders, which have advantages and disadvantages for less technical users:

• More visually appealing
• All available options presented visually (types of controls, groups, etc.)
• Less likely to make a mistake and produce an invalid form
• Cumbersome user interface slows down data entry

Unfortunately, none of these designers were able to import an existing form in XForms format, which means that the modifiable “source code” of the form must be maintained in a “proprietary” format in each case, and it’s difficult to switch between tools.

You can download the conversion tools, and the Excel spreadsheet with the completed questionnaire as we delivered it to RuralNet, here. RuralNet staff, please use the latest version of the spreadsheet that you can find locally. To use the tools, you will need to download and install Python 2.7 and Java (JRE). Then download the tools as a ZIP file and extract it somewhere. I recommend that you keep the master copy of the spreadsheet in Dropbox to ensure that it’s backed up, and it’s always clear what the latest version is.

For help in building surveys using XLS2XForm, please see the documentation. In addition to the question types listed there, we have used the following shortcuts, which also work in this customised version of XLS2XForm:

• text is short for add text prompt (a text field, such as a person’s name)
• note is short for add note prompt (a read-only field, providing additional information for the user)
• time is a time field without a date (for example, survey start and end times)

To compile the spreadsheet into an XForms form, run the build_and_validate.py script by double-clicking on it. If it works, it will show the message “Success!”, otherwise it will show an error message, usually caused by a mistake in the Excel spreadsheet. If it works, it will create (replace) the file called zambia-ranq-round3.xml in the same directory. If your spreadsheet has a different name, you can create a shortcut to call build_and_validate_custom.py with the name of the spreadsheet on the command line.

Software components

ODK Aggregate is the software that powers the Internet server. It is a repository for blank forms (designs) and completed forms (data). Our server is located at http://partimob.appspot.com/. This server is currently paid for by us, and will need to transfer to RuralNet at some point.

ODK Collect is the application runs on the device, and users interact with it to complete the survey. It’s essentially a user interface for XForms. It can download blank forms (designs) from an ODK Aggregate server, and upload completed forms (data) to the Aggregate server as well.

ODK Briefcase is the software that downloads completed forms (data) from the Aggregate server and convert them into CSV (spreadsheet) format, which can be loaded into

Customised ODK Collect

We are using a custom version of ODK Collect. You can download the source code for it here, or the compiled application here. You can also find it in the ZIP file download. If you prefer, you can use the latest official version of ODK Collect. The two are compatible, but our version adds the following useful features:

• Use supplied login and password by default to save a round trip and a prompt.
• Add keyboard navigation, useful for form filling on android-x86 because the mouse interface is pretty clunky.
• Restore ability to modify completed and submitted forms on the device, which was removed from the official version in 1.1.7.
• Improved error messages and progress indication during form uploads.
• Allow setting the instance name on the first page of the survey.
• Allow saving incomplete surveys on required questions (in case a survey is interrupted; almost all of our questions are required).

There are several ways to install ODK Collect on a device:

• Download it from the Android Market (official version only, not our customised version)
• Copy the APK file onto a microSD card, insert the card into the device, and use the My Files application find and open it from the SD card.
• Attach the USB cable from the device to a computer, enable mass storage mode on the device, and on the computer, drag and drop the APK file onto the device’s internal memory, then use the My Files application to find and open it.
• Attach the USB cable from the device to a computer, and use ADB‘s install command to install the APK file.

It’s useful to put the application onto the device’s desktop. To do that, open the Applications list, find ODK Collect, and press and hold it with your finger for a few seconds. The background will change to the desktop; release your finger to drop the application there.

It’s also useful to remove all the other junk from the desktop. For each icon and widget on the desktop, press and hold it with your finger for a few seconds, until the trashcan icon appears, then drag your finger to the trashcan and release it there.

Form management on the device

There are several ways to put blank forms (designs) onto the tablets:

• Download them from the ODK Aggregate server using ODK Collect.
• Copy them onto a microSD card, insert the card into the device, and use the My Files application to copy them from the SD card to the /sdcard/odk/forms directory.
• Attach the USB cable from the device to a computer, enable mass storage mode on the device, and on the computer, drag and drop the form into the /sdcard/odk/forms directory.
• Attach the USB cable from the device to a computer, and use ADB or DDMS to push the file onto the device, into the /sdcard/odk/forms directory.

Of these methods, ADB or DDMS is recommended for rapid development, and using the Aggregate server is recommended for production use, since the form must be installed on the Aggregate server for it to be able to accept submissions.

Similarly there are several ways to copy completed forms (data) off the device:

• Upload them to the ODK Aggregate server using ODK Collect.
• Use the My Files application to copy them from /sdcard/odk/instances to a microSD card, then remove the card and connect it to the computer, and drop the files into the ODK Briefcase data directory.
• Attach the USB cable from the device to a computer, enable mass storage mode on the device, and on the computer, drag and drop the files from the /sdcard/odk/instances directory to the ODK Briefcase data directory.
• Attach the USB cable from the device to a computer, and use ADB or DDMS to pull the file from the device’s /sdcard/odk/instances directory to the ODK Briefcase data directory.

Of these methods, using ODK Aggregate is recommended for development and production use.

Since the Aggregate server is on the Internet, this method requires that the device have Internet access. So it either needs a valid SIM card installed with credit and a data bundle, or a WiFi network connected. We had many problems with using SIM cards for data, so WiFi is preferred if possible.

The directories mentioned above will not exist until ODK Collect is installed on the device and run for the first time. Forms downloaded from the Aggregate server will also be placed in the /sdcard/odk/forms directory. Forms completed on the device will be placed in the /sdcard/odk/instances directory.

Configuring ODK Collect

Collect needs to know the details of the ODK Aggregate server to log into it, download blank forms and upload completed forms.

Open the ODK Collect application, press the Settings button and click on Change Settings. Click on URL and enter https://partimob.appspot.com. Similarly, complete the Username and Password using the details that you’ve been given by the Aggregate server operator, or the account that you’ve created on the Aggregate server. This account should only have Data Collector permissions, no more. Press the Back key to get back to the main menu of ODK Collect.

Downloading forms using ODK Collect

Open ODK Collect on the device, and click on the Get Blank Form button. Collect will try to log into the Aggregate server using the details that you’ve provided, and get a list of forms on the server that have the Downloadable box ticked. This is on by default for newly uploaded forms.

Tick the box next to all the forms that you want to download, and click on the Get Selected button.

Filling forms on the device

Open ODK Collect on the device, and click on the Fill Blank Form button. All the forms in the device’s /sdcard/odk/forms directory should be listed. Choose the form that you want to complete.

You will see an introductory screen showing how to move between questions by swiping your finger across the screen, from right to left or left to right. This screen has a text box at the bottom, which you can use to name the form that you’re completing. Naming forms is useful if your data collection is interrupted and you need to resume it later. It’s much easier to identify the form using its name, rather than opening it and flicking through to find some identifying information. You might name the form based on the household code that you’re surveying.

Depending on your answers to some questions, others may be hidden, or their text might change.

At the end of the form there is another chance to Name this form, and a tickbox to Mark form as finalized. Before you can upload the form to the Aggregate server, this box must be ticked, and you must press the Save Form and Exit button. Otherwise Collect will consider that the form is incomplete.

Sending completed forms to Aggregate

Open ODK Collect on the device, and click on the Send Finalized Form button on the main menu. Tick the box next to all the forms that you want to upload to Aggregate, and click on Send Selected. After the upload is complete, you should see the Upload Results message. Every form should have “Success” next to it, otherwise it was not sent successfully.

Downloading forms using Briefcase

We are using a customised version of ODK Briefcase with the following changes:

• Fix the export of repeated groups, which before only worked for the first row (issue 461).
• Shorten exported column names, to allow the CSV file to be imported into Access.
• Allow the server name, username and password to be provided on the command line (or via a shortcut).

You can find the source code here and the pre-compiled version here, as an executable JAR file. You can also find it in the ZIP file download. If you make changes to the source and want to build the executable JAR again, install Maven and use the mvn package command.

To download the completed forms, open Briefcase by double-clicking on the briefcase-1.0-jar-with-dependencies.jar file. On the Transfer tab, click on the Connect button. For the URL, enter https://partimob.appspot.com, and for the user name and password, give the details of an ODK Aggregate account with Data Viewer permissions.

Then you should see a list of forms appear under the heading Forms to Transfer. Tick the box next to the one that your users have been completing, and then click on the Transfer button. If you do this after all the completed forms (data) have been submitted to the ODK Aggregate server, you will not need to do it again for that form template (design).

Now switch to the Transform tab and see if the form appears in the Form list. If it doesn’t, then exit and restart the Briefcase application (issue 464).

For Output Type, choose .csv and media files. For Output Directory, choose the directory where you’d like to save the CSV files. Note that any previous files exported to that directory from the same form will be overwritten without warning, even if they have been modified (cleaned). Click on the Output button to write the CSV files.

Cleaning data in Excel

You can find the Excel spreadsheet that we use for data storage and cleaning here. Note that Excel is a long way from the best way to store and manipulate data like this. Microsoft Access would be far more appropriate. Yet again I wish there was a sufficiently powerful open source alternative desktop application to Access, allowing ordinary people (not developers) to develop and maintain their databases themselves.

Because the spreadsheet contains cleaned data, which is “better” than the raw data which is included in the CSV export, we don’t want to overwrite existing rows. For the main section of the questionnaire (the so-called Single Responses) you can include only the new data like this:

• Open the main spreadsheet and switch to the Single Responses tab
• Highlight all rows from 3 down to the bottom, and Sort them by the SubmissionDate column.
• Note the last submission date on this spreadsheet.
• Open the newly exported CSV file for the single responses (something like RANQ-2011-Round-4-v5.csv).
• Sort this file by the SubmissionDate column as well.
• Highlight and copy all the rows whose submission date is later (more recent) than the last one in the main spreadsheet.
• Paste them at the bottom of the Single Responses tab of the main spreadsheet, below the other data.

For the other tables, this process needs to be done completely manually at present.

You can then check and clean the data by viewing and modifying it in Excel. Note that each sheet has one or two columns at the end, which are filled by formulae that look up values from the Single Responses sheet, such as the Household Code.

Using the Android x86 Emulator

To be written.

Lessons learned

Project Goals

The actual aim of the project was never clear, because all the stakeholders wanted different things. But if it was to help our partners work more efficiently, then we could have attacked other parts of the process that would have yielded bigger improvements more quickly, such as helping with the output processing (data analysis and report writing) rather than data collection. If the project had incorporated a systems analyst early on, we would have identified and addressed these needs better.

Nobody wanted to be the product owner, to take responsibility for setting priorities for the project. We normally refuse to do this ourselves, because we see our role as assisting someone else to achieve their goals, and that person will need to maintain ownership of the project after our work is done. But in this case we had no choice but to become the product owners, because we couldn’t function without one.

Development Process

We found that workflow elicitation was useful for generating user stories in the agile sense, but we did not have a clear map of the workflow during the development process.

During development we collectively discounted the need for data cleaning, because we thought incorrectly that all the errors were introduced during transcription. Data cleaning might be less necessary if we had collected more data, since the errors would tend to average out, but it was still essential, and not planned for, and there was no workflow to make it happen, so we had to hack something together in the field.

The agile process meant that some of the tasks that we had identified were deprioritised, and never completed due to lack of time. This was supposed to be agreed by all the stakeholders, but some were unfortunately not involved. Some project members expected us to have automated these tasks, and were surprised and angry that this had not been done. We had not automated the production of graphs from the data, and we had to hack this together in the field in a hurry. Some project members had unfulfilled expectations about how the data would be presented for review and cleaning.

We also had some disagreements about the definition of “automated”. Some of the project team thought it meant that they would press buttons on their tablets and graphs would magically, instantly appear on the Internet. We did not have time to develop this level of automation, and the need for data cleaning made it logically impossible. We were able to produce graphs directly and instantly during data cleaning in Excel.

Procurement

We had difficulty actually purchasing equipment in country. Samsung Zambia could not accept credit card payments over the phone. In the end, we had to bring large amounts of cash into Zambia and change it locally. This resulted in late and risky procurement of the equipment.

We received disappointing service from the retailer of the tablets, including repeated failure to deliver the tablets to their shop for purchase despite prior agreements and assurances, and supplying us with devices in unsealed boxes. We suspect that some of the tablets had been used as demonstration models in shops, or returned by other customers.

The supplier’s warranty was only one week, after which we would have to return the tablets to Samsung in China.

In future we would:

• Ensure equipment is new (in sealed boxes)
• Have a trusted in-country agent pick up and test the equipment — this agent must take responsibility for functioning goods
• Ensure that equipment is in warranty, and if it fails tests it must be returned

We expected that Samsung would supply high quality, reliable tablets at a high price. We were disappointed with their reliability and performance. We could have spent much less money on the tablets for a similar level of performance, and had more spares.

Hardware

We had only a limited number of hardware devices, and our user experience developer did not even have access to an Android phone or an emulator.

Two tablets had hardware problems: short battery life and failure to connect to the mobile network. These were discovered in the field, too late to return the devices to the supplier for replacement.

We had reports from enumerators that the touch screen became less sensitive as the battery drained.

Most tablets failed to submit data over the wireless network at all. We had many problems with data communications in Mufulira, which we expected, but we thought we would at least be able to submit small amounts of data wirelessly, and this turned out not to work.

Survey Instrument

The survey instrument itself was unclear, complex, and not designed for electronic use. The wording of the questions was unclear and we had many arguments over it. Some questions would have benefited from having a calculator embedded, to help with subtraction and conversion of units, such as calculating change in land area owned by a household.

We had many problems with the form logic, in particular skipping questions depending on the answers to previous questions. This was very difficult to get right, and took us a long time, especially when using XLS2XForm to input the survey. It would have been easier if we could have visualised the flows through the form. We suspect that we did not have time to provide enough training that the local staff would be comfortable maintaining the survey in the XLS2XForm spreadsheet by themselves.

ODK Software

We had problems with ODK Collect crashing many times while enumerators were entering data. This always resulted in loss of the completed survey, unlike paper forms. We debugged and patched several bugs in Collect in the field. This required us to have an Android and ODK development environment already set up, because there was no way to download that software in Mufulira. At least we were able to fix the bugs, as this was open source software; some other products would have left us powerless.

Enumerators reported that Collect would sometimes register a different response than intended. Perhaps touching the screen in the wrong place, between questions, or an unclear/lazy touch, might activate the wrong answer. I noticed several times that Collect did not register an answer at all, and I found myself repeatedly jabbing at the screen in frustration. I put this down to hardware problems with the tablets.

For some questions, a grid with three columns (Question, Yes and No checkboxes) would have been a faster way to enter data than repeatedly touching an answer and then dragging left to flip pages. This was not a component that we had available to us in Collect.

Data Analysis

We had intended to load the survey output into Microsoft Access for data processing and storage for future use. However we were not able to make the data load into Access successfully. ODK Briefcase outputs CSV files with field names (column headings) too long for Access to import. Importing 20 separate CSV files into Access was very painful. I strongly recommend building much better integration between ODK and Access in future. We have an open question as to whether Access is the best data storage and management solution, but it would fit the needs and fit within the comfort zone of the project members who would be carrying this process forward in future.

We identified the need to check that we could import multiple spreadsheets into SPSS during the development phase, but we ran out of time and did not actually complete this task. When we got around to processing the data, we discovered that it was not possible, which seriously disrupted our assumptions and plans for the data analysis.

In addition, our backup plan to process the data and produce graphs using Google Fusion Tables, also failed because Aggregate was unable to export the data successfully to Fusion Tables, and editing (cleaning) the data online turned out to be too difficult (complex, inconvenient and awkward).

Positive Outcomes

All the enumerators reported they would prefer to use tablets than paper forms in future.

We also realised that interviewing people can create social change by encouraging them to think in new directions, and question previous assumptions about how things worked. Being listened to is also empowering.

One of our enumerators took photos of the people he interviewed, using the tablet camera, without even being asked! Well done that man. I think the photos were one of the most valuable outputs from the project, more than the data collected. Politicians, all of us can relate to photos, they tell a much more powerful and personal story than numbers.

I would really like to see technology used to empower local people to reach out to their politicians and hold them to account.

When we were asked to recommend a forum to be integrated into the suite of websites we were developing for the RIMI4AC Project, Askbot was the clear favourite, due to its large feature set, ease of customisation, active development team and wide user base. The only drawback was that the templating engine used by Askbot was Jinja2, which would make it difficult for Askbot to be embedded into the websites. Up until then these websites had been developed using standard Django templates.

We came up with the following options to solve this problem:

Create an Askbot skin using Jinja2, which would mimic our existing templates

This would be easy to implement but would incur high maintenance costs, as any changes to the standard Django templates would need to be made also to the Jinja2 templates. This could possibly be scripted to make this easier.

Embed Askbot in an iframe

Again this would be simple to implement but iframes introduce a number of problems themselves with navigation and rendering.

Rewrite Askbot to use Django templates

This would be a lot of work and as we would effectively be forking Askbot we would incur the costs of maintaining our own version.

Rewrite the rest of our websites to use Jinja2 templates

This would be quite a bit of work and any new components we wanted to integrate into our websites would also need to use Jinja2.

Choose a different forum that used Django templates

We really didn’t want to do this as we had had good reasons for choosing Askbot.

Some way of rendering Django templates from within Jinja2

Although Jinja2 supports extensions and we could possibly have written one to render Django templates, this seemed to be the opposite of what we wanted – to embed Askbot as a component of our website and not the other way around.

Some way of rendering Jinja2 templates from within Django templates

This looked like the most promising solution. Fortunately the Askbot developers were good enough to name their views, which meant that we could provide our own urls.py with views of the same name and then any reverse() lookups within Askbot would just work. Any view that rendered into a Jinja2 template by calling the function render_into_skin() would be replaced with this wrapper function:

def render_jinja2_into_django_template(request, jinja2_view, *args, **kwargs):
    response = jinja2_view(request, *args, **kwargs)

    return render_to_response("forum_container.html",
        {'forum_content':response.content},
        context_instance=RequestContext(request))

The wrapper function would call the original Jinja2 view function and we could pull the raw content out of the returned response object. This would go into the forum_content variable that would be passed to our own Django template and simply written out from within there.

Because we wanted Askbot to appear as a component within a page rather than its own standalone application, we would also need to remove the headers and footers from the Askbot Jinja2 templates. Askbot’s skin customisation made this straightforward. The Django container template would need to include any stylesheets or scripts that we had removed from the headers.

With some fixes to the CSS we had successfully embedded Askbot in our website. There would be some maintenance costs in that any future changes to the Askbot views would need to be reflected in our own Askbot views, but this would be far preferable to having to maintain our own set of templates.

I must admit to being skeptical about any such innovation, and I’ve been listening to both sides of the debate on the BytesForAll mailing list. Despite my skepticism, Osama’s questions have some answers, and I’d like to present them for comment.

• India has one of the lowest ratio of teachers—just 456 teachers per million people.
• Seventy-two percent of our primary schools have only three teachers or less.
• 25% of teachers were absent from school, and only about half were teaching, during unannounced visits to a nationally representative sample of government primary schools.

How is the $35 tablet going to solve any of these problems?

Of course technology on its own is not going to solve these problems. It is just a valuable weapon in the armoury of those who would launch an all-out war on poverty (and other abstract nouns).

Kentaro Toyama, an ex-Microsoft guru turned ICT4D researcher, says that “technology is [just] an amplifier of human intent and capacity.” And when faced with a task that’s possible but simply too large, an amplifier is exactly what we need. It doesn’t need to be high tech. Tanzania did just fine with radio, one of the oldest, simplest and most inclusive ICTs:

About ten years after independence, Tanzania decided to move towards universal primary education, almost doubling the number of children in school. The government estimated that it needed an extra 40,000 teachers. As the existing training colleges were producing only 5,000 new teachers a year, it was decided to recruit secondary school leavers and train them on an apprenticeship model, partly on the job and partly through distance education. Over a period of three years, they were posted in schools where they had a reduced teaching load. They then followed correspondence courses backed by radio programmes; they were supervised and tested on their classroom practices, and passed their examinations. Two evaluations found that they ended up reasonably competent in the classroom (Chale, 1993; quoted by Perranton, 2000; retrieved from UNU)

If India were to launch a massive teacher education programme, they would find it cheaper to implement that programme using technology. For example, they might distribute radios, TVs, portable audio players or even (heaven forbid!) computers to trainee teachers. It might take longer for those teachers to reach high standards, and more might drop out, without the personal connection and feedback of face-to-face training. Even so, one could train more teachers for more time and achieve a similar number of fully trained teachers at a lower cost.

In the business sector, more than 70% micro, small and medium enterprises (MSMEs) are not connected to information society to leverage opportunities of business and efficiency. How will the $35 tablet help in the financial inclusion of MSMEs, which are largely situated in small towns and remote areas?

It’s unfortunate that the tablet doesn’t include a long-range wireless network (such as GPRS), which must surely cover most of India as it does Africa. Even without an Internet connection, it can still provide useful services such as record keeping, business accounting and stock tracking to small enterprises. The tablet is based on Android, but the marketplace has been disabled, and this is a serious limitation. I think it’s likely to be overcome soon. When that happens, India’s many skilled software developers will be free to create localised applications for a potentially huge local market.

Most of India’s 3.3 million non-governmental organisations (NGOs) are also located in remote areas—70% of them lack any sort of information and communication technology (ICT) infrastructure or connectivity, and have no websites.

How can the $35 tablet help these NGOs’ global outreach efforts or aid the millions of people working with them in rural areas?

You probably know the answer to this question as well as I do: The same way as computer and phones can, only more so. Helping people to communicate and to do their work is exactly what ICTs do. All of them. With the possible exception of Angry Birds. A computer can help us to make leaflets, track visits to patients and beneficiaries, diagnose illnesses, improve farming techniques, or learn about anything we wish to know in the whole world of knowledge.

Can it bring transparency in governance at this level?

Good question. Not by itself, sure. Transparency comes from open data. The people might get together to publish what the government would rather hide, or pressure the government to release the data, but a $35 tablet won’t help them much.

When they do release that data, however, the usual problem is how to make use of it. Government data tends to be massive and unwieldy, and answering difficult questions takes much time and significant skill even with the best of data. I think that free, open, widecast media provide the biggest opportunity to make real use of transparency, and our use of the Internet as an enabler of democracy is the best example of that. Potentially, a simple but powerful Internet device could help bring people together to investigate and answer those difficult questions. But by the sound of it, this tablet is not quite there yet. Hopefully it will be soon.

Since a large population of our country communicate verbally, and cannot read and write with ease, their preferred medium of content consumption and content production is audio-visual… But to make use of good multimedia content, you need powerful machines, not cheap and underperforming ones.

I disagree with that. I grew up with “multimedia content” on BBC Micros: simple games, moving blocks around a screen, simple word processors and spreadsheets and databases and graphics. A picture is worth a thousand words, and a simple, clear diagram can be worth far more than a complex, confusing one. Advanced graphics are no substitute for a visual designer’s ingenuity and skill. Wikipedia is “multimedia content” that is perfectly suited to a $35 tablet.

If the $35 tablet can do anything good to education in India, the only way is by handing them to each and every teacher and school management staff to monitor the workings and functioning of the school and its teachers…

Monitoring is an interesting application, and a double-edged sword. Robert Chambers, the inventor of participatory rural appraisal, told us a story at the recent ICT4D Finale event in Cambridge of a hospital in India where the nurses were given mobile phones “to collect data at the source.” But the director of the hospital used it to monitor what they were doing, effectively spying on them. The nurses went on strike and eventually the director was fired. I think that for monitoring to have a positive benefit, it must be done with consent and a shared vision to use the data to improve performance, not to criticise and control.

rather than assuming that each student will buy Aakash and India will become digitally literate overnight.

I have to agree with that sentiment, although I’m not sure who raised it. Kapil Sibal, who takes the credit for inventing the $35 tablet, merely said:

This low cost device is part of our national mission on education through information and communication technology (NME-ICT) which will connect over 1,000 institutions across the country, enabling tonnes of web-based course content for free.

Now that doesn’t sound so far-fetched, does it?

For a while I’ve felt that the Agile methodologies from the software development world share a similar outlook to the Participatory methodologies from the international development world.

So we came up with an idea for an event. Wouldn’t it be great to get an expert from each discipline and have them talk to each other, in front of an audience?

Last night, thanks to support from the Humanitarian Centre, and our two esteemed guests, our idea became reality.

Alistair Cockburn, Agile guru, sat on a sofa next to Robert Chambers, expert on Participatory approaches, in front of an audience.

I thought it was fantastic and we’ve had a lot of positive feedback about the event. It was so good, I found myself afterwards wondering if this is in general a good format for an event.

So I wanted to write a post about the form of the event, rather than the content.

After the event I was chatting with Alistair and he’d already been thinking along similar lines. We called it a “Jazz Talk”. We were drawing an analogy with two jazz musicians improvising.

Jazz Talks

Here’s the format -

1) Get two affable speakers from different disciplines
2) Sit them on a sofa in front of an audience
3) Let them talk about the relationship between their disciplines
4) Periodically interrupt them with “Kibitzers”

Kibitzer

A “kibitzer” is a person who comments on the conversation.

“Kibitzer” was a term Alistair came up with. I had to look it up, literally it means an observer of a card game who gives (unwanted) commentary.

There’s two types of Kibitzer. A “content kibitzer” gives comment on the content of the conversation. In the event last night I played the role of one of the kibitzers and asked the question “How do we get funders to engage with agile / participatory proposals?”.  All of our kibitzers last night were content kibitzers.

Talking to Alistair afterwards, he was keen to push the idea of a “form kibitzer”. This is someone who gives a commentary on the form of the conversation, not the subject matter. For instance, “I liked how speaker-A extended speaker-B’s questions to the audience”, or “Can we hear more from speaker-A?”. I think form kibitzing is less natural but likely to be shorter. It also potentially plays a facilitatory role in guiding the conversation and could help address issues like one speaker dominating the conversation.

Perhaps a mix of both types could work. Each commentary would start with a short form kibitz followed by a content kibitz.

Timing

Here’s a suggested recipe:

• a 90 minute conversation
• kibitzing every 15 minutes (eg 5 interruptions)

When you have a classroom full of children with computers, each working at their own pace on speech or language tasks, they need private sound rather than the built-in speakers of their laptops. Otherwise the cacophony would make learning much harder for all of them.

Headphones (or headsets) are the normal solution for language labs in UK schools. But they’re not great for use with primary school kids in a dirty, dusty environment. They’re extremely fragile, hard to clean, uncomfortable to wear for long periods, and can spread ear infections.

A bluetooth headset would work, and would be nice in theory, but much more expensive, and would need charging often.

The most obvious solution seems to be something that looks like a mobile phone, but attaches to a computer with a cable. They’re very hard to find. It seems that everyone wants tiny, delicate, wireless or in-ear headsets. So manufacturers don’t bother making the kind of big, clunky, bulletproof handsets I’m thinking of.

First, after long and fruitless searching, I discovered that what I’m looking for is actually called a handset (because you hold it in your hands) and not a headset (that fits over your head).

And then I found them:

Unfortunately the cheapest I’ve found so far is £10 ($14) through Maplin, which is about ten times the cost of the cheap, fragile headsets we’d like to replace.

If you know of any others, or a cheaper bulk supplier than Maplin (such as their supplier in China) please let us know!