System Imaging for Free using G4L

By Chris Wilson on 22 July 2010

This is a copy of the notes that I wrote at AfNOG 2010 as a guide to using system imaging at future workshops. Unfortunately that wiki is not accessible without signing up for an account, so I'm posting the information here too.

How to Install Computer Labs

If you ever need to set up a large number of computers in identical configurations, you have a few options:

• Install each one individually by hand
• Automate the standard install process, for example using:
  • AIK for Windows;
  • Kickstart for Red Hat Linux, CentOS and Ubuntu; or
  • sysinstall and mfsroot for FreeBSD.
• Configure one machine exactly how you like it, and then exactly duplicate the hard disk to the others (disk imaging)

The first option (manual installation) is extremely slow, tedious, error-prone, unlikely to result in identical machines, and does not speed up future installations or reinstallations.

The second option requires using rarely-used and less tested parts of the installer, slows down badly with multiple simultaneous installations (due to limited network bandwidth and bugs in the inetd TFTP server), and places limits on what you can customise. For example, it seems impossible to customise /etc/rc.conf using the installer on FreeBSD, and pre-installing SSH keys is tricky. I spent days writing a sysinstall script to automate the process. It would have taken just half an hour to set one machine up perfectly by hand, and then copy the system image onto all the other PCs in a few unattended hours.

Therefore I prefer the third option, system imaging.

What is System Imaging

Imaging is the process of making exact copies of one machine's hard disk, including all partitions, onto another. This only works when the second hard disk is at least as large as the first. It works best when all the PCs are identical.

Imaging is independent of the operating system. You can image Windows, FreeBSD, any version of Linux, dual-boot and triple-boot installations, whatever you like.

We successfully used imaging to set up the PCs for these workshops:

• AfNOG 2009, Cairo, all tracks
• KENET BMO Training, Nairobi 2009
• AfNOG 2010, Kigali, for three tracks: SA-E, SS-E and SI-F

How to Image

Many systems administrators have heard of Norton Ghost and Acronis True Image, two of the most popular commercial applications.

However, open source alternatives such as G4L (Linux-based) and its ancestor G4U (FreeBSD-based) are pretty good, and completely free. G4L however lacks a website, and it's not obvious how best to use it, hence this post.

G4L is quite similar to G4U, and I could have used G4U instead. But I find the Linux kernel's hardware support a bit better than FreeBSD's, and G4L supports multicasting, which enables it to install many machines at the same time with good performance.

Using Ghost for Linux (G4L)

I've successfully used Ghost 4 Linux (G4L) versions 0.27 and 0.33 for this process. 0.33 has multicast support, which allows setting up an entire room in one go, without wasting network bandwidth copying the same 4 GB disk image to each of 50 machines independently.

Set up an FTP server on your network with an account that supports downloads and uploads (e.g. on a local server on your network). Make sure it has plenty of disk space free, perhaps 40 GB. Create an "img" directory under the FTP user's home directory for the images.

Download G4L and burn some CDs, maybe about five copies, or set up network booting (this conflicts with FreeBSD PXE installation and may require BIOS setup changes to enable PXE).

It's a good idea to explore G4L and get used to the options, but please be very careful, as it has the potential to wipe your hard disk! So please use a machine with a fresh hard disk or which you don't mind wiping.

To boot into G4L (you will need to do this several times below, but not yet, unless you just want to explore):

• Reboot or power up the machine
• Press the key to choose boot device
• If CD-ROM is not on the list, reboot, go into the BIOS and enable booting from CD-ROM
• Choose to boot from the CD
• Choose the default kernel at the GRUB screen (just press Enter)
  • If for some reason the default kernel doesn't work, the machine hangs or crashes or doesn't detect the network interface, then try one or two other kernels
• Wait for the kernel and initrd to be loaded (two long lines of dots)
• Then you can remove the CD, about one minute from cold boot, and start booting another PC
• Press space to skip each of the information/advertising screens (about 8 of them)
• Enter g4l at the prompt (if you go past this and get a shell, just type g4l at the shell prompt)
• You can access other consoles with Ctrl-Alt-F1 to F4, log in as g4l with no password, and run g4l, ifconfig, ping or whatever
• Choose Network Use (default)
• Choose Raw Mode (default)
• Check that you have an IP address (option B) or try again to acquire one by DHCP
• If you can't get an IP address by DHCP, check your cabling and DHCP server

Create a Restore Image (optional)

Back up one of your PCs if necessary (if you plan to restore the PCs later) by:

• Follow the procedure above to get into Ghost for Linux
• Enter the FTP server's IP address, username and password
• Choose an image name, e.g. backup_original_2010_07_22.img
• Choose the back up option
• Press Space to select the entire disk (mark it with an asterisk [*])
• Start backing up the image

This process can take 1-2 hours. In the mean time...

Set up the Master PC

If you don't already have a master computer set up, it's a good idea to WIPE THE DISK first. This makes the image much smaller, and transfer much faster. Please DO NOT do this if you have anything valuable on the master computer, for example an existing operating system installation that you want to keep.

Boot G4L on the PC that you will use as the master. Use DD to wipe the entire disk with zeroes:

dd if=/dev/zero of=/dev/sda bs=1M

Install FreeBSD or whatever operating system(s) on the master PC, and set it up exactly the way you want all of the PCs to be. Examples include:

• Install Gnome (gnome/gnome2)
• Install Xorg (x11/xorg)
• Install Firefox (www/firefox35)
• Install Xpdf (print/xpdf)
• Enable gnome and sshd in /etc/rc.conf, and add templates for the IP address configuration (this saves typing when setting all the machines to static IPs):

  hostname="pc01.sse.ws.afnog.org"
  ifconfig_bge0="dhcp"
  # ifconfig_bge0="196.200.219.101/24"
  defaultrouter="196.200.219.254"
  gnome_enable="YES"
  sshd_enable="YES"
  

• Create a user account (e.g. username afnog, password afnog)
• Log into Gnome, add firefox, terminal and the Downloads folder to your toolbar, and remove epiphany and evolution
• Edit /etc/fstab and add the proc filesystem:

  proc /proc procfs rw 0 0
  

  (this allows GDM to display the user list and shut down and restart the machine)
• Edit /etc/profile and set the default pager to less by adding:

  PAGER=less; export PAGER
  

• Set the timezone by softlinking /etc/localtime to something like /usr/share/zoneinfo/Africa/Kigali
• Create /etc/rc.local and have it run /usr/sbin/ntpd -qg to set the time once at boot

I recommend using DHCP on this machine. Otherwise all the imaged machines will boot up with the same IP address, causing IP address conflicts, and you will have to reconfigure them before you can access the Internet at all, or reconfigure them automatically.

Create some SSH keys for use in administering the machines. You may wish to set up the local server already and generate the keys there for security. I recommend adding the keys to /root/.ssh/authorized_keys. Please test that they work, and that sshd comes up automatically after boot!

Imaging the other PCs

On all the PCs (master and clones):

• Boot G4L as above
• Check that it has an IP address (option B)

Once a master is online, all the PCs will show "press any key to start". Pressing any key on any computer will start all the machines imaging. If any PCs are not ready yet, you will have to cancel the imaging process on all of them and start again, or image those PCs later. So:

Start the master last! (when all the other PCs are ready)

Start the clones first, by following these steps on each one:

• Choose UDP Multicast Client (option U)
• Select the entire disk, /dev/sda with the space key
• Say yes, you're sure
• When it says "Compressed UDP receiver", it's ready and waiting for a master to appear on the network

Then start the master:

• Get ALL the clones ready, as above, before doing this!
• On the master, choose UDP Multicast Server (option W)
• Select the entire disk, /dev/sda, with the space key
• Leave the options blank
• Say yes, you're sure
• The master start accepting connections from clients, which will happen automatically. The screens on the clients will also change.
• Please check that every client says "Press any key to start".
• If not, please check it for network problems, etc.
• DO NOT stop or kill the server now, unless you want to visit every client again!
• You can press Ctrl+C on the client and run g4l again to check the IP address, retry DHCP, and try the UDP Multicast Client option again.
• This is your last chance to join any remaining clients to the group for this imaging session!
• When all the clients are ready, press a key on the master to start transfer.

The master will show progress of the transfer, and an error line if any clients fail to respond. Clients that cause too many errors will be kicked out of the group and appear to "finish" early.

It's difficult to tell if the imaging process finished successfully or failed on the clients. However it appears that FreeBSD is very good at detecting filesystem corruption, and will fail to boot if the image was not completely transferred. So you can test them by trying to boot FreeBSD and seeing if it boots completely or stops with a filesystem error. Ideally this would be improved in future versions of G4L.