Colossus on a Raspberry Pi

One of only nine known photographs of Colossus

Like many oldies who ordered a Raspberry Pi, I was more taken that such a thing now existed rather than having any kind of specific plans for it. Never mind, I thought, something will come along. Sure enough it did. A little background first.

In 2007, Tony Sale and the volunteers at The National Museum of Computing at Bletchley Park had completed their magnum opus: The rebuild of Colossus. This saw a major milestone after many years of works by the team as they toiled to recreate the world’s first programmable computer in as authentic a fashion as possible.

Colossus was the ultimate expression of necessity being the mother of invention. In 1941, Bletchley Park was struggling with deciphering the German ‘Fish’ codes. These were high-command transmissions using an unknown enciphering system seemingly much more complex than Enigma. A breakthrough by John Tiltman, helped along unwittingly by some German radio engineers, had given the codebreakers vital clues into how the system worked. After three months obsessive work on the problem, Bill Tutte correctly ascertained how the traffic was being encrypted. They were now able to build an analogue of the deduced twelve-rotor system and had a solid method to crack the messages. The problem was the process took six to eight weeks per message.

The race was on to build machines that could speed up this process. After a few false starts, Tommy Flowers, who was working for the General Post Office’s research department at Dollis Hill, constructed a monster that could cut though the lion’s share of codebreaking work in just a few hours. This machine, Colossus, had such an impact on Bletchley Park’s abilities to read ‘Fish’ traffic that a total of ten were built towards the end of WWII. It wasn’t until after the war that the German secret machine was revealed, the Lorenz SZ40 & SZ42 ‘secret writers’.

So, back to 2007. A rebuilt Colossus partakes in the ‘Great Cypher Challenge’. A Lorenz cypher machine was taken to the Heinz Nixdorf museum in Paderborn, Germany. Over the course of a day, messages were enciphered using the device and transmitted over teleprinter code (‘non-morse’) just as they were in WWII. In Bletchley, modern-day ‘Y’ station operators listened in on old radio sets trying to record the signal. At about 3pm, a clear message was received and Colossus was set to work. A few hours later, and with a little help from it’s modern-day codebreakers, all twelve rotors settings for Lorenz had been found and the message decrypted. Champagne everywhere.

However.

In Bonn, a young cryptology enthusiast by the name of Joachim Schueth was also listening (fair enough, the competition was open to anyone). He had written some software that would not only act as an analogue for Colossus’ work (identifying the first five rotor settings) but would in fact handle the entire process from interception to revealing the plaintext message. He intercepted the message first time and had an answer 46 seconds later.

Jo has been kind enough to allow others to download his code, along with recordings of all three messages he intercepted, along with generous instructions on how to proceed. So, there was my first Raspberry Pi challenge, could it become a modern-day Colossus? It seemed appropriate after all, to see this little wonder of 2012 pit itself against it’s Granddad.

After a few tweaks I was able to get the whole process up and running on Eben and David’s little baby. Yes, the Raspberry Pi, running the standard Debian ‘Squeeze’ image, can not only replicate Colossus’ work but the whole process from interception to plaintext. Let’s call it Pilossus.

I thought others might like to have a go at running this themselves, so I’ve provided a simple tar ball that will get you up and running. Get your Pi booted up and we’ll start from a terminal prompt in your home directory.

Before you go any further, you must run the following command (especially if it’s a clean build of Debian), or the installation will not work).

$ sudo apt-get update

Now you can…

Download Pilossus 0.1 (tgz 12MB)

or, directly from your RasPi, run this:

pi@raspberrypi:~$ wget http://mrpjevans.com/downloads/pilossus-0.1.tar.gz

Once  you’ve got the package into your ‘pi’ home directory via wget, scp, usb stick, etc we should have this:

pi@raspberrypi:~$ ls
pilossus-0.1.tar.gz

First, uncompress the package in a suitable directory:

$ mkdir pilossus
$ cd pilossus
$ tar xvf ../pilossus-0.1.tar.gz

Now, run the script that will ensure you have all the bits and pieces you need:

$ ./install.sh

This will use apt-get to install or update any dependancies. It’ll probably take a few minutes and does require an Internet connection. Once downloaded, the binaries will be compiled from the Ada sources. There will be a couple of warnings, but these are nothing to worry about.

Once you see ‘Pilossus is ready’, the binaries required are now installed in /usr/local/bin and the scripts are ready to run from your current directory.

The cypher challenge comprised three messages. These are known here as 1200, 1600 and 1700 simply based on time of transmission. They had some variations in transmission type and how much information was given away in advance.

You can process and break these three transmissions from their raw audio form by running the script like so:

$ ./pilossus.sh 1200
$ ./pilossus.sh 1600
$ ./pilossus.sh 1700

Each process will take some time but will end with a lot of information including the results of analysis (the analogue for Colossus), the calculated wheel settings for Lorenz and the resulting plaintext. If you scroll up to the start of the plaintext output you will see a time reading. This shows how long it took to run the section that replicates the work of Colossus. I won’t spoil it for you but Colossus itself took around three hours.

If you would like to hear the original recordings, you can cd into the 1200UTC, 1600UTC or 1700UTC directories and enter ‘play A.mp3′.

If you want to run the codebreak again, you can ‘reset’ the system by running the ./clean.sh script. This removes all output files from a previous run.

Jo has released the source code and recordings of the original transmissions for anyone to download. However, the comments in the source files attribute copyright to DL2KCD and Joachim Schueth. Unfortunately I have been unable to contact Jo to ask permission. I am assuming that what I am doing here is simply an extension of the public release of his source and I make no assertions over copyright of this code. The scripts I have added (pilossus.sh and install.sh) may be considered Public Domain.

So, why not download the package and see how your Raspberry Pi compares to the granddaddy of them all, Colossus. The result is quite remarkable considering it’s a £20 computer the size of a credit card. How far we have come. Got a time reading? I would to hear about it in the comments.

 More information on the cypher challenge, and how he did it, can be found on Jo’s website
 
 

PJ

 

6 thoughts on “Colossus on a Raspberry Pi

  1. For 1200UTC:

    221.51user 0.29system 3:42.65elapsed 99%CPU (0avgtext+0avgdata 6000maxresident)k
    0inputs+0outputs (3major+441minor)pagefaults 0swaps

    1. Yes, so the Colossus analogue running on the Raspberry Pi find the 5 rotor settings in a little under four minutes. The original machine took about 3 hours and my 2012 iMac managed to run the same software as the Pi in 3.9 seconds!

  2. And for 1600UTC:

    210.25user 0.28system 3:31.34elapsed 99%CPU (0avgtext+0avgdata 6000maxresident)k
    0inputs+0outputs (2major+442minor)pagefaults 0swaps

    What an amazing little beast the RPi is. Thanks for this code…

  3. And just to be completist about this, 1700UTC gave the following on my plucky little RPi:

    216.61user 0.32system 3:37.77elapsed 99%CPU (0avgtext+0avgdata 6000maxresident)k
    0inputs+0outputs (1major+443minor)pagefaults 0swaps

Comments are closed.