Celebrating Joan Clarke on International Women's Day
Monday 8th March is International Women’s Day, a global event celebrating women's social, economic, cultural and political achievements.
I wanted to take the opportunity to write a few words celebrating Joan Clarke, a little known cryptanalyst who served at Bletchley Park during the Second World War.
Her work saved countless lives. She was a pioneer, an innovator and commanded the respect of some of the greatest minds of the 20th century. Working alongside Alan Turing, she was his friend, confidant and briefly his fiancee.
Born on the 24th June 1917 in London, she won a scholarship to study mathematics at Cambridge University in 1936. There she achieved a double-first degree but was denied a full degree as they were only awarded to men at the time.
At the start of the Second World War, she was one of the top four mathematicians recruited to the Government Code and Cipher School (GCCS) at Bletchley Park in Buckinghamshire. Initially, Clarke was assigned to an all-women group referred to as “The Girls” who performed administrative duties.
Cryptology was not work considered suitable for women to perform, and according to Clarke, she only knew of one other woman working in that field at Bletchley.
Soon after, she was assigned to Hut 8 and dedicated her time to cracking the Enigma machine that the Nazis had assumed unbreakable.
Enigma encrypted plaintext messages into ciphertext by passing an electrical current through a set of rotors (either three or four, depending on the model). Two enigma machines with identical starting settings would produce the same encryption/decryption results allowing the plaintext to be read by the recipient.
During use in the field, the operator would know the rotor starting position for the day and randomly select a message indicator for the individual message they wanted to send. By choosing a random starting position for each message, the cryptanalysts trying to decode the message were denied clues about the day's rotor starting position.
Let’s imagine I want to send an encrypted message using Enigma. I know the rotor position for the day is “AOH.”
I would start by setting the rotors to the correct position for the day - “AOH”. I would then choose a random indicator for the message - let’s say “EIN” (or any three-letter combination). I would then encrypt the message indicator twice in succession (“EINEIN” to the ciphertext “XHTLOA”). With that encrypted message indicator, I would reset the rotors to position “EIN” and then encrypt my message.
When transmitting the ciphertext over the open radio waves, I would start by sending XHTLOA followed by the message that was encrypted with the message indicator.
The radio operator on the receiving side would note the first six characters of the message and decode them using the rotor positions for the day (“AOH”), understand that the first six characters decode from “XHTLOA” to “EINEIN” and then have the message indicator.
By resetting the rotor position to “EIN”, the operator can decrypt the message.
Each message has a new message indicator. The rotor starting positions change every day.
The critical point is that Enigma would never encrypt a plaintext character to the same letter in the ciphertext. “A” would never be encrypted to “A”. Each press of a key moves the rotors to the next position. This small chink in the cipher was exploited by the Bletchley Park codebreakers to build a system to decrypt Enigma messages.
Enigma also had an electrical plugboard that could be set to reroute the electrical signal around the rotors, dramatically adding complexity to the number of possible combinations.
In total there are 158,962,555,217,826,360,000 unique starting positions for the machine - almost 159 quintillion
Clarke specialised in a code-breaking process named Banburismus, which Alan Turing devised. Banburismus was used to predict the starting positions of centre and right-hand rotors, which would drastically reduce the time required to crack the code for that day.
The rest of the code-cracking process was done on electro-mechanical machines called bombes that would calculate the set of rotors in use and the starting and middle rotor position. Without Clarke’s work, the time taken on the bombe machines meant that messages wouldn’t have been decrypted in time for useful action to be taken based on the plaintext message.
Banburismus was used against the enemy Naval forces that had an operational weakness - they chose the message indicators from a list rather than randomly for each message, meaning that multiple messages could be observed that had been encrypted with the same indicator. The Navy chose message indicators from a sequential list (“EIN” then “EIO” then “EIP”)
The principle of Banburismus is that repeating patterns of characters in ciphertext reveal clues about which rotor starting positions might be likely. Although the rotors started in a new position for each message, it was possible that one message of sufficient length would contain the rotor positions that would be used to start the next message.
The process involves examining how often characters in the same position in a message would repeat if the text was truly randomised text which would be a 1 in 26 chance (3.8%). If two messages are written in a spoken language like English or German, you would find that the same character appears in the same position in a message at a higher occurrence rate of 1 in 17 times or 5.9%
By examining encrypted messages and shifting the positions of the messages relative to each other, it was possible to find two messages that were “in-depth” of each other.
In the image below (from Wikipedia), you can see two encrypted messages with the second offset against the first by 8 characters.
By sliding the second message against the first, this position seems to put the messages “in-depth” of each other. There are 9 repeats within a message block of 56 characters - a 16% overlap.
Once two messages were identified as having been “in-depth” of each other, comparisons could be made to exploit the fact that Enigma could never encrypt a plaintext character to the same encrypted character (“A” never encrypts to “A”)
By analysing the repeating patterns of encrypted texts all transmitted on the same day (with the same starting positions), the probability of the rotor positions could be measured, weakening the cipher to a point where brute-forcing the remaining combinations was feasible on the bombe machines.
The work to analyse messages was performed on punch-cards that were printed in Banbury in Oxfordshire. They became known as 'banburies' at Bletchley Park, and hence the procedure using them: Banburismus
Joan Clarke became the deputy head of Hut 8 in 1944 but was prevented from progressing further within GCCS because of her gender.
The work of Joan Clarke and the Bletchley Park cryptanalysts shortened the duration of the war and possibly changed the outcome altogether.
She lived in Oxford until her death at the age of 79, and in 2014 was portrayed by Keira Knightly in the film “The Imitation Game.”
On International Women’s Day, I celebrate Joan Clarke's achievements and many like her that achieved so much in science, technology, and maths.
If you haven’t already today, take time to recognise a woman you admire in the comments below.