The Thinking Machine Chronicles #0006: Colossus: The Secret Electronic Brain That Cracked Lorenz

Era 1 · The Foundations (1936–1955) The war demands a machine that thinks faster than any human. A telephone engineer obliges.

The World in 1943

The year 1943 was the hinge on which the Second World War turned. In February, the German Sixth Army surrendered at Stalingrad, the first major Axis defeat on the Eastern Front, costing some 300,000 soldiers and ending Hitler's dream of a quick victory over the Soviet Union. In North Africa, the Axis forces capitulated in May, surrendering 275,000 troops to Allied command and opening the way for the invasion of Sicily in July and the Italian mainland in September. At the Tehran Conference in November, Roosevelt, Churchill, and Stalin met for the first time to coordinate the Allied strategy that would become D-Day. In the Pacific, American forces began the slow, brutal island-hopping campaign that would eventually bring the war to Japan's doorstep. The world was consuming men and materiel at a rate no previous conflict had approached, and the outcome hung on intelligence as much as arms.

In Britain, at a nondescript Victorian mansion called Bletchley Park in Buckinghamshire, the most concentrated collection of mathematicians, crossword enthusiasts, chess champions, and linguists ever assembled was working in secret to read the enemy's mind. Turing's Bombe had cracked the Wehrmacht's Enigma traffic. But by 1942, the Germans had begun routing their highest-level strategic communications through a different, far more formidable cipher system, one that no Bombe could touch. The cryptanalysts at Bletchley called it Fish. The engineers called it a nightmare. A Post Office telephone engineer named Tommy Flowers believed he knew how to build a machine that could crack it.

The Machine Behind Fish: Lorenz SZ-40/42

Enigma was a field cipher used by the army and navy. The Lorenz SZ-40 and its successor the SZ-42 were something altogether different: a teleprinter cipher machine used for communication between Hitler's headquarters and his generals in the field. Where Enigma encrypted letters one at a time using three or four rotors, Lorenz encrypted the five-bit Baudot teleprinter codes of an entire typed message, operating at speeds far beyond human manipulation.

The Lorenz machine worked by XOR addition. Every character of plaintext $P$ was combined with a keystream character $K$ to produce ciphertext $C$:

$C = P \oplus K$

where $\oplus$ denotes bitwise XOR over the five Baudot bits. Decryption was identical: $P = C \oplus K$, since XOR is its own inverse. The keystream $K$ was itself the XOR of two independent streams:

$K = \chi \oplus \psi$

The $\chi$ (chi) stream came from five chi wheels, each a toothed wheel with a different number of positions: 41, 31, 29, 26, and 23 teeth respectively, giving a combined period of $41 \times 31 \times 29 \times 26 \times 23 = 22{,}041{,}682$. The $\psi$ (psi) stream came from five psi wheels of 43, 47, 51, 53, and 59 teeth, but the psi wheels did not step at every character. Their advance was controlled by two "motor" wheels (37 and 61 teeth), making the psi contribution highly irregular.

The total key period was therefore astronomically large:

$41 \times 31 \times 29 \times 26 \times 23 \times 43 \times 47 \times 51 \times 53 \times 59 \times 37 \times 61 \approx 1.6 \times 10^{19}$

Not even the most exhaustive brute-force attack imaginable at the time could cycle through that. Breaking Lorenz required something subtler.

Bill Tutte's Miracle

The cryptanalysis of Lorenz began with a catastrophic German mistake. On 30 August 1941, an operator in Athens sent a message of roughly 4,000 characters. The receiving station at Vienna did not receive it cleanly and asked for a retransmission. The Athens operator sent the same message again, but this time slightly abbreviated and using the same wheel settings. He had, in effect, sent the plaintext $P$ encrypted with the same key $K$ twice. The two ciphertexts were:

$C_1 = P_1 \oplus K \quad \text{and} \quad C_2 = P_2 \oplus K$

XORing them together eliminates the key entirely:

$C_1 \oplus C_2 = P_1 \oplus P_2$

From $P_1 \oplus P_2$, a brilliant young Cambridge mathematician named Bill Tutte, working without ever seeing a Lorenz machine,deduced the machine's complete logical structure over the following months: the number of wheels, the number of teeth on each, and the role of the motor wheels. This is one of the most remarkable feats of pure deduction in the history of cryptanalysis. Tutte worked out the architecture of a machine he had never seen, from a stream of ones and zeros.

The attack on individual messages, once the wheel structure was known, worked by searching for the starting positions of the chi wheels. The key statistical insight was that the XOR of consecutive characters in the cipherstream, a quantity called the "delta",has non-random properties when the chi stream is correctly aligned. Specifically:

$\Delta C = C_i \oplus C_{i+1} = \Delta P \oplus \Delta\chi \oplus \Delta\psi$

Since $\Delta\psi$ is zero approximately 50–60% of the time (the psi wheels frequently do not step), the term $\Delta C \oplus \Delta\chi$ is a biased estimator of $\Delta P$. The chi wheel starting positions that maximise the count of "dot" (zero) characters in $\Delta C \oplus \Delta\chi$ are likely correct. This is the statistical break: not cryptographically decisive, but probabilistically overwhelming when applied to thousands of characters simultaneously.

The problem was speed. Computing these character-by-character statistics across 4,000-character messages for all possible starting positions of five chi wheels, each with up to 59 positions,required evaluating millions of XOR operations per message setting attempt. Human counters working by hand could manage one attempt per minute at best. The chi wheels alone provided $41 \times 31 \times 29 \times 26 \times 23 = 22{,}041{,}682$ possible starting combinations. At one attempt per minute, a complete search would take forty-two years.

Tommy Flowers and the Electronic Gamble

The predecessor to Colossus, a machine called Heath Robinson (built by engineer Frank Morrell and mathematician Max Newman in 1943), attacked this problem by reading two punched paper tapes in synchrony, one carrying the ciphertext and one simulating the chi wheel stream,and counting matches electronically. It worked, after a fashion. The mechanical tape-reading mechanism could achieve about 2,000 characters per second, and the synchronisation between the two tapes was maddeningly unreliable.

Tommy Flowers was a senior engineer at the General Post Office's Dollis Hill Research Station in north London, where he had spent years working on telephone exchange switching circuits using thermionic valves (vacuum tubes). When he was brought in to consult on Heath Robinson's reliability problems, he formed a radical opinion: scrap the second tape entirely. Replace it with electronic valve circuits, registers, counters, and logic gates,that could generate the chi wheel sequence internally at electronic speed, synchronised perfectly with the ciphertext tape. Newman and the cryptanalysts were sceptical. Valves were known to be unreliable; 1,500 of them in a single machine seemed like a recipe for constant breakdown.

Flowers disagreed. His experience at Dollis Hill had taught him that valves failed almost exclusively at switch-on, and that a machine kept permanently powered could run for months without a failure. In February 1943, with no formal authorisation from Bletchley and largely at his own initiative, Flowers began designing and building the machine at Dollis Hill. He called it Colossus.

Colossus: What It Was

Colossus Mark 1 arrived at Bletchley Park on 18 January 1944 and processed its first message on 5 February. It used 1,500 thermionic valves. The paper tape containing the ciphertext was mounted on a series of pulleys and driven at 5,000 characters per second, so fast that the tape formed a complete loop in the air as it circulated. At this speed, a complete message could be read once per second. The chi wheel sequence was generated internally by electronic shift registers.

The machine could perform five operations simultaneously on the five Baudot bits, counting the occurrences of chosen Boolean functions of the bits as the tape flew past. The operator set the chi wheel starting positions using a panel of switches; the machine ran through all positions of one or two wheels automatically, printing counts on an attached teleprinter. A skilled operator could find the correct chi settings for a message in a matter of hours.

Colossus Mark 2, delivered in June 1944, deliberately timed to be operational before D-Day on 6 June,used 2,400 valves and ran at 25,000 characters per second. It could handle all five chi wheels simultaneously. Eventually ten Colossi were built, operating around the clock at Bletchley Park.

What made Colossus historically important was not merely its size. It was the first machine to use conditional branching, the ability to execute different counting operations depending on intermediate results. It was not stored-program (the program was set by switches and plugboards, not by instructions on a tape), but it was programmable in a precise sense: its logical behaviour could be reconfigured for each message by the operator. It was, in the words of historian Jack Copeland, "the world's first large-scale programmable electronic computer."

The Code

The Lorenz cipher and the Colossus attack are surprisingly approachable in Python. Below is the core of the depths_attack, the XOR that cancels the key when an operator reuses settings:

def depths_attack(c1: list[int], c2: list[int]) -> list[int]:
    """
    C1 = P1 XOR K,  C2 = P2 XOR K
    C1 XOR C2 = P1 XOR P2  — key cancels out entirely.
    """
    return [a ^ b for a, b in zip(c1, c2)]

And the statistical count Colossus ran for each candidate chi position:

def colossus_count(ciphertext, machine, chi_pos):
    chi_str = generate_chi_stream(machine, chi_pos, len(ciphertext))
    dc   = delta(ciphertext)   # ΔC[i] = C[i] XOR C[i+1]
    dchi = delta(chi_str)      # Δχ[i] = χ[i] XOR χ[i+1]
    count = 0
    for dc_char, dchi_char in zip(dc, dchi):
        xored = dc_char ^ dchi_char
        count += (5 - bin(xored).count("1"))  # count zero bits
    return count   # maximise this over all starting positions

The full project includes the complete SZ-40/42 machine with all 12 wheels and correct motor-controlled psi stepping, the depths attack demo recreating the Athens/Vienna 1941 mistake, the delta brute-force chi-finder, and a key period calculator that prints exactly how many years an exhaustive search would have taken.

Why It Mattered

Colossus mattered for three reasons that cascade across the history of computing.

It proved electronic computation worked at scale. The engineering community's scepticism about valve reliability was demolished by Colossus. Flowers ran his machines continuously for months without a valve failure caused by the machine itself (one machine was damaged when a worker accidentally kicked a valve from its socket). The principle, keep the valves powered,was vindicated. ENIAC, built in the United States and unveiled in 1946, used 18,000 valves and was influenced by the same engineering logic, though its designers did not know about Colossus.

It compressed time. The intelligence produced by reading Hitler's strategic communications, his orders to Rommel in North Africa, his response to the deception operations preceding D-Day, his insistence that the real Allied landing would come at Pas-de-Calais and not Normandy,has been assessed by historians as shortening the war in Europe by at least two years. Those two years mean millions of lives not lost. The computation done by Colossus is among the most consequential ever performed.

It established the gap between invention and credit. John Mauchly and J. Presper Eckert, builders of ENIAC, are widely credited as the creators of the first general-purpose electronic computer. The legal and historical record is murky: a US patent dispute ultimately credited prior work by John Atanasoff. Colossus predates ENIAC by two years but was unknown to the world for three decades. This is a cautionary lesson that still resonates: the history of computing, particularly its wartime and classified chapters, is full of invention that could not be acknowledged.

What Came Next

Even as Colossus hummed through its last wartime messages, a Hungarian-American mathematician named John von Neumann was drafting the architectural blueprint that every computer built since has followed. His 1945 "First Draft of a Report on the EDVAC" described a machine in which program instructions and data share the same memory, the stored-program architecture that Turing had described theoretically in 1936. Where Colossus was programmable by switch and plug, von Neumann's architecture would make the program itself just another kind of data. That story is next: The Thinking Machine Chronicles #0007: The Blueprint of Every Computer Ever Built.

---

References

1. Copeland, B. J. (ed.). (2006). Colossus: The Secrets of Bletchley Park's Codebreaking Computers. Oxford University Press. The definitive history; contains Flowers's own account and the declassified engineering documents.
2. Flowers, T. H. (1983). The Design of Colossus. Annals of the History of Computing, 5(3), 239–252. Flowers's first public technical account, written after declassification.
3. Good, I. J., Michie, D., & Timms, G. (2000). General Report on Tunny. GCHQ / Public Record Office. The original wartime Bletchley Park report on the Lorenz cipher and Colossus, declassified 2000. Available at The National Archives (HW 25/4, HW 25/5).
4. Tutte, W. T. (1998). Fish and I. Lecture at the University of Waterloo, 19 June 1998. Tutte's own account of reconstructing the Lorenz machine.
5. Copeland, B. J. (2004). The Essential Turing. Oxford University Press. Contains primary sources on the relationship between Turing's theoretical work and the wartime machines.
6. Sale, A. E. (2003). The Colossus of Bletchley Park, The German Lorenz Cipher System. Resurrection: The Bulletin of the Computer Conservation Society, 26. Technical overview of the Lorenz cryptanalysis.