Advances in Cryptography since World War II
World War II cryptography
By World War II mechanical and electromechanical cryptographic cipher machines were in wide use, although where these were impractical manual systems continued to be used. Great advances were made in both practical and mathematical cryptography in this period, all in secrecy. Information about this period has begun to be declassified in recent years as the official 50-year (British) secrecy period has come to an end, as the relevant US archives have slowly opened, and as assorted memoirs and articles have been published.
The Germans made heavy use (in several variants) of an electromechanical rotor based cipher system known as Enigma. Marian Rejewski, in Poland, attacked and 'broke' the early German Army Enigma system (an electromechanical rotor cipher machine) using theoretical mathematics in 1932. It was the greatest breakthrough in cryptanalysis in a thousand years and more. The Polish break into Enigma traffic continued up to '39, when changes in the way the German Army used Enigma required more resources to continue the break than the Poles had available. They passed their knowledge, some sample machines, on to the British and French that summer. Even Rejewski and his fellow mathematicians and cryptographers from the Biuro Szyfrow ended up with the British and French after the German blitzkrieg. The work was extended by Alan Turing, Gordon Welchman, and others at Bletchley Park leading to sustained breaks into several other of the Enigma variants and into the message traffic on the assorted networks for which they were used.
US Navy cryptographers (with cooperation from British and Dutch cryptographers after 1940) broke into several Japanese Navy crypto systems. The break into one of them, JN-25, famously led to the US victory in the Battle of Midway. A US Army group, the SIS, managed to break the highest security Japanese diplomatic cipher system (an electromechanical 'stepping switch' machine called Purple by the Americans) even before WWII began. The Americans referred to the intelligence resulting from cryptanalysis, perhaps especially that from the Purple machine, as 'Magic'. The British eventually settled on 'Ultra' for intelligence resulting from cryptanalysis, particularly that from message traffic enciphered by the various Enigmas. An earlier British term for Ultra had been 'Boniface'.
The German military also deployed several mechanical attempts at a one-time pad. Bletchley Park called them the Fish ciphers, and Max Newman and colleagues designed and deployed the world's first programmable digital electronic computer, the Colossus, to help with their cryptanalysis. The German Foreign Office began to use the one-time pad in 1919; some of this traffic was read in WWII partly as the result of recovery of some key material in South America that was insufficiently carefully discarded by a German courier.
Allied cipher machines used in WWII included the British TypeX and the American SIGABA; both were electromechanical rotor designs similar in spirit to the Enigma, though with major improvements. Neither is known to have been broken by anyone during the war. Troops in the field used the M-209 and less secure M-94 family. British SOE agents initially used 'poem ciphers' (memorized poems were the keys), but later in the war, they began to switch to one time pads.
The Poles had prepared for wartime by building the LCD Lacida cipher device, which was kept secret even to Rejewski. When in July 1941 its security was verified by Rejewski, it took him just a few hours to crack it. Operational Procedures for LCS Lacida, which had no reflector rotor, were changed in a hurry. The relevance of messages sent with Lacida were, however, not comparable to enigma traffic, but interception could have meant the end of the crucial polish cryptanalyst effort.
The era of modern cryptography really begins with Claude Shannon, arguably the father of mathematical cryptography. In 1949 he published the paper Communication Theory of Secrecy Systems in the Bell System Technical Journal and a little later the book, Mathematical Theory of Communication, with Warren Weaver. These, in addition to his other works on information and communication theory established a solid theoretical basis for cryptography and for cryptanalysis. And with that, cryptography more or less disappeared into secret government communications organizations such as the NSA. Very little work was again made public until the mid '70s, when everything changed.
An encryption standard
The mid-1970s saw two major public (i.e., non-secret) advances. First was the publication of the draft Data Encryption Standard in the U.S. Federal Register on 17 March 1975. The proposed DES was submitted by IBM, at the invitation of the National Bureau of Standards (now NIST), in an effort to develop secure electronic communication facilities for businesses such as banks and other large financial organizations. After 'advice' and modification by the NSA, it was adopted and published as a Federal Information Processing Standard Publication in 1977 (currently at FIPS 46-3). DES was the first publicly accessible cipher to be 'blessed' by a national agency such as NSA. The release of its specification by NBS stimulated an explosion of public and academic interest in cryptography.
DES was officially supplanted by the Advanced Encryption Standard (AES) in 2001 when NIST announced FIPS 197. After an open competition, NIST selected Rijndael, submitted by two Flemish cryptographers, to be the AES. DES, and more secure variants of it (such as 3DES or TDES; see FIPS 46-3), are still used today, having been incorporated into many national and organizational standards. However, its 56-bit key-size has been shown to be insufficient to guard against brute force attacks (one such attack, undertaken by the cyber civil-rights group Electronic Frontier Foundation, succeeded in 56 hours -- the story is in Cracking DES, published by O'Reilly and Associates). As a result, use of straight DES encryption is now without doubt insecure for use in new cryptosystem designs, and messages protected by older cryptosystems using DES, and indeed all messages sent since 1976 using DES, are also at risk. Regardless of its inherent quality, the DES key size (56-bits) was thought to be too small by some even in 1976, perhaps most publicly by Whitfield Diffie. There was suspicion that government organizations even then had sufficient computing power to break DES messages; clearly others have achieved this capability.
The second development, in 1976, was perhaps even more important, for it fundamentally changed the way crypto systems might work. This was the publication of the paper New Directions in Cryptography by Whitfield Diffie and Martin Hellman. It introduced a radically new method of distributing cryptographic keys, which went far toward solving one of the fundamental problems of cryptography, key distribution, and has become known as Diffie-Hellman key exchange. The article also stimulated the almost immediate public development of a new class of enciphering algorithms, the asymmetric key algorithms.
Prior to that time, all useful modern encryption algorithms had been symmetric key algorithms, in which the same cryptographic key is used with the underlying algorithm by both the sender and the recipient who must both keep it secret. All of the electromechanical machines used in WWII were of this logical class, as were the Caesar and Atbash ciphers and essentially all cipher and code systems throughout history. The 'key' for a code is, of course, the codebook, which must likewise be distributed and kept secret.
Of necessity, the key in every such system had to be exchanged between the communicating parties in some secure way prior to any use of the system (the term usually used is 'via a secure channel') such as a trustworthy courier with a briefcase handcuffed to a wrist, or face-to-face contact, or a loyal carrier pigeon. This requirement is never trivial and rapidly becomes entirely unmanageable when the number of participants increases beyond some (very!) small number, or when (really) secure channels aren't available for key exchange, or when, as is sensible cryptographic practice, keys are frequently changed. In particular, a separate key is required for each communicating pair if, as part of the crypto system design, no third party including another user is to be able to decrypt their messages. A system of this kind is known as a secret key, or symmetric key cryptosystem. D-H key exchange (and succeeding improvements and variants) made operation of these systems much easier, and more secure, than had ever been possible before.
In contrast, with asymmetric key encryption, there is a pair of mathematically related keys for the algorithm, one of which is used for encryption and the other for decryption. Some, but not all, of these algorithms have the additional property that one of the keys may be made public since the other cannot be (at least by any currently known method) deduced from the 'public' key. The other key in these systems must be kept secret and is usually called, somewhat confusingly, the 'private' key. An algorithm of this kind is known as a public key or asymmetric key system.
For those using such algorithms, only one key pair is needed per recipient (regardless of the number of senders) since possession of a recipient's public key (by anyone whomsoever) does not compromise the 'security' of messages so long as the corresponding private key is not known to any attacker (effectively, this means not known to anyone except the recipient). This quite surprising property of these algorithms made possible, and made practical, widespread deployment of high quality crypto systems which could be used by anyone at all.
Asymmetric key cryptography, Diffie-Hellman key exchange, and the best known of the public key / private key algorithms (i.e., what is usually called the RSA algorithm), all seem to have been independently developed at a UK intelligence agency before the public announcement by Diffie and Hellman in '76. GCHQ has released documents claiming that they had developed public key cryptography before the publication of Diffie and Hellman's paper. Various classified papers were written at GCHQ during the 1960s and 1970s which eventually led to schemes essentially identical to RSA encryption and to Diffie-Hellman key exchange in 1973 and 1974. Some of these have now been published, and the inventors (James Ellis, Clifford Cocks, and Malcolm Williamson) have made public (some of) their work.
This in turn broke the near monopoly on cryptography held by government organizations worldwide (see S Levy's Crypto for a journalistic account of the policy controversy in the US). For the first time ever, those outside government organizations had access to cryptography not readily breakable by government. Considerable controversy, and conflict, both public and private, began immediately. It has not yet subsided. In many countries, for example, export of cryptography is subject to restrictions. Until 1996 export from the U.S. of cryptography using keys longer than 40 bits was sharply limited. As recently as 2004, former FBI Director Louis Freeh, testifying before the 9/11 Commission, called for new laws against public use of encryption.
The most notable player in the advocacy of strong encryption for public use was Phil Zimmermann with his release of PGP (Pretty Good Privacy) in 1991. He distributed a freeware version of PGP when he felt threatened by legislation then under consideration by the US Government that would require back doors to be created in all cryptographic solutions developed within the US. His efforts in releasing PGP worldwide earned him a long battle with the Justice Department for the alleged violation of export restrictions. The Justice Department eventually dropped its case against Zimmerman, and the freeware distribution of PGP made its way around the world to become an open standard. (RFC2440 or OpenPGP)
While modern ciphers like AES are considered unbreakable, poor designs are still sometimes adopted and there have been notable cryptanalytic breaks. Three notable examples of crypto designs which have been broken are the first Wi-Fi encryption scheme WEP, the Content Scrambling System used for encrypting and controlling DVD use, and the A5/1 and A5/2 ciphers used in GSM cell phones. In addition, none of the mathematical ideas underlying public key cryptography have been proven to be 'unbreakable' and so some future advance might render systems relying on them insecure. No competent observer foresees such a breakthrough, however. And the key size recommended for security keeps increasing as improved computer power becomes cheaper and incremental improvements in such things as integer factorization are made. The contest between crypto designers and cryptanalysts has hardly ended.