Okay. The title might be click bait, just a little, but after you finish reading this post, I think you'll be a bit more careful picking your HMAC keys. After learning this, I know I will be. However, HMAC is not broken . It just has an interesting ... property that's worth knowing about.
First off, let's remind ourselves what HMAC is. HMAC, or Hashed Message Authentication Codes, are the ability to authenticate a cryptographic message. This is done through an asymmetric key agreement protocol, such as Diffie-Hellman, where two parties securely share symmetric keys. These keys are used to encrypt messages as well as authenticate data. HMAC tags prevent chosen plaintext attacks, where the attacker can insert malicious data into the payload (send $1,000,000 to my account), and HMAC tags prevent adaptive chosen ciphertext attacks where the attacker can send encrypted data to the server, and learn what is being protected (is "password" in the payload?).
Authenticated messages are absolutely essential to modern cryptographic software. If you're writing cryptographic software, and you're not authenticating your ciphertext, you're doing it wrong. It doesn't matter if the data is at rest or in motion. Authenticate your ciphertext. This is where HMAC fits in.
So, best practice, when not using native AEAD ciphers, such as AES-GCM, or ChaCha20-Poly1305, is to encrypt the plaintext then authenticate it with HMAC, and prepend or append the digest (called a "MAC tag" or just "tag") to the ciphertext.
Something like this pseudocode:
ciphertext = AES-256-CTR(nonce, plaintext, key1)tag = HMAC-SHA-512(ciphertext, key2)
ciphertext = tag || ciphertext
Then we ship off the newly bundled ciphertext and MAC tag. When it arrives at our destination, the recipient can verify if the ciphertext is what it should be, before decrypting it, by verifying the MAC tag, which is the whole point:
tag1 = ciphertext[:64]data = ciphertext[64:]
tag2 = HMAC-SHA-512(ciphertext, key2)
hmac_check = 0
for char1, char2 in zip(tag1, tag2):
hmac_check |= ord(char1) ^ ord(char2)
if hmac_check == 0:
plaintext = AES-256-CTR(nonce, data, key1)
else:
return False
return True
Notice that we're doing constant time comparison of the shipped tag and the calculated tag. Last thing we want is to introduce a timing attack by doing "if tag1 != tag2". However, after the constant time comparison, if tag1 and tag2 match, we can decrypt the data, and retrieve the plaintext.
So, now you have the background on HMAC, let's look at some examples with python, and see where HMAC breaks. After all, that's the click bait title, no?
>>> import os >>> import hmac >>> import hashlib >>> key = os.urandom(16) >>> msg = os.urandom(256) >>> hmac.new(key, msg).hexdigest() 'd5a94f051b1e6ff67065b6f4c3a60130'In this example, the default HMAC is HMAC-MD5. HMAC-MD5 is still considered cryptographically secure, even though vanilla MD5 is broken. Regardless, it'll suffice for this example, and we'll look at SHA-1 and SHA-2 also.
In RFC 2104, where HMAC is standardized, section 3 has this odd little tidbit (emphasis mine):
The key for HMAC can be of any length (keys longer than B bytes are
). However, less than L bytes is stronglydiscouraged as it would decrease the security strength of the
function. Keys longer than L bytes are acceptable but the extra
length would not significantly increase the function strength. (A
longer key may be advisable if the randomness of the key is
considered weak.)
The "B bytes" length is the block size of the underlying HMAC operations. If the key is shorter than this block size, zeroes need to be appended to the key. If the key is longer than this block size, then it is hashed with the HMAC cryptographic hash.
The block size is 64 bytes for the following HMACs:
HMAC-MD5 HMAC-RIPEMD128 HMAC-RIPEMD160 HMAC-SHA1In other words, HMAC wants to key to be exactly one block in length. If it's longer, it's hashed with zeros appended to fit exactly into one block.
Can we test this?
>>> key = os.urandom(65) # longer than one block >>> msg = os.urandom(256) >>> hmac.new(key, msg).hexdigest() 'f887a4146e94ed47405c97931798885d' >>> hmac.new(hashlib.md5(key).digest(),msg).hexdigest() 'f887a4146e94ed47405c97931798885d'We have a collision. In other words:
For:* 'H' a cryptographic hash
* 'k' a private key
* 'm' a message
* 'B' an HMAC block size
HMAC(k, m) == HMAC(H(k), m)
for all 'k', where len(k) > B
Does this work with HMAC-SHA1?
>>> key = os.urandom(65) >>> msg = os.urandom(256) >>> hmac.new(key, msg, hashlib.sha1).hexdigest() '1070312944223b36928382d7a53ca54f7204ad4a' >>> hmac.new(hashlib.sha1(key).digest(), msg, hashlib.sha1).hexdigest() '1070312944223b36928382d7a53ca54f7204ad4a'How about all the SHA-2 functions? (SHA-224, SHA-256, SHA-384, & SHA-512)?
SHA-224:
>>> key = os.urandom(65) >>> msg = os.urandom(256) >>> hmac.new(key, msg, hashlib.sha224).hexdigest() '9ea8c3f667e55e6e9c5d63c5dd1b569ca69e2cc69f5e3fa3f87e94ba' >>> hmac.new(hashlib.sha224(key).digest(), msg, hashlib.sha224).hexdigest () '9ea8c3f667e55e6e9c5d63c5dd1b569ca69e2cc69f5e3fa3f87e94ba'SHA-256:
>>> key = os.urandom(65) >>> msg = os.urandom(256) >>> hmac.new(key, msg, hashlib.sha256).hexdigest() '2aa02e678fcfe7ecaa1475efb70fe284fe91cc81e5a9c543433b70f5f5112c4b' >>> hmac.new(hashlib.sha256(key).digest(), msg, hashlib.sha256).hexdigest () '2aa02e678fcfe7ecaa1475efb70fe284fe91cc81e5a9c543433b70f5f5112c4b'SHA-384:
>>> key = os.urandom(65) >>> msg = os.urandom(256) >>> hmac.new(key, msg, hashlib.sha384).hexdigest() '0941e6502233a72d01beeec729eaa7db2469f8ce96339cd5b3b2c9a4684501e6a7025fac 6c9c20a511c48df76b453ec3' >>> hmac.new(hashlib.sha384(key).digest(), msg, hashlib.sha384).hexdigest () '5380905fb89fee68836be076ebfccff600e3b89c6554840fe61fed01b049d6a6a77423d2 f5f4be1afb9d1c6f63b8b7fc'SHA-512:
>>> key = os.urandom(65) >>> msg = os.urandom(256) >>> hmac.new(key, msg, hashlib.sha512).hexdigest() '36063bdc2d02ce8ea4b01b40ba040094c640959e0cc5716f7a75f119cbc348aa93d555f8 6bfcdaee5dad4ec2e5d53ed4362f9df0720ec0e1272288d49a912f7e' >>> hmac.new(hashlib.sha512(key).digest(), msg, hashlib.sha512).hexdigest () '8bfe675e3ca35a8680243d3747b5d3ce7ded1731e1a307cf5d1b00ae9243395ab94039f2 2585b417d7cbdf09f3d8dcf39c85ce147ff77c901c1a21f8de981b6a'So it appears that the block size is indeed 64 bytes for MD5, SHA-1, SHA-224, and SHA-256, but for SHA-384 and SHA-512, it doesn't appear to be working. That is because the block size has changed to 128 bytes for these two functions. So, if our key is 129 bytes, we should be able to replicate collisions:
SHA-384 with a 129-byte key:
>>> key = os.urandom(129) >>> msg = os.urandom(256) >>> hmac.new(key, msg, hashlib.sha384).hexdigest() 'bda2b586637e3bd73a27919601d7d5a1c1743f1f9f5cb72a0aa874f832046f4bc396ff8e 307f9318dc404c4b432ca491' >>> hmac.new(hashlib.sha384(key).digest(), msg, hashlib.sha384).hexdigest () 'bda2b586637e3bd73a27919601d7d5a1c1743f1f9f5cb72a0aa874f832046f4bc396ff8e 307f9318dc404c4b432ca491'SHA-512 with a 129-byte key:
>>> key = os.urandom(129) >>> msg = os.urandom(256) >>> hmac.new(key, msg, hashlib.sha512).hexdigest() 'd0153f8bb6a549539abbcff8ee5ac7592c48c082bbbb7b3cc95dcb2166f162e5c59bb7bb 3316e65d1481bd8697e8d3bc91deb46ad44845b972c57766f45c54bd' >>> hmac.new(hashlib.sha512(key).digest(), msg, hashlib.sha512).hexdigest () 'd0153f8bb6a549539abbcff8ee5ac7592c48c082bbbb7b3cc95dcb2166f162e5c59bb7bb 3316e65d1481bd8697e8d3bc91deb46ad44845b972c57766f45c54bd'This isn't a poor Python implementation of HMAC. Try it in your favorite language, and you should be able to replicate the collisions. This is a "bug" in HMAC. If the key is longer than the block size, it's hashed with the HMAC cryptographic hash, then appended with zeros to fit the single block.
So what does this mean? It means that when choosing your HMAC keys, you should stay within one block size of bytes- 64 bytes or less for MD5, RIPEMD-128/160, SHA-1, SHA-224, SHA-256, and 128-bytes or less for SHA-384 and SHA-512. If you do this, you'll be fine.
Then again, you should probably be using NaCl or libsodium rather than piecing these cryptographic primitives manually yourself. These sorts of pitfalls are already handled for you.
This post is the result of a Twitter discussion between Scott Arciszewski, myself, and some others.
@adamcaudill @ErrataRob Aside from the fact that HMAC-MD5(m, k) and HMAC-MD5(m, MD5(k)) collide, when len(k) > 16
― Scott Arciszewski (@CiphperCoder) July 28, 2016
UPDATE:I'm not aware of any actual security implications where this is a problem that two distinct inputs produce the same HMAC digest. Ultimately, what are you trying to get out of HMAC? It's used as an authentication and integrity mechanism. So what if a long key, and the hash of that key produce the same HMAC digest? At 64 bytes, or 512-bits, the amount of work required at guessing the right key is anything but practical. Still, this is interesting.
