Updating OpenAM’s encryption

Updated 30th March 2017 to reflect updated information (see comments), add additional links and add some clarifying text about why misuse-resistance is useful.

With the impending release of the ForgeRock Identity Platform, I thought I’d spend some time writing up a few of the bits of OpenAM 14 that I was directly involved with creating. One of my last acts before leaving FR to go solo, was to put in place the first phase of modernising AM’s aging system credential encryption scheme. Before I start, I should say that this encryption scheme is not used for encrypting user passwords (which are hashed by the LDAP user store, not AM). Instead, this scheme is used for encrypting various system credentials (passwords for SMTP servers, HMAC shared secrets, etc) in the config store and in exported system configurations and in a few other places.

The original (and still default) encryption method was first mentioned in Dante’s Inferno. Actually it dates from the original iPlanet codebase from the mid-90s, and uses correspondingly ancient cryptographic algorithms (MD5 and DES). It is best to regard it as providing only limited obfuscation of credentials, rather than any true security guarantees, and the advice has always been to secure the config store by traditional means (TLS, access controls) rather than rely on this encryption. Still, we can do much better than this now, so AM 14 ships with a new AESWrapEncryption scheme that provides significantly improved security:

• Key derivation using PBKDF2–HMAC–SHA1 using at least 10,000 rounds (configurable) and random 128-bit salts per encrypted item as per latest NIST guidelines. (You can also configure PBKDF2 to use SHA2 if your JRE supports it – not all of ForgeRock’s supported platforms do out of the box).
• Authenticated encryption using NIST-approved AES Key Wrapping (RFC 3394) with support for 128-, 192- or 256-bit key encryption key (KEK).

Why AES Key Wrapping?

AES Key Wrapping is a slightly unusual mode that isn’t used much, so why do we use it here? It provides deterministic authenticated encryption (DAE) of key material and does not require any random IV or nonce. The deterministic part is not currently utilised (but would be needed if you intended to use this scheme in conjunction with session ID encryption in the CTS, which relies on a deterministic encryption scheme Update: I am informed that the new release uses SHA-256 hashing rather than deterministic encryption to hide session storage keys). However, the lack of a random IV requirement provides misuse resistance that means that the encryption scheme cannot be compromised by programmer error in selecting a random IV, or by poor quality/compromised random number generators. This comes at a cost of reduced performance, but is worth the effort to protect keys and other credentials because these are both  high-value—compromise of a key can have severe consequences—and short in length, so the increased cost is negligible in overall processing.

The deterministic output of the construction appears to be a weakness against chosen plaintext attacks, as the same plaintext encrypted with the same key will produce the same output. However, this problem is defeated by the use of random salts in the key derivation process (PBKDF2): i.e., we will never (with high probability) reuse the same key to encrypt two different payloads. Failure to provide random/non-repeating salt to the KDF will result in a potential information leak, but still provide DAE security, unlike (say) a nonce reuse in AES-GCM that can result in a catastrophic loss of security. The payloads in this case are also themselves random keys and other credentials, so the plaintext is itself unpredictable. This means that both the key and the plaintext are highly unlikely to be reused.

AES Key Wrapping therefore provides a strong basis for encrypting system credentials, but it does also have some drawbacks. It is quite inefficient, requiring several passes of AES (6 if memory serves) over each block of input data, although this is less of a concern given we are just encrypting relatively short keys. It’s also a rather unusual mode that is not very widely used as far as I can tell (Update 18th April: it turns out that Apple use it quite extensively in iOS [pdf] for key wrapping). There are more modern misuse-resistant modes suitable for key wrapping, such as AES SIV mode or its GCM updated version. However, AES-KW has the advantage of being widely implemented by ForgeRock’s supported JREs, which avoids hand-rolling the crypto or spending a lengthy period evaluating a move to Bouncy Castle or another third-party crypto provider.

When wouldn’t I use this?

I mentioned at the start that this is the first phase of providing improved credential encryption, and it is not yet a full solution. There are a number of drawbacks that must still be addressed by ForgeRock in the future:

• There is no upgrade path to re-encrypt existing configuration values—the alternative encryption mode must be enable at installation time and only for fresh installs.
• Some functionality, such as CTS session encryption will not work yet.
• It is highly likely that there will be compatibility problems with policy agents in some configurations, but I haven’t tested these personally yet.
• OpenIG password replay functionality will likely not work with the new scheme. Update: OpenIG password replay uses a different mechanism for password encryption.

OK, so there is still quite a bit of work to do (I’ve no idea if FR have done some of this since I left). In short, you can enable this if you are using OpenAM in a relatively self-contained way, for instance as a standalone OAuth2 provider.

How do I turn it on?

If you think that your use-case will fit in with the above caveats, then you can enable it by setting the com.iplanet.security.encryptor system property to the value org.forgerock.openam.shared.security.crypto.AESWrapEncryption in your container startup properties (e.g. in Tomcat’s setenv.sh) before beginning installation of OpenAM. The following system properties can then be used to control the behaviour of the encryption algorithm and key derivation:

• org.forgerock.openam.encryption.key.size – determines the size of AES key to derive for encryption, can be 128, 192 or 256. Key sizes larger than 128 bits will need the JCE Unlimited Strength policy files to be installed (on Mac with Homebrew just brew cask install jce-unlimited-strength-policy). The default is 128-bit encryption.
• org.forgerock.openam.encryption.key.digest – determines the underlying message digest (hash) function to use with PBKDF2-HMAC to derive the key. The default is SHA1 as this is widely available. It is recommended to use SHA-512 where supported (NB: you need to use “SHA512” without the hyphen to configure this—JCE avoids naming consistency wherever possible).
• org.forgerock.openam.encryption.key.iterations – the number of PBKDF2 iterations to use. Must be at least 10,000, which is also the default.

Shameless plug: If you would like to see this work back-ported to older versions of OpenAM or expanded to cover some of the missing functionality, feel free to contact me as an independent consultant to discuss your requirements.