Least privilege with less effort: Macaroon access tokens in AM 7.0

Part 1: Lightweight PoP

One of the major changes between OAuth 1 and OAuth 2 was the decision to drop the requirement that requests had to be signed using a secret key associated with each access token. This signing process was notoriously hard to get right and dropping it from OAuth 2 enormously simplified the life of client and resource server developers. In OAuth 2, access tokens are pure bearer tokens, a bit like passwords: if you know the value of the token you can use it, without needing any other credentials. This is very easy for developers, but is definitely less secure. The rise of single-page apps (SPAs) and other browser-based clients leaves access tokens more exposed to theft or accidental leakage compared to traditional server-side clients.

Drinking the PoP

Signed requests in OAuth 1 are an example of proof-of-possession (PoP) tokens rather than bearer tokens. If bearer tokens are like cash, PoP tokens are like Chip-and-PIN. Even if you steal an OAuth 1 token you can’t use it without the associated secret key used to sign requests. The only problem was it turned out that even if you did have the secret key you often couldn’t use the token either, because it was just too hard to get request signing to work reliably. Some brave souls periodically try and revive this idea.

To get around the problems with request signing, various alternative schemes have been proposed. Brian Campbell gave a nice overview of the history of OAuth PoP schemes at Identiverse recently, called The Burden of Proof. It’s a good talk and worth watching. The latest such scheme is DPoP, which tries to keep things as simple as possible: rather than signing the whole request, you just sign enough basic details of the request (the URI and HTTP method) to be able to stop the most likely attacks and rely on TLS to protect the content of the request.

DPoP is good, and for some use-cases I think it’s really good, but it still has some downsides:

  • Although the signature process itself is simplified, the client still has to generate and manage this secret key securely, which adds complexity that many client developers will avoid unless they have to.
  • The client, authorization server, and resource servers all have to know about DPoP. It’s quite hard to incrementally deploy.
  • It relies on fresh public key signatures on every request. Public key crypto is slow and expensive, so you generally want to do it as little as possible. I could imagine a rollout of DPoP requiring a significant increase in front-line servers to handle the extra load.
  • DPoP is based on JWS, which supports a wide range of signature algorithms, with no guidance as to which one to use. This means the resource servers have to potentially support them all (including some really expensive ones like ES512), or risk rejecting some clients. Clients will also have to guess which algorithms the server supports when they generate the private key. In practice this means that DPoP will only be used in closed deployments, or else everyone will gravitate to a lowest common denominator option like 2048-bit RSA with PKCS#1 v1.5 padding.
  • Public key signature algorithms are fragile. The point of a signature is that it can be verified by anybody, but in the case of an OAuth request the DPoP token only needs to be verified by exactly one party: the server that you’re sending the request to. There are much simpler and safer ways to achieve request authentication in this case.

Taking the biscuit

OK, so if DPoP is still too complex, what would a better solution look like? The risk that PoP solutions are trying to prevent is an access token being stolen from one request and used to authorize completely different requests that the client didn’t intend. A different way to solve this problem would be to have individual access tokens for every request that just authorizes that one specific request and no others. For example, it could have an audience restriction limiting it to only be used for requests to one specific server, and a single scope that authorizes just the specific type of API call that is being made. The expiry time could be limited to just a few seconds in the future, and so on. In current OAuth implementations it’s pretty hard to restrict an access token to exactly one request, but you can get pretty close.

Of course, this all sounds like a nightmare. The client can’t keep going back to the user to authorize access tokens for every little request! 

It turns out there is a way to get lots of little access tokens for individual requests. It’s easy for developers, relatively simple to deploy, and cheap. (Much cheaper than digital signatures). It’s also quite delicious.

Chocolate coated macaroons

Enter macaroons

Apart from being a tasty treat, macaroons are also a new token format invented by the boffins at Google. At first glance, a macaroon is an authenticated token a bit like a JWT using the HS256 algorithm. But the additional yummy goodness that macaroons add is the ability to add caveats. A caveat is a restriction on how the token can be used. For example, you might limit the scope of a token, or reduce its expiry time. They always reduce, never expand, the authority of the token. The really cool thing is that adding a caveat gives you a new token, leaving the original token unchanged. It’s also really cheap: a few microseconds typically. This means you can have a single powerful access token and then derive multiple more restricted access tokens from it. This lets us have our cake and eat it (sorry, my metaphors are all over the place today): you can get a single token approved by the user but then derive individual access tokens for every single API call you make, with just the perfect amount of privilege for that one call.

A picture of macaroon access tokens, showing how new tokens can be derived with reduced scope, expiry, or audience.

This gives us the benefits of having lots of tiny individual access tokens for each request, but we can generate these tokens on the fly from the original token. This is what the original macaroons paper refers to as contextual caveats: specific details of the context in which an API call is taking place that we add to a token to restrict its use. That way, if the token does get stolen or misused, the scope of what you can do with it is severely limited by the caveats attached to it. The genuine client still has the original access token though, so they can still do whatever they like within the scope of the original grant.

If we restrict the tokens enough then they become pretty hard to misuse. We get much of the same benefits of a PoP approach but with much less complexity. Where you might have had code like the following to call an API:

fetch('https://api.example.com/v1/kittens', {
    method: 'POST',
    headers: {
        'Authorization': 'Bearer ' + token
    },
    body: JSON.stringify(kittenDetails)
});

The new macaroon-enhanced version becomes something as simple as the following with an appropriate library:

fetch('https://api.example.com/v1/kittens', {
    method: 'POST',
    headers: {
        'Authorization': 'Bearer ' + token.restrict({
            aud: 'api.example.com',
            exp: nowPlus5Seconds(),
            scope: 'upload_cute_kitten_picture'
        })
    },
    body: JSON.stringify(kittenDetails)
});

Macaroon OAuth tokens in ForgeRock AM 7.0

At ForgeRock we love OAuth and we also love macaroons. We think they make a great combination. OAuth benefits from the simple but flexible approach to least privilege that macaroons enable, and macaroons benefit from the framework and tooling that exists around OAuth. I strongly believe that macaroons provide about 90% of the benefit of PoP for about 20% of the effort.

That’s why we’ve included support for issuing OAuth access and refresh tokens as macaroons in the upcoming 7.0 release of ForgeRock Access Management. So how does it work? Well, mostly you just go into the OAuth2 Provider Settings and turn it on:

Image of toggle button to turn on "Use Macaroon Access and Refresh Tokens"

Now when you complete an OAuth flow you get issued with a macaroon access token and optional refresh token. They look just like normal access tokens, but are a little bit longer:

Screenshot of Postman showing macaroon access and refresh tokens

You can call the standard token introspection endpoint to see the details of the token, just like you could with a normal access token:

Screenshot showing JSON response from the OAuth token introspection endpoint, showing the scope and other token details.

You can then add a bunch of caveats just like in the example earlier, to reduce the scope, expiry time, and audience and introspect it again to see the difference:

The updated JSON response from the introspection endpoint, showing the reduced scope, expiry, and audience due to the caveats added to the token.

Repeating the token introspection request a few seconds later, the restricted access token has expired:

Screenshot showing the active: false response

But the original access token is still valid. In fact, all the details of the original token in the database (AM’s Core Token Service) are exactly as they were when the token was first issued.

Everything is handled transparently through the token introspection endpoint, making it really easy to deploy this incrementally:

  1. In step 1, you simply turn on macaroon access tokens at the Authorization Server. So long as your resource servers are using token introspection, everything can carry on as normal. Clients and resource servers need no changes.
  2. In step 2, your clients start adding caveats to tokens using a macaroon library. Resource servers carry on introspecting the tokens, but now the introspection responses take into account the caveats on the tokens.
  3. There is no step 3.

That’s a little taster of macaroon access tokens in ForgeRock AM 7.0 and why I think they will be a game changer. Least privilege with less effort. In part 2, I’ll show you some more exotic things you can do with macaroon access tokens using 3rd-party caveats.

Convergence in web and API security

Every now and then technologies that initially appear to be distinct end up converging on a common approach from opposite directions. I believe that something like that is happening right now in approaches to web and API authentication around the use of tokens and cookies.

Continue reading “Convergence in web and API security”

Can you ever (safely) include credentials in a URL?

URLs are a cornerstone of the web, and are the basic means by which content and resources are shared and disseminated. People copy and paste URLs into Slack or WhatsApp to share interesting links. Google crawls the web, discovering and indexing such links. But what happens when the page you want to link is not public and requires credentials in order to view or interact with it? Suddenly a URL is no longer sufficient, unless the recipient happens to already have credentials. Sometimes they do, and everything is fine, but often they do not. If we really do want to give them access, the problem becomes how to securely pass along some credentials with the URL so that they can access the page we have linked.

A commonly desired approach to this problem is to encode the credentials into the URL itself. While convenient, this solution is fraught with dangers and frequently results in credentials being exposed in insecure contexts. In this article, we’ll look at various ways to accomplish this, the ways that things can go wrong, and conclude with a set of guidelines for cases where this can be made secure, and actually improve security overall.

Continue reading “Can you ever (safely) include credentials in a URL?”

Public key authenticated encryption and why you want it (Part II)

In Part I, I made the argument that even when using public key cryptography you almost always want authenticated encryption. In this second part, we’ll look at how you can actually achieve public key authenticated encryption (PKAE) from commonly available building blocks. We will concentrate only on approaches that do not require an interactive protocol. (Updated 12th January 2019 to add a description of a NIST-approved key-agreement mode that achieves PKAE).

Continue reading “Public key authenticated encryption and why you want it (Part II)”

Key-driven cryptographic agility

One of the criticisms of the JOSE/JWT standards is that they give an attacker too much flexibility, by allowing them to specify how a message should be processed. In particular, the standard “alg” header tells the recipient what cryptographic algorithm was used to sign or encrypt the contents. Letting the attacker chose the algorithm can lead to disastrous security vulnerabilities. Continue reading “Key-driven cryptographic agility”

Java KeyStores – the gory details

Java KeyStores are used to store key material and associated certificates in an encrypted and integrity protected fashion. Like all things Java, this mechanism is pluggable and so there exist a variety of different options. There are lots of articles out there that describe the different types and how you can initialise them, load keys and certificates, etc. However, there is a lack of detailed technical information about exactly how these keystores store and protect your key material. This post attempts to gather those important details in one place for the most common KeyStores.
Continue reading “Java KeyStores – the gory details”

So how *do* you validate (NIST) ECDH public keys?

Updated 20th July 2017 to clarify notation for the point of infinity. A previous version used the symbol 0 (zero) rather than O, which may have been confusing.

Updated 28th May 2020: in step 4 of the full validation check, n is the order of the prime sub-group defined by the generator point G, not the order of the curve itself. This is critical for security if you are performing this check because small-order points will satisfy the order of the curve (which is h * n), but not the order of G.

In the wake of the recent critical security vulnerabilities in some JOSE/JWT libraries around ECDH public key validation, a number of implementations scrambled to implement specific validation of public keys to eliminate these attacks. But how do we know whether these checks are sufficient? Is there any guidance on what checks should be performed? The answer is yes, but it can be a bit hard tracking down exactly what validation needs to be done in which cases. For modern elliptic curve schemes like X25519 and Ed25519, there is some debate over whether validation should be performed at all in the basic primitive implementations, as the curve eliminates some of the issues while high-level protocols can be designed to eliminate others. However, for the NIST standard curves used in JOSE, the question is more clear cut: it is absolutely critical that public keys are correctly validated, as evidenced by the linked security alert.

Continue reading “So how *do* you validate (NIST) ECDH public keys?”

Updating OpenAM’s encryption

Updated 30th March 2017 to reflect updated information (see comments), add additional links and add some clarifying text about why misuse-resistance is useful.

With the impending release of the ForgeRock Identity Platform, I thought I’d spend some time writing up a few of the bits of OpenAM 14 that I was directly involved with creating. One of my last acts before leaving FR to go solo, was to put in place the first phase of modernising AM’s aging system credential encryption scheme. Before I start, I should say that this encryption scheme is not used for encrypting user passwords (which are hashed by the LDAP user store, not AM). Instead, this scheme is used for encrypting various system credentials (passwords for SMTP servers, HMAC shared secrets, etc) in the config store and in exported system configurations and in a few other places.

The original (and still default) encryption method was first mentioned in Dante’s Inferno. Actually it dates from the original iPlanet codebase from the mid-90s, and uses correspondingly ancient cryptographic algorithms (MD5 and DES). It is best to regard it as providing only limited obfuscation of credentials, rather than any true security guarantees, and the advice has always been to secure the config store by traditional means (TLS, access controls) rather than rely on this encryption. Still, we can do much better than this now, so AM 14 ships with a new AESWrapEncryption scheme that provides significantly improved security:

Continue reading “Updating OpenAM’s encryption”

Should you use JWT/JOSE?

In the wake of some more recent attacks against popular JSON Web Token (JWT)/JSON Object Signing and Encryption (JOSE) libraries, there has been some renewed criticism of the JWT/JOSE standards themselves (see also discussion on lobste.rs with an excellent comment from Thomas Ptacek summarising some of the problems with the standard). Given these criticisms, should you use JOSE at all? Are articles like my recent “best practices” one just encouraging adoption of bad standards that should be left to die a death?

Certainly, there are lots of potential gotchas in the specs, and it is easy for somebody without experience to shoot themselves in the foot using these standards. I agree with pretty much all of the criticisms levelled against the standards. They are too complicated with too many potentially insecure options. It is far too easy to select insecure combinations or misconfigure them. Indeed, much of the advice in my earlier article can be boiled down to limiting which options you use, understanding what security properties those options do and do not provide, and completely ignoring some of the more troublesome aspects of the spec. If you followed my advice of using “headless” JWTs and direct authenticated encryption with a symmetric key, you’d end up not far off from the advice of just encrypting a JSON object with libsodium or using Fernet.

So in that sense, I am already advocating for not really using the specs as-is, at least not without significant work to understand them and how they fit with your requirements. But there are some cases where using JWTs still makes sense:

  • If you need to implement a standard that mandates their use, such as OpenID Connect. In this case you do not have much of a choice.
  • If you need to interoperate with third-party software that is already using JWTs. Again, in this case you also do not have a choice.
  • You have complex requirements mandating particular algorithms/parameters (e.g. NIST/FIPS-approved algorithms) and don’t want to hand-roll a message format or are required to use something with a “standard”. In this case, JWT/JOSE is not a terrible choice, so long as you know what you are doing (and I hope you do if you are in this position).

If you do have a choice, then you should think hard about whether you need the complexity of JWTs or can use a simpler approach that takes care of most of the choices for you or store state on the server and use opaque cookies. In addition to the options mentioned in the referenced posts, I would also like to mention Macaroons, which can be a good alternative for some authorization token use-cases and the existing libraries tend to build on solid foundations (libsodium/NaCl).

So, should you use JWT/JOSE at all? In many cases the answer is no, and you should use a less error-prone alternative. If you do need to use them, then make sure you know what you are doing.

Ephemeral elliptic curve Diffie-Hellman key agreement in Java

If you like this post, you might like my book: API Security in Action (use discount code fccmadden to get 37% off when ordering).

Update 2 (17th May, 2017): I’ve written some notes on correctly validating ECDH public keys.

Update (20th April, 2017): I’ve noticed that this article gets by far the most daily hits on my blog. This worries me that people are using this code as a template for building real ECDHE key agreement, when it was only intended as a guide to the Java API. There are a lot of details in safe construction of such a protocol. More secure alternatives than to trying to roll this yourself include the various complete protocols listed at the end of the article. With that said, we’ll get back to the original article:

Diffie-Hellman key agreement (DH) is a way for two parties to agree on a symmetric secret key without explicitly communicating that secret key. As such, it provides a way for the parties to negotiate a shared AES cipher key or HMAC shared secret over a potentially insecure channel. It does not by itself provide authentication, however, so it is vulnerable to man-in-the-middle attacks without additional measures. There are several ways to provide these additional measures (e.g. signing the ephemeral public keys using a CA-issued certificate, or using a protocol like OTR), but we will not discuss them here, or go into the details of how the key agreement works. Java provides support out-of-the-box for both original discrete log DH and elliptic curve (ECDH) key agreement protocols, although the latter may not be supported on all JREs. ECDH should be preferred for any new applications as it provides significantly improved security for reasonable key sizes.

elliptic curve
An elliptic curve defined by y2 = x3 – 2x + 2

As is often the case in Java, the use of these classes can be a bit convoluted. Here we demonstrate simple Java code for ECDH key agreement on the command line. We only demonstrate ephemeral key agreement, in which the two parties generate unique public/private key pairs at the start of the protocol and throw them away once the shared secret has been negotiated. This can form the basis for perfect forward secrecy.

WARNING: the code here is not a complete security protocol and should be used for reference on the Java API only.

Continue reading “Ephemeral elliptic curve Diffie-Hellman key agreement in Java”