As some of you may already know, I have been working on a book on API security for Manning in my spare time: API Security in Action. The book has now reach a point where the publisher are happy for it to go into early-access release (MEAP as they call it), so you can now pre-order and download the first 3 chapters on their website. Use discount code fccmadden to get 37% off when ordering.
The book covers the basics of securing remotely accessible APIs (REST) taking a ground-up approach, and then moves on in the later chapters to look at the specifics of securing microservice APIs in Kubernetes and even APIs for the Internet of Things (IoT). It covers lots of things you’d expect, like JSON Web Tokens and OAuth 2, and some things you perhaps wouldn’t like Waterken-style capability URLs and Macaroons. I’ve also taken an opinionated approach, which will come as no surprise to anybody who knows me. JWT is covered because its an important technology, but that doesn’t mean it gets a free pass. I’ve tried to separate the good parts from the bad, and warn you away from the real foot-guns.
The principle I’ve followed so far is that good practical advice (the “in Action” part) requires having a good understanding of how things actually work and what security properties each component provides. So rather than just throwing together a JWT library and an API gateway and showing a few deployment patterns, you’ll learn some of the deeper principles involved first. I start off with some basic secure development techniques and common attacks against REST APIs. We then look at the basic security controls of authentication, rate-limiting, access control, and audit logging. You’ll build primitive (but secure) versions of all these mechanisms from scratch before moving on to look at more fully-featured alternatives as the book progresses.
I hope you take a look. If you do, please leave feedback (good or bad) in the forum or email me personally. It’s been a much harder process writing it than I originally expected, trying to balance the desire to teach everything with a need to keep it simple enough to engage readers. Hopefully I’ve struck the right balance, but let me know either way. And of course, if you spot anything outright wrong then let me know about that too so I can fix it.
There is a common story that has played out many times in the history of computer science. A software engineer learns regular expressions for the first time, feels like a god, and then immediately over-applies them to situations they are not great at, like parsing HTML. What then usually happens is that a more experienced developer patiently explains why HTML cannot be parsed with regular expressions, because HTML is (at least) a context-free language, and you should use a proper HTML parser instead. In this post, I’ll explain why the advice is right, but the reasoning is wrong.
Note: in this post, I am talking about regular expressions as defined in theory, not the heavily extended regexes of Perl etc.
Continue reading “Why you really can parse HTML (and anything else) with regular expressions”
Hot on the heels of my dives into authenticated encryption and key-driven cryptographic agility, I thought I’d lay out a sketch of what I think a simpler core JOSE/JWT spec might look like. We’ll take in misuse-resistant crypto and Macaroons along the way. And cats. But first, a brief diversion into JSON.
Continue reading “Simplifying JOSE”
In Part I, we saw that authenticated encryption is usually the security goal you want in both the symmetric and public key settings. In Part II, we then looked at some ways of achieving public key authenticated encryption (PKAE), and discovered that it is not straightforward to build from separate signing and encryption methods, but it is relatively simple for Diffie-Hellman. In this final part, we will look at how existing standards approach the problem and how they could be improved.
Continue reading “Public key authenticated encryption and why you want it (Part III)”
Almost everyone hates Java’s checked exceptions. Even those that still use them will admit that they lead to a lot of boilerplate. Everyone has code along the following lines: Continue reading “A small tweak to checked exceptions”
If you need an unguessable random string (for a session cookie or access token, for example), it can be tempting to reach for a random UUID, which looks like this:
This is a 128-bit value formatted as 36 hexadecimal digits separated by hyphens. In Java and most other programming languages, these are very simple to generate:
String id = UUID.randomUUID().toString();
Under the hood this uses a cryptographically secure pseudorandom number generator (CSPRNG), so the IDs generated are pretty unique. However, there are some downsides to using random UUIDs that make them less useful than they first appear. In this note I will describe them and what I suggest using instead.
Continue reading “Moving away from UUIDs”
I’m seeing a large spike in attempted comment spam, so I’ve turned off all commenting for now.
Updated 20th July 2017 to clarify notation for the point of infinity. A previous version used the symbol 0 (zero) rather than O, which may have been confusing.
Updated 28th May 2020: in step 4 of the full validation check, n is the order of the prime sub-group defined by the generator point G, not the order of the curve itself. This is critical for security if you are performing this check because small-order points will satisfy the order of the curve (which is h * n), but not the order of G.
In the wake of the recent critical security vulnerabilities in some JOSE/JWT libraries around ECDH public key validation, a number of implementations scrambled to implement specific validation of public keys to eliminate these attacks. But how do we know whether these checks are sufficient? Is there any guidance on what checks should be performed? The answer is yes, but it can be a bit hard tracking down exactly what validation needs to be done in which cases. For modern elliptic curve schemes like X25519 and Ed25519, there is some debate over whether validation should be performed at all in the basic primitive implementations, as the curve eliminates some of the issues while high-level protocols can be designed to eliminate others. However, for the NIST standard curves used in JOSE, the question is more clear cut: it is absolutely critical that public keys are correctly validated, as evidenced by the linked security alert.
Continue reading “So how *do* you validate (NIST) ECDH public keys?”
I am sometimes asked whether doing a PhD was worth it, given that I left academia and research to become a full-time software developer. My answer is an unequivocal “yes”, despite the fact that my thesis is about as relevant to what I do now as a book on the sex lives of giraffes.
By far the most important skill I learnt during that time was not any particular technical knowledge, but rather a general approach to critical thinking—how to evaluate evidence and make rational choices. In a profession such as software engineering, where we are constantly bombarded with new technologies, products and architectural styles, it is absolutely essential to be able to step back and evaluate the pros and cons to form sensible technology choices. In this post I’ll try and summarise the approach I take to making these decisions.
Continue reading “Critical thinking for software engineers”
A new month a new job. I’m now working as a Java Application Developer for a large software consultancy. I’ll be developing Java EE applications for this job, which will be an interesting experience. I’m keen to really get my hands dirty with what is currently the de facto standard for building large business apps.
My first challenge is to get to grips with the Java Persistence API (JPA), which provides a standardised object-relational mapping (ORM) for Java applications. Basically, it takes a bunch of annotated Java classes and determines how best to map these to a relational database, taking care of foreign key constraints, inheritance, joins, and so forth. On the surface, this is all very good. Current best practice in developing these kinds of apps is to separate out the data model of the domain into a layer of simple Java beans (i.e., objects with properties and little custom behaviour), so something like JPA makes this much easier to knock together than using JDBC directly. (Although, it’s not that hard to use JDBC). Continue reading “Relations ain’t “flat””