Implementing AWS authentication for your own REST API

If you need to build an authentication mechanism for an HTTP-based REST API, a common approach is to use HTTP Basic – it’s simple, all clients have it built-in, it’s easy to test from the browser, and you can store passwords as hashes. The downside is that your credentials are transmitted in (nearly) plain text, which makes SSL (with its associated security restrictions and computational cost) a necessity.

If you’d like to implement a simple scheme for a non-sensitive API that doesn’t require SSL, this is more complicated. HTTP Digest requires storing unhashed passwords on the server, and requires a challenge-response conversation between server & client. Schemes like Kerberos and three-legged oath require hanging your hat on a third-party authentication provider, and are awkward to implement in a client.

Luckily, in software if you hit a problem you can usually copy somebody else’s solution. This is what the Microsoft Azure team did when implementing their API authentication – “Let’s just copy what Amazon does“. Amazon probably copied someone else. Who am I to argue with that approach?

The general concept behind these schemes is relatively simple:

  1. Come up with a way to generate API & secret keys for a client. These are usually base64-encoded cryptographically generated random byte arrays.
  2. For each request, hash a string containing the requested URL and specific headers (including the Date header) with the secret key using HMAC-SHA1.
  3. Add an Authorize header with a custom scheme name (e.g. ‘AWS’), containing the access key & base64-encoded signature separated by a colon.

The client goes through this process to generate the Authorize header, then the server performs a reverse of the procedure using the stored secret key to authenticate the request. Additionally, the server checks the Date header value against server time to check for replay attacks.

Below is the source for an AuthorizeAttribute subclass (for an MVC3 REST API). The code is easily adaptable to other frameworks, such as an OpenRasta PipelineContributor. The injected Session property in this instance is an NHibernate Session, and the ApiKey class is mapped to a database table. Successful authentication adds a custom IPrincipal to the HttpContext. Note that none of the x-aws headers are being used.

public class ApiAuthenticateAttribute : AuthorizeAttribute
    private static System.Text.UTF8Encoding utf8 = new System.Text.UTF8Encoding();
    public ISession Session { private get; set; }

    public override void OnAuthorization(AuthorizationContext filterContext)
        var request = filterContext.HttpContext.Request;
        IPrincipal principal = null;

        if (request.Headers["Authorization"] != null && request.Headers["Authorization"].StartsWith("AWS "))
            // Amazon AWS authentication scheme.
            var credential = filterContext.HttpContext.Request.Headers["Authorization"].Substring(4).Split(':');
            var apiKey = Session.Query<ApiKey>().Where(k => k.AccessKey == credential[0]).FirstOrDefault();
            if (apiKey != null && !apiKey.IsDisabled && credential.Count() > 1)
                // check the date header is present & within 15 mins
                DateTime clientDate;
                if (request.Headers["Date"] != null
                    && DateTime.TryParseExact(request.Headers["Date"], "R", DateTimeFormatInfo.CurrentInfo, DateTimeStyles.AdjustToUniversal, out clientDate)
                    && Math.Abs((clientDate - DateTime.UtcNow).TotalMinutes) <= 15)
                    // build the signature & check for match
                    var stringToSign = String.Format("{0}\n{1}\n{2}\n{3}\n{4}",
                        request.Headers["Content-MD5"] ?? "",
                        request.Headers["Content-Type"] ?? "",
                        request.Headers["Date"] ?? "",

                    var hmac = new HMACSHA1(utf8.GetBytes(apiKey.SecretKey));
                    var signature = Convert.ToBase64String(hmac.ComputeHash(utf8.GetBytes(stringToSign)));
                    if (signature == credential[1])
                        principal = apiKey.ToPrincipal();

        if (principal == null)
             filterContext.Result = new HttpUnauthorizedResult();
            filterContext.HttpContext.User = principal;

Generating API Keys can be done like so:

public ApiKey()
    // Generate random keys by using RNGCryptoServiceProvider & Base64 encoding the output
    // Key lengths match AWS keys.
    var rngProvider = RNGCryptoServiceProvider.Create();
    var bytes = new byte[15];
    // Do some magic to ensure we have uppercase & digits only.
    AccessKey = Convert.ToBase64String(bytes).ToUpper().Replace("+", "0").Replace("/", "9");
    bytes = new byte[30];
    SecretKey = Convert.ToBase64String(bytes);

For the client, most languages have freely available Amazon client code that can be easily adapted. Reusing a popular scheme like this saves a lot of time & energy over rolling a completely custom solution, particularly where a number of disparate client platforms are likely to be used.


Building a REST application server in Haskell Part 1

As mentioned previously, I’m boning up on Haskell. I prefer to learn by doing, so I’m attempting to build a functioning application as part of the process. I’ve struggled to find much entry level (real application) source code, so I’ll be posting my progress as I go in the hope that it will help others undertaking the same journey. Note the emphasis is on ‘entry level’ — I’m not pretending to be a Haskell guru; if you are one, pointing out what I’m doing wrong would be much appreciated.

This application is a hypothetical REST-based continuous integration status aggregator — CI servers publish build results to the server, then mobile clients view the results. It seemed a little bit more interesting than another blog app tutorial. I’ll be using the popular happstack web application framework, serving JSON over its built-in web server.

First, the setup. Assuming you have a GHC distribution and cabal installed on your computer (on OS X, the Haskell platform binary installer includes these), you’ll need to install happstack and JSON:

cabal update
cabal install happstack
cabal install json

If you get happstack build errors, see my previous post for a possible resolution.

First step is returning some JSON from the server. The following code is a minimal implementation that serialises some hard-coded test data and returns it in response to a web request:

{-# LANGUAGE DeriveDataTypeable #-}
module Main where

import Happstack.Server
import Text.JSON.Generic

data Project =
	Project {
		name :: String,
		lastStatus :: String,
		builds :: [Build]
	deriving (Data, Typeable)

data Build =

	Build {
		buildName :: String,
		status :: String
	deriving (Data, Typeable)

testProjectJSON = encodeJSON (Project "New Project" "Succeeded" [Build "2010-10-02 build 2" "Succeeded", Build "2010-10-01 build 1" "Failed"])

main = simpleHTTP nullConf $ ok $ toResponse testProjectJSON

I’m using Text.JSON.Generic and the DeriveDataTypeable language extension to automatically generate mappings from my custom algebraic data types (Project & Build) to JSON. Using the standard Text.JSON library would require implementing showJSON and readJSON functions manually, eg something like:

instance JSON Project where
	showJSON (Project name lastStatus builds) = showJSON makeObj [("name", toJSString name), ("lastStatus", toJSString lastStatus), ("builds", showJSONs builds)]

My gut feel, although this is probably the OO developer in me talking, is that automatic generation would be better suited to simple serialisation problems, and explicit serialisation more useful in instances where C#/Java devs would think ‘DTO’ (eg producing a flattened, version-tolerant data contract).

Running the application should result in the following highly enlightening data being served from http://localhost:8000/

{"name":"New Project","lastStatus":"Succeeded","builds":[{"buildName":"2010-03-23","status":"Succeeded"},{"buildName":"2010-10-01","status":"Failed"}]}