Selecting a HAProxy backend using Lua

Once you’ve learned the basics of using Lua in HAProxy, you start to see a lot of places the scripting language could be useful. At Cloudant, one of the places we saw that we could make use of Lua was when selecting from the various backends to which our frontend load balancers direct traffic. We wrote a simple proof of concept, which I wanted to document here along with some of the problems we hit along the way.

Say we wanted to choose a backend based on the first component of the request path (i.e., a in /a/something/else). We actually don’t do this at Cloudant, but it is a simple, not-quite-totally-trivial demo.

When using HAProxy 1.5, you’d do something like this:

frontend proxy
  ... other settings ...

  # del-header ensures that we're using 'new' headers
  http-request del-header x-backend
  http-request del-header x-path-first

  http-request set-header x-path-first path,word(1,/)

  acl is_backend_set hdr_len(x-backend) gt 0
  acl path_first_a %[req.hdr(x-path-first)] -m a
  acl path_first_b %[req.hdr(x-path-first)] -m b

  http-request set-header x-backend a if path_first_a !is_backend_set
  http-request set-header x-backend b if path_first_b !is_backend_set
  http-request set-header x-backend other if !is_backend_set

  http-request del-header x-path-first

  use_backend %[req.hdr(x-backend)]

backend a
  ...

backend b
  ...

backend other
  ...

In outline, this code uses a couple of temporary headers to store the first path component and the backend we choose, combined with ACLs as guards to make sure that the right ordering priority is used for backends. In particular, the is_backend_set ACL prevents us always using the other backend.

This is fairly concise, but in my experience gets complicated quickly. Moreover, it hides the fact that the logic is essentially an imperative if...else if...else statement.

Thankfully, HAProxy 1.6 introduces both variables and Lua scripting, which we can use to make things clearer and safer, if not particularly shorter.

Variables

We can use variables to replace the use of headers for temporary data. Setting and retrieving looks like this:

http-request set-var(req.path_first) path,word(1,/)
acl path_first_a %[var(req.path_first)] -m a

This isn’t any shorter, but it does reduce the chance of a malicious request slipping in a header that affects processing.

Variables all have a scope: req variables are only available in HAProxy’s request phase; res in the response phase; and txn are stored and available in both.

Lua

Variables are nice, but are a fairly straightforward feature. Lua allows us to get a bit more interesting. Instead of the header/acl dance, we can now write the backend-switching logic more explicitly.

Assuming that we put the Lua code in a file called select.lua alongside the HAProxy configuration file:

global
  lua-load select.lua
  ... other settings ...

frontend proxy
  ... other settings ...

  # Store the backend to use in a variable, available in both request
  # and response (txn-scope)
  http-request set-var(txn.backend_name) lua.backend_select()

  # Use the backend_name txn variable
  use_backend %[var(txn.backend_name)]

Here, we use a Lua sample fetch function. Sample fetch is a HAProxy term for any function – whether in-built or written in Lua – that processes the HTTP transaction and returns a value calculated using the transaction details. The Lua function is automatically passed details of the request as part of the transaction details.

The backend returned is put into a variable in case it’s needed elsewhere. A txn scoped variable can be used in both request and response phases; using one, you could add a header to the response containing the chosen backend, for example. If this wasn’t needed, you could put the backend_select fetch directly into the use_backend line.

Warning: One thing that we found when trying out this code is that we couldn’t do what we used to and store the return value from the Lua code in a HTTP request header. If we did that, for some reason HAProxy returned a 503 status code, that is, the use_backend statement appeared to be trying to use a non-existent backend. Swapping to a variable fixed this.

The Lua code contained in select.lua ends up being straightforward:

-- Work out the backend name for a given request's HTTP path
core.register_fetches("backend_select", function(txn)

  # txn.sf contains HAProxy's in-built sample-fetches, like the HTTP path
  local path = txn.sf:path()
  local path_first = string.match(path, '([^/]+)')

  if path_first == 'a' then
    return 'a'
  elseif path_first == 'b' then
    return 'b'
  else
    return 'other'
  end
end)

In outline:

core is a class exposed globally by HAProxy. One of the uses of core is to register Lua functions for use in HAProxy. The register_fetches call registers our sample fetch under the name backend_select. The sample fetch is a Lua function, declared inline in the call.
The first part of the sample fetch function uses the txn argument. HAProxy provides this argument automatically to all Lua functions registered as sample fetches. The txn argument provides access to both the request context and a lot of the in-built HAProxy fetches for accessing data from the request. We use one of the fetches, path, to retrieve the path.
We take the first part of the path using Lua’s match function, which we can make perform a split-like behaviour.
Finally, we can do the if/else statement and return the backend name to use.

For me, after learning the basics of Lua, the most complicated part of this was figuring out what’s available on the txn variable. The Lua documentation directs you towards the standard HAProxy documentation, but I found it a bit hard to generate quite the right Lua code to access the fetches that HAProxy exposes (probably due to my unfamiliarity with terms like sample fetch when I started this proof of concept and that I’m new to Lua).

And there you have it. Once you get the right code, it’s quite short, but it took a few days to figure out all the moving parts from scratch.