Selecting a HAProxy backend using Lua
Once you’ve learned the basics of using Lua in HAProxy, you start to see a lot of places the scripting language could be useful. At Cloudant, one of the places we saw that we could make use of Lua was when selecting from the various backends to which our frontend load balancers direct traffic. We wrote a simple proof of concept, which I wanted to document here along with some of the problems we hit along the way.
Say we wanted to choose a backend based on the first component of the request path (i.e.,
/a/something/else). We actually don’t do this at Cloudant, but it is a simple, not-quite-totally-trivial demo.
When using HAProxy 1.5, you’d do something like this:
frontend proxy ... other settings ... # del-header ensures that we're using 'new' headers http-request del-header x-backend http-request del-header x-path-first http-request set-header x-path-first path,word(1,/) acl is_backend_set hdr_len(x-backend) gt 0 acl path_first_a %[req.hdr(x-path-first)] -m a acl path_first_b %[req.hdr(x-path-first)] -m b http-request set-header x-backend a if path_first_a !is_backend_set http-request set-header x-backend b if path_first_b !is_backend_set http-request set-header x-backend other if !is_backend_set http-request del-header x-path-first use_backend %[req.hdr(x-backend)] backend a ... backend b ... backend other ...
In outline, this code uses a couple of temporary headers to store the first path component and the backend we choose, combined with ACLs as guards to make sure that the right ordering priority is used for backends. In particular, the
is_backend_set ACL prevents us always using the
This is fairly concise, but in my experience gets complicated quickly. Moreover, it hides the fact that the logic is essentially an imperative
if...else if...else statement.
Thankfully, HAProxy 1.6 introduces both variables and Lua scripting, which we can use to make things clearer and safer, if not particularly shorter.
We can use variables to replace the use of headers for temporary data. Setting and retrieving looks like this:
http-request set-var(req.path_first) path,word(1,/) acl path_first_a %[var(req.path_first)] -m a
This isn’t any shorter, but it does reduce the chance of a malicious request slipping in a header that affects processing.
Variables all have a scope:
req variables are only available in HAProxy’s request phase;
res in the response phase; and
txn are stored and available in both.
Variables are nice, but are a fairly straightforward feature. Lua allows us to get a bit more interesting. Instead of the header/acl dance, we can now write the backend-switching logic more explicitly.
Assuming that we put the Lua code in a file called
select.lua alongside the HAProxy configuration file:
global lua-load select.lua ... other settings ... frontend proxy ... other settings ... # Store the backend to use in a variable, available in both request # and response (txn-scope) http-request set-var(txn.backend_name) lua.backend_select() # Use the backend_name txn variable use_backend %[var(txn.backend_name)]
Here, we use a Lua sample fetch function. Sample fetch is a HAProxy term for any function – whether in-built or written in Lua – that processes the HTTP transaction and returns a value calculated using the transaction details. The Lua function is automatically passed details of the request as part of the transaction details.
The backend returned is put into a variable in case it’s needed elsewhere. A
txn scoped variable can be used in both request and response phases; using one, you could add a header to the response containing the chosen backend, for example. If this wasn’t needed, you could put the
backend_select fetch directly into the
Warning: One thing that we found when trying out this code is that we couldn’t do what we used to and store the return value from the Lua code in a HTTP request header. If we did that, for some reason HAProxy returned a
503 status code, that is, the
use_backend statement appeared to be trying to use a non-existent backend. Swapping to a variable fixed this.
The Lua code contained in
select.lua ends up being straightforward:
-- Work out the backend name for a given request's HTTP path core.register_fetches("backend_select", function(txn) # txn.sf contains HAProxy's in-built sample-fetches, like the HTTP path local path = txn.sf:path() local path_first = string.match(path, '([^/]+)') if path_first == 'a' then return 'a' elseif path_first == 'b' then return 'b' else return 'other' end end)
coreis a class exposed globally by HAProxy. One of the uses of
coreis to register Lua functions for use in HAProxy. The
register_fetchescall registers our sample fetch under the name
backend_select. The sample fetch is a Lua function, declared inline in the call.
- The first part of the sample fetch function uses the
txnargument. HAProxy provides this argument automatically to all Lua functions registered as sample fetches. The
txnargument provides access to both the request context and a lot of the in-built HAProxy fetches for accessing data from the request. We use one of the fetches,
path, to retrieve the path.
- We take the first part of the path using Lua’s
matchfunction, which we can make perform a split-like behaviour.
- Finally, we can do the
if/elsestatement and return the backend name to use.
For me, after learning the basics of Lua, the most complicated part of this was figuring out what’s available on the
txn variable. The Lua documentation directs you towards the standard HAProxy documentation, but I found it a bit hard to generate quite the right Lua code to access the fetches that HAProxy exposes (probably due to my unfamiliarity with terms like sample fetch when I started this proof of concept and that I’m new to Lua).
And there you have it. Once you get the right code, it’s quite short, but it took a few days to figure out all the moving parts from scratch.