Selecting a HAProxy backend using Lua
Once you’ve learned the basics of using Lua in HAProxy, you start to see a lot of places the scripting language could be useful. At Cloudant, one of the places we saw that we could make use of Lua was when selecting from the various backends to which our frontend load balancers direct traffic. We wrote a simple proof of concept, which I wanted to document here along with some of the problems we hit along the way.
Say we wanted to choose a backend based on the first component of the request path (i.e., a
in /a/something/else
). We actually don’t do this at Cloudant, but it is a simple, not-quite-totally-trivial demo.
When using HAProxy 1.5, you’d do something like this:
frontend proxy
... other settings ...
# del-header ensures that we're using 'new' headers
http-request del-header x-backend
http-request del-header x-path-first
http-request set-header x-path-first path,word(1,/)
acl is_backend_set hdr_len(x-backend) gt 0
acl path_first_a %[req.hdr(x-path-first)] -m a
acl path_first_b %[req.hdr(x-path-first)] -m b
http-request set-header x-backend a if path_first_a !is_backend_set
http-request set-header x-backend b if path_first_b !is_backend_set
http-request set-header x-backend other if !is_backend_set
http-request del-header x-path-first
use_backend %[req.hdr(x-backend)]
backend a
...
backend b
...
backend other
...
In outline, this code uses a couple of temporary headers to store the first path component and the backend we choose, combined with ACLs as guards to make sure that the right ordering priority is used for backends. In particular, the is_backend_set
ACL prevents us always using the other
backend.
This is fairly concise, but in my experience gets complicated quickly. Moreover, it hides the fact that the logic is essentially an imperative if...else if...else
statement.
Thankfully, HAProxy 1.6 introduces both variables and Lua scripting, which we can use to make things clearer and safer, if not particularly shorter.
Variables
We can use variables to replace the use of headers for temporary data. Setting and retrieving looks like this:
http-request set-var(req.path_first) path,word(1,/)
acl path_first_a %[var(req.path_first)] -m a
This isn’t any shorter, but it does reduce the chance of a malicious request slipping in a header that affects processing.
Variables all have a scope: req
variables are only available in HAProxy’s request phase; res
in the response phase; and txn
are stored and available in both.
Lua
Variables are nice, but are a fairly straightforward feature. Lua allows us to get a bit more interesting. Instead of the header/acl dance, we can now write the backend-switching logic more explicitly.
Assuming that we put the Lua code in a file called select.lua
alongside the HAProxy configuration file:
global
lua-load select.lua
... other settings ...
frontend proxy
... other settings ...
# Store the backend to use in a variable, available in both request
# and response (txn-scope)
http-request set-var(txn.backend_name) lua.backend_select()
# Use the backend_name txn variable
use_backend %[var(txn.backend_name)]
Here, we use a Lua sample fetch function. Sample fetch is a HAProxy term for any function – whether in-built or written in Lua – that processes the HTTP transaction and returns a value calculated using the transaction details. The Lua function is automatically passed details of the request as part of the transaction details.
The backend returned is put into a variable in case it’s needed elsewhere. A txn
scoped variable can be used in both request and response phases; using one, you could add a header to the response containing the chosen backend, for example. If this wasn’t needed, you could put the backend_select
fetch directly into the use_backend
line.
Warning: One thing that we found when trying out this code is that we couldn’t do what we used to and store the return value from the Lua code in a HTTP request header. If we did that, for some reason HAProxy returned a 503
status code, that is, the use_backend
statement appeared to be trying to use a non-existent backend. Swapping to a variable fixed this.
The Lua code contained in select.lua
ends up being straightforward:
-- Work out the backend name for a given request's HTTP path
core.register_fetches("backend_select", function(txn)
# txn.sf contains HAProxy's in-built sample-fetches, like the HTTP path
local path = txn.sf:path()
local path_first = string.match(path, '([^/]+)')
if path_first == 'a' then
return 'a'
elseif path_first == 'b' then
return 'b'
else
return 'other'
end
end)
In outline:
core
is a class exposed globally by HAProxy. One of the uses ofcore
is to register Lua functions for use in HAProxy. Theregister_fetches
call registers our sample fetch under the namebackend_select
. The sample fetch is a Lua function, declared inline in the call.- The first part of the sample fetch function uses the
txn
argument. HAProxy provides this argument automatically to all Lua functions registered as sample fetches. Thetxn
argument provides access to both the request context and a lot of the in-built HAProxy fetches for accessing data from the request. We use one of the fetches,path
, to retrieve the path. - We take the first part of the path using Lua’s
match
function, which we can make perform a split-like behaviour. - Finally, we can do the
if/else
statement and return the backend name to use.
For me, after learning the basics of Lua, the most complicated part of this was figuring out what’s available on the txn
variable. The Lua documentation directs you towards the standard HAProxy documentation, but I found it a bit hard to generate quite the right Lua code to access the fetches that HAProxy exposes (probably due to my unfamiliarity with terms like sample fetch when I started this proof of concept and that I’m new to Lua).
And there you have it. Once you get the right code, it’s quite short, but it took a few days to figure out all the moving parts from scratch.