Embedding files in Go binaries
I recently ported github-to-omnifocus to Go. One new thing I learned is that it’s easy to embed files into Go binaries, extending Go’s “single file” deployment simplicity to include required non-Go resources.
In this case, I needed to call JXA scripts from Go. In github-to-omnifocus, these are required to interface with Omnifocus itself; they create, read, update and complete tasks. I wanted a single binary executable still, and feared I’d need to embed the JXA scripts as strings within the binary. That would make them hard to debug as I’d not be able to run them standalone. Fortunately, Go 1.16 introduced compiler support for embedding files into Go binaries, alongside an API to read them at runtime.
This is handled using the
embed package. The way that
embed works is that you sit the files to embed alongside your source file, and use a compiler directive inside your code to tell the compiler to embed the file. Within your code, the embedded resource can be read using a variable associated with the resource.
Embedding a single file
import "embed" //go:embed hello.txt var f embed.FS
go:embed compiler directive tells the compiler to embed the file
hello.txt that is in the same directory as the Go file. This file is made available to your code through the
f variable that follows the
To use the file, one uses the
embed.FS variable to “read” the data:
data, _ := f.ReadFile("hello.txt") print(string(data))
Embedding the contents of a directory
To embed a directory, the
go:embed directive uses the path to the directory. Finding this, the compiler will embed all the files in the directory. In
github-to-omnifocus, I have a directory of JXA scripts in a
jxa folder alongside my Go file. In
omnifocus.go, I embed them using:
import "embed" var ( //go:embed jxa jxa embed.FS )
I can then read each file using a file path, including the original
jsCode, _ := jxa.ReadFile("jxa/ofaddnewtask.js")
If you are curious, this code shows how I execute a script by calling the
More complex scenarios
go:embed directive accepts patterns, and multiple file patterns, matched using
path.Match. There can be multiple lines of
go:embed before the variable that is used to access them. The example in the
embed package tells you what you need to know:
import "embed" // content holds our static web server content. //go:embed image/* template/* //go:embed html/index.html var content embed.FS
In addition, a single file can be embedded “directly”: if the variable type is
byte instead of
embed.FS and the
go:embed is a single file reference, the compiler will ensure the variable is initialised with the contents of the file:
import _ "embed" //go:embed hello.txt var s string
Note the use of
_ in the
import line, which tells the compiler not to balk at the “unused”
embed package – though it isn’t in the source file, it is used to initialise the variable so needs to be there. A wart, nothing more.
I was pleasantly surprised by the ease of embedding and using files within Go. The API feels very well thought out, and very concise. I can see how it’d make it easy to embed all the resources of a web service within the code, as shown by the more complex example.
Alongside Go, I write Python. Since I started to write Go four years ago, I’ve always been impressed by its deployment story: copy a single binary to where you want to run it. This is bliss compared to deploying a Python application. The simple, yet effective, way that Go allows embedding resources extends this simplicity in deployment. While I’d like to learn more about the inner workings – to understand how embedding large amounts of files, or large files, would affect performance and resource usage – for now, I’m content to leave that for another day.