I ported my toy database index to Rust and I liked it

  • rust coding database-diy •  6 minutes

I like Go a lot, and have been writing production Go for about seven years now. It runs some critical bits of Cloudant.

But I thoroughly enjoyed rewriting my Go toy document database into a Rust toy document database. So I can say now that I also like Rust a lot.

(My view on Go, Rust, Zig, Swift and other recent compiled-but-nice languages is that we have an embarrassment of riches available to us these last few years compared to what went before. For so long systems programming was “C/C++ or go home”. In my Rust book there’s a quote, “Rust: systems programmers can have nice things too”. I think that’s correct.)

(But back to what we were talking about.)

At this point, I have just finished getting the Rust code to feature equivalence with the Go code. It’s slightly better, even: the Rust code can handle arrays, which I never got to for the Go code. The Rust codebase is 1,076 lines of code while the Go codebase is 1,210 lines of code — pretty comparable. While a thousand lines of code isn’t very much, I feel that I’ve learned a lot of Rust along the way. I’d feel comfortable writing Rust in a production environment, albeit hopefully with some other people who were able to correct my still-a-novice screw-ups.

There were several places I found Rust outshone Go for this project. So instead of talking about what I’ve learned — which would be confusing for people who don’t know Rust and boring for those who do know it — I’ll talk about the parts of Rust that made this rewrite so enjoyable.

Those were:

  • Strictness
  • Representing JSON
  • Result and Option
  • Expressions all return things

It’s all a bit honeymoon period, I think; Rust and me right now. I’m sure there are frustrating things with Rust, but this project hasn’t exposed them (yet). So be generous. Spare me a moment to enjoy the honeymoon 🌇, and let’s suspend our inner cynics and talk about these things I liked.

Strictness

I found this while I was writing mdanchored, my first, very small excursion into rust. There’s something about the way the compiler — more the borrow-checker, but also the sophisticated type system that’s put to such good use throughout Rust codebases — is so damn strict. It feels fresh somehow, after working for a long time in the liberal worlds of Go and Python, to have a compiler that’s such a pedant.

In the main, my Go and Python programs crashed all the freakin’ time during development (arguably I should read more about my programming practice than Go and Python into that, but 🤷). When building mdanchored and docdb-rust, by contrast, they don’t crash at all. Of course I still wrote a bunch of bugs, but they didn’t cause my program to blow up in an ugly mess.

It helps that I believe the Rust compiler to be enforcing a set of rules that I want my code to follow already. And that I know that it’s hard work to ensure that I’m following these self-imposed rules. So when it takes me a long time to write code the compiler will accept, I feel that the compiler is helping me write the code that I wanted to write anyway, and that it would’ve taken me even longer to get right without the compiler’s help. (And I’d have probably found out about the problems when it crashed in production 😬).

I’m still torn about whether this strictness is an overall accelerant. It does make progressing through coding a bit slower. But I think that what I end up with is less likely to cause problems later. My suspicion is still that fewer interruptions later to debug probably increases overall throughput.

Representing JSON

Go’s type system is easy to learn, but isn’t that expressive. A clear-cut case of this is how I had to represent generic JSON as map[string]interface{}, a type that not even its mother can call handsome. Working with interface{} is a pain in the arse, quite frankly, as it can be literally anything; I just have to assume the user hasn’t given me some other junk. And it couldn’t represent some valid JSON texts, like [1,2,3] or "just a string".

Contrast this with Rust, which is able to use its enum type to represent all possible JSON structures within its type system. It’s a thing of sparse beauty:

enum Value {
    Null,
    Bool(bool),
    Number(Number),
    String(String),
    Array(Vec<Value>),
    Object(Map<String, Value>),
}

This is from serde_json, the defacto JSON parsing package in Rust. The ability to represent arbitrary JSON as a strongly-typed object made so many things easier and more obvious.

I ended up using this union-type pattern in a few other places. It’s fab.

Result and Option

I still don’t quite get how to use them ergonomically in my own code, but Result and Option feel like the Right Way to handle return values that are not straightforward.

I like Go’s thing, err = ... pattern, I think the regularity is great. Result and Option take this to the next level, and my code is better for it. No more to say here really.

Expressions all return things

I like this in Erlang, and it’s one of the few things I like in Ruby. Expressions like match, case, if or what-have-you all return values. There are many places where this offers a jump in readability.

let ids = match qp {
    QP::E { p, v } => lookup_eq(db, p, v)?,
    QP::GT { p, v } => lookup_gt(db, p, v)?,
    QP::GTE { p, v } => lookup_gte(db, p, v)?,
    QP::LT { p, v } => lookup_lt(db, p, v)?,
    QP::LTE { p, v } => lookup_lte(db, p, v)?,
};

For me, this code communicates very clearly that the whole point of the match is to choose the right lookup function for the query predicate (QP) type. This is hidden more in the Go code, because it’s a big if statement where each branch has its own ids, err = ... line. The repetition obscures the fact that the same variable is being assigned each time.

Plus, used well, it’s just elegant.

Okay, one perhaps-bad thing

The thing I like about Go is that Go doesn’t give you many ways to do things. One go codebase works much like another. With Rust, I can feel that the flexibility of the language must invite different approaches.

A good example is that most Go HTTP frameworks look the same. Basically everything is method chaining. In contrast, Rocket and axum look very different to each other.

Rocket relies on deriving traits:

#[put("/<id>", data = "<msg>")]
fn update(db: &Db, id: Id, msg: Json<Message<'_>>) -> Value {
    /* do something */
}

Whereas axum, like Go frameworks, relies on method chaining:

async fn get_foo() { /** do something **/ }

async fn main() {
    let app = Router::new().route("/", get(get_foo));
    // ... 
}

The approaches are really different compared to the Go ecosystem. Perhaps it’s bad, perhaps it’s not. But it does mean that Rust codebases are not quite so uniform.

But it’s basically good

Overall, however, as I said, I’m super-happy I learned more Rust, but I still have a big place in my heart for Go (and Python, and Erlang, and others; I like languages).

For this toy database experiment, I’ve decided to continue in Rust. It’s a really great learning project, and I want to get to a future-me where I can confidently say “yup, Rust’s the right choice for this project” — which means writing more of it.

Long may bring your own database index continue!

← Older
Found languages: Inko and Gleam
→ Newer
In defense of defensiveness