Links: September 2023

The challenges of building things with language models, problems with peer reviewing of scientific papers, MongoDB’s new query execution engine and comparing operational automation with Iron Man.

Enjoy.

• Squish Meets Structure: Designing with Language Models

  I really enjoyed Maggie Appleton’s talk about designing with large language models. It’s a fascinating discussion of how we are trying to understand how to work with these new tools, which work in different ways to how we are used to.

  We’re trying to make an unpredictable and opaque system adhere to our rigid expectations for how computers behave. We currently have a mismatch between our old and new mental models for computer systems.

  And the slides are somehow beautiful.

• The rise and fall of peer review - by Adam Mastroianni

  Peer review in the sciences has come in for a bit of a bludgeoning recently. This piece argues that it’s adding little value. If anything, that peer review is actively hostile to new ideas and serves mostly to entrench existing hierarchies (and a large publishing industry).

  Can we do better?

• Inside New Query Engine of MongoDB | Nikita Lapkov

  In my day job at Cloudant, I think a lot about how we could make our database better. I enjoy any and all deep dives into how other systems work. I find it helps create a library of patterns that I can later match against as I dig into problems.

  Here we learn how MongoDB created an idea called slots, which they used to significantly improve the efficiency of moving data values through their query execution pipeline.

• Automation Should Be Like Iron Man, Not Ultron - ACM Queue

  Also at Cloudant, we’ve found automation that increases and magnifies the abilities of our operators to be the best kind of automation. It allows us to manage more and more machines, while also increasing the flexibility we have in managing load across the system. Because it is magnifying our abilities, instead of replacing them, it allows us to maintain understanding of the system. Our understanding comes in very useful when things go wrong that automation does not cover.

  This article sits well with this experience, and I can see myself referring to it over time to explain our approach.