Like coding more, I’d like to go see more live music this year. Just after new year, I poked around gig listings for Bristol and came across The Future Icons Tour feat George O’Hanlon & Lizzie Esau + How Nice. While I’d not heard of any of them before, I enjoyed the music in the videos on the page, listened to a few more tracks on Spotify and convinced a friend to take a punt on the gig with me.

Finally, after 10 years, 146 lines of my own in CouchDB 💪.

I joined Cloudant just over ten years ago, when it was still a startup of thirty. And only now have I made my first PR to CouchDB, the open-source database that undergirds Cloudant’s service. Why so long? Well, I spent those ten years playing my part by building many parts of the service around the database.

But I’ve always harboured a desire to work more on the database. Reflecting over Christmas, I felt that I’d really like to make this year the one where I build up my skills in both Erlang and the database.

Having made this decision, this PR is a nice thing to have gotten done within the first month of 2023. While not groundbreaking, it’s not trivial and so I’m pleased with it for a first effort. The PR improves query processing by shifting some work down to the machines hosting the database shards, reducing cluster network traffic. The only real complexity lies in not breaking things while the cluster is in a mixed-version upgrade state – the query is distributed so can be being processed simultaneously by old-version and new-version nodes. Ensuring that behaviour is always correct as nodes are upgraded at different times across the cluster took a little head-scratching.

My favourite thing about the work is that it lays some groundwork for other improvements in query performance, such as those described in this RFC I’ve written. After that, I’m not yet sure where to look. I’ve been digging into a few deeper bits on database querying which could be interesting. But there are many other places I could usefully contribute.

It’s nice to be writing more challenging code again.

In Evaluating conversations with ChatGPT I wondered about how much we can rely on AI to help us do stuff when it has a somewhat gung ho relationship with the truth. I came across the beginnings of the academic research into this area in Lin et al, 2021. I also found a more recent paper, Wei et al, 2022, that discusses the ways that increased scale of models have produced unexpected step changes in this area and others.

Things are improving shockingly quickly.

I enjoyed, and found a kind of solace, in All Human Systems are Enormous Trash Fires.

Realizing this can be revelatory. Once you recognize that all human systems are enormous trash fires, you stop trying to figure out how to switch to a system that isn’t an enormous trash fire, since they don’t exist. Instead, you ask better questions about your current trash fire. Like, “Am I doing everything I can to contain this enormous trash fire, even though I know it will never go out?”; “Do the people in charge recognize that this whole place is an enormous trash fire?”; and, most importantly, “Am I surrounded by a team of firefighters or a team of arsonists?”

We’re imperfect beings in the extreme, and the organisations we create are as often the sum of the imperfections as they are of our better attributes. But that doesn’t mean we can’t look to have left things a little better each time we step away.

Leaving the enormous trash fire functioning better, just a bit.

When using ChatGPT, I had an idea to ask it to summarise an article. Seeing it do well, I wondered about other uses of summarisation. One item that struck me is using generative AI to improve how we interact with search, for example in apps like Obsidian or Evernote.

It went like this. Search hasn’t changed much in a long time. We’ve got a bit better at ranking results, but the experience of search is a list of results. Each must be examined to see if it answers the question. What if instead search results could be presented in a summarised way? This would be particularly useful for queries whose underlying goal was “tell me what I know about X?”.