Sometimes you have to write down an intent publicly to make sure you carry it through to the end. And so here is one such statement, to encourage me to complete this vital part of toykv.
I’ve started work on Simple Compaction · Issue #24 · mikerhodes/toykv. I’d been putting it off for a while — it felt big, and tackling it in the short evening blocks of time I usually have seemed a bit overwhelming.
To make it more manageable, I spent the first hour dividing the work into several “versions”, creating an iterative path to the simplest compaction scheme: merge all sstables into a single sorted run. This broke the back of the task: three hours in, I’ve now reached version 2 of 7.
The top-level method is deceptively, but pleasingly, simple:
// Attempt to run a compaction on the stored data. The exact
// compaction carried out is opaque to the caller.
pub fn compact(&mut self) -> Result<(), ToyKVError> {
// v2 compaction just compacts all existing sstables
// in L0 into a single L0 table. This is implemented
// with SimpleCompactionPolicy.
let policy = SimpleCompactionPolicy::new();
// Guard ensures only one compaction can happen at a time.
let _guard = CompactionGuard::acquire(self.state.clone())?;
// Get a table compaction task from the sstables structure.
let c_task = {
let state = self.state.read().unwrap();
state.sstables.build_compaction_task_v2(policy)?
};
// Compaction task is safe to run outside state lock, to
// allow reads and writes to continue while we compact.
let c_result = c_task.compact_v2()?;
// Commit the compacted table while holding write lock.
{
let mut state = self.state.write().unwrap();
state.sstables.try_commit_compaction_v2(c_result)?;
}
self.metrics.compacts.fetch_add(1, Ordering::Relaxed);
Ok(())
}
And so here is my stake in the ground, my statement to say, “Behold, for this is something I intend to complete!” 😬