Using Generative AI in note-taking

When using ChatGPT, I had an idea to ask it to summarise an article. Seeing it do well, I wondered about other uses of summarisation. One item that struck me is using generative AI to improve how we interact with search, for example in apps like Obsidian or Evernote.

It went like this. Search hasn’t changed much in a long time. We’ve got a bit better at ranking results, but the experience of search is a list of results. Each must be examined to see if it answers the question. What if instead search results could be presented in a summarised way? This would be particularly useful for queries whose underlying goal was “tell me what I know about X?”.

I had thought that one might do this by doing the search, then using heuristics to grab paragraphs within the results, and finally pushing those relevant paragraphs through a generative API to create a more readable summary.

I was excited to see that (unsurprisingly!) others had this same idea, and had actually started experimenting rather than just thinking:

So I made a chatbot that will answer questions using my own directory of personal notes!

It’s ugly as heck and there’s so much more I want to add, but it performs fantastically so far! I have 20k notes so being able to extract info this easily is 👍

Head over to the tweet for a quick demo video.

I also learned of a couple of useful tools from the tweet:

1. GPT Index (Getting started guide):

   GPT Index is a project consisting of a set of data structures designed to make it easier to use large external knowledge bases with LLMs.

2. Pinecone vector database. What’s a vector database? I found a good summary at Cloud Database Report, though I’d like to dig deeper to understand the details of their use in machine learning:

   A vector database stores, searches, and retrieves vectors, which are long strings of numbers representing documents, images, and other data types used in machine learning applications. Use cases include recommendations, personalization, image search, and deduplication of records.

I’m not sure I’ll get a chance to play with this idea as much as I’d like to, so I’m excited to see what others come up with. I have enjoyed seeing straight-forward integrations with Obsidian that take a prompt from within Obsidian and insert ChatGPT’s reponses. I can’t help, however, feeling that the real value is in integrating ML more subtlely, more under-the-hood, to give improvements to our everyday experiences, much as ML is used heavily in the computational photography that invisibly improves our smartphone photos. I’m sure it’s coming, and I look forward to seeing the directions it takes.