This is part 1 of the series "Adopting RAG — Grow your AI on your own accumulated judgment."

Series contents

1. Adding RAG: store your own judgment and grow your AI (this article)
2. What RAG should search, and what an index should answer (coming soon)

My Claude Code setup loaded a 32KB index of its own notes at the start of every session, and it still kept making the mistakes those notes were written to prevent. "Take the screenshots for the English version in the English UI." "This article is too long." Each correction was saved as a memory note the first time I made it. One of those notes still quotes me asking how many times I have to say it.

The notes were there. What was missing was the moment. When the situation came up mid-task, the AI did not go back and look. That is not specific to Claude Code: any AI coding agent with saved notes, Codex included, has to decide to search them, and that decision is the step that gets skipped.

So I added RAG. RAG (retrieval-augmented generation) means that before the AI answers, something searches your own records and hands it the closest ones. Mine runs on every message I send: it searches my past notes and conversations by meaning and hands the closest few to the AI before it answers. We call it memory recall. It is part of ELN workflow, the in-house plugin suite we use with Claude Code. It relies on a Claude Code hook (a script Claude Code runs automatically at fixed moments, here every time you send a message), so today it runs on Claude Code only. The idea would work for any agent.

Nothing gets retrained. What changes is what the AI is handed. Every decision and correction I write down becomes something it can be shown the next time the situation comes up, so the more of my own judgment I store, the more the AI has to work with. That is what I mean by growing the AI with RAG.

An earlier article covered a related hook that hands the AI a list of our architecture decisions once, at session start (The real reason AI ignores existing decisions). Resending the same list on every message was wasteful, so that hook moved to session start. This one works per message: what it hands over changes with what you just said, it hands over nothing or a short list when the match is weak, and it never exceeds 2,048 bytes.

The short version

• Every message triggers a RAG search. It looks through the current project's memory notes, the titles of past conversations in the same project, and our shared engineering rules, and hands the closest matches to the AI before it answers.
• Setting it up takes five steps. ELN workflow is in-house and not published, but every part it uses is public. The details, and a prompt you can hand to Claude Code as is, are in "Setting it up yourself" near the end.
  1. Write each correction or decision down as its own memory note, one note per file.
  2. Turn the notes into embeddings with bge-m3 and store them in a local index.
  3. Keep a search process running with the model loaded, embed each message with the same model, and rank the index by closeness.
  4. Query it from the Claude Code hook that runs on every message (UserPromptSubmit), and hand the headlines it returns to the AI.
  5. Decide how much to hand over by how far the top hit stands out from the rest.
• Stored judgment keeps paying off. A correction written down once becomes a candidate whenever a related message comes in, and is handed over when it is close enough. The model stays the same; the records it gets keep growing.
• It matches meaning, not words. A question with no words in common with the note still finds it. Because the model is multilingual, an English question even finds a note written in Japanese.
• It hands over headlines, not documents. At most five items and 2,048 bytes per message. Conversations go in as date and title only, never their content. The block is labeled as reference, not instructions.
• It injects nothing or only a short list when the match is weak. Similarity scores alone could not tell related messages from unrelated ones, so it looks at how far the top hit stands out from the rest, and injects nothing, a little, or a bit more.
• The search runs on the Mac. The embedding model (which turns text into numbers that can be compared by meaning), the index and the search process all live locally. The search sends nothing to a paid embedding API, and it costs nothing.
• It never holds up the conversation. If the search does not answer within 300 ms, the message goes through without it. Sixty repeat queries had a median of 69 ms and a worst case of 228 ms.

The screens below are real output from the live index on my Mac. The tool's own injection text is in Japanese only, so for this version I asked English questions and used jq (a command-line JSON filter) to print just the fields that matter. The command lines drop the script's install path, and the home folder in the output is shown as ~/.

An English question, a Japanese note

Here is the idea in one screen. I ask, in English, when the firm was founded. Searching the notes folder for "firm" or "founded" as whole words finds nothing. Searching by meaning puts the note about my name and company first, well ahead of the next one. That note records the founding year in Japanese, as 創業 2000年 ("founded in 2000").

Neither word from the question appears anywhere in that note. The note does contain some English, such as the company name and the file name, but the part about the founding year is written only in Japanese. A text search can only find the exact characters. The search here turns both the question and every note into embeddings (lists of numbers arranged so that texts with similar meaning end up close together) and compares those. The model is bge-m3, published by BAAI, which handles many languages, including English and Japanese, in the same space.

Long notes are split into 700-character pieces, and a note scores by its best piece, so that a detail further down than the first few lines still has a chance to be found.

Scores alone could not separate related from unrelated

Handing notes to the AI on every message is only useful if it holds back when nothing is relevant. Otherwise the context fills with noise. This was the part that surprised me.

In testing, the highest-scoring unrelated question scored 0.597. The median question that did have a correct note scored 0.589. Any cutoff on the score would either inject notes into unrelated messages or miss real ones.

What did separate them was th