🤖 AI 资讯

Three weeks back , i posted SHADOW-250M here. It got 360 upvotes, 293 on r/LocalLLaMA and 94 GitHub stars. Thank you.

That model was 60 MB, ran around 400 tok/s on CPU and could retrieve records from an archive on disk. What it couldn’t do reliably was reason over what it retrieved or compute. So I built a smaller one to experiment with those two problems.

SHADOW-50M is actually 44M parameters, trained from scratch on 45B tokens. 19.8 MB complete model, ~1,900 tok/s on laptop CPU, ~41 MB RAM, ternary {-1,0,+1} weights, 73,880-token vocabulary represented by fixed 512-bit fingerprints instead of a trained embedding, and a 159 KB compiled kernel. It runs completely offline. The same kernel compiled to WebAssembly runs in a browser tab at around 500 tok/s.

This is a proof of concept, not a product.

If SHADOW decides something needs calculation, it writes something like [calc]347*86[eq]. A fixed circuit at the readout takes over and fills in the correct digits in the same token stream. No calculator API, no tool call and nothing pasted back into the prompt. I added circuits for arithmetic, percentages, dates, weekdays, units, counting, sorting, comparisons and a small program machine.

When SHADOW stores a record, it reads it once and writes its attention state to disk at 1 bit, 288 bytes/token. Later it can write [need]condition of Patient P-204, the index finds the record in roughly a microsecond and the stored attention state goes directly back into the model in about 0.03 ms. The text isn’t re-read.

The index is 22 bytes/token and doesn’t use a vector database or embedding model. At 100M tokens, the archive is 28.8 GB plus a 2.2 GB index on disk, while the process uses about 28 MB RAM because everything is memory-mapped and a question only touches the pages it needs.
The index also has a small persistent trail. When retrieved information gets used, that record is reinforced in the index itself. On repeated questions, measured top-1 went from 0.571 to 0.743 without training the model.

I also put it next to Supra-50M-Reasoning, a 51.8M parameter Llama-style bf16 model.
Supra beats SHADOW on every standard benchmark I tested. ARC-Easy: 0.435 vs SHADOW 0.307. PIQA: 0.600 vs 0.570. WikiText-2 perplexity: 165 vs 186.
I’m publishing that because I don’t want to hide where SHADOW is weak.
But those tests aren’t what I trained this model for. Here are some actual greedy outputs, one fresh process per question:
“Tell me a short joke about computers.”
SHADOW: Why did the computer go to the doctor? Because it had a virus.
Supra: Thought for 189 tokens about what computers are and stopped without a joke.
“I have 3 books and I bought 5 more. How many books do I have now?”
SHADOW: You have 8 books.
Supra: Books often come from collections of short stories, poems, or other forms of literature.
“My bill is 240 dollars. What is 15 percent of that?”
SHADOW: 36 dollars.
Supra: 15% ($240) is $250. It’s $150 because 20% of it is $200.
“What date is 45 days after December 20, 2026?”
SHADOW: February 3, 2027.
Supra: The German government introduced a new date called March 22nd that precede February 25th in 1900.
Eight records stored, one of them “Patient P-204 condition: asthma”.
“What is the condition of Patient P-204?”
SHADOW: From disk: The condition of Patient P-204 is asthma.
Supra, with the records in its prompt: The patient experiences asthma when they get too much sleep.
“What is the condition of Patient Z-999?” No such record exists.
SHADOW: There is no record of Patient Z-999.
Supra: The condition of an allergic reaction occurs when the immune system mistakenly attacks specific cells on the body.
All 11 side-by-side questions, scripts and traces are in the repo.
On size, Supra is 103.6 MB in bf16 and 56.2 MB at 8-bit. Below that it starts losing quality. Int4 takes its perplexity from 165 to 193 and ternary breaks it. Its vocabulary is 32,000 tokens. SHADOW carries 73,880 tokens in a 4.7 MB frozen table.

Something unexpected happened with that frozen table too.
The first SHADOW-50M release was missing about 8,600 English word pieces. Lowercase “fitzgerald”, for example, could reach the model as “fitz”. I tried fixing it through fine-tuning, but every run that learned the missing words broke something else.
So I added the 8,600 missing rows directly to the frozen fingerprint table. No training. Same weights.
All 34/34 previously published answers stayed unchanged, while the model could now read many of the new pieces. The table scores 0.594 Spearman against human word-similarity ratings versus -0.057 for random codes.
A trained embedding can’t simply accept thousands of new rows without training. A frozen table can.

I also built four harnesses that put this tiny model next to larger models.
My favourite is video memory. Gemma 3 4B watches a ten-minute film once, one frame every two seconds, and writes 298 little descriptions such as “Moment M-0039 scene: A chubby white rabbit reaches for a purple butterfly.”
Then Gemma leaves.
SHADOW keeps those 298 moments as memory on disk. Afterwards I can ask the 20 MB model what happened at a particular moment, by number, by time or by what appeared in it. It answers from memory with the record quoted, without the film and without Gemma. It scored 55-56/60 across those query types on a laptop in about 42 MB RAM.
The other three experiments: an inventory of 1,600 records got 159/160, with all 20 questions about items never stored correctly returning “no record”; SHADOW as a draft model for Qwen3-32B took llama.cpp generation from 19.7 to 28.5 tok/s while Qwen still chose every final token; and an MCP memory server let Qwen3-14B store facts mid-chat and later retrieve 5/5 with the original records quoted.

There are plenty of shortcomings. General knowledge is thin. Creative writing isn’t good. Seven-digit operands sometimes get copied incorrectly. A large archive can occasionally pull an unrelated record into a question carrying a number. They’re documented in the repo next to the successful results.

And one thing happened after my last post that I really didn’t expect.
Someone called engram-forge sent a pull request to SHADOW-250M containing a CUDA engine, a quantization tutorial, and then a talking Peppa Pig plush toy with SHADOW inside it.
Microphone, small speaker, ~$35 board. You talk to the toy, it listens, SHADOW generates the answer locally and the toy talks back. No cloud, no account, no internet.
I haven’t merged the ~11,000 lines yet because I can’t properly verify that much CUDA myself. When the demo is finished I’ll keep it under engram-forge’s name.
I never imagined one of these models living inside a stuffed toy on someone’s shelf .Thanks

I’m not saying a 20 MB model beats normal LLMs. It doesn’t. I’m trying to find out how much useful behaviour can fit into a tiny local model when computation and persistent memory are treated differently.
Everything is MIT licensed. The master weights and fine-tuning/export kit are public. Next I’m releasing the training code, dataset, frozen table and a proper write-up of how I built it, including the costs, failed experiments and mistakes.

Code:
https://github.com/QLNI/SHADOW-50M-Instruct
Weights:
https://huggingface.co/QLNI/shadow-50m-instruct
Run it in your browser:
https://qlni.github.io/SHADOW-50M-Instruct/web