⚡ Unlimited Context LLM

Give your AI superpowers with unlimited context for Ollama — a billion+ token memory for any LLM. Local-first, on your own machine, free.

An open project from Aether · Apache-2.0 · Install

---

Your context window didn't get bigger. Its reach did. Unlimited Context is virtual memory for an LLM. The model keeps its small window; the engine keeps a vast store on your disk and pulls the right slice back in while the model reasons. A small local model stays coherent across runs that would blow past any context window.

⚠️ Giving a model durable memory is powerful — read the safety measures first. Real concerns (runaway agents, grounding drift, an agent's own notes becoming its rules) and what we do about them: Aether AI — Ethical & Safety Measures. The project is open under Apache-2.0 and governed by the Acceptable Use Policy — these are not recommendations. Anyone using the project must comply with its terms, and by using the project you already agree to and are bound by them.

---

The shape of it

Four steps, start to reach:

1. Pick a model — any local model (Ollama, llama.cpp, Hugging Face) or your own API-backed one.
2. Start a session — one object wraps the model and its memory.
3. Allocate disk — choose a pool size; that disk is the memory.
4. Reach 1,000×+ further — overflow is encoded to the pool and the exact slice is recovered the moment the model needs it.

That's it. A 5 GB pool gives a small model ~1.16B tokens of reach — about 9,000× a 128K window — on your own machine, offline.

---

The problem

Long agentic runs all die the same way. The model fills its window, starts compressing its own history, silently drops the one detail that mattered three steps ago — and drifts. You've seen it: the runaway PR, the agent that rewrites a function it already wrote, the build that falls apart at hour two. Bigger windows just delay it, and a crammed 1M-token window rots in the middle anyway.

The fix isn't a bigger window — it's to stop throwing the overflow away. Instead of summarizing what spills over, Unlimited Context encodes it to a local pool on your disk and recovers the right slice exactly when it's needed. Nothing load-bearing is silently lost.

How it works

It's virtual memory, for attention. Map it to an OS and it clicks:

OS	Unlimited Context
RAM	the resident window the model sees now (small, fast)
Disk	the context pool — your encoded memory (~5 GB ≈ ~1B tokens)
Pager	the slice loader — prefetches the next slice from what the model is reasoning about right now
Page-replacement	the retention policy — useful slices stay, stale ones fade, anything relevant again comes back

All of it runs while the model generates, so reaching the pool adds no wall-clock. → full explainer in docs/how-it-works.md.

What you get

• 🧠 Unbounded reach — ~1B tokens of encoded context in ~5 GB on disk; the model reaches it in slices.
• 🧩 MPO context chain — recall pulls the whole connected thread, not isolated nearest-neighbors (5× connected-context recall in-bench).
• ⚡ Zero added latency — the pager runs concurrently with generation, hidden behind the model's own thinking.
• 🪟 Curated beats crammed — a small, relevant resident window outperforms a stuffed one (no lost-in-the-middle) — and costs less.
• 🔒 Local-first — your context never leaves your machine. Free storage, full privacy, works offline.
• 🤖 Any model — Llama, Qwen, Mistral, Phi — via Ollama, llama.cpp, or HF, or your own API-backed model.
• 📉 Coherence you can measure — ship the head-to-head: same model, engine on vs off, watch the drift rate fall off a cliff.

Pick your memory size

First run drops you into a slider — pick how much your model gets to remember:

$ aether-context init
──────────────────────────────────────────────────────────────────
  ⚡ choose your context pool          encoded reach · not a window
──────────────────────────────────────────────────────────────────
  ▸  5 GB   ████░░░░░░░░░░░░   ~1.16B tokens   a big project   (floor)
     10 GB  ████████░░░░░░░░   ~2.33B tokens   a large monorepo + docs
     15 GB  ████████████░░░░   ~3.49B tokens   multiple repos / long runs
     20 GB  ████████████████   ~4.65B tokens   massive corpus / power user
──────────────────────────────────────────────────────────────────
  reach ≈ pool_GB × 233M tokens     custom: --pool 12  (any size ≥ 5 GB)
  ↑/↓ slide      ↵ confirm

  pool [5]: 10
  ✓ 10 GB  →  your model can now reach ~2.33 billion tokens

What that buys you in coding time

The real win isn't the token count — it's that the wall disappears. A typical ~128K context window fills after well under an hour of active agent work, then starts compacting and forgetting. A 5 GB pool is ~9,000× bigger.

Rough ballpark — assuming a busy coding agent encodes ~300K–1M keep-worthy tokens/hour (chatty swarms burn more, careful single agents less):

Pool	Reach	Active autonomous coding before it even fills*
5 GB	~1.16B	~1,200–3,900 hrs — weeks of nonstop building
10 GB	~2.33B	~2,300–7,800 hrs
15 GB	~3.49B	~3,500–11,600 hrs
20 GB	~4.65B	~4,700–15,500 hrs

For color: 5 GB of reach ≈ ~100M lines of code, or a shelf of ~8,000 books — you won't fill it in one sitting.

* Rough order of magnitude. Because the retention policy fades stale slices, the pool never hard-stops anyway — it just keeps what's relevant. Run a build as long as you want; it won't lose the plot. The per-session RAM math is in RAM footprint below.

Running many sessions

Running more than one agent? How the pool is shared is the single biggest RAM lever:

• --pool-mode shared — one pool, one index, all sessions reach the same memory. The index is paid once; each extra session adds only ~30 MB, so RAM barely moves as you add sessions. Best for related work (same project) or max concurrency on a small machine. Trade-off: sessions can see each other's context (no isolation).
• --pool-mode separate (default) — each session gets its own pool + index, fully isolated and private. Clean, but you pay one index per session, so RAM scales with N × pool. Best for unrelated tasks or when isolation matters.

How many actually fit:

Pool	8 GB · shared	8 GB · separate	16 GB · shared	16 GB · separate
5 GB	dozens¹	~13	dozens¹	~33
10 GB	dozens¹	~7	dozens¹	~18
15 GB	dozens¹	~4	dozens¹	~12
20 GB	dozens¹	~3	dozens¹	~9

Reserves: ~2.5 GB held back on an 8 GB machine, ~6 GB on 16 GB — the rest stays for your OS and editor. ¹ With a shared pool, RAM stops being the limit (50–70+ sessions fit); you're bounded by CPU and good sense, not memory.

The math, per tier

Derived, not vibes:

Pool	Slices	Encoded reach	Slider
5 GB (floor)	2.27M	~1.16B tokens	████░░░░░░░░░░░░
10 GB	4.55M	~2.33B tokens	████████░░░░░░░░
15 GB	6.82M	~3.49B tokens	████████████░░░░
20 GB	9.09M	~4.65B tokens	████████████████

How those numbers come out: ~2.2 KB per slice (a 256-dim vector + compressed text + metadata) ÷ 512 tokens per slice → ~455K slices/GB → ~233M tokens of reach per GB. So reach ≈ pool_GB × 233M. 5 GB is the floor; bump anytime with aether-context --pool 20.

Honest: that's encoded reach, retrieved in slices — not a bigger attention window, and it rides on retrieval hit rate. A bigger pool buys more reachable codebase/corpus per session — not more concurrent sessions (those are RAM-bound, ~30 on 8 GB either way).

---

The proof

Not a synthetic micro-benchmark — a real, paid, end-to-end run. A reasoning model (deepseek-v4-pro, via OpenRouter) driven through a 40-turn agent session that overflows its window (2,000-token window, 60 real microsoft/vscode issues), measured engine off vs on — one live run, $0.19, 2026-06-14.

• The model stops forgetting. Recall of early facts after they fall out of the window: 0.15 → 1.00. The baseline drifts and forgets; the engine holds every early fact — zero drift.
• Failure turns into success on the real work. Tasks completed correctly: 3 / 20 → 20 / 20. The job is only done right with the engine.
• Cheaper, not just better. −24% total cost, −54% in the back half — the engine sends a compact recalled slice instead of dragging the whole transcript into every call.

Metric	Off (baseline)	On (engine)	Change
Recall coherence (early facts still correct)	0.15	1.00	6.7×
Work outcome (tasks done right)	3 / 20	20 / 20	3 → 20
Cost — full session	$0.0711	$0.0542	−24%
Cost — back half (recall phase)	$0.00117/turn	$0.00053/turn	−54%

Committed data: full write-up · raw artifacts (api_eval_results.json, api_eval_series.csv, api_eval_plot.png, RESULTS.md) · reproduce with python -m bench.api_eval --model deepseek/deepseek-v4-pro --repo microsoft/vscode --arms off,on,on_chain --plot

Scope, honestly: this measures the engine (retrieve-on-overflow memory), not the MPO chain — on this single-fact recall task the chain ties plain recall (both 1.00); its multi-slice edge is synthetic-only so far (bench/chain_recall.py: connected-context recall 0.15 → 0.78), with the live thread run pending, not yet claimed. T