Hey HN! I am Roan, and I built Hillock: a 100% local, offline neuro-symbolic memory engine built for consumer hardware (runs in <1.2 GB VRAM on a GTX 1070 or on pure CPU laptops).
Why not standard vector RAG?
Dense vector databases and 8B+ extraction models are heavy on local hardware, suffer from semantic drift, and still hallucinate when out of context. Hillock explores a deterministic alternative:
SQLite Knowledge Graph: Stores ground-truth facts as relational Subject-Predicate-Object triples (zero vector drift).
Hebbian Plasticity Engine: Gradient-free associative learning across turns to surface primed context.
10,000-D Hyperdimensional Computing (VSA/HDC): Sub-millisecond similarity gating directly on CPU in discrete bipolar space.
TALON Extraction Stack: 3-stage local extraction (Fastcoref + MiniLM + GLiREL) with type-constrained schema validation, running offline without cloud calls.
Refusal is Control Flow (Not Prompting)
Instead of prompting an LLM to "only answer if you know", Hillock's gate is an actual programmatic check. If candidate facts fail our similarity threshold with positive predicate intent, it returns a hardcoded refusal immediately. The local LLM (Ollama) is never called with un-evidenced context, saving 100% of GPU compute on unanswerable queries.
What is New in v0.6 (HYDRA & HyperGraph)
HYDRA (Bipolar MaxSim): Adapted ColBERT-style token-level MaxSim to discrete bipolar vectors. A 2,000-D Sub-Dimensional Projection Cascade early-rejects ~95% of candidates on CPU in ~0.5ms.
HYPERGRAPH-HDC: Uses positional cyclic shifts to break binding commutativity, encoding 2-hop and 3-hop relational paths without combinatorial RAM explosion.
Benchmarks (Unseeded 32-query run on a laptop CPU)
Answerable Retrieval Accuracy: 54.5% (Fast-eval in 1.16s on CPU)
Hard-Negative Block Rate: 60.0% (stops trick queries cold)
Extraction Recall: 59.1% | Pooled Gate Accuracy: 56.2%
Known Limitations
Retrieval quality is strictly bounded by extraction recall: if the extractor misses a fact during ingestion, the gate honestly refuses (producing a false block).
The project is AGPL-3.0 licensed with 1-click quickstart launchers (run.bat / run.sh) and a standalone 21-point CPU verification suite (verify_hillock.py).
Hi Jacob, thanks for sharing! Witchcraft looks super interesting—starring the repo now.
The late interaction (ColBERT / MaxSim token-level) approach is fascinating for local search. Retaining per-token interaction matrices gives incredible semantic retrieval precision without losing fine-grained token context.
From an architectural standpoint, the main trade-off I explored with Hillock was moving away from dense token-embedding matrices entirely toward an explicit neuro-symbolic split:
1. Symbolic Grounding: Hard Subject-Predicate-Object (SPO) triples in relational SQLite tables to eliminate vector drift for exact factual memory.
2. Subsymbolic VSA Gating: A 10,000-dimensional Vector Symbolic Architecture (HDC) hypervector space on CPU using subword n-grams and GloVe SimHash projections for <1ms gating and pronoun resolution.
3. Edge VRAM Footprint: Keeping the whole engine under 1.2 GB VRAM on a GTX 1070 by using a CUDA bi-encoder pipeline (Fastcoref + MiniLM + GLiREL) for non-generative document parsing.
I'm curious—how do you manage the index footprint and per-token memory overhead in Witchcraft when scaling to larger local document collections?
There is no memory overhead, in that everything resides on disk/in the sqlite database, and is read from there on demand. That said, sqlite for blob storage is a bit inefficient, so the next version will use tightly packed structs on disk. I also managed to finetune an embedding down to 96d, so in the coming version the storage for the index is getting close to parity with the size of the text being indexed.
I built Hillock (https://github.com/roandejager/Hillock), a 100% local, offline neuro-symbolic memory engine designed to replace heavy Vector DBs and token-hungry LLM document ingestion pipelines on edge hardware.
### The Problem with Local RAG
Standard local RAG is surprisingly heavy. Running dense vector databases alongside an 8B+ generative LLM just to parse documents and maintain conversational context burns VRAM, chokes mid-range GPUs (like a GTX 1070), and still hallucinates when asked about missing or unverified information.
### The Architecture
Instead of using autoregressive LLMs to generate JSON extractions, Hillock decouples memory into three lightweight tiers:
1. SQLite Knowledge Graph (database.py): Ground-truth Subject-Predicate-Object (SPO) triples stored in relational tables. Zero vector drift for factual data.
2. Hebbian Plasticity Engine (plasticity.py): Gradient-free co-activation associative learning that updates synaptic weights between entity nodes across chat turns.
3. 10,000-D VSA/HDC Reservoir (reservoir.py): A Vector Symbolic Architecture hypervector space using Subword N-Grams (3,4,5-grams) and Locality-Sensitive Sign Random Projections (SimHash) over a static 10MB GloVe dictionary. It handles pronoun resolution and context similarity gating on CPU in under 1 millisecond.
Ingestion completes in ~3.9–5.0 seconds for a 32-sentence academic document (6.3–8.0 sentences/sec pure GPU rate) using <1.2 GB VRAM on a GTX 1070. Generative LLMs (via local Ollama) are invoked ONLY for final response rendering once a query passes the HDC similarity gate.
### What's New in v0.4
In the v0.4 release series, we tackled zero-shot extraction noise and inverted facts:
- O(1) Type-Constrained Schema Validation: Set-based domain/codomain filtering over GLiREL outputs.
- Automatic Direction Correction: Detects and swaps inverted extractions (e.g., turning [Budapest born_in John von Neumann] into [John von Neumann born_in Budapest]).
- Precompiled Entity Sanitization: Strips possessive artifacts ('s), trailing action verbs, and prepositional tails.
- Raw extraction precision improved from 11.5% to 15.5% (+35% cleaner graph).
### Current Benchmark Status (Honest Baseline)
On a fixed 32-sentence benchmark (20 answerable queries, 10 hard-negative trick queries):
- Retrieval Accuracy: 55.0%
- Extraction Recall: 50.0%
- Extraction Precision: 15.5%
- Fast-Eval Retrieval Duration: ~1.27s for 30 queries (~0.04s per query)
- Unanswerable Queries: 0 GPU tokens wasted (CPU similarity gate shuts down LLM calls instantly).
