🧠 notion-second-brain

A fully local RAG agent over your Notion workspace — no cloud APIs, no subscriptions, no data leaving your machine.

A local-first "second brain" agent. Ingests Notion exports (and if needed live Notion pages), processes PDFs/images into markdown, indexes everything into Qdrant, and serves RAG queries via CLI (without memory) or Streamlit GUI (with memory within and between runs). All inference runs locally through Ollama. Designed to run well on native Windows.

Built as a complete local RAG stack: hybrid dense+sparse retrieval, cross-encoder reranking, sentence-aware chunking (with atomic code/table handling), file-based persistent memory, an anchored-rubric evaluation harness, and (optionally) Phoenix observability — all wired together with pydantic-ai. Optimised for a single 12 GB VRAM / 32 GB RAM system.

📖 The problem

I use Notion heavily for knowledge management, collecting and organizing my thoughts, notes, and research. As my notes have grown, it has become increasingly beneficial to summarize key sections or pages. While Notion includes a native AI package, I wanted to avoid it due to security, privacy and cost concerns. So I built this instead.

🖼️ Demo

The Streamlit GUI, with memory enabled. A CLI REPL without memory is also available — see Pipeline stages.

✨ Features

🔍 Hybrid retrieval — dense (nomic-embed-text) + sparse (BM25) with RRF fusion
🔁 Cross-encoder reranking — BAAI/bge-reranker-v2-m3 for precision on top of recall
✂️ Sentence-aware chunking — atomic handling of code blocks and tables (v4)
🧠 Persistent memory — file-based per-session and long-term memory injected at runtime
📊 Evaluation harness — hand-rolled + pydantic_evals with 4-criterion anchored rubric
🔭 Phoenix observability — optional OTel tracing via Arize Phoenix
🏠 Fully local — Ollama inference, Qdrant vector store, zero cloud dependencies

📋 Requirements

Python 3.12+
Ollama running locally
Docker for Qdrant + (optionally) Phoenix
Hardware baseline: 12 GB VRAM, 32 GB RAM (tested on Windows; cross-platform)
Disk: ~30 GB for Ollama models + Qdrant persistence

For lower-end hardware, adjust model size accordingly.

🚀 Quickstart

Assumes you already have cleaned markdown in data/clean/. To build that from scratch, see Pipeline stages.

docker compose up -d
python -m venv .venv && .venv\Scripts\activate   # Windows
# python -m venv .venv && source .venv/bin/activate  # macOS/Linux
pip install -r requirements.txt
ollama pull nomic-embed-text    # or pull any suitable embedding model
ollama pull gemma4:latest       # or pull any agentic model
python -m scripts.run_rag     # index existing data/clean/
python -m assistant.cli         # start chatting in CLI (no memory)
streamlit run assistant/app.py  # start chatting in Streamlit (with memory)

⚙️ Setup

One-time. Install the environment, start the services, connect Notion, then fill in .env.

Install

Single venv. One conflict: openai version swap to move between RAG/agent mode and marker mode (see OpenAI conflict).

With pip

python -m venv .venv
.venv\Scripts\activate          # Windows
# source .venv/bin/activate     # macOS/Linux
pip install -r requirements.txt

requirements.txt pulls torch+cu130 — about 2.5 GB. Expect first install to download ~3–4 GB of wheels.

With uv (alternative)

uv venv .venv && .venv\Scripts\activate
uv pip install -r requirements.txt

The fully project-managed uv flow (with pyproject.toml + uv.lock) is theoretically cleaner, but encountered setup issues on this stack — uv pip or plain pip is the recommended path until those are resolved.

Services (Docker Compose)

docker compose up -d          # starts both Qdrant and Phoenix
docker compose up -d qdrant   # Qdrant only
docker compose up -d phoenix  # Phoenix only

services:
  qdrant:
    image: qdrant/qdrant:latest
    ports:
      - "32768:6333"
      - "6334:6334"
    volumes:
      - ./data/qdrant:/qdrant/storage
    restart: unless-stopped

  phoenix:
    image: arizephoenix/phoenix:latest
    ports:
      - "6006:6006"
      - "4317:4317"
    volumes:
      - ./data/phoenix:/mnt/data
    restart: unless-stopped

🗒️ Notion setup

Relies on the notion-to-md-py library. You need two separate Notion integrations because they serve different code paths:

NOTION_TO_MD_AUTH_TOKEN — used by scripts/run_etl.py to bulk-fetch pages as markdown during ETL.
NOTION_ASSISTANT_AUTH_TOKEN — used by the agent's fetch_notion_page tool to live-fetch a single page on demand.

If you don't need live-fetch, skip the second integration.

Creating the integrations

Go to https://www.notion.so/profile/integrations → + New integration
Name it (e.g. Second Brain — Notion-to-MD) → Read content capabilities only
Copy the Internal Integration Secret (ntn_...) into .env
Optionally repeat for a second integration (NOTION_ASSISTANT_AUTH_TOKEN)

Granting page access

For each integration, open each page in Notion → ⋯ → Connections → add the integration. For workspace-wide access: Settings → Connections → add at workspace level.

Without this step the integration returns empty results for every page.

🔧 Configure `.env`

# ── Notion ───────────────────────────────────────────────────────────────────
NOTION_ASSISTANT_AUTH_TOKEN=ntn_...
NOTION_TO_MD_AUTH_TOKEN=ntn_...

# ── ETL mode ─────────────────────────────────────────────────────────────────
LOAD_MODE="notion"            # "files" or "notion"
# ETL_PAGE_NAME="AI/ML/Data Science"  # limit to one page for first-run testing

# ── Marker (PDF / image OCR) ─────────────────────────────────────────────────
MARKER_USE_LLM=false
MARKER_FORCE_OCR=true
MARKER_WORKERS=2
MARKER_DISABLE_IMAGES=false

# ── Qdrant ───────────────────────────────────────────────────────────────────
QDRANT_URL=http://localhost:32768
FORCE_REINDEX=true            # set true on first run, or after schema changes
ENABLE_RERANK=true

# ── Ollama ───────────────────────────────────────────────────────────────────
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_MODEL=gemma4:latest
# Use gemma4 for the agent — qwen3 has a large KV-cache offload that
# spills to CPU and tanks throughput. Qwen3 is fine as eval judge.

# ── Memory ───────────────────────────────────────────────────────────────────
ENABLE_MEMORY=true
RECENT_LOG_DAYS=2

# ── Evaluations ──────────────────────────────────────────────────────────────
JUDGE_MODEL=qwen3:8b

Verify GPU offload

ollama ps

Expected: 100% GPU. If you see any CPU %, set OLLAMA_NUM_GPU=99 at the shell level (not just .env), or recreate the model via a Modelfile with num_gpu 99.

🔄 Pipeline stages

Run stages 1, 2 and 4 to build the index, then stage 5 to query it. Stage 3 (marker) is only needed when you have new PDFs or images to convert.

1 · ETL — Notion / files → raw markdown

python -m scripts.run_etl

Fetches from Notion (LOAD_MODE=notion) or reads local files (LOAD_MODE=files). Fetches text, images and documents. Does not crawl through links embedded in Notion. Writes to data/raw/raw_md/.

First run: set ETL_PAGE_NAME="Some Page" to test the Notion connection on a single page before pulling your whole workspace. Notion rate limits are aggressive on large workspaces.

Rate limits: the Notion API allows roughly three requests per second per integration, and a page with many blocks costs many requests. Pulling a whole workspace in one pass will throttle and can fail partway through. The practical approach is to loop ETL_PAGE_NAME over your top-level pages and run ETL once per page — slower in wall-clock terms, but each run is independently restartable and you don't lose an hour of fetching to a 429 near the end.

2 · Cleaning — raw → clean markdown

python -m scripts.run_clean_md

LLM-based cleanup of data/raw/raw_md/ → data/clean/clean_md/.

3 · Marker — PDF + image → markdown

python -m scripts.run_marker

Converts PDF documents and images into (clean) markdown.

Env vars: MARKER_STEP (pdfs / images / all), MARKER_TEST_SUBDIR (debug subset).

This is the slow stage. On my corpus a full marker pass runs on the order of hours. It is largely CPU-bound at the default settings, so runtime scales with core count and with how many documents you feed it, not with GPU. Use MARKER_TEST_SUBDIR to validate on a handful of files first, and treat full runs as something you start and walk away from. It is also incremental in practice — you only need to rerun it for new source files.

Alternatives that avoid the openai conflict: PyMuPDF4LLM (lightest, no OCR), MinerU, Kreuzberg, Docling. Fair warning: all of them are smoother on Linux than on native Windows. Expect extra setup — system-level OCR or image libraries, occasionally WSL — where marker mostly works out of the box here. If you are already on Linux, PyMuPDF4LLM is the cheapest way to sidestep the dependency swap entirely.

4 · RAG indexing — clean markdown → Qdrant

python -m scripts.run_rag

Applies sentence-aware chunking → hybrid embeddings → optional reranking → Qdrant upload with RRF fusion. Collection visible at http://localhost:32768/dashboard#/collections.

Key env vars: FORCE_REINDEX=true (schema changes), RETRIEVAL_TOP_N=10, RERANK_TOP_K=3.

Qdrant indexing note: indexed_vectors_count: 0 with points_count > 0 is normal — HNSW indexing is deferred until data exceeds indexing_threshold. Lower the threshold in pipelines/rag/indexer.py for immediate indexing.

5 · Agent — query your second brain

CLI REPL (no memory)

python -m assistant.cli
python -m assistant.cli "What should I focus on this quarter?"

Streamlit GUI (with memory)

streamlit run assistant/app.py

Always run from the repo root.

🏗️ How it works

Notion pages / local files ──► ETL ──► raw md ──► LLM cleanup ──┐
                                                                ├──► chunk ──► dense + sparse embed ──► Qdrant
PDFs / images ──────────────► marker ──────────────────────────┘                                          │
                                                                                                          ▼
                        agent (CLI / Streamlit)  ◄──  rerank  ◄──  RRF fusion  ◄──  hybrid retrieval  ◄────┘
                                   │
                                   └──► memory/  +  live Notion fetch

⏱️ Performance and cost

Running fully local means the money cost is zero and the cost shows up as wall-clock time instead. Roughly where it goes:

ETL (stage 1) — bounded by Notion's API rate limits rather than by your hardware, which is why page-by-page runs are the sane default.
Cleaning (stage 2) — local LLM pass over every raw file; scales with corpus size and model size.
Marker (stage 3) — by far the heaviest. Hours for a large document set, CPU-bound, one-off per batch of new files.
Indexing (stage 4) — fast once the markdown exists.
Query (stage 5) — interactive, provided the model sits fully in VRAM. Check ollama ps; any CPU offload is the difference between a few seconds and an unusable REPL.

The one-time ingest is the expensive part. Steady-state use is cheap.

📁 Project layout

notion-second-brain/
├── assistant/                  # Agent runtime
│   ├── agent.py                # pydantic-ai Agent + per-call instantiation
│   ├── app.py                  # Streamlit GUI (with memory)
│   ├── cli.py                  # CLI REPL (no memory)
│   ├── memory.py               # file-based memory
│   └── tools.py                # retrieve_knowledge, fetch_notion_page
├── pipelines/
│   ├── etl/                    # Notion/files → raw markdown
│   ├── rag/                    # chunker, embeddings, reranker, indexer
│   ├── utils/
│   └── models.py
├── scripts/                    # Pipeline entry points
│   ├── run_marker.py
│   ├── run_clean_md.py
│   ├── run_etl.py
│   └── run_rag.py
├── evals/                      # Evaluation suite
│   ├── cases.py                # golden + adversarial + distribution cases
│   ├── rubrics.py              # 4 anchored 1–5 rubrics
│   ├── judges.py               # LLM-as-judge (Ollama)
│   ├── run_evals.py            # hand-rolled runner (canonical)
│   └── run_pydantic_evals.py   # pydantic_evals runner
├── extras/                     # Optional / reference
│   ├── run_deepeval.py         # DeepEval showcase (see deepeval_info.md)
│   ├── run_phoenix.py          # Phoenix OTel tracing
│   ├── deepeval_info.md        # DeepEval setup notes
│   ├── llm-eval-patterns.md    # Eval methodology reference
│   └── prompt-eval-designer.md # Rubric design protocol
├── memory/                     # Conversation memory (gitignored)
├── data/                       # All data files (gitignored)
├── images/                     # README screenshots
├── docker-compose.yml
└── requirements.txt

📂 Data layout

data/
├── raw/
│   ├── documents/<page-name>/   # raw Notion exports
│   ├── images/<page-name>/      # downloaded page images
│   └── raw_md/                  # raw Notion text exports
├── crawled/                     # (not yet used) web crawler output
├── clean/
│   ├── pdfs_md/                 # PDF → md via marker
│   ├── images_md/               # image OCR via marker
│   └── clean_md/                # raw_md → cleaned md
└── pages.txt                    # page list for live Notion fetch
memory/
├── MEMORY.md                    # long-term distilled context
└── YYYY-MM-DD.md                # daily conversation logs

🧠 Memory

File-based memory at memory/:

MEMORY.md — long-term distilled context (curated facts, preferences)
YYYY-MM-DD.md — daily conversation logs

Each turn is appended as:

### HH:MM
User: ...
Assistant: ...

The agent loads MEMORY.md + the most recent RECENT_LOG_DAYS daily logs into its system prompt per call, implemented via per-call Agent instantiation (pydantic-ai 1.x freezes system_prompt after construction).

Toggle: ENABLE_MEMORY=true|false.

📊 Evaluations

Two runners, same anchored rubric and golden test set — run either or both for cross-validation:

Script	Framework	Purpose
`python -m evals.run_evals`	Hand-rolled	Canonical. 4-criterion rubric (relevance, correctness, citation_quality, safety), LLM-as-judge via Ollama.
`python -m evals.run_pydantic_evals`	pydantic_evals	Same dataset + rubric via `Evaluator` + `EvaluationReason`. Runs all 3 tiers by default.

Results written to evals/results/*.json. Set judge model via .env: JUDGE_MODEL=qwen3:8b.

Eval methodology

Follows the frameworks in extras/llm-eval-patterns.md and extras/prompt-eval-designer.md — moving from vibes-based to statistically anchored evaluation (pointwise rubrics, 3-tier test suites, CI gates). Built on: ai-engineering-from-scratch.

🔭 Observability — Arize Phoenix (optional)

extras/run_phoenix.py wires Phoenix OTel tracing. Every agent.run(...), LLM call, and tool call appears as a span in the Phoenix UI at http://localhost:6006.

pip install openinference-instrumentation-pydantic-ai opentelemetry-sdk opentelemetry-exporter-otlp opentelemetry-api
docker compose up -d phoenix
python -m extras.run_phoenix "your query here"

Don't run Phoenix tracing and DeepEval's DeepEvalInstrumentationSettings simultaneously — both wrap the same pydantic-ai OTel hooks.

🔗 Related approaches

Karpathy's LLM wiki (April 2026) is the closest idea in spirit and the sharpest contrast in mechanism. Shared assumptions: markdown is the right substrate, the corpus is personal rather than enterprise, and knowledge should accumulate instead of being re-derived on every question. The difference is when knowledge gets assembled. The wiki pattern compiles it up front — an agent incrementally maintains a set of interlinked pages that sit between you and the raw sources, and queries are answered by loading those pages into context, no vector store involved. This project assembles at query time: chunk, embed, retrieve, rerank.

Which one is better depends almost entirely on corpus size. If your notes fit comfortably in a context window, the wiki wins on simplicity — no Qdrant, no embedding pipeline, no chunking strategy to tune, and nothing to re-index. Past that point you are paying to stuff irrelevant pages into every prompt, and retrieval starts earning its complexity. My Notion workspace is well past it, which is why this repo looks the way it does. The memory/MEMORY.md layer here is a small nod to the same idea: distilled, human-readable, loaded wholesale rather than retrieved.

Supermemory is the pragmatic alternative if you would rather not run any of this. It's a memory and context engine with a Notion connector, hybrid search, and multi-modal extraction, available both hosted and self-hosted. It solves most of what this repo does, with far less setup and a much better ingest story. The tradeoff is the one from The problem: the hosted path means your notes leave your machine, which was the constraint I started from. Worth knowing about before you commit a weekend to marker.

🚧 Limitations

The dependency conflict is unresolved. marker-pdf and pydantic-ai cannot coexist in one environment; the workaround is a manual openai version swap (below).
No headline eval numbers. The harness, rubrics and test tiers ship with the repo, but scores are corpus-dependent — run python -m evals.run_evals to generate your own. Grading is LLM-as-judge (JUDGE_MODEL), not human review.
Web crawling is not implemented. data/crawled/ is reserved but unused, and the link-crawling block in scripts/run_etl.py is commented out. Notes that lean on outbound links therefore index as bare URLs with no content behind them. Stretch goal rather than a near-term fix — the pieces I'd reach for are anydoc for turning arbitrary fetched pages into markdown and pdf-inspect for inspecting linked PDFs before committing them to the marker pipeline.
The project-managed uv flow does not work yet. pyproject.toml + uv.lock hit setup issues on this stack; uv pip or plain pip is the supported path.
One tested configuration. Windows, 12 GB VRAM / 32 GB RAM. Other platforms should work but are unverified; smaller GPUs need smaller models.

⚠️ OpenAI conflict (marker-pdf vs pydantic-ai)

The single biggest setup friction. pip check surfaces it as:

marker-pdf 1.10.2 requires openai<2.0.0, but you have openai 2.41.0

marker-pdf pins openai<2.0.0
pydantic-ai pulls openai>=2.0.0
They are incompatible in the same environment

Workaround: swap versions as needed (~10 seconds):

# Before running marker / ETL scripts
pip install "openai<2.0.0,>=1.65.2"

# Before running the agent / evals
pip install "openai>=2.0.0"

Only needed when you have new PDFs or images to OCR. Re-indexing already-cleaned markdown (run_rag.py) doesn't touch marker and doesn't need the swap.

Decision tree

New PDFs / images to OCR?          → marker mode  (pip install openai<2, run_marker.py)
Re-index existing cleaned markdown? → agent mode   (pip install openai>=2, run_rag.py)
Chat / develop / run evals?        → agent mode

🛠️ Troubleshooting

Symptom	Fix
Notion returns nothing	Pages not shared with integration — open page → ⋯ → Connections → add integration
Notion rate limits / 429s	Use `ETL_PAGE_NAME="Single Page"` for first-run validation
GPU offload (CPU %)	Set `OLLAMA_NUM_GPU=99` at shell level, or recreate model via Modelfile
pydantic-ai 404 against Ollama	Use `OllamaProvider(base_url="http://localhost:11434/v1")` — the `ollama:<model>` shorthand routes to the wrong path
Streamlit import errors	Always `cd` to repo root before running
Eval JSON missing	Default output: `evals/results/*.json`; override with `--output PATH`
Memory not used	Check `ENABLE_MEMORY=true`; `memory/MEMORY.md` is auto-created on first run
Stale chunks after schema change	Set `FORCE_REINDEX=true` to drop and rebuild the Qdrant collection

License

MIT — see LICENSE. This repo is meant to be used, adapted and improved upon based on individual user needs and system capabilities.

Name		Name	Last commit message	Last commit date
assistant		assistant
evals		evals
extras		extras
images		images
pipelines		pipelines
scripts		scripts
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
docker-compose.yml		docker-compose.yml
main.py		main.py
requirements.txt		requirements.txt

Latest commit

History

Folders and files

Repository files navigation