PageIndex: reasoning-based retrieval for long documents

TL;DR

PageIndex (Vectify AI, MIT, Python SDK pageindex) replaces chunk-and-embed RAG with a per-document table of contents the model navigates. Indexing builds a section tree (title, page range, LLM summary per node); retrieval is an agent loop over three tools: list documents, read the tree, fetch tight page ranges. It reads the right pages instead of the nearest chunks.

It earns its keep on long, structured, professional documents (10-Ks, contracts, regulations, manuals, textbooks) where the answer depends on where something sits, not what it sounds like. It is not a cheap retriever: per query it costs several agent turns with a strong model, so it wins on accuracy and traceability, not price, against a well-tuned vector setup. Read as a context-engineering pattern, it is progressive disclosure for documents: load the map, then only the pages the map points to.

How it works

Two phases: build a tree once per document, then let an agent walk it per question. Read from main at commit 2ac8694 (5 Nov 2026), about 33K lines of Python in pageindex/.

Index (once per document).

1. Structure. Default is Flash (pageindex/flash/), which extracts the outline with no LLM: a char-level pdfium parser, header/footer and TOC-boilerplate classification, font-style heading detection and outline assembly. It prefers embedded PDF bookmarks when present; toc_source records detected, bookmarks, hybrid, pages (no hierarchy found) or unreadable. The older classic path (page_index_classic.py) asks an LLM to find, verify and repair the TOC, with retries. A Markdown path splits on headings.

2. Optimize. tree_optimize.py merges thin nodes (keeping their titles as key_items) and uses an LLM to expand oversized ones.

3. Summarize. A cheap model writes a summary per node. Every node carries node_id, title, 1-based inclusive start_index/end_index pages, summary and children. Every page belongs to some node (a Preface covers leading pages).

Retrieve (per question). The model gets tools, not chunks (agent_tools.py, mirroring the cloud MCP contract so prompts port across local and cloud):

1. browse_documents lists the library (with relevance sort) to pick the document.

2. get_document_structure returns the tree, required first for documents over 20 pages and paged in parts under a 100K-character response cap.

3. get_page_content fetches tight page ranges the tree points to ("never the whole document").

4. A grounding prompt forces answers only from pages actually read, cited as [doc, p. N] (block-level on cloud).

The agent loop runs on LiteLLM, the OpenAI Agents SDK or the Claude Agent SDK; there are drop-in integrations for both SDKs and a claude-code/ model prefix that bills a Claude subscription through Claude Code. Cloud adds OCR, images, metadata filters, folders, an MCP server and a corpus-level "File System" tree; the newest product, a hosted Page Search API, is closed and not in this repo.

Evidence

The most useful number is in the repo's own notebook (pageindex-vs-vector-10k): 63 questions over five NVIDIA 10-Ks, graded blind by an LLM judge, run 16 Oct 2026. PageIndex scored 100% at about $110 per 1,000 queries. Vector search only gets close by stuffing 100 chunks into the prompt, which costs more.

Other vendor claims, unverified here:

• Indexing about $0.001 per page with a basic model; 9 to 1,098 pages took 13 s to 4.5 min. A 1,098-page textbook used about 1.6M input and 0.65M output tokens in Flash's own benchmark.

• Versus whole-PDF-in-context, native input cost 2.1× more at 52 pages and 16.6× at 420; 805 pages did not fit (prompt caching excluded, which flatters PageIndex).

• FinanceBench 98.7% versus about 50% for vector RAG, via Vectify's Mafin 2.5 product, a 2025 result.

Caveats: every benchmark is financial filings, every one is run by the vendor, and the 10-K set is small (63 questions; the notebook says each run moves by a question or two). Vector baselines use naive fixed chunks with no reranker, hybrid search or contextual chunking, so the gap against a tuned stack is probably smaller.

Tradeoffs and failure modes

The design moves cost from index time to query time and bets that the document's own structure is a better index than embeddings.

• Latency and cost per query. Retrieval is several sequential tool calls with the best model you can afford (the README's advice). Expect seconds, not milliseconds; this is not a search box backend. Prompt caching across turns of one question is what keeps it near $0.11 a query.

• Structure is a single point of failure. If the outline is wrong, the agent navigates confidently to the wrong pages. Weak spots: flat or badly formatted PDFs (toc_source: pages gives a Page N tree, which the local client refuses past 10 pages), scanned documents (no OCR locally; cloud only), and tables or figures whose meaning is not in headings.

• Questions that cut across structure. "Every place the contract mentions indemnity" is a keyword or vector job; a tree walk can miss a passage filed under an unrelated heading. Cloud adds keyword and metadata search as escalation paths, which concedes this.

• Scale. Locally it reasons over a handful of documents; the corpus-level tree ("File System") and Page Search API are cloud-only and closed. Index refresh is per document, which suits stable filings and suits fast-changing corpora poorly.

• Traceability is the real win. Each answer comes with the nodes visited and page citations, so a wrong answer can be debugged as a wrong turn in the tree rather than an opaque similarity score.

• Vendor gravity. The open repo is the local, text-PDF tier. OCR, block-level citations, MCP and corpus search sit behind a paid API key.

When to use it

Use it when a few long, well-structured documents must be answered accurately with page citations, and a few seconds and about ten cents a question are acceptable. Keep vectors (or hybrid search) for large, fast-changing corpora, short documents, and recall-style "find every mention" queries. A common hybrid: vectors or keywords pick the document, PageIndex reads it.

Review checklist before adopting it:

• Spot-check toc_source and the tree on your own documents; count how many come back pages or unreadable.

• Re-run the 10-K notebook's method on your own questions, against a tuned baseline (hybrid search plus reranker), not naive top-k.

• Measure p50 and p95 latency and turns per question with your chat model, with caching on.

• Check that answers cite pages that were actually fetched (the grounding prompt asks for it; verify it holds).

• Decide local versus cloud: scanned or image-heavy input and corpus-wide search need the paid API.

• Plan re-indexing: when a document changes, its tree and summaries are rebuilt.

Related entries

Filed as R010 under 4.3 Retrieval over long documents in the project's research index.

• R002 (memory layers): a retrieval design for semantic memory. "Similarity is not relevance" is the failure R002 describes when the read step returns the near but wrong fact.

• R006 and R005: retrieval as tool calling and context management (R006), and as the context layer, "what it sees" (R005), rather than a preprocessing step.

Sources

• VectifyAI/PageIndex: README, pageindex/ source, Flash README and cookbook notebooks, cloned at commit 2ac8694.

• pageindex-vs-vector-10k notebooks for the accuracy and cost table.

• Not opened: the PageIndex docs site, blog, FinanceBench and Mafin 2.5 write-ups, and the Page Search benchmark repo.