R039 · AI's effect on people › Human verification of AI output

LLM reliability layers sort answers; they don't fix them

Read the original

agent-reliabilityevalsbenchmarksdeterministic-workflowshuman-oversightretrieval

Sources: Pushpdeep Mishra's LinkedIn post resharing InfoQ's post; InfoQ article, 2026-11-10 by Wejdan Bagais, Pushpdeep Mishra and a third author (InfoQ Certified Architect Program capstone; reviewer Hien Luu).

In one paragraph

The authors tested the common assumption that stacking checks around an LLM makes it more accurate. On an adverse-drug-event extraction task (the CADEC corpus: 1,250 forum posts, each reaction a span coded in SNOMED CT), they ran six layers on the same records with each one priced: a deterministic vocabulary check, self-correction on provable failures, three-sample voting, a second-model judge, refusal that holds doubtful records for a person, and the person. The layers did not add up. They sorted answers by how far to trust them and corrected almost none. Voting cost about 420K tokens a run and flipped as many right answers to wrong as the reverse. Self-correction fired two or three times a run. The judge separated right from wrong well once it was shown the same candidate menu the extractor saw, but no later layer read its verdict. The real ceiling was detection: no layer can recover a mention the extractor never quoted.

The system under test

Measure the noise floor first

Three identical runs at temperature zero differed by four F1 points. Of 233 grouped mentions, only 70% matched on both span and code across all three runs. Four other model families (llama3.1:8b, mistral:7b-instruct, qwen3:8b, granite4:micro-h) were byte-identical across repeats; the mixture-of-experts extractor was the only one that varied, and only inside a full run. From then on, every change was reported as a range across three runs.

What each layer bought

Why the layers don't stack

Does it generalize?

The authors re-ran on seven more corpora and five model families. "Layers sort, they don't fix" held. The free check held for clinical vocabularies (the endorsed lane was right 76% to 100% of the time) but reversed for gazetteers, where a place name can match and still be the wrong place (7% to 26%). It never fired on financial tag names. On the financial corpus the model read well but coded badly, mostly by habitually picking the menu's first line. Seventeen matrix cells ran with the wrong prompt or model, caught only because each run saved its received configuration.

Tradeoffs and caveats

Review checklist for a reliability stack

Connections

See R039 on the knowledge map

Part of the Research notes on nowucca.com. Figures and claims are the source's own unless the summary says otherwise.