Per-field confidence and the claim-to-evidence path

Flattening destroys the evidence. A financial or ESG table converted to a text blob loses the row-column relationship that made a number mean something. The figure survives; what it is a figure of does not. Any pipeline that normalises to plain text before extraction has already lost the case.

Four things must all be right, and being wrong about any one produces a plausible answer. Entity, period, unit, framework. A retrieved passage stating a correct number for the wrong subsidiary, the wrong reporting year, in thousands rather than millions, or against a superseded framework version is not partially correct — it is wrong in a way no downstream reader can detect.

Framework mapping is a versioning problem, not a classification problem. GRI, CSRD, SDGs and internal KPI sets change. Two disclosures reporting the same underlying fact against different versions of the same framework have not reported the same number, and comparing them without tracking the mapping version silently produces a trend that does not exist.

And extraction confidence has to be a first-class output. A pipeline that returns values without certainty forces the analyst into a binary — trust everything or re-derive everything — and they will rationally choose the second, which is the work the system existed to remove.

• Preserve document structure through parsing, so tables stay tables

• Carry entity, period, unit and framework as hard metadata, never inferred at query time

• Version framework mappings, because comparability depends on it

• Score every extracted field, so uncertainty routes to review rather than into a dashboard

A containerised pipeline from raw document to governed, queryable intelligence, with five layers that each preserve something the next one needs.

Layout-aware parsing takes each document through intake, OCR and language processing, producing structured chunks, sections, tables and footnotes with rich metadata. The design goal is machine-readability without removing the original structure — which is the opposite of the usual normalise-to-text instinct, and is the decision the rest of the system depends on.

ESG-aware extraction pulls entities, metrics, units, time periods, tables and narrative claims, each with a confidence score. That score is not a diagnostic. It is a routing signal: low-confidence fields are flagged for review rather than passed downstream as fact.

Framework mapping aligns KPIs and statements to SDGs, GRI, CSRD or a custom framework through classifiers plus rule logic, with traceable, version-controlled mappings. Rules and models together, because the rules encode the parts that are definitional and must not be probabilistic.

A knowledge graph plus vector search links entities and KPIs across reports. The graph carries the relationships — this entity, this period, this framework — and vector search carries the semantic reach. Either alone fails: pure vector search cannot express "the same subsidiary two years earlier," and pure graph traversal cannot find the narrative claim that was never structured.

A governed analytics layer serves it, over a versioned store with role-based access, audit logs and lineage to source pages.

Design decision: research questions are decomposed rather than retrieved once. The system selects sources, combines graph and vector evidence, constructs claims, checks them, and attaches provenance — because the questions analysts actually ask are comparative, and one retrieval cannot answer a comparison.

Parse without flattening

Intake, layout-aware OCR and language processing produce structured chunks, sections, tables and footnotes. Table structure and footnote attachment survive, because a metric detached from its qualifier is a liability.

Extract with confidence

Entities, metrics, units, periods, tables and narrative claims come out typed, each with a certainty score that determines whether it proceeds or routes to a human.

Map to frameworks, versioned

Classifiers and rules align each signal to the applicable framework, recording which mapping version was applied — so a later comparison can tell a real change from a definitional one.

Link across the corpus

Entities and KPIs join the knowledge graph; passages join the vector index. Cross-report and time-series questions become traversals rather than repeated reads.

Decompose the research question

A question is broken into parts, sources are selected per part, graph and vector evidence are combined, claims are constructed, checked against the existing knowledge base, and provenance is attached.

Serve under governance

Dashboards, heatmaps, benchmarks and anomaly views over a versioned store with role-based access and audit logging.

Governance

Source authority, document metadata, reporting period, framework mapping, role-based access and lineage together determine what may be used as a fact. Low-confidence extraction cannot silently become accepted data. Generated analysis must retain citations and source-page links — an unsupported conclusion does not get presented as a finding, which is enforced at generation rather than reviewed afterwards.

Orchestration

Intake, OCR, extraction, mapping, graph construction, retrieval, analysis, fact-checking, presentation. The fact-checking step is the one most pipelines omit: a new filing is checked against what the knowledge base already holds, which is how a restatement or a contradiction surfaces as a flag instead of as two coexisting truths.

Observability

The observable unit is the claim-to-evidence path: document, page, parsed structure, extracted entity or metric, confidence, retrieval decision, framework mapping, generated conclusion.

Versioning is what makes this useful over time. Comparing two revisions answers the question that actually gets asked in an audit — did this number move because the company changed, because a document was restated, or because we changed how we read it?

The dangerous failure is a confident extraction of a correct number under wrong metadata. Everything downstream passes: the value is real, the citation resolves, the page is right. It is simply attributed to the wrong entity, period, unit or framework version. This is why those four travel as hard metadata from parsing rather than being inferred at query time — inference at query time makes the error unrecoverable.

Scanned and irregular documents are the volume problem: annexures, rotated tables, footnotes that qualify a figure three pages away. The correct behaviour is low confidence and routing to review, not a best guess, and the tuning work is mostly about making sure low confidence actually fires rather than producing a confident wrong parse.

Framework version drift is the slow failure. A mapping that was right last year silently produces a broken comparison this year, which is why mappings are version-controlled and the version is recorded on the mapped fact rather than held globally.

• Analysis cycles from weeks to hours, replacing manual extraction and spreadsheet assembly at high document volume

• Every answer citation-grounded and fact-checked against the existing knowledge base rather than generated from one retrieval

• Low-confidence extraction routed to review, so uncertainty is visible instead of averaged into a dashboard

• Claim-to-evidence path inspectable end to end, from generated conclusion back to the source page

Directly demonstrable: grounded research with citations, fact-checking against the knowledge base, custom evaluations, the generated analytical interfaces, per-field confidence scoring, and human review of flagged extractions.

Internal working results requiring confirmation: the weeks-to-hours cycle-time change, and any universal statement about complete lineage. Lineage holds as designed; "every metric, always" is a stronger claim than we have measured, so we do not make it.

If you are building document intelligence over institutional filings, the two decisions that are expensive to reverse are whether you flatten structure during parsing, and whether extraction emits confidence. Both look like refinements you can add in a second pass. Neither is — the first destroys information you cannot recover, and the second changes what analysts are willing to build on top of.