Hybrid retrieval with an evidence lock

Entity resolution across four source types, none of which share a schema. A person on LinkedIn, a name in a press release, a signatory on a filing and a bio on a company site are the same human, described four ways, with roles that may be current in one source and stale in three. Get this wrong and every downstream inference is confidently about the wrong person.

Company questions are not uniformly semantic. "How do they make money" is a meaning problem — the answer is phrased a hundred ways and never in those words. "Are they using Snowflake" is a keyword problem, and a pure vector search will happily return a passage about a different warehouse because it is semantically adjacent. A retrieval design that picks one loses badly on the other half of the questions.

Free-running generation over scraped text invents things, and in this domain the asymmetry is brutal. A research tool that is right 90% of the time is not 90% useful, because a single invented detail spoken on a call costs more than the whole profile was worth.

And repeated runs cost real money. A research engine is used against overlapping accounts, repeatedly. Naive execution re-scrapes and re-reasons every time, and the cost of that determines whether the tool gets used freely or rationed.

• One typed object from four source types, with entities actually resolved

• Hybrid retrieval, because the question set is genuinely mixed

• Generation constrained to retrieved evidence, not steered toward it

• Deterministic and cache-first, so a rerun is nearly free and gives the same answer

Ingestion is schema-first, not text-first. Signals from LinkedIn, company sites, news and filings normalise into one metadata schema at the point of entry — roles, company facts, strategy cues, revenue signals, funding history — rather than being stored as text to be interpreted later. Interpreting later means interpreting repeatedly and inconsistently.

Retrieval is hybrid over a vector database, fed by a custom chunker. Semantic and keyword search run together rather than as a fallback chain, because the failure modes are complementary: semantic search finds the passage that means the right thing, keyword search finds the passage that says the exact thing. Company research needs both on the same profile.

Reasoning runs in deterministic structured loops that derive business model, ICP fit, strategic moves, risk profile and tech ecosystem. Each loop consumes retrieved evidence and emits typed output. Determinism is a requirement rather than an aesthetic — a research tool that gives two different reads of one company across two runs will not be trusted twice.

The evidence lock sits at generation. Output is constrained to what retrieval returned. A claim without a supporting retrieved passage does not appear — not softened, not hedged, absent.

The runtime is cache-first and streaming, so repeated runs against overlapping accounts reuse work rather than re-paying for it.

Design decision: the engine is permitted to return less. The failure mode we designed against is not an incomplete profile — it is a complete-looking profile with one invented line in it, because that one is undetectable until someone says it out loud.

Aggregate into one schema

Four source types normalise into unified metadata at ingestion. Entity resolution happens here, while the source context that makes it possible is still attached.

Chunk for retrieval, not for storage

A custom chunker prepares passages sized and bounded for the retrieval step rather than for convenient storage — chunk boundaries that split a role from its company are a retrieval bug created at write time.

Retrieve hybrid

Semantic and keyword search over the vector database, returning evidence-backed passages per reasoning step rather than one context blob for everything.

Reason in typed loops

Structured loops derive the analytical layer — business model, ICP fit, strategy, risk, tech ecosystem — each consuming retrieved evidence and emitting typed output that the next step can rely on.

Generate against the lock

A reason-and-act loop produces pitch variants, tone analysis, objection handling and follow-ups, each tied to specific retrieved evidence.

Cache everything worth reusing

Deterministic run rules plus caching mean a repeated or overlapping run reuses prior work, which is what makes the tool cheap enough to run without thinking about it.

Governance

The evidence lock is the governing control: retrieval scope defines what can be asserted, and generation cannot exceed it. Schema strictness is the second control — typed output means a downstream step consumes a known shape rather than re-parsing prose, so an error surfaces as a validation failure instead of propagating as plausible text.

Orchestration

Ingest, normalise, chunk, retrieve hybrid, reason in loops, generate under the lock, cache. Each reasoning step retrieves for itself rather than sharing one context, which is what keeps evidence attributable per claim instead of per profile.

Observability

Every claim resolves to the passage supporting it and that passage to its source document. The distillation ratio is worth watching in its own right — on one run, 420 collected items reduced to 6 used as evidence, and a run where that ratio collapses is usually a retrieval problem rather than a source problem.

Cache hit rate is the cost metric. It is what determines whether the engine is economical to run repeatedly, which is the only way a research tool actually gets used.

Entity resolution is the failure that matters and the one that hides. A misresolved person produces a profile that is internally coherent, well-sourced and about the wrong human — every downstream check passes, because each individual claim is genuinely supported.

Retrieval returning too little is the safe failure by construction: the lock means the output says less rather than filling the gap from parametric memory. This is deliberately the behaviour we biased toward, because a short profile is visibly short and an invented one is not.

Stale sources are the third case. A role that changed six months ago is still on the page that was scraped, and there is no signal in the text itself that it is out of date — which is why recency is carried as metadata from ingestion rather than inferred at reasoning time.

Measured on our own pipeline, running our own outreach research.

• Full-profile analysis time down 80–90%, from parallelised extraction and retrieval caching

• 420 collected items reduced to 6 grounded sources on one observed run, each traceable

• No output drift across repeated runs, from deterministic loops and evidence-locked generation

• An estimated 50–60% fewer external API calls on repeated runs, via cache-first execution

Directly demonstrable: multi-source ingestion into a strict schema, the hybrid retrieval engine over a vector database, the evidence lock at generation, and the deterministic cache-first runtime.

Internal measurements, ours alone: the 80–90% time reduction against our own prior manual process, the 420-to-6 ratio from one observed run rather than an average, and the 50–60% API reduction as an estimate from our usage rather than a benchmark.

Status, stated plainly: this is Bigcircle's own prospect-intelligence engine, used on our own pipeline. It is not a deployed client product.

If you are building retrieval-backed research, the two decisions worth making early are hybrid-versus-semantic retrieval — decide by looking at your actual question distribution, not by preference — and whether your generation step is constrained by retrieval or merely encouraged by it. The second determines whether you can put the output in front of a customer.