# Bigcircle

> Bigcircle supplies production-proven AI engineers. They embed in your team or lead the build, and every one of them has already shipped real AI in production: agents that act on live systems, retrieval-augmented generation grounded in your own data, document intelligence, and voice AI. Engagements run from a single embedded engineer to a full delivery pod.

Bigcircle is an AI engineering studio. The site has four kinds of page worth reading:

- **Services and roles** describe what can actually be bought, and who does the work.
- **Case studies** are write-ups of systems that reached production, with the constraints and the numbers.
- **Guides** are long-form, opinionated engineering explainers on inference, retrieval, agents, evals, and reliability.
- **The glossary** holds 80 plain-English definitions of LLM concepts, each written by someone who has shipped it.
- **The roadmap** is the implementation skill ladder itself, nine layers deep, each layer a page listing the competencies, concepts, technologies, and production practices it takes to ship that layer.

Contact: hello@bigcircle.ai. Full text of every page: https://bigcircle.ai/llms-full.txt

## Start here

- [Bigcircle: production-proven AI engineers](https://bigcircle.ai/): Engineers who have already shipped real AI in production, embedded in your team or leading the build.
- [Hire elite Gen AI engineers](https://bigcircle.ai/hire): The top 3% of generative-AI engineers, production-proven and embedded in your team in days.
- [Production AI engineering services](https://bigcircle.ai/services): 20 production AI engineering services, from building agents through getting them to production, models and infrastructure, and the work around the AI.
- [Case studies](https://bigcircle.ai/work): Production AI our engineers have shipped: agents acting on live systems, citation-grounded document intelligence, sub-second voice.
- [AI engineering guides](https://bigcircle.ai/guides): Long-form, opinionated guides to LLM inference, retrieval, agents, evals, and production AI engineering.
- [AI engineering glossary](https://bigcircle.ai/glossary): Plain-English definitions of the LLM concepts behind real AI systems, written by engineers who have shipped them.
- [The AI engineer roadmap](https://bigcircle.ai/roadmap): The implementation skill ladder behind Bigcircle's engineers, foundations through fine-tuning, serving, and production.
- [AI readiness assessment](https://bigcircle.ai/assessment): Score your organization across six AI readiness dimensions and get a maturity report with tailored next steps.
- [Can your team ship production GenAI?](https://bigcircle.ai/hire/assessment): A free assessment for engineering leaders: score your team's ability to build and ship production GenAI.
- [Contact Bigcircle](https://bigcircle.ai/contact): Talk to the engineers who would do the work. Reply within two hours on weekdays.
- [About Bigcircle](https://bigcircle.ai/about): How the Bigcircle team is built, why it was set up that way, and where to check the work.
- [Privacy](https://bigcircle.ai/privacy): What the contact form, newsletter, and assessments store, which analytics scripts load, and how to ask about a record.
- [Terms](https://bigcircle.ai/terms): Using bigcircle.ai, what a form submission is for, and the line between this site and a signed project.

## Who writes this

- [Tirth Gajjar, Founder & CTO](https://bigcircle.ai/team/tirth-gajjar): Founder and CTO of Bigcircle. He hires and trains the engineering team, leads the production AI work, and writes the guides on this site.

## Services

- [AI agent development](https://bigcircle.ai/services/ai-agents): Multi-step agents for workflows where a wrong action is expensive to undo.
- [Document extraction](https://bigcircle.ai/services/document-intelligence): Document extraction and search for teams who have to show their source.
- [Production AI engineering](https://bigcircle.ai/services/production-ai-engineering): Evals, routing, and audit trails for teams whose demo cannot survive real traffic.
- [Cloud and DevOps](https://bigcircle.ai/services/cloud-devops): Cloud changes made safe to apply, for teams postponing the risky ones.
- [AI voice agents](https://bigcircle.ai/services/voice-ai): Sub-second voice agents for teams whose customers are waiting on hold.
- [AI product engineering](https://bigcircle.ai/services/product-engineering): The product surface around your model, for teams who have the AI but not the app.
- [Agent orchestration](https://bigcircle.ai/services/agent-orchestration): Multi-agent systems, for work that outgrew a single agent.
- [RAG systems](https://bigcircle.ai/services/rag-systems): Retrieval over your own data, for answers that have to come from it.
- [AI features in an existing product](https://bigcircle.ai/services/ai-in-your-product): AI inside a product you already have, for teams who do not want a second product.
- [Agent harness engineering](https://bigcircle.ai/services/harness-engineering): The runtime around the model, for teams whose agent works in a notebook and not on real traffic.
- [LLM evaluation](https://bigcircle.ai/services/llm-evaluation): Eval suites and regression testing, for teams shipping on opinion instead of a number.
- [AI observability](https://bigcircle.ai/services/ai-observability): Tracing and monitoring for LLM and agent systems, for teams who cannot explain what happened.
- [Model routing and cost control](https://bigcircle.ai/services/model-routing): Routing across models by task, for teams paying frontier prices for routine work.
- [Guardrails and safety layers](https://bigcircle.ai/services/guardrails): Policy checks around model output and agent actions, for systems where a bad action is expensive.
- [PII redaction and data boundaries](https://bigcircle.ai/services/pii-redaction): A data boundary in front of the model, for teams who cannot send raw records to a provider.
- [Fine-tuning](https://bigcircle.ai/services/fine-tuning): Fine-tuning and adapters, for behavior a prompt cannot hold.
- [On-device and edge inference](https://bigcircle.ai/services/on-device-inference): Models running locally, for data that cannot leave the machine.
- [Self-hosted model deployment](https://bigcircle.ai/services/self-hosted-models): Open models on your own infrastructure, for teams who will not call an API.
- [MCP servers and tool layers](https://bigcircle.ai/services/mcp-servers): MCP servers that expose your systems to agents, without handing over credentials.
- [Data pipelines for AI](https://bigcircle.ai/services/data-pipelines): Ingestion and preparation, for data that is not retrieval-ready.

## Engineering roles you can hire

- [Hire Generative AI Engineers](https://bigcircle.ai/hire/generative-ai-engineers): All-rounders who ship a whole AI feature end to end, or anchor a pod of the specialists below.
- [Hire LLM Engineers](https://bigcircle.ai/hire/llm-engineers): Model specialists who fine-tune, quantise, and serve the model itself.
- [Hire AI Agent Engineers](https://bigcircle.ai/hire/ai-agent-engineers): Builders of agents that plan and take real actions in your systems, safely.
- [Hire RAG & Retrieval Engineers](https://bigcircle.ai/hire/rag-engineers): Retrieval specialists who ground every answer in your data.
- [Hire LangChain Engineers](https://bigcircle.ai/hire/langchain-engineers): Engineers who know which LangChain abstractions to keep and which to drop.
- [Hire LlamaIndex Engineers](https://bigcircle.ai/hire/llamaindex-engineers): Retrieval engineers who work the ingestion path, not only the vector store.
- [Hire CrewAI Engineers](https://bigcircle.ai/hire/crewai-engineers): Multi-agent engineers who bound run cost and delegation before anything runs unattended.
- [Hire MLOps & AI Infra Engineers](https://bigcircle.ai/hire/mlops-engineers): The infra people who keep AI fast, efficient, and observable in production.
- [Hire Voice AI Engineers](https://bigcircle.ai/hire/voice-ai-engineers): Realtime voice agents that answer in under a second.
- [Hire Data Engineers for AI](https://bigcircle.ai/hire/data-engineers): The pipelines that turn raw data into retrieval-ready, training-ready fuel.
- [Hire Prompt & Evaluation Engineers](https://bigcircle.ai/hire/prompt-evaluation-engineers): The people who turn AI quality into a number you can ship against.

## Case studies

Each case study is published in two lanes: the business read at `/work/<name>`, and the engineering read at `/work/<name>/build`. They are different documents about one system — outcome and operating change in the first, architecture and failure behaviour in the second.

- [Series A LegalTech Built an AI Trademark Drafting Platform](https://bigcircle.ai/work/trademark-drafting-fine-tuned-llm): A drafting engine for trademark clearance opinions. It holds the firm's five-part structure on every draft, can only cite authorities from a curated index, and rejects its own output when a citation does not resolve, so the attorney reviews and signs rather than writing from scratch.
- [Making a fabricated citation structurally impossible](https://bigcircle.ai/work/trademark-drafting-fine-tuned-llm/build): The engineering write-up behind Brandiligence. Why prompting cannot solve citation integrity, how the four constraint layers compose, and the 0-10 legal-integrity harness that made quality measurable rather than asserted.
- [Seed-Stage DevOps SaaS Built an AI Cloud Operations Agent](https://bigcircle.ai/work/devops-agent-live-aws): A DevOps agent for cloud infrastructure teams. It reads the real account before it plans, proves the change is valid before it applies, shows an engineer a diagram of what will exist, and tags everything it creates back to the request that asked for it.
- [Treating the agent as untrusted on live AWS](https://bigcircle.ai/work/devops-agent-live-aws/build): The engineering write-up behind Skionis. Why a generate-then-apply loop is unusable against CloudFormation rollback times, how live quota and region reads change planning, what the validation loop actually catches, and how tagging makes post-hoc attribution possible.
- [Venture-Backed B2B SaaS Built an AI Chief-of-Staff Agent](https://bigcircle.ai/work/executive-ops-desktop-agent): A 24/7 AI chief-of-staff for founders and leadership teams. The leader states the outcome they want; the agent works the sequence across every system the routine touches, checks its own output, and comes back with finished work or the one thing it could not resolve.
- [Dynamic tool discovery across 100+ integrations](https://bigcircle.ai/work/executive-ops-desktop-agent/build): The engineering write-up behind Solarpunk. Retrieval-before-invocation over a registry of 100+ tools, a planner-critic loop with bounded retry, an on-device credential vault outside model context, and code execution instead of summarisation for spreadsheet work.
- [Series A SalesTech Built an On-Device AI Meeting Intelligence Platform](https://bigcircle.ai/work/on-device-meeting-transcription): A real-time meeting-intelligence stack for account executives. It notices the call has started, transcribes it on the rep's own machine, guides them while they are still talking, and hands back a structured CRM-ready summary the moment the call ends.
- [Live-meeting detection and CPU-only transcription](https://bigcircle.ai/work/on-device-meeting-transcription/build): The engineering write-up behind AnyTeam. OS-event meeting detection with recovery from partial states, native loopback-aware capture, a quantized on-device speech model with vocabulary-biased decoding, and streaming guidance under real-time on commodity laptops.
- [Institutional ESG Analytics Built an AI Filing Intelligence Platform](https://bigcircle.ai/work/esg-filings-citation-extraction): An ESG-intelligence pipeline that ingests dense, inconsistent filings at scale, extracts framework-aligned metrics with per-field confidence and source lineage, connects them across reports and years, and serves them through a governed analytics layer where a low-confidence number goes to review rather than into a dashboard.
- [Per-field confidence and the claim-to-evidence path](https://bigcircle.ai/work/esg-filings-citation-extraction/build): The engineering write-up behind DocVerse. Why flattening a filing to text destroys the evidence, why per-field confidence is the feature that makes output reusable, and how decomposed research over graph plus vector evidence beats single-retrieval RAG on institutional documents.
- [Enterprise Insurer Built an AI Voice Claims Platform](https://bigcircle.ai/work/insurance-voice-agent): An insurance voice agent that handles routine policy, renewal, claim and payment calls end to end. Fast enough that callers stay in the conversation, and built so every action runs through a validated tool and every call leaves a structured record.
- [A 900ms voice loop with a hard tool boundary](https://bigcircle.ai/work/insurance-voice-agent/build): The engineering write-up behind AVOX. Where the latency budget actually goes in a voice loop, why turn detection is the component that breaks first on code-switched audio, and how a typed tool contract turns compliance from a model property into a system property.
- [Institutional Fund Manager Built an AI Reconciliation Platform](https://bigcircle.ai/work/fund-operations-reconciliation): An institutional fund and portfolio operations platform. It ingests statements and brokerage data from every custodian, normalises and reconciles them automatically, surfaces breaks while they are still small, and keeps the source lineage a property of the data rather than a report assembled afterwards.
- [Lineage as a data property, not a report](https://bigcircle.ai/work/fund-operations-reconciliation/build): The engineering write-up behind FinSight. Why per-custodian normalisation is the hard part, why an unresolved break must be a state rather than a difference, and what it costs to make lineage structural instead of reconstructed.
- [Bigcircle Built an AI Prospect Intelligence Engine](https://bigcircle.ai/work/prospect-intelligence-retrieval): A reasoning-first prospect intelligence engine that turns a company or LinkedIn URL into a structured, intent-aware profile. Multi-source ingestion and hybrid retrieval surface business model, ICP fit, strategy and risk, with generation constrained to evidence the system actually retrieved.
- [Hybrid retrieval with an evidence lock](https://bigcircle.ai/work/prospect-intelligence-retrieval/build): The engineering write-up behind Prospex. Why entity resolution across LinkedIn, sites, news and filings is the real problem, why hybrid beats pure vector search on company questions, and how cache-first deterministic execution makes a research tool affordable to run repeatedly.
- [PropTech Platform Built an AI Voice Property Discovery Agent](https://bigcircle.ai/work/real-estate-voice-agent): A conversational agent for real-estate discovery. It turns free-form buyer conversation into structured, constraint-aware property search against current listings in English, Hindi and Hinglish, with no human qualification step, on the same runtime built for insurance.
- [Porting a voice runtime to a second industry](https://bigcircle.ai/work/real-estate-voice-agent/build): The engineering write-up behind AVOX Realty. Which layers of a voice agent are domain-independent, why conversational search needs accumulated constraint state rather than query extraction, and how Hinglish code-switching changes the turn-detection problem.
- [Bigcircle R&D Built an AI Sales Roleplay Coaching Platform](https://bigcircle.ai/work/sales-roleplay-voice-agent): A voice-first sales coaching simulator. A real-time agent runs roleplay calls that raise objections and counter-questions rather than answering politely, and every session is scored against the same rubric, so feedback is consistent across reps instead of depending on who listened.
- [An adversarial roleplay loop with normalised scoring](https://bigcircle.ai/work/sales-roleplay-voice-agent/build): The engineering write-up behind RepliCoach. Why an adversarial roleplay loop is a harder real-time problem than a cooperative one, how structured session capture makes scoring possible at all, and what normalisation does to make rubric scores comparable across reps and time.

## Guides

- [How LLM inference works](https://bigcircle.ai/guides/how-llm-inference-works): A practical guide to the systems work behind fast, affordable LLM serving.
- [Why RAG systems fail in production](https://bigcircle.ai/guides/why-rag-systems-fail-in-production): Most RAG failures look like model failures until teams measure retrieval, context assembly, and generation separately.
- [How to build production AI agents](https://bigcircle.ai/guides/practical-guide-to-ai-agents): Useful agents are not magic workers. They are constrained systems that decide, act, verify, and stop.
- [Why enterprise AI pilots fail](https://bigcircle.ai/guides/how-to-derisk-your-ai-strategy): Most enterprise AI pilots fail for predictable, well-documented reasons. This guide walks through what the evidence says and the decisions that separate the few that reach production value from the rest.
- [When your AI product needs write access](https://bigcircle.ai/guides/ai-product-write-access): A practical guide to choosing the first action an AI product can perform inside an authority boundary that engineering, product, and security can review.
- [How to scope AI agent permissions](https://bigcircle.ai/guides/ai-agent-permissions): How to bind one agent action to a principal, task, target, policy, credential, approval, expected state, and recovery path.
- [Why AI agents report unfinished work as done](https://bigcircle.ai/guides/why-ai-agents-report-work-as-done): A production guide to expected-state contracts, independent verification, idempotency, reconciliation, retry safety, partial failure, and causal audit evidence.
- [How to roll out AI agent autonomy in five levels](https://bigcircle.ai/guides/rolling-out-ai-agent-autonomy): How to move one agent action from read-only to bounded execution through explicit authority levels, exit evidence, demotion rules, and shared release ownership.
- [RAG vs fine-tuning vs agentic search](https://bigcircle.ai/guides/rag-vs-fine-tuning-vs-agentic-search): Most teams pick a technique and then look for the problem. Start from the failure you can observe in your logs and the choice is usually forced.
- [How to build LLM evals](https://bigcircle.ai/guides/how-to-build-llm-evals): You cannot write the rubric first. Criteria emerge from reading your own failures, and everything downstream depends on doing that step properly.
- [How to build an audit trail for AI decisions](https://bigcircle.ai/guides/ai-decision-audit-trail): A tool trace can show what ran without explaining the accepted business decision. This guide defines the record an AI workflow needs when a case is challenged or its evidence changes.
- [Embedded AI engineer vs hire vs agency](https://bigcircle.ai/guides/embedded-ai-engineer-vs-hire-vs-agency): Embed a production-proven engineer when you need AI depth in your repo without a permanent headcount bet. Hire full-time when AI is a multi-year competency you can manage. Use an agency for a bounded deliverable with acceptance tests, and not for ongoing ownership inside your systems.
- [Embed vs CTO-led pod vs fixed-scope](https://bigcircle.ai/guides/embed-vs-cto-led-pod-vs-fixed-scope): Embed an engineer when you can direct the work and need AI depth in your repo. Take a CTO-led pod when you have a roadmap and no senior AI leadership to run it. Buy a fixed-scope build when you want the working system, and not the hiring or the management.

## The AI engineer roadmap

- [Foundations: the AI engineer roadmap](https://bigcircle.ai/roadmap/foundations): The engineering bedrock under every AI system. Before the model, the fundamentals that keep it honest in production.
- [LLM applications: the AI engineer roadmap](https://bigcircle.ai/roadmap/llm-applications): Turning a raw model into a dependable product surface, with output you can actually build on.
- [Retrieval and RAG: the AI engineer roadmap](https://bigcircle.ai/roadmap/retrieval-rag): Grounding answers in real data, with citations, so the system says what is true rather than what is plausible.
- [AI agents: the AI engineer roadmap](https://bigcircle.ai/roadmap/ai-agents): Systems that plan and act across real tools, check their own work, and recover when a step goes wrong.
- [Multimodal and voice: the AI engineer roadmap](https://bigcircle.ai/roadmap/multimodal-voice): Vision, generation, and real-time voice, built to stay fast and grounded on a live call.
- [Evaluation and error analysis: the AI engineer roadmap](https://bigcircle.ai/roadmap/evaluation-observability): The core production discipline. Not generic metrics, but looking at your own data, naming failures, and measuring the things that actually break.
- [Fine-tuning and adaptation: the AI engineer roadmap](https://bigcircle.ai/roadmap/fine-tuning): When prompting and retrieval run out, reshape the model itself. Done right, a small fine-tune can match a frontier model on your task at a fraction of the cost.
- [Serving and inference: the AI engineer roadmap](https://bigcircle.ai/roadmap/serving-inference): What it takes to run a model yourself, fast and affordably. The two-phase nature of inference governs your real cost and tail latency, and it is where most teams have no depth at all.
- [Production and LLMOps: the AI engineer roadmap](https://bigcircle.ai/roadmap/production): Running it for real, inside a latency and cost budget, observable, defended against hostile input, and reliable enough that customers feel it.

## Glossary

- [Agent Memory](https://bigcircle.ai/glossary/agent-memory): How an agent carries state across steps and sessions when the work outgrows the context window, using scratchpads, episodic stores, compaction, and retrieval back into context.
- [AI Agent](https://bigcircle.ai/glossary/ai-agent): An LLM running in a loop that decides its own actions toward a goal using tools and feedback, instead of producing a single answer.
- [Attention](https://bigcircle.ai/glossary/attention): The mechanism that lets each token weigh every other token in the sequence when deciding what comes next, and the reason long contexts cost quadratically more.
- [Batch API](https://bigcircle.ai/glossary/batch-api): Submit large non-urgent LLM workloads asynchronously for a 50% per-token discount, in exchange for results that arrive within a 24-hour window instead of seconds.
- [Benchmark Contamination](https://bigcircle.ai/glossary/benchmark-contamination): Benchmark contamination happens when evaluation examples or close variants appear in training data. Scores then measure memorization or overfitting instead of real generalization.
- [Catastrophic Forgetting](https://bigcircle.ai/glossary/catastrophic-forgetting): When fine-tuning on a narrow dataset overwrites general capabilities the base model already had, trading broad competence for a single task.
- [Chain-of-Thought](https://bigcircle.ai/glossary/chain-of-thought): Prompting a model to work through intermediate reasoning steps before answering, which improves accuracy on hard multi-step tasks at the cost of extra output tokens and latency.
- [Chunking](https://bigcircle.ai/glossary/chunking): Chunking is how you split documents into passages before indexing, and it sets a hard ceiling on what retrieval can ever find.
- [CodeAct](https://bigcircle.ai/glossary/codeact): CodeAct agents act by writing executable code instead of emitting one JSON tool call at a time, so composition, loops, and error handling happen in a single action inside a sandbox.
- [Computer Use & GUI Agents](https://bigcircle.ai/glossary/computer-use): GUI agents control software the way a person does, by reading screenshots and issuing clicks and keystrokes, which unlocks the long tail of applications that have no API. By mid-2026 the best agents roughly match human success rates on the OSWorld benchmark, but latency, compounding errors, and prompt injection still demand sandboxes and confirmation gates.
- [Constitutional AI & RLAIF](https://bigcircle.ai/glossary/constitutional-ai): Constitutional AI aligns models against a written set of principles by having models critique, revise, and label outputs. RLAIF uses AI-generated preference labels in place of some human preference work.
- [Constrained Decoding](https://bigcircle.ai/glossary/constrained-decoding): Constrained decoding masks invalid next tokens during generation so the model can only emit text accepted by a schema, grammar, or regex. It turns "please return JSON" into an enforceable decoding contract.
- [Context Window](https://bigcircle.ai/glossary/context-window): The fixed token budget a model can attend to in one call, shared by the prompt and the generated output, and the hardest constraint in LLM application design.
- [Continuous Batching](https://bigcircle.ai/glossary/continuous-batching): A serving scheduler that rebuilds the batch every decode step, swapping finished sequences out and waiting requests in, so the GPU never idles on completed slots.
- [Data Poisoning](https://bigcircle.ai/glossary/data-poisoning): Data poisoning plants malicious examples in training, fine-tuning, retrieval, or tool data so a model learns a hidden behavior. The same idea can attack pretraining corpora, RAG indexes, and agent observations.
- [Diffusion Models](https://bigcircle.ai/glossary/diffusion-models): Diffusion models generate images by learning to reverse noise, starting from pure static and denoising step by step toward a picture that matches your prompt.
- [Disaggregated Serving](https://bigcircle.ai/glossary/disaggregated-serving): Disaggregated serving runs prefill and decode on separate GPU pools, then transfers the KV cache between them. It improves utilization because prefill is compute-heavy while decode is memory-bandwidth-heavy.
- [Distillation](https://bigcircle.ai/glossary/distillation): Training a small, cheap student model to mimic a large teacher model on your task, transferring its behavior while cutting inference cost and latency by an order of magnitude.
- [DPO (Direct Preference Optimization)](https://bigcircle.ai/glossary/dpo): A preference-tuning method that optimizes a model directly on chosen and rejected response pairs with a simple classification-style loss, skipping the separate reward model and RL loop that RLHF requires.
- [Embeddings](https://bigcircle.ai/glossary/embeddings): Embeddings map text to vectors so that semantic similarity becomes geometric closeness you can search, cluster, and classify.
- [Evals](https://bigcircle.ai/glossary/evals): Evals are task-specific test suites that tell you whether a prompt or model change actually helped, replacing generic benchmarks and gut feel with a measurable gate.
- [Few-Shot & In-Context Learning](https://bigcircle.ai/glossary/in-context-learning): Models learn a task from examples placed in the prompt at inference time, with no weight updates or training. More and better-chosen examples improve format reliability and edge-case handling, at a token cost.
- [Fine-Tuning (SFT)](https://bigcircle.ai/glossary/fine-tuning): Continuing a model's training on curated input-output examples so it reliably adopts a behavior, format, or style that prompting alone cannot pin down.
- [FlashAttention](https://bigcircle.ai/glossary/flashattention): FlashAttention is an exact attention algorithm that avoids materializing the full attention matrix in high-bandwidth memory. It makes attention faster by reducing memory traffic, not by changing the model's math.
- [Foundation & Frontier Models](https://bigcircle.ai/glossary/foundation-models): Foundation models are large models pretrained once on web-scale data and then adapted to many downstream tasks. "Frontier" names the most capable current generation of those models.
- [GraphRAG](https://bigcircle.ai/glossary/graphrag): GraphRAG builds an entity and relationship graph from a corpus, then retrieves through that graph for questions that need multi-hop reasoning or corpus-level synthesis.
- [Groundedness (Faithfulness)](https://bigcircle.ai/glossary/groundedness): Groundedness asks whether every factual claim in an answer is supported by the retrieved passages. It is the generation-side counterpart to retrieval recall, and it is checkable in a way that "is this answer good" is not.
- [GRPO](https://bigcircle.ai/glossary/grpo): GRPO is a reinforcement-learning method that removes PPO's separate value model by scoring a group of sampled answers relative to each other. It became prominent through DeepSeek-R1 style reasoning training.
- [Guardrails](https://bigcircle.ai/glossary/guardrails): Guardrails are programmable input and output filters that sit around the model to block unsafe, off-topic, or policy-violating content, ideally failing closed.
- [Hallucination & Grounding](https://bigcircle.ai/glossary/hallucination): Hallucination is when a model states something fluent and confident but false, and grounding answers in retrieved evidence with citations is how production systems contain it.
- [Hybrid Search](https://bigcircle.ai/glossary/hybrid-search): Hybrid search runs keyword (BM25) and vector search in parallel and fuses their rankings, so retrieval catches exact terms and paraphrases at the same time. It is the default retriever for production RAG.
- [Judge Alignment (TPR, TNR, Kappa)](https://bigcircle.ai/glossary/judge-alignment): A model judge is only worth its cost if it agrees with the person whose standard it replaces. Raw agreement hides a judge that passes everything, so alignment is measured with separate error rates and an agreement score corrected for chance.
- [Knowledge Cutoff](https://bigcircle.ai/glossary/knowledge-cutoff): The date past which a model has no training knowledge, which is why retrieval, tools, and web search exist to fill the gap.
- [KV Cache](https://bigcircle.ai/glossary/kv-cache): The cache of attention keys and values that lets each new token skip recomputing the whole sequence, and the main memory limit on how many requests a GPU can serve at once.
- [Late Interaction & ColBERT](https://bigcircle.ai/glossary/late-interaction): Late interaction retrieval keeps token-level vectors and compares them at query time, avoiding the information loss of one embedding per passage. ColBERT is the canonical architecture.
- [LLM Observability & Tracing](https://bigcircle.ai/glossary/observability): LLM observability records every prompt, retrieval, tool call, token count, and latency as spans in a trace, so you can debug non-deterministic chains from evidence instead of guesswork.
- [LLM-as-a-Judge](https://bigcircle.ai/glossary/llm-as-judge): Using a strong model to grade other models' outputs at scale is powerful but bias-prone, so it works best as binary or pairwise judgments calibrated against human-labeled examples.
- [Logprobs & Confidence](https://bigcircle.ai/glossary/logprobs): Per-token log-probabilities expose how confident a model was in each choice, powering cheap confidence scores, abstention, routing, and evals.
- [LoRA & QLoRA](https://bigcircle.ai/glossary/lora-qlora): Parameter-efficient fine-tuning that freezes the base model and trains tiny low-rank adapter matrices, making customization cheap, fast, and swappable. QLoRA runs the same trick on a 4-bit quantized base so it fits on one GPU.
- [Mechanistic Interpretability](https://bigcircle.ai/glossary/mechanistic-interpretability): Mechanistic interpretability tries to reverse-engineer the computations inside neural networks. Instead of asking what a model says, it asks which features and circuits caused the behavior.
- [Mixture-of-Experts (MoE)](https://bigcircle.ai/glossary/mixture-of-experts): An architecture where a router activates only a few expert sub-networks per token, so a model can hold huge total knowledge while paying inference compute for a small fraction of it.
- [Model Context Protocol (MCP)](https://bigcircle.ai/glossary/mcp): An open standard, started by Anthropic and now governed by the Linux Foundation, that gives AI applications a uniform way to connect to tools and data sources, replacing custom point-to-point integrations.
- [Model Routing & Cascading](https://bigcircle.ai/glossary/model-routing): Send easy requests to a cheap model and escalate only the hard ones to a frontier model, optimizing cost per successful answer while holding a quality floor.
- [MQA, GQA & MLA](https://bigcircle.ai/glossary/mqa-gqa-mla): MQA, GQA, and MLA reduce the size of the KV cache by sharing or compressing keys and values across attention heads. They are a major reason modern long-context models can serve many requests on finite GPU memory.
- [Multi-Agent Orchestration](https://bigcircle.ai/glossary/multi-agent-orchestration): Splitting a task across specialized agents coordinated by a supervisor, trading roughly 15x token cost for parallelism, context isolation, and separation of concerns.
- [Next-Token Prediction](https://bigcircle.ai/glossary/next-token-prediction): An LLM is an autoregressive next-token predictor. At every step it scores its entire vocabulary, sampling picks one token, and the loop repeats, which is why generation is sequential and output tokens dominate latency and cost.
- [PagedAttention](https://bigcircle.ai/glossary/pagedattention): The memory management technique from vLLM that stores the attention KV cache in small paged blocks, like operating system virtual memory, so fragmentation no longer caps how many requests fit on a GPU.
- [Parameters & Model Size](https://bigcircle.ai/glossary/parameters-model-size): The weight count, like 8B or 70B, that loosely tracks a model's capability but precisely predicts its memory footprint and serving cost. Multiply parameters by bytes per parameter and you know what hardware a model needs.
- [Plan-and-Execute](https://bigcircle.ai/glossary/plan-and-execute): An agent architecture that drafts the whole plan up front with one frontier-model call, executes the steps cheaply, and replans only when reality disagrees with the plan.
- [Prefill vs Decode](https://bigcircle.ai/glossary/prefill-vs-decode): The two structurally different phases of LLM inference, why one is compute-bound and the other memory-bound, and what that split means for latency metrics, batching, and serving architecture.
- [Pretraining vs Post-Training](https://bigcircle.ai/glossary/pretraining-post-training): Pretraining builds a model's raw knowledge by predicting the next token on web-scale text, and post-training turns that raw model into an instruction-following assistant.
- [Prompt Caching](https://bigcircle.ai/glossary/prompt-caching): Reusing the model's computed work on a repeated prompt prefix to cut input cost by up to 90% and slash time-to-first-token on cache hits.
- [Prompt Engineering & System Prompts](https://bigcircle.ai/glossary/prompt-engineering): Structuring instructions, roles, and context in the prompt so a frozen model reliably produces the output you want, and knowing when prompting alone stops being enough.
- [Prompt Injection](https://bigcircle.ai/glossary/prompt-injection): Untrusted text that hijacks a model's instructions, ranked the number one LLM application security risk by OWASP. It cannot be fully patched in the prompt, only contained by architecture.
- [Quantization](https://bigcircle.ai/glossary/quantization): Storing model weights, and sometimes activations and KV cache, in fewer bits such as 8-bit or 4-bit to shrink memory and speed up inference, trading a small measurable slice of quality for large efficiency gains.
- [ReAct (Reason + Act)](https://bigcircle.ai/glossary/react-pattern): A prompting pattern that interleaves reasoning traces with tool actions so a model plans, acts, and adjusts in the same loop. Modern tool-calling agents are its direct descendants.
- [Realtime Voice Agents](https://bigcircle.ai/glossary/voice-agents): Realtime voice agents hold a spoken conversation, listening, thinking, and speaking with human-like turn-taking under a latency budget of well under one second.
- [Reasoning & Extended Thinking](https://bigcircle.ai/glossary/reasoning-models): Models that spend extra inference compute thinking before they answer, trading latency and token cost for accuracy on hard, multi-step problems.
- [Red-Teaming & Jailbreaks](https://bigcircle.ai/glossary/red-teaming): Red-teaming is adversarially probing a model to find jailbreaks and failures before users or attackers do, turning safety into a measured attack success rate.
- [Reflexion & Self-Correction](https://bigcircle.ai/glossary/reflexion-self-correction): Agents that critique their own output and retry, turning a failed attempt into verbal feedback that improves the next one. Works best when the critique is anchored to external checks like tests and validators.
- [Reranking](https://bigcircle.ai/glossary/reranking): Reranking is a second-stage cross-encoder that reorders the top retrieval results for precision before they reach the model.
- [Retrieval Metrics (Recall@k, MRR, NDCG)](https://bigcircle.ai/glossary/retrieval-metrics): Retrieval is measured by whether the right passage arrived and how high it ranked, which are separate questions. Recall@k answers the first, MRR and NDCG the second, and a system can be strong on one while failing the other.
- [Retrieval-Augmented Generation (RAG)](https://bigcircle.ai/glossary/rag): RAG fetches relevant documents at query time and feeds them into the prompt so the model answers from your facts, not its memory.
- [RLHF](https://bigcircle.ai/glossary/rlhf): Reinforcement learning from human feedback turns a raw base model into a helpful assistant by training a reward model on human preference pairs and optimizing the policy against it with PPO.
- [RLVR (Verifiable Rewards)](https://bigcircle.ai/glossary/rlvr): RLVR trains models with rewards that can be checked by a verifier, such as exact math answers or passing unit tests. It avoids learned reward models but only works where correctness is objectively checkable.
- [RoPE & Positional Encoding](https://bigcircle.ai/glossary/rope): Positional encodings tell attention where tokens sit in a sequence. RoPE is the rotary method behind many long-context models because it represents relative position inside the query and key vectors themselves.
- [Scaling Laws](https://bigcircle.ai/glossary/scaling-laws): Scaling laws describe the predictable relationship between model loss, compute, data, and parameter count. They are how labs estimate whether a bigger run is worth the money before spending it.
- [Speculative Decoding](https://bigcircle.ai/glossary/speculative-decoding): A latency technique where a small draft model proposes several tokens and the large target model verifies them in one parallel pass, typically cutting decode latency 2x to 3x while provably preserving the output distribution.
- [Speech-to-Text (ASR)](https://bigcircle.ai/glossary/speech-to-text): Speech-to-text turns audio into written words, in one batch pass for recordings or streaming low-latency partials for live voice apps.
- [Structured Output & Constrained Decoding](https://bigcircle.ai/glossary/structured-output): Forcing a model to emit schema-conformant JSON every time, so its output can drive code instead of being parsed by hand. Constrained decoding makes invalid tokens unreachable during generation.
- [Synthetic Training Data](https://bigcircle.ai/glossary/synthetic-data): Synthetic training data is model-generated data used to train or tune another model. It is useful when human data is scarce, expensive, private, or not diverse enough.
- [Temperature & Sampling](https://bigcircle.ai/glossary/temperature-sampling): Temperature, top-p, top-k, and min-p control how deterministic or creative an LLM's output is by reshaping or truncating the next-token distribution. Temperature 0 means greedy decoding, though even that is not fully deterministic on real serving stacks.
- [Tensor & Pipeline Parallelism](https://bigcircle.ai/glossary/model-parallelism): Tensor and pipeline parallelism split a model across GPUs when one device cannot hold or run it efficiently. Tensor parallelism slices layers; pipeline parallelism slices the stack of layers.
- [Text-to-Speech (TTS)](https://bigcircle.ai/glossary/text-to-speech): Text-to-speech synthesizes spoken audio from text, with neural voices that are hard to tell from recordings and stream out fast enough for live voice agents.
- [Tokens & Tokenization](https://bigcircle.ai/glossary/tokenization): Models read, generate, and bill in tokens, which are subword chunks learned by byte pair encoding rather than characters or whole words. Token counts drive cost, latency, and context limits, and the same text counts differently across models and languages.
- [Tool / Function Calling](https://bigcircle.ai/glossary/tool-calling): How an LLM emits a structured call to a function you defined so your code can run it and feed the result back into the conversation.
- [Transformer](https://bigcircle.ai/glossary/transformer): The neural-network architecture, built on attention, behind virtually every modern large language model, and the reason scaling up data and compute keeps working.
- [TTFT vs TBT (Latency Metrics)](https://bigcircle.ai/glossary/latency-metrics): LLM latency splits into time to first token (TTFT, prefill-bound) and time between tokens (TBT, decode-bound), two metrics with different causes that need separate budgets and percentile-level measurement.
- [Vector Search & Vector Databases](https://bigcircle.ai/glossary/vector-search): Vector search finds the stored embeddings nearest to a query using approximate nearest-neighbor indexes like HNSW, trading a sliver of recall for millisecond lookups at scale.
- [Vision-Language Models (VLMs)](https://bigcircle.ai/glossary/vision-language-models): Vision-language models read images and text together by turning pixels into tokens, so one model can answer questions about charts, screenshots, and documents, and even point to where the answer is.

## Optional

- [RSS feed](https://bigcircle.ai/rss.xml): guides and case studies, newest first.
- [Sitemap](https://bigcircle.ai/sitemap.xml): every indexable URL.