Reflexion & Self-Correction

Reflexion is an agent technique in which the model, after a failed attempt, writes a plain-language note on what went wrong, keeps that note in agent memory, and tries again with the note in its prompt. Nothing is retrained. Noah Shinn and colleagues introduced it in March 2023 and called it verbal reinforcement learning, because the feedback is a sentence rather than a change to the model's weights. Self-correction is the wider family: generate an answer, check it, turn the failure into specific feedback, revise.

The part most explanations skip is the check. A retry loop improves results when something outside the model decides the attempt failed: a test suite, a schema validator, the actual state of the system the agent was meant to change. When the model grades itself, the same loop can make answers worse and cost several times as much.

Attempt, check, critique, retry#

The loop has four moves. The agent produces a candidate, say a function that should pass a test suite. The candidate is checked, by running the tests, compiling the code or validating the JSON. The failure becomes a written diagnosis, for example "attempt 1 failed test_empty_input because the function indexes into an empty list; guard the empty case". That diagnosis goes into the prompt for the next attempt, so the model is answering a description of its own last error rather than guessing again. A pass ends the loop, and a retry cap ends it when the output will not pass.

Reviewing a finished answer is easier than writing a perfect one in one go, and each retry is aimed at a named failure, so attempts get better rather than just different. That is where Reflexion's headline numbers come from. It reached 91% pass@1 on the HumanEval coding benchmark, where GPT-4 alone scored 80%, and on the ALFWorld household-task benchmark a Reflexion agent built on ReAct completed 130 of 134 tasks over 12 rounds of trying, 22 points better than ReAct alone. Self-Refine, where one model drafts, critiques and revises, averaged about 20 points of absolute improvement across seven tasks. Every one of those results had either an outside signal, unit tests or an environment reporting that the task failed, or a task where critique is plainly easier than writing.

When the model grades itself#

Huang and colleagues showed at ICLR 2024 that intrinsic self-correction, where the model reviews its own reasoning with no outside feedback, struggles and at times lowers accuracy. Asked to review its answer, a model will change some correct answers to wrong ones, because the review comes from the same habits that produced the mistake. A model that confused two API versions in the attempt will confuse them in the review. The CRITIC paper found the same from the other side. With search or a code interpreter supplying the feedback, revisions improved results. With the tools removed, the critiques added little on question answering and were unstable on maths.

Two failures follow in production. The first is a false pass: the model declares a wrong answer correct and leaves the loop early, which is worse than having no loop, because the retry budget bought confidence the answer did not earn. The second is churn: with no reliable stop signal, the model keeps rewriting output that was already fine and burns tokens doing it. One attempt plus five retries is up to six model calls, and more than six times the tokens once each retry carries the previous critiques. Self-grading also inherits the known biases of an LLM judge, with the added problem that the judge wrote the work.

Check against something the model cannot argue with#

In the three builds below, Bigcircle put the check outside the model, because an agent that grades itself reports success on work that never happened. On the executive operations desktop agent, a critic decides after each step whether to continue, retry or escalate, and the build compares what the agent claims it finished against the real state of the target application. That comparison catches an agent reporting success on work that did not land. Work it cannot complete comes back as a flagged exception rather than a confident, false "done".

On the DevOps agent that runs on live AWS, the check is a validation loop of structural and semantic checks against the live account, and the validation errors are fed back until the template is clean. Retries sit in two places there, inside that loop and around the provider call, because they recover from different things: a malformed template in one case, a transient provider failure in the other. The logged validation failures are the system's own record of what the model gets wrong, and that record improves the loop more than a prompt edit would.

Code is the easy case. Run the tests, the compiler or the type checker, and paste the literal error output in as the critique, because a failing traceback is a fact and "this code looks correct" is an opinion. Elsewhere the same rule holds: validate structured output against its schema, check extracted numbers against the source document, run the SQL against a sandbox, confirm a cited link resolves.

Bound the retry, and say what it may use#

The retries in those builds are bounded, because an agent that retries indefinitely against a broken vendor is worse than one that stops: it burns cost and time while looking busy. After the bound, the work escalates with its trace attached, so a person can see whether the failure was the plan or the provider. On the DevOps agent, a template that will not come clean after bounded revision goes to a person as a request the agent cannot safely fulfil. Our page on loop engineering applies the same cap to every loop around an agent, not only the retry.

The retry also needs limits on what it may draw on. On the trademark drafting platform, a draft that fails citation validation is retried using only retrieved evidence. A free regeneration would bring back the failure being corrected, an invented citation, from the model's memory.

For a coding agent, a reasonable starting cap is three to five attempts, with the literal error and a one-line reflection added before each retry. Record pass rate by attempt number in your evals so you can see what each retry buys. If nearly all the gain arrives on the first retry, lower the cap. If attempt three scores below attempt one, the critique has lost its outside anchor.