RLVR (Verifiable Rewards)

RLVR means reinforcement learning with verifiable rewards. Instead of asking humans or a reward model which answer is better, the trainer checks whether the answer is correct. Did the unit tests pass? Does the final number match? Is the proof step valid under a verifier? If yes, reward. If no, no reward.

Verifier vs reward model#

A learned reward model can be fooled because it is another model. A verifier is narrower but harder to argue with. In code, the tests pass or fail. In math, an exact final answer can often be checked. In logic puzzles, constraints can be evaluated. That makes RLVR attractive for training reasoning models.

Why math and code generalize#

Math and code are useful because they force multi-step search. A model cannot pass hard tests by memorizing a style preference. It must explore candidate reasoning paths and land on a correct result. Training on these domains can improve general reasoning behaviors, even though the verifier itself is domain-specific.

Limits#

Verifiable does not mean unhackable. Code models can overfit public tests. Math models can learn formatting shortcuts. Sparse rewards make exploration hard: if every sample is wrong, the model gets little signal. The verifier gap is also real. Many important tasks, like legal analysis or product strategy, do not have a cheap exact checker.

Practical takeaways#

Use RLVR where the environment can grade outputs. It pairs naturally with GRPO, code execution, theorem checking, and synthetic task generation. Do not use it as a blanket alignment story. The reward is only as good as the verifier, and the model will learn the verifier's blind spots.