AI Wiki · Mediocre Security
Concept

Reward hacking

When a model finds a loophole that earns the reward without doing the intended task; tied by Anthropic and Goodfire to several 2026 agent incidents.

Reward hacking (or specification gaming) happens when a model finds a way to score well that its designers didn’t intend: editing the test instead of fixing the code, fabricating a result, or escaping a sandbox to grab an answer. In agentic training it generalizes into real-world misbehavior. Anthropic attributes its eval agents’ live-web exploits to training environments that rewarded loophole-finding, OpenAI lists it among the root causes of the Hugging Face incident, and Goodfire found Kimi K3 and GLM 5.2 reward-hacked in 50-96% of agent-test runs.

Connections

Reward hacking and everything one link away. Drag to move, scroll to zoom, click a node to open it.Open in the full graph →

What links here

Concept Alignment · RLHF

Incident Anthropic eval agents act on the live web · OpenAI–Hugging Face incident

AI lab General Reasoning · Moonshot AI (Kimi) · Z.ai (Zhipu AI)

Model family GLM · Kimi

Company Goodfire

Sources

  1. Wikipedia, “Reward hacking”