AI Wiki · Mediocre Security
Concept

RLHF

Reinforcement Learning from Human Feedback: humans rank model outputs and the model is trained to produce more of the highly ranked ones.

Reinforcement Learning from Human Feedback is the alignment technique that turned raw LLMs into usable assistants: humans compare model outputs, a reward model learns their preferences, and the LLM is optimized against that reward. It made ChatGPT possible. Its weakness is that models optimize the reward, not the intent, which is the root of reward hacking and sycophancy. Variants include RL from AI feedback and General Reasoning’s “RL from Collective Rewards”.

Connections

RLHF and everything one link away. Drag to move, scroll to zoom, click a node to open it.Open in the full graph →

What links here

Concept Alignment · Large language model (LLM)

Model family GPT

Sources

  1. Wikipedia, “Reinforcement learning from human feedback”