AI Wiki · Mediocre Security
Concept

Distillation

Training a smaller "student" model to mimic a larger "teacher" model's outputs; also a form of model theft when done against a competitor's API.

Distillation trains a smaller, cheaper student model to reproduce a larger teacher model’s outputs, keeping most of the capability at a fraction of the cost. Labs distill their own models routinely; Nanbeige designs its 33B model specifically as a teacher for smaller agents.

Done against someone else’s API, it becomes adversarial distillation, a terms-of-service violation and a form of IP theft. OpenAI accused Moonshot-linked users of a 16,000-request campaign to extract its models’ hidden reasoning, and Anthropic reports a 90-99% drop in distillation harvesting after countermeasures. Microsoft AI advertises “no distillation” in MAI-Base-2’s training data. OpenCode described its credential theft as a “distillation attack”.

Industrial use in China

A Reuters review of 300+ Chinese papers, patents, filings, job ads and government documents (Nov 17 2026) found distillation of US model outputs has become mainstream industrial practice. Shanghai Jiao Tong researchers used about 5,000 OpenAI o1 answers, translated with GPT-4o mini, to further train Alibaba’s Qwen2.5-72B; Alibaba researchers used GPT-4 and GPT-4o to score training data; and a Feb 2026 paper with Meituan used GPT-4o for agent training data. A US official told Reuters Washington sees “industrial-scale distillation” as the main reason Chinese models are closing the gap.

Timeline

Every dated event in the wiki that involves Distillation, newest first.

Connections

Distillation and everything one link away. Drag to move, scroll to zoom, click a node to open it.Open in the full graph →

What links here

AI lab OpenAI

Product OpenCode

Sources

  1. Wikipedia, “Knowledge distillation”
  2. Reuters, “China taps US AI models to train homegrown rivals as chip curbs bite”, Nov 17 2026 (via Wccftech)