Distillation
Training a smaller "student" model to mimic a larger "teacher" model's outputs; also a form of model theft when done against a competitor's API.
Distillation trains a smaller, cheaper student model to reproduce a larger teacher model’s outputs, keeping most of the capability at a fraction of the cost. Labs distill their own models routinely; Nanbeige designs its 33B model specifically as a teacher for smaller agents.
Done against someone else’s API, it becomes adversarial distillation, a terms-of-service violation and a form of IP theft. OpenAI accused Moonshot-linked users of a 16,000-request campaign to extract its models’ hidden reasoning, and Anthropic reports a 90-99% drop in distillation harvesting after countermeasures. Microsoft AI advertises “no distillation” in MAI-Base-2’s training data. OpenCode described its credential theft as a “distillation attack”.
Industrial use in China
A Reuters review of 300+ Chinese papers, patents, filings, job ads and government documents (Nov 17 2026) found distillation of US model outputs has become mainstream industrial practice. Shanghai Jiao Tong researchers used about 5,000 OpenAI o1 answers, translated with GPT-4o mini, to further train Alibaba’s Qwen2.5-72B; Alibaba researchers used GPT-4 and GPT-4o to score training data; and a Feb 2026 paper with Meituan used GPT-4o for agent training data. A US official told Reuters Washington sees “industrial-scale distillation” as the main reason Chinese models are closing the gap.
Timeline
Every dated event in the wiki that involves Distillation, newest first.
Reuters reviewed 300+ Chinese papers, patents, filings, job ads and government documents and found distillation of US model outputs has moved into mainstream industrial use. Examples: Shanghai Jiao Tong researchers used ~5,000 o1 answers, translated with GPT-4o mini, to further train Alibaba's Qwen2.5-72B (Nov 2024); Alibaba researchers used GPT-4 and GPT-4o to score training data; a Feb 2026 paper with Meituan used GPT-4o. Distillation is not illegal, but OpenAI and Anthropic terms forbid it; a US official said Washington sees it as the main driver of China narrowing the gap.
Connections
Distillation and everything one link away. Drag to move, scroll to zoom, click a node to open it.Open in the full graph →
What links here
AI lab OpenAI
Product OpenCode
Sources
- Wikipedia, “Knowledge distillation”
- Reuters, “China taps US AI models to train homegrown rivals as chip curbs bite”, Nov 17 2026 (via Wccftech)