Concept
Inference
Running a trained model to produce an output, as opposed to training it; the cost center that inference chips, gateways and decision models compete on.
Inference is running a trained model to get outputs. As usage exploded, inference became the dominant cost of AI and the target of a whole industry: dedicated chips (Etched, Fractile, Taalas, Maia 300), serving frameworks (Nvidia Dynamo, DeepSeek’s DeepInfer), optimization startups (Simplismart), gateways (LiteLLM, Respan) and fast, cheap decision models. See also MoE, which cuts inference compute.
Connections
Inference and everything one link away. Drag to move, scroll to zoom, click a node to open it.Open in the full graph →
What links here
Concept Mixture of experts (MoE)
Sources
- Wikipedia, “Inference engine”