Scaling Out
The default answer to "I need more from my AI" is still a bigger model on a bigger GPU. But every piece of the other answer, horizontal scaling for inference, already exists in the research. Here's how small, specialized, cache-friendly workers make AI cheap instead of scarce.
When a system runs out of headroom, you've got two options: buy a bigger box, or run more boxes. We settled that question for the whole internet a long time ago; the web runs on adding cheap…