Once an enterprise can see what it is spending on tokens, the next instinct is usually the wrong one: hand the invoice to engineering with instructions to bring the number down. That produces a scramble of point fixes, applied under pressure, to systems that were never designed with cost as a consideration in the first place. The enterprises that actually get token spend under control treat optimisation the way they treat performance or security: as an architecture discipline, owned at design time, not a retrofit applied after someone in finance asks an uncomfortable question.
The Default Model Is Rarely the Right One
The single largest, easiest lever most enterprises have not pulled is model selection. The default pattern in most AI implementations is to route every call through the most capable model available, because it was the model the team was already using for the hardest problem in the application, and nobody revisited the decision for the easier ones. A customer service triage classification and a complex multi-step reasoning task do not need the same model, and the cost difference between them is not marginal — it is frequently an order of magnitude.
Right-sizing model selection to task complexity requires two things enterprises usually lack: a clear-eyed assessment of which tasks in an application actually need frontier-model reasoning versus which are well within the capability of a smaller, cheaper model, and the engineering discipline to route calls accordingly rather than defaulting to one model across an entire application for simplicity. The organisations doing this well are not making a one-time decision. They are building routing logic that treats model selection as a configurable, monitored parameter, revisited as task requirements and model capabilities both change.
Prompt and Context Engineering as a Financial Decision
The second lever is less visible and more habitual: how much context an application sends with every call. It is easy, when building an AI feature under deadline pressure, to include more context than the task strictly requires, because more context reduces the risk of the model missing something, and the cost of that extra context is invisible in the moment it is written. That habit compounds. An application that sends twice the necessary context on every call pays twice the token cost on every call, indefinitely, until someone deliberately goes back and re-engineers the prompt.
Treating prompt and context design as a cost decision, not just a quality decision, means asking at build time what context a task actually requires, not what context is convenient to include. It means designing prompts to be reviewed periodically the way SQL queries get reviewed for performance, rather than written once and left alone as long as the output looks correct. Quality and cost are not opposed here — a tighter, more deliberately constructed prompt is frequently both cheaper and more reliable than a bloated one, because the model is not being asked to find the signal inside noise it did not need.
Caching and Retrieval as Cost Architecture
The third lever is architectural rather than habitual: avoiding paying for the same reasoning twice. Caching frequently repeated queries, reusing embeddings instead of regenerating them, and designing retrieval systems that fetch precisely the context a task needs rather than everything that might be relevant, are all standard performance engineering techniques that have an equally significant cost dimension that gets less attention. An application that recomputes the same expensive call for near-identical inputs, because nobody built a caching layer, is paying repeatedly for work it has already paid for once.
This is where token optimisation most resembles conventional systems architecture rather than a new discipline. The techniques are familiar. What is new is treating token cost as a first-class reason to apply them, alongside latency and infrastructure cost, rather than an afterthought that only gets attention once the bill is large enough to notice.
Optimisation Owned at Design Time
The pattern across all three levers is the same: they are cheap and natural to build in at the point a feature is designed, and expensive and disruptive to retrofit once a feature is live and generating an invoice nobody wants to explain. A team that considers model selection, context design, and caching architecture during the design review for a new AI feature makes better decisions than a team optimising under pressure after the fact, because the design-time team is choosing between architectures, while the retrofit team is constrained by an implementation already in production.
This is why token optimisation belongs in the same review process as any other architecture decision, with the same seniority of ownership. It is not a task to hand to whichever engineer has spare capacity once the feature ships. It is a design constraint, alongside latency budgets and security requirements, that shapes how the feature gets built in the first place.
The Discipline That Compounds
None of these levers are individually difficult. What makes them consequential is that they compound across every AI feature an enterprise ships, and the gap between an organisation that builds this discipline in and one that does not widens with every feature added, not narrows. The technology leaders who make token optimisation a standing part of architecture review now are building a cost structure that scales sublinearly with AI adoption. The ones who wait are building a cost structure that scales with it directly, and will eventually have to unwind years of unexamined defaults under exactly the pressure this discipline was meant to avoid.
