For most enterprises, 2026 is the year AI spending stopped being an experiment line item and became a material operating cost. Agents run multi-step tasks, reasoning models think at length before answering, and AI features are embedded in dozens of SaaS products. Unit prices for many models continue to fall, yet total bills keep climbing. The FinOps community has responded: the FinOps Foundation has extended its framework to cover AI, and in June 2026 the Linux Foundation announced a new Tokenomics Foundation at FinOps X to work on token economics questions alongside it.
This playbook translates those ideas into concrete steps for the teams that run enterprise AI platforms: how to meter consumption, connect it to value, allocate costs fairly, and apply engineering levers without degrading quality.
Why AI Costs Behave Differently
Cloud FinOps taught organizations to manage elastic infrastructure. AI adds new dynamics that make familiar techniques insufficient on their own. In a May 2026 FinOps Foundation article on token economics, J.R. Storment describes several of them:
- Input and output tokens are priced separately, often at different rates. Input includes prompts, retrieved context and conversation history; output includes responses, tool calls and reasoning.
- Consumption is non-linear. System prompt overhead, context and memory, model choice, output length and retry or orchestration overhead all multiply usage. The article notes that agentic and reasoning workloads can increase token use by one to two orders of magnitude.
- Falling unit prices do not mean falling spend. Cheaper tokens encourage more modalities, more agent autonomy and longer reasoning chains, so aggregate spend can rise.
- The subsidy phase is ending. The article argues that providers are moving away from below-cost pricing, with continued per-token declines mainly in commodity tiers and new commercial models that combine seats with committed consumption.
- Tokens are only part of the stack. Compute, storage, networking, SaaS-embedded AI, engineering and governance, data licensing, shadow AI and training also contribute.
The implication is that AI cost management has to work at two levels at once: the token level, where engineering decisions drive consumption, and the portfolio level, where business value justifies spend.
The FinOps for AI Framework in Brief
The FinOps Foundation positions FinOps for AI as an extension of its existing framework rather than a separate discipline. In the token economics article, it maps AI practices onto familiar domains:
- Understand usage and cost: add token-level metering by model, application, team and use case.
- Quantify business value: introduce AI-specific unit metrics such as cost per inference, token yield rate and cost relative to outcome.
- Optimize usage and cost: apply semantic caching, model tiering, prompt optimization and goodput-aware routing.
It also recommends normalizing cost and usage data with FOCUS, the open FinOps Open Cost and Usage Specification, so that model and provider costs can be compared and reconciled against underlying infrastructure.
Step 1: Meter Everything at the Platform Layer
You cannot manage what you cannot attribute. The most effective place to meter AI usage is a shared platform layer that all applications pass through, typically an AI gateway or model proxy. For every model call, capture:
- Application, team, environment and use case identifiers.
- Model, provider, region and pricing tier.
- Input, cached input, output and reasoning token counts.
- Latency, errors and retries.
- For agents, the parent task or session identifier so that multi-step costs roll up to one business transaction.
Enforce tagging at the gateway: requests without valid metadata are rejected or routed to a default cost center that someone must answer for. Export data in a FOCUS-aligned format to your FinOps tooling, and reconcile monthly against provider invoices.
Do not forget the hidden layers
Self-hosted models carry GPU, storage and networking costs that must be allocated across tenants. SaaS products with embedded AI often hide token consumption behind subscriptions; the FinOps Foundation article specifically flags AI-native developer tools as token aggregators. Include these in your inventory even when you cannot meter them at token level, and run discovery for shadow AI usage on corporate cards and expense reports.
Step 2: Define Unit Economics Tied to Outcomes
Token volume alone is a poor management metric. A support agent that uses more tokens but resolves twice as many tickets is a better investment. Define a unit of value for each significant use case and track cost against it:
- Customer support: cost per resolved conversation, alongside resolution rate and satisfaction.
- Document processing: cost per document correctly extracted, alongside straight-through processing rate.
- Software engineering: AI cost per merged pull request or per story point, alongside defect rates.
- Sales enablement: cost per qualified proposal generated.
The FinOps Foundation describes a token yield rate, the share of generated tokens that drive a business action, and the concept of goodput, output that meets defined latency and throughput objectives. Both help distinguish productive consumption from waste such as failed retries or responses nobody reads.
Optimize cost per outcome, not cost per token. Cutting tokens that drive value creates false economies; cutting tokens that drive nothing is pure gain.
Step 3: Allocate and Show Back
Once metering and unit metrics are in place, make costs visible to the people who drive them:
- Start with showback. Publish monthly dashboards by team and use case showing spend, unit costs and trends. Visibility alone changes behavior.
- Allocate shared costs transparently. Split platform overhead, such as gateways, observability and reserved capacity, using simple, explainable rules like proportion of tokens or requests.
- Move to chargeback where appropriate. Once numbers are trusted, charge business units for consumption so AI spend competes with other investments on equal terms.
- Set budgets and alerts. Configure per-application budgets and anomaly alerts at the gateway, including hard limits for experimental workloads.
Step 4: Apply the Engineering Levers
Most AI cost reduction comes from engineering choices that reduce tokens per outcome. The FinOps Foundation article lists several levers that platform teams can turn into shared capabilities:
Route to the cheapest capable model
Use model tiering or cascades: send simple requests to small, inexpensive models and escalate only when confidence is low or the task is complex. Maintain evaluation suites so that routing decisions are based on measured quality, not assumptions.
Cache aggressively
Use provider prompt caching for repeated system prompts and context, and semantic caching for repeated questions. The article describes caching and prompt optimization as among the cheapest optimizations available.
Shrink context
Compress conversation history, filter retrieved content to what is relevant, and use hybrid retrieval with long-context escalation only when needed. For agents with many tools, load tool definitions selectively or use code-execution patterns instead of listing every tool in every prompt.
Control output
Set sensible maximum output lengths, request structured formats, and choose efficient data representations. The article notes that compact formats such as CSV can be cheaper than JSON for tabular data passed to models.
Tame agent loops
Agents can spiral into repeated tool calls and retries. Set step limits, timeouts and budget caps per task, and monitor tasks that exceed expected token envelopes.
Choose the right deployment archetype
The FinOps Foundation distinguishes SaaS-embedded, API-consumed and self-hosted deployments, noting that cost crossover points occur at predictable volumes. Model your demand to decide when committed capacity, provisioned throughput or self-hosting becomes cheaper than pay-as-you-go APIs.
Step 5: Govern the Portfolio
Engineering levers cut waste; governance ensures money goes to the right places. Practical controls include:
- ROI gates for new AI initiatives, with expected unit economics and value hypotheses reviewed before production.
- Quarterly portfolio reviews that compare actual unit costs and outcomes with plans, retiring use cases that do not deliver.
- Demand forecasting for tokens and GPU capacity, treated like cloud capacity planning, especially as providers move toward committed-consumption pricing.
- Model and provider policy defining approved models per data classification and use case, enforced at the gateway.
- Shadow AI discovery through expense analysis, network monitoring and SaaS management tools.
Building an AI Cost Dashboard People Actually Use
Dashboards fail when they show everything to everyone. Design separate views for the audiences that make different decisions:
- Executives and finance: total AI spend by business unit and category (API, self-hosted, SaaS-embedded), forecast versus budget, and a handful of unit costs for flagship use cases.
- Product owners: cost per outcome for their features, quality and adoption metrics alongside cost, and the effect of recent changes.
- Engineers: tokens per request by prompt version, cache hit rates, model mix, retries, and the most expensive sessions or agent tasks of the week.
- Platform team: gateway coverage (share of calls with complete tags), reconciliation gaps against invoices, and capacity utilization for reserved or self-hosted resources.
Link cost data to traces wherever possible. Managed agent platforms increasingly emit OpenTelemetry-compatible telemetry; AWS, for example, describes AgentCore Observability as OTEL compatible. When a cost spike appears, an engineer should be able to click through from the dashboard to the specific sessions and steps that caused it.
Who Does What
AI FinOps is a team sport. A workable division of responsibilities looks like this:
- Platform team: owns the gateway, metering, tagging enforcement, shared optimization features such as caching and routing, and data feeds to FinOps tools.
- FinOps practitioners: own reporting, allocation rules, forecasting, anomaly management and commitment purchasing.
- Product and engineering teams: own unit economics for their use cases, prompt and context efficiency, and evaluation suites.
- Finance and leadership: own budgets, ROI gates and portfolio decisions.
- Security and risk: own model and provider policies and shadow AI discovery in partnership with procurement.
Hypothetical Example: A Software Company Tames Its AI Bill
Imagine a hypothetical mid-sized software company whose monthly AI spend has tripled in two quarters. Usage is spread across a customer support agent, an internal coding assistant, a document summarization feature in its product and various team experiments.
The platform team routes all model calls through a gateway with mandatory tags and builds a FOCUS-aligned dashboard. Within a month it finds that the support agent consumes the largest share, mostly because every conversation includes the full product manual as context. Introducing retrieval with filtered chunks and prompt caching sharply reduces input tokens per conversation without affecting resolution rate in evaluation. The summarization feature is moved from a frontier model to a smaller model after evaluations show equivalent quality for that task. Experiments receive monthly budgets with hard caps. Finance receives showback by product line and introduces an ROI gate for new AI features. In this illustration, the company does not cut AI usage; it raises value per token and makes spend predictable.
Pitfalls to Avoid
- Measuring only provider invoices. Invoices arrive late and lack attribution. Meter at the platform layer.
- Optimizing without evaluation. Switching to cheaper models without quality tests can quietly damage outcomes.
- Ignoring reasoning tokens. Reasoning models may generate substantial hidden tokens. Track them explicitly.
- Treating AI cost as purely technical. Product and business owners decide what features exist and how they are used; involve them early.
- Waiting for perfect data. Start with showback on what you can measure today and improve coverage over time.
Outlook: Token Economics Through 2030
Several trends are likely to shape AI FinOps over the next few years. Standards work through FOCUS and the new Tokenomics Foundation should make cross-provider cost comparison easier. Pricing models will continue to diversify, with seats, commitments and outcome-based pricing alongside per-token rates. Agent platforms will add native budget controls and cost-aware routing. And as AI becomes embedded in most enterprise software, the distinction between AI cost and software cost will blur, making value-based governance more important than token counting alone.
A 90-Day Starting Plan
- Days 1–30: inventory all AI spend sources, including SaaS and shadow usage; route production model calls through a gateway; enforce basic tags; publish a first showback report, even if incomplete.
- Days 31–60: define unit metrics for the top five use cases; build evaluation suites for them; introduce caching and context filtering where evaluations show no quality loss; set budgets and alerts for experimental workloads.
- Days 61–90: implement model tiering for suitable use cases; reconcile metered data with invoices in a FOCUS-aligned format; launch an ROI gate for new AI initiatives; agree the first quarterly portfolio review.
By the end of the quarter, leadership should be able to see what AI costs, who drives it, and what value each major use case returns, which is the foundation for every later decision about scale.
Conclusion
AI platforms need FinOps that understands tokens, agents and value. Meter every call at the platform layer, define unit economics tied to business outcomes, make costs visible to the teams that drive them, and turn engineering levers such as routing, caching and context control into shared platform capabilities. Govern the portfolio so that investment flows to use cases that pay back. Done well, FinOps for AI does not slow adoption; it gives leadership the confidence to scale it.
How Hanoi Technology Can Help
Hanoi Technology helps platform teams implement AI cost metering, showback and optimization levers as part of well-governed AI platforms, through projects or managed services.