Cost per Inference
Definition
Measures the cost incurred for a single inference (i.e., when an AI model processes an input and generates an output). Useful for applications like chatbots, recommendation engines, or image recognition systems. Used for tracking the operational efficiency of deployed AI models, especially for high-volume applications. Helps optimize resource allocation and identify cost spikes due to inefficient code or infrastructure.
Formula, as published
Cost Per Inference = Total Inference Costs / Number of Inference Requests
- Data sources:
- Cloud billing data Logs from AI platforms (e.g., OpenAI, Vertex AI).
- Example:
- If the total inference cost is $5,000 and the system processes 100,000 inference requests, the cost per inference is:$5,000/100,000 = $0.05 per request.
Calculate it
Example values to start with; replace them with yours.
How to measure it
AI spend per request. Divide the whole inference cost (tokens, plus any provisioned throughput and hosting), not tokens alone, or a self-hosted model looks free.
Drop in your OpenAI or Claude usage export and see where the AI spend goes.
From the FinOps Foundation
More Reporting & Analytics KPIs
Definition and formula: Cost per Inference, FinOps KPI library, FinOps Foundation, licensed CC BY 4.0, read 16 Oct 2026. Changed: HTML removed, dashes and British spellings changed, formula split from its data sources and examples. The calculator and the notes are Finitizer’s.
Measured once, a KPI is a number. Tracked daily, it is a practice.
Finitizer computes the KPIs your billing data supports every day, by team and product, across AWS and Google Cloud.
See how Finitizer works