Vertex AI Generative Model Tokens
Token charges for hosted generative models on Vertex AI, billed separately for input and output. Output tokens cost substantially more than input tokens, and agentic workloads invert the usual intuition by generating enormous input volume as they re-read their own context on every step.
Billed: Per 1,000 or 1,000,000 tokens, priced separately for input and output. Output is typically several times the input rate.
What triggers this charge
- Every prompt and completion, counted in tokens rather than characters or requests.
- Long context windows, where the entire prompt is re-billed on each call in a multi-step conversation.
- Agentic loops, which consume many times the tokens of a single chat exchange for the same task.
- Retries and failed generations, which are still billed.
How to reduce it
Turn on context caching
Cached input tokens are billed at a substantially lower rate. For agents and RAG systems that resend the same system prompt and documents, this is the largest single lever.
Route by difficulty
Most requests do not need the largest model. Sending the easy majority to a smaller one and escalating only when needed changes the blended rate dramatically.
Use batch mode for anything not interactive
Batch prediction is billed well below interactive rates, and most enrichment and classification jobs do not need a synchronous answer.
Cap output length
Output tokens are the expensive half. Setting a sensible max output length bounds the worst case.
Rates
Now find out how much of this you are actually paying.
Finitizer breaks your Google Cloud bill down to the SKU, shows how each one has moved since last week, and separates rate changes from usage changes — so you know whether to talk to engineering or to procurement.