Skip to content
Finitizer
Expert help

What will it cost to run your AI model?

Find cloud GPU configurations that may fit your model, compare their monthly cost, and see what the price includes.

Multi-cloudCalculator60 seconds
New here? See what this tool does and how to use it.

Learn more

GPU Cost Calculator

What this free tool does for you

Find out which cloud GPU machines have enough memory for your model, and what each would cost a month as rented: every GPU in the machine, the disk, and the hours it runs.

How to use it

  1. Pick your Model and Model format. The example is filled in; replace it with yours.
  2. Answer How long will the instance stay running? and pick a Location. Add a Monthly budget if you have one.
  3. Open Adjust workload assumptions to set the longest request and how many are answered at once.
  4. Read the three cards, then Configurations with enough memory and Why this option appears for any row you select.

Good to know

  • Running is billed, busy or not. A machine bills every hour it runs, however little the GPU does. Fewer hours only cost less when the machine is stopped.
  • Whole machines. Some GPUs only come eight to a machine. A model that needs one still pays for all eight, so check the per-GPU rate and the 1-GPU sizes.
  • Memory is not speed. A model that fits can still answer too slowly under load. Benchmark the shortlist with your own prompts.
  • Save or share. Use Copy link to this result to get a link with your numbers filled in.

Start with your model and running schedule. We estimate memory needs and compare the actual configurations you would pay for.

How do you want to start?

The weights as the model was published: the most memory, and the quality the model was tested at.

How long will the instance stay running?Include idle time. Low GPU usage does not automatically mean fewer billed hours.

730 hours per planning month.

Adjust workload assumptionsShow
GPU memory kept freeA margin for memory spikes. 10% is a common setting.
Example. Llama 3.1 8B Instruct, 16-bit, as released, requests up to 8,192 tokens, 2 requests at once, 10% of GPU memory kept free, always on (730 hours), United States, 100 GiB disk, sized for the model. Prices read 30 Oct 2026.
Estimated GPU memory needed
19 GiB
15 GiB for the model, 2.0 GiB for 2 requests in progress, 2.0 GiB for the serving software. One GPU needs at least 21 GiB with 10% kept free.
Lowest estimated monthly cost among suitable listed options
$525.99/mo
g2-standard-4, Google Cloud, Iowa (us-central1): 1 x NVIDIA L4, 22 GiB each. Compute $515.99 plus disk $10.00.
Selected configuration
g2-standard-4
Google Cloud, Iowa (us-central1). 1 x NVIDIA L4, 22 GiB each. $525.99/mo. The lowest listed; pick another in the table to compare.

Configurations with enough memory

Lowest estimated cost among the listed configurations that meet the selected memory estimate comes first. Each price is for the whole machine you would rent.

Memory fit does not guarantee response time or throughput. Benchmark shortlisted configurations with your workload.

Provider and regionConfigurationMemory fitMonthly costSelect
Google CloudIowa (us-central1)g2-standard-41 x NVIDIA L4, 22 GiB eachFits on one GPU, using 95% of the usable memory
$525.99/mo$0.71 an hour for the machine
AWSUS East (N. Virginia)g6.xlarge1 x NVIDIA L4, 22 GiB eachFits on one GPU, using 95% of the usable memory
$595.50/mo$0.80 an hour for the machine
AWSUS East (N. Virginia)g5.xlarge1 x NVIDIA A10G, 22 GiB eachFits on one GPU, using 95% of the usable memory
$742.38/mo$1.01 an hour for the machine
AWSUS East (N. Virginia)g6e.xlarge1 x NVIDIA L40S, 44 GiB eachFits on one GPU, using 47% of the usable memory
$1,366.53/mo$1.86 an hour for the machine
Google CloudIowa (us-central1)g2-standard-242 x NVIDIA L4, 22 GiB eachThis workload needs 1 GPU. This offer rents 2 GPUs together, and all 2 are included in the price shown.Fits on one GPU, using 95% of the usable memory
$1,470.61/mo$2.00 an hour for the machine, $1.00 per GPU-hour
AzureEast US 2Standard_NV36ads_A10_v51 x NVIDIA A10, 22 GiB eachFits on one GPU, using 95% of the usable memory
$2,353.92/mo$3.20 an hour for the machine
Google CloudIowa (us-central1)a2-highgpu-1g1 x NVIDIA A100 40GB, 40 GiB eachFits on one GPU, using 52% of the usable memory
$2,691.57/mo$3.67 an hour for the machine
AzureEast US 2Standard_NC24ads_A100_v41 x NVIDIA A100 80GB, 80 GiB eachFits on one GPU, using 26% of the usable memory
$2,699.21/mo$3.67 an hour for the machine
Why this option appears: g2-standard-4Show
Memory against the estimate
Needs 19 GiB on one GPU: 15 GiB for the model, 2.0 GiB for the requests in progress, 2.0 GiB for the serving software. Each NVIDIA L4 has 22 GiB; with 10% kept free, 20 GiB is usable. Sold as 24 GB; AWS documents 22 GiB usable.
GPUs rented
1 x NVIDIA L4, 22 GiB each. All of them are used.
Hours priced
730 hours at $0.71 an hour: $515.99/mo for compute.
Included in the price
The GPUs, 4 vCPUs and 16 GiB of memory. 100 GiB Balanced persistent disk: $10.00/mo, billed every hour whether the machine runs or not.
Not included, or not known
Data sent out to the internet, load balancers, snapshots, licensed operating systems, support plans and tax. Whether the region has capacity when you ask, and whether your account has GPU quota there: new accounts often have none until they request it.
Checked
Price read 30 Oct 2026 from Google Cloud's published list prices; GPU and machine details read 29 Oct 2026. Model figures read 29 Oct 2026 from meta-llama/Llama-3.1-8B-Instruct.

Explore a lower-cost scenario

Each one changes a single assumption and keeps the rest. They are separate: the savings do not add up.

  • Running schedule: Always on (730 hours) to Workdays (176 hours)
    Selected configuration, everything else as it is.
    Now
    $525.99/mo
    g2-standard-4, Google Cloud Iowa (us-central1)
    With this change
    $134.40/mo
    g2-standard-4, Google Cloud Iowa (us-central1)
    $391.59 a month less

    The model is unavailable while the machine is stopped, and each start reloads it. Not for a service that must answer at any hour.

Spot prices are not included. Spot machines cost less but can be taken back with 2 minutes’ notice on AWS and 30 seconds on Google Cloud and Azure: only for work that can stop and restart.

Before you choose: what to test

  • Run your own prompts on the shortlisted configurations, at the number of requests you expect at once, and time the answers.
  • Check your account has GPU quota in the region, and that the machine is available when you ask for it.

What this model assumes

Show
  • Memory, not speed. The estimate adds the model’s weights, the working memory (the KV cache) for the requests in progress at full length, and 2 GiB on each GPU for the serving software. It says nothing about response time or throughput, and it does not estimate tokens per second.
  • Several GPUs are not one big GPU. A model uses several GPUs only when the serving software splits each layer across GPUs in the same machine (tensor parallelism), and only when the GPU count divides the model’s attention heads. Each GPU then needs its share plus its own overhead.
  • Formats. 8-bit means FP8; 4-bit means AWQ or GPTQ with groups of 128, about 4.16 bits a weight. Both keep the input embedding and output layer at 16-bit, as published versions do. The format you pick is never changed for you.
  • Whole machines, list prices. On-demand Linux prices with no discounts, for the machine as rented: all its GPUs, vCPUs and memory. Committed use, savings plans, reservations and Spot are left out; Google’s automatic sustained-use discount does not apply to these machines. A month is 730 hours; workdays are 176.
  • Stopped means stopped. A shorter schedule only lowers the bill if the machine is stopped the rest of the time. Low GPU usage on a running machine is billed in full. The persistent disk bills every hour, and local NVMe or temporary disks are erased when the machine stops.
  • Usable GPU memory. L4, A10G and A10 count as 22 GiB and L40S as 44 GiB, the usable figures AWS documents for cards sold as 24 GB and 48 GB, the same on every cloud.
  • Not covered: training and fine-tuning, mixture-of-experts models, data transfer, load balancers, snapshots, support plans, tax, quota and capacity. Google offers A2 and A3 machines in Europe but publishes their prices for Iowa only, so Europe on Google is L4 here; Google’s H200 machines (A3 Ultra) are not offered on demand, and are left out.
Rates used

Prices verified 30 Oct 2026. Public list prices in USD for 6 regions on AWS, Google Cloud and Azure, before any discount you have negotiated.

Already running models on cloud GPUs?

Cloud Cost Audit: We review your GPU machines, schedules and disks against what your models need. From $1,500, fixed. Nothing paid upfront.

Talk to Finitizer

Questions

How much GPU memory do I need to run an LLM?

At 16-bit, 2 bytes per parameter for the model, plus memory for the requests being answered and about 2 GiB for the serving software. Llama 3.1 8B needs 14.96 GiB for the model and 2 GiB for two 8,192-token requests at once: about 19 GiB, which fits one NVIDIA L4 with 10% kept free. Llama 3.3 70B at 16-bit needs about 138 GiB, more than any single listed GPU holds with 10% kept free (an H200 leaves 127 GiB), so it is split across two 80 GB GPUs in one machine.

What is the lowest-cost cloud GPU for an 8B model?

Among the configurations listed here in the United States, Google Cloud g2-standard-4, with one NVIDIA L4, has the lowest estimate for Llama 3.1 8B at 16-bit: $515.99 a month for 730 hours of compute plus $10.00 for a 100 GiB disk, $525.99 in all. AWS g6.xlarge, also one L4, comes to $595.50. These are on-demand list prices; benchmark your own prompts before you choose.

Why does the price include 8 GPUs when I need one?

Some GPUs are only rented as whole machines. Google Cloud a3-highgpu-8g and Azure ND96isr_H100_v5 come on demand only with eight H100s, and AWS p5.48xlarge has eight at $55.04 an hour, $6.88 per GPU-hour. A workload that needs one GPU on such a machine still pays for all eight. AWS also rents p5.4xlarge, with one H100, at $6.88 an hour.

Does running the GPU only on workdays lower the bill?

Only if the machine is stopped the rest of the time. A running machine bills every hour, however little the GPU does. Stopped outside 176 workday hours, the one-L4 example falls from $525.99 to $134.40 a month: compute drops by $391.59 and the disk keeps billing $10.00. On Azure, stop the machine from the portal or command line; shutting it down from inside keeps billing.

Can I run a model on two smaller GPUs instead of one larger one?

Sometimes. GPU memory does not pool: the serving software splits each layer across GPUs in the same machine, and only by a number that divides the model's attention heads. Phi-4 splits two ways but not four or eight. Each GPU then holds its share plus its own overhead, and the GPUs talk to each other on every request, so benchmark the split before you rely on it.

Related tools

LLM Token Cost Calculator

Compare monthly AI spend on Claude, GPT, Gemini and more before the invoice arrives.

Open

LLM Provisioned Throughput and Service Tier Planner

PTUs, GSUs or pay as you go: when reserved LLM capacity pays, and what Flex or Batch saves.

Open

AI Carbon Calculator

The carbon of AI prompts and GPUs, from the figures the makers have published.

Open

More tools for: AI and model usage, Compute and utilization

An estimate is a shortlist. Your bill shows what you run.

Finitizer reads your AWS and Google Cloud billing data every day, shows which GPU machines and disks drive the bill, and which ones run idle.

See how Finitizer works

Rate this tool

How would you rate the GPU Cost Calculator?