A filter that misses the first cluster column saves far less
Clustering prunes from its first column inward, and not through a function.
Filter on the first column, and compare it directly.
Check one of your own queries for this
Paste a query and the calculator estimates what it scans and flags the patterns it can detect statically. It runs in your browser.
What it is
A clustered table keeps its data sorted by the cluster columns, in the order they were declared. BigQuery skips blocks only when a filter compares those columns directly, starting with the first.
Why it costs money
- Measured by dry run on one day of wikipedia.pageviews_2024, clustered on wiki then title: no filter 5.68 GiB, wiki 1.00 GiB, title alone 2.07 GiB, both 0.11 GiB.
- Wrapping the first column in a function loses the saving: UPPER(wiki) = 'EN' read 6.47 GiB, no better than no filter.
- A dry run gives an upper bound on a clustered table. The completed job shows the real bytes.
Worked example
Expensive
SELECT title, views
FROM `bigquery-public-data.wikipedia.pageviews_2024`
WHERE DATE(datehour) = '2024-06-01'
AND title = 'BigQuery'Cheaper
SELECT title, views
FROM `bigquery-public-data.wikipedia.pageviews_2024`
WHERE DATE(datehour) = '2024-06-01'
AND wiki = 'en'
AND title = 'BigQuery'The arithmetic. At 1,000 runs a month, the first bills 2,070 GiB ($12.63) and the second 107 GiB ($0.65), in the US.
How to fix it
- 1Filter on the first cluster column, as well as any later ones.
- 2Compare the bare column. Move any function to the other side of the comparison.
- 3When creating a table, put the column queries filter on most first.
The calculator detects this one. Paste your query into the BigQuery Query Cost Calculator and it will point at the exact line.
Fixing one query is satisfying. Fixing the pattern is the win.
Finitizer finds every instance of this across your BigQuery job history, ranks them by what they actually cost, and keeps checking after you have fixed them.
See how Finitizer works