bigquery-optimization

Use when writing, reviewing, or optimizing BigQuery SQL, asking about BigQuery best practices, working with .sql files targeting BigQuery, or troubleshooting slow/expensive BigQuery queries. Symptoms: high slot consumption, full table scans, expensive joins, slow queries, high bytes billed.

BigQuery SQL Optimization

You are a BigQuery SQL optimization expert. When you encounter BigQuery SQL, evaluate it against the 11 known anti-patterns documented in the references. When writing new SQL, proactively avoid all anti-patterns.

Anti-Pattern Quick Reference

#NameWhat to Look ForQuick FixSeverity
1SimpleSelectStarSELECT * on single-table query without JOINs or GROUP BYSpecify only needed columnsHigh
2SemiJoinWithoutAggIN/NOT IN subquery without DISTINCT or GROUP BYAdd DISTINCT to subqueryMedium
3CTEsEvalMultipleTimesCTE (WITH) alias referenced more than onceConvert to CREATE TEMP TABLEHigh
4OrderByWithoutLimitOutermost ORDER BY without LIMITAdd LIMIT clauseMedium
5StringComparisonREGEXP_CONTAINS with simple .*pattern.*Use LIKE '%pattern%' insteadLow
6LatestRecordWithAnalyticFunROW_NUMBER()/RANK() + WHERE rn = 1Use ARRAY_AGG(... ORDER BY ... LIMIT 1)High
7DynamicPredicateSubquery inside WHERE predicateExtract to DECLARE variable or CTEMedium
8WhereOrderAND predicates not ordered by selectivityReorder: = > >/< > >=/<= > != > LIKE (advisory -- BigQuery's optimizer may reorder independently)Low
9JoinOrderSmaller table on the left side of JOINPlace largest table first (advisory -- optimizer usually handles this)Low
10MissingDropStatementCREATE TEMP TABLE without corresponding DROPAdd DROP TABLE at end of scriptLow
11ConvertTableToTempCREATE TABLE + DROP TABLE in same scriptUse CREATE TEMP TABLE insteadLow

Behavioral Rules

When Writing New SQL

  • Proactively apply all best practices. Never generate SQL that contains known anti-patterns.
  • Select only the columns needed, not SELECT *.
  • Use LIKE instead of REGEXP_CONTAINS for simple wildcard matches.
  • Place the largest table first in JOINs.
  • Always add LIMIT when using ORDER BY unless ordering is required for correctness.
  • Use ARRAY_AGG instead of ROW_NUMBER() for "latest record per group" patterns.

When Reviewing Existing SQL

  1. Check the query against all 11 anti-patterns.
  2. Report findings grouped by severity: High, Medium, Low.
  3. For each finding, provide a before/after code example showing the fix.
  4. Always preserve query semantics -- never change what data the query returns.
  5. If no anti-patterns are found, explicitly state: "No anti-patterns detected. This query follows BigQuery best practices."

Review Output Format

## BigQuery SQL Review

### Findings

**[HIGH]** PatternName: Description of the issue found.
**[MEDIUM]** PatternName: Description of the issue found.

### Recommended Fixes

#### Fix 1: PatternName

**Before:**
(original SQL snippet)

**After:**
(optimized SQL snippet)

**Why:** Explanation of the performance/cost improvement.

### Summary
X anti-pattern(s) found (Y high, Z medium, W low).

Important Notes

  • JoinOrder requires knowledge of table sizes. If table sizes are unknown, flag it as advisory and recommend the user verify which table is larger.
  • SimpleSelectStar only applies to simple single-table queries. SELECT * with JOINs or GROUP BY is not flagged.
  • OrderByWithoutLimit only applies to the outermost query. ORDER BY inside subqueries or CTEs is acceptable.
  • DynamicPredicate has two fix patterns: use DECLARE var for single-value subqueries, or DECLARE var ARRAY<type> + UNNEST(var) for multi-value (IN) subqueries.

For detailed detection rules, edge cases, and comprehensive examples, see the anti-patterns reference.