Athena Monitoring From S3 Access Logs: Cold Partitions, Stale Results, and Scan Waste
Athena scan costs scale directly with how much data each query touches. Cold partitions, stale query results, and unbounded table scans all inflate your bill, but they’re invisible in Athena’s own metrics. S3 access logs tell a different story.
The problem with Athena’s built-in monitoring
Athena provides per-query scan metrics and query history, but both are scoped to individual query execution. They don’t tell you which partitions are being repeatedly scanned, whether your results cache is actually helping, or what proportion of your total scan cost comes from cold data that rarely gets queried.
CloudWatch Athena metrics (TotalExecutionTime, ProcessedBytes) are aggregate. They’re useful for budget tracking but not for understanding where the waste is concentrated.
What S3 access logs reveal about Athena behavior
S3 activity shows where Athena scan cost is concentrated.
Cold partitions: the biggest source of Athena waste
Partitions that your queries include in scope without needing them add to every scan even though they contribute no useful results. This happens when partition pruning isn’t working correctly, or when queries use date ranges broader than necessary.
Stale results and query result reuse
Athena has a query result reuse feature that can serve cached results for identical queries, but Athena’s metrics don’t show whether it is working.