Skip to content

Expand sql_planner benchmark for benchmarking physical and logical optimization.#17276

Merged
alamb merged 2 commits into
apache:mainfrom
Omega359:planning_perf
Aug 24, 2025
Merged

Expand sql_planner benchmark for benchmarking physical and logical optimization.#17276
alamb merged 2 commits into
apache:mainfrom
Omega359:planning_perf

Conversation

@Omega359

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Part of #17261

Rationale for this change

Adding/updating benchmarks to allow for making it easier for others to investigate planning performance concerns.

What changes are included in this PR?

sql_planner benchmark update.

Are these changes tested?

Yes.

Are there any user-facing changes?

No

@github-actions github-actions Bot added the core Core DataFusion crate label Aug 21, 2025
@Omega359

Omega359 commented Aug 21, 2025

Copy link
Copy Markdown
Contributor Author

Here is an instrumented output from running the new logical_plan_optimize benchmark. Compared to the results I see with my actual code the type_coercion in execute_and_check is much faster (I suspect it's because the benchmark isn't using a ListingTable with many files behind it) however the pushdown filter rules takes more time.

Adjusting the column count up in the benchmark increases the time significantly.

Benchmarking logical_plan_optimize: Warming up for 3.0000 s[2025-08-21T20:17:34Z INFO  datafusion_optimizer::analyzer] Analyzer::execute_and_check rule resolve_grouping_function took 464 ms
[2025-08-21T20:17:37Z INFO  datafusion_optimizer::analyzer] Analyzer::execute_and_check rule type_coercion took 2953 ms
[2025-08-21T20:17:37Z INFO  datafusion_optimizer::analyzer] Analyzer::execute_and_check rule processing took 3417 ms
[2025-08-21T20:17:37Z INFO  datafusion_optimizer::analyzer] Analyzer::execute_and_check took 3619 ms
[2025-08-21T20:17:38Z INFO  datafusion_optimizer::optimizer] Optimization for rule eliminate_nested_union took > 50ms: 180 ms
[2025-08-21T20:17:40Z INFO  datafusion_optimizer::optimizer] Optimization for rule simplify_expressions took > 50ms: 2038 ms
[2025-08-21T20:17:40Z INFO  datafusion_optimizer::optimizer] Optimization for rule replace_distinct_aggregate took > 50ms: 158 ms
[2025-08-21T20:17:40Z INFO  datafusion_optimizer::optimizer] Optimization for rule eliminate_join took > 50ms: 159 ms
[2025-08-21T20:17:41Z INFO  datafusion_optimizer::optimizer] Optimization for rule decorrelate_predicate_subquery took > 50ms: 293 ms
[2025-08-21T20:17:41Z INFO  datafusion_optimizer::optimizer] Optimization for rule scalar_subquery_to_join took > 50ms: 202 ms
[2025-08-21T20:17:41Z INFO  datafusion_optimizer::optimizer] Optimization for rule decorrelate_lateral_join took > 50ms: 158 ms
[2025-08-21T20:17:41Z INFO  datafusion_optimizer::optimizer] Optimization for rule extract_equijoin_predicate took > 50ms: 159 ms
[2025-08-21T20:17:41Z INFO  datafusion_optimizer::optimizer] Optimization for rule eliminate_duplicated_expr took > 50ms: 159 ms
[2025-08-21T20:17:42Z INFO  datafusion_optimizer::optimizer] Optimization for rule eliminate_filter took > 50ms: 159 ms
[2025-08-21T20:17:42Z INFO  datafusion_optimizer::optimizer] Optimization for rule eliminate_limit took > 50ms: 159 ms
[2025-08-21T20:17:42Z INFO  datafusion_optimizer::optimizer] Optimization for rule propagate_empty_relation took > 50ms: 160 ms
[2025-08-21T20:17:42Z INFO  datafusion_optimizer::optimizer] Optimization for rule eliminate_one_union took > 50ms: 162 ms
[2025-08-21T20:17:42Z INFO  datafusion_optimizer::optimizer] Optimization for rule filter_null_join_keys took > 50ms: 163 ms
[2025-08-21T20:17:42Z INFO  datafusion_optimizer::optimizer] Optimization for rule eliminate_outer_join took > 50ms: 160 ms
[2025-08-21T20:17:43Z INFO  datafusion_optimizer::optimizer] Optimization for rule push_down_limit took > 50ms: 160 ms
[2025-08-21T20:19:24Z INFO  datafusion_optimizer::optimizer] Optimization for rule push_down_filter took > 50ms: 101005 ms
[2025-08-21T20:19:24Z INFO  datafusion_optimizer::optimizer] Optimization for rule single_distinct_aggregation_to_group_by took > 50ms: 290 ms
[2025-08-21T20:19:24Z INFO  datafusion_optimizer::optimizer] Optimization for rule eliminate_group_by_constant took > 50ms: 277 ms
[2025-08-21T20:19:25Z INFO  datafusion_optimizer::optimizer] Optimization for rule common_sub_expression_eliminate took > 50ms: 946 ms
[2025-08-21T20:19:30Z INFO  datafusion_optimizer::optimizer] Optimization for rule optimize_projections took > 50ms: 4573 ms
[2025-08-21T20:19:30Z INFO  datafusion_optimizer::optimizer] Processing time for pass 0 was 112145ms
[2025-08-21T20:19:30Z INFO  datafusion_optimizer::optimizer] Optimization for rule eliminate_nested_union took > 50ms: 267 ms
[2025-08-21T20:19:32Z INFO  datafusion_optimizer::optimizer] Optimization for rule simplify_expressions took > 50ms: 1898 ms
[2025-08-21T20:19:32Z INFO  datafusion_optimizer::optimizer] Optimization for rule replace_distinct_aggregate took > 50ms: 145 ms
[2025-08-21T20:19:32Z INFO  datafusion_optimizer::optimizer] Optimization for rule eliminate_join took > 50ms: 145 ms
[2025-08-21T20:19:33Z INFO  datafusion_optimizer::optimizer] Optimization for rule decorrelate_predicate_subquery took > 50ms: 242 ms
[2025-08-21T20:19:33Z INFO  datafusion_optimizer::optimizer] Optimization for rule scalar_subquery_to_join took > 50ms: 170 ms
[2025-08-21T20:19:33Z INFO  datafusion_optimizer::optimizer] Optimization for rule decorrelate_lateral_join took > 50ms: 144 ms
[2025-08-21T20:19:33Z INFO  datafusion_optimizer::optimizer] Optimization for rule extract_equijoin_predicate took > 50ms: 144 ms
[2025-08-21T20:19:33Z INFO  datafusion_optimizer::optimizer] Optimization for rule eliminate_duplicated_expr took > 50ms: 144 ms
[2025-08-21T20:19:33Z INFO  datafusion_optimizer::optimizer] Optimization for rule eliminate_filter took > 50ms: 144 ms
[2025-08-21T20:19:34Z INFO  datafusion_optimizer::optimizer] Optimization for rule eliminate_limit took > 50ms: 144 ms
[2025-08-21T20:19:34Z INFO  datafusion_optimizer::optimizer] Optimization for rule propagate_empty_relation took > 50ms: 291 ms
[2025-08-21T20:19:34Z INFO  datafusion_optimizer::optimizer] Optimization for rule eliminate_one_union took > 50ms: 101 ms
[2025-08-21T20:19:34Z INFO  datafusion_optimizer::optimizer] Optimization for rule filter_null_join_keys took > 50ms: 94 ms
[2025-08-21T20:19:34Z INFO  datafusion_optimizer::optimizer] Optimization for rule eliminate_outer_join took > 50ms: 94 ms
[2025-08-21T20:19:34Z INFO  datafusion_optimizer::optimizer] Optimization for rule push_down_limit took > 50ms: 94 ms
[2025-08-21T20:19:35Z INFO  datafusion_optimizer::optimizer] Optimization for rule push_down_filter took > 50ms: 291 ms
[2025-08-21T20:19:35Z INFO  datafusion_optimizer::optimizer] Optimization for rule single_distinct_aggregation_to_group_by took > 50ms: 99 ms
[2025-08-21T20:19:35Z INFO  datafusion_optimizer::optimizer] Optimization for rule eliminate_group_by_constant took > 50ms: 98 ms
[2025-08-21T20:19:35Z INFO  datafusion_optimizer::optimizer] Optimization for rule common_sub_expression_eliminate took > 50ms: 337 ms
[2025-08-21T20:19:37Z INFO  datafusion_optimizer::optimizer] Optimization for rule optimize_projections took > 50ms: 1628 ms
[2025-08-21T20:19:37Z INFO  datafusion_optimizer::optimizer] Processing time for pass 1 was 6904ms
[2025-08-21T20:19:37Z INFO  datafusion_optimizer::optimizer] Optimization for rule eliminate_nested_union took > 50ms: 107 ms
[2025-08-21T20:19:38Z INFO  datafusion_optimizer::optimizer] Optimization for rule simplify_expressions took > 50ms: 711 ms
[2025-08-21T20:19:38Z INFO  datafusion_optimizer::optimizer] Optimization for rule replace_distinct_aggregate took > 50ms: 84 ms
[2025-08-21T20:19:38Z INFO  datafusion_optimizer::optimizer] Optimization for rule eliminate_join took > 50ms: 81 ms
[2025-08-21T20:19:38Z INFO  datafusion_optimizer::optimizer] Optimization for rule decorrelate_predicate_subquery took > 50ms: 124 ms
[2025-08-21T20:19:38Z INFO  datafusion_optimizer::optimizer] Optimization for rule scalar_subquery_to_join took > 50ms: 92 ms
[2025-08-21T20:19:38Z INFO  datafusion_optimizer::optimizer] Optimization for rule decorrelate_lateral_join took > 50ms: 85 ms
[2025-08-21T20:19:38Z INFO  datafusion_optimizer::optimizer] Optimization for rule extract_equijoin_predicate took > 50ms: 83 ms
[2025-08-21T20:19:38Z INFO  datafusion_optimizer::optimizer] Optimization for rule eliminate_duplicated_expr took > 50ms: 82 ms
[2025-08-21T20:19:38Z INFO  datafusion_optimizer::optimizer] Optimization for rule eliminate_filter took > 50ms: 82 ms
[2025-08-21T20:19:39Z INFO  datafusion_optimizer::optimizer] Optimization for rule eliminate_limit took > 50ms: 82 ms
[2025-08-21T20:19:39Z INFO  datafusion_optimizer::optimizer] Optimization for rule propagate_empty_relation took > 50ms: 82 ms
[2025-08-21T20:19:39Z INFO  datafusion_optimizer::optimizer] Optimization for rule eliminate_one_union took > 50ms: 82 ms
[2025-08-21T20:19:39Z INFO  datafusion_optimizer::optimizer] Optimization for rule filter_null_join_keys took > 50ms: 82 ms
[2025-08-21T20:19:39Z INFO  datafusion_optimizer::optimizer] Optimization for rule eliminate_outer_join took > 50ms: 82 ms
[2025-08-21T20:19:39Z INFO  datafusion_optimizer::optimizer] Optimization for rule push_down_limit took > 50ms: 82 ms
[2025-08-21T20:19:39Z INFO  datafusion_optimizer::optimizer] Optimization for rule push_down_filter took > 50ms: 285 ms
[2025-08-21T20:19:39Z INFO  datafusion_optimizer::optimizer] Optimization for rule single_distinct_aggregation_to_group_by took > 50ms: 80 ms
[2025-08-21T20:19:39Z INFO  datafusion_optimizer::optimizer] Optimization for rule eliminate_group_by_constant took > 50ms: 79 ms
[2025-08-21T20:19:40Z INFO  datafusion_optimizer::optimizer] Optimization for rule common_sub_expression_eliminate took > 50ms: 283 ms
[2025-08-21T20:19:41Z INFO  datafusion_optimizer::optimizer] Optimization for rule optimize_projections took > 50ms: 1261 ms
[2025-08-21T20:19:41Z INFO  datafusion_optimizer::optimizer] Processing time for pass 2 was 4183ms
[2025-08-21T20:19:41Z INFO  datafusion_optimizer::optimizer] Processing time for 3 passes was 123504ms
[2025-08-21T20:19:41Z INFO  datafusion_optimizer::optimizer] Optimizer took 123599 ms

@alamb alamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @Omega359

@alamb
alamb merged commit 8c65a41 into apache:main Aug 24, 2025
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate

2 participants