Overview
In the Optimizer, we currently use the fixed logical plan provided by the user and generate samples with that logical plan. A more optimal strategy for plans with filter(s) may be to:
- First determine the optimal ordering of filters based on a sample estimate of their selectivities
- Run the optimizer's usual search procedure with the logical plan that has the optimal filter ordering
Acceptance Criteria
Add a function call to Optimizer.optimize() which is invoked where self.heuristic_optimization(final_group_id) is currently commented out. This function should take the dataset (and possibly the final_group_id) as input and return an updated version of the dataset which has filter operators ordered optimally based on filter selectivity estimates.
Please speak with @mdr223 before starting this issue, as I may be able to point you in the right direction for getting started.
Overview
In the Optimizer, we currently use the fixed logical plan provided by the user and generate samples with that logical plan. A more optimal strategy for plans with filter(s) may be to:
Acceptance Criteria
Add a function call to
Optimizer.optimize()which is invoked whereself.heuristic_optimization(final_group_id)is currently commented out. This function should take thedataset(and possibly thefinal_group_id) as input and return an updated version of thedatasetwhich has filter operators ordered optimally based on filter selectivity estimates.Please speak with @mdr223 before starting this issue, as I may be able to point you in the right direction for getting started.