Clone
rebase <builds@mapr.com>
committed
on 16 Mar 18
DRILL-6381: (Part 3) Planner and Execution implementation to support Secondary Indexes
  1. Index Planning Rules and Plan generators
    - D… Show more
DRILL-6381: (Part 3) Planner and Execution implementation to support Secondary Indexes

  1. Index Planning Rules and Plan generators

    - DbScanToIndexScanRule: Top level physical planning rule that drives index planning for several relational algebra patterns.

   - DbScanSortRemovalRule: Physical planning rule for index planning for Sort-based operations.

    - Plan Generators: Covering, Non-Covering and Intersect physical plan generators.

    - Support planning with functional indexes such as CAST functions.

    - Enhance PlannerSettings with several configuration options for indexes.

  2. Index Selection and Statistics

    - An IndexSelector that support cost-based index selection of covering and non-covering indexes using statistics and collation properties.

    - Costing of index intersection for comparison with single-index plans.

  3. Planning and execution operators

    - Support RangePartitioning physical operator during query planning and execution.

    - Support RowKeyJoin physical operator during query planning and execution.

    - HashTable and HashJoin changes to support RowKeyJoin and Index Intersection.

    - Enhance Materializer to keep track of subscan association with a particular rowkey join.

  4. Index Planning utilities

    - Utility classes to perform RexNode analysis, including conversion to and from SchemaPath.

    - Utility class to analyze filter condition and an input collation to determine output collation.

    - Helper classes to maintain index contexts for logical and physical planning phase.

    - IndexPlanUtils utility class for various helper methods.

  5. Miscellaneous

    - Separate physical rel for DirectScan.

    - Modify LimitExchangeTranspose rule to handle SingleMergeExchange.

   - MD-3880: Return correct status from RangePartitionRecordBatch setupNewSchema

Co-authored-by: Aman Sinha <asinha@maprtech.com>

Co-authored-by: chunhui-shi <cshi@maprtech.com>

Co-authored-by: Gautam Parai <gparai@maprtech.com>

Co-authored-by: Padma Penumarthy <ppenumar97@yahoo.com>

Co-authored-by: Hanumath Rao Maduri <hmaduri@maprtech.com>

Conflicts:

        exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/HashJoinPOP.java

        exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScanBatch.java

        exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashPartition.java

        exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashTable.java

        exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashTableTemplate.java

        exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/HashJoinBatch.java

        exec/java-exec/src/main/java/org/apache/drill/exec/planner/common/DrillRelOptUtil.java

        exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/Materializer.java

        exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillMergeProjectRule.java

        exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillOptiq.java

        exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillPushProjectIntoScanRule.java

        exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillScanRel.java

        exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/BroadcastExchangePrel.java

        exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/DrillDistributionTrait.java

        exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/HashJoinPrel.java

        exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/PrelUtil.java

        exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SystemOptionManager.java

        exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetPushDownFilter.java

        exec/java-exec/src/main/resources/drill-module.conf

        logical/src/main/java/org/apache/drill/common/logical/StoragePluginConfig.java

Resolve merge comflicts and compilation issues.

Show less

master + 3 more