Clone Tools
  • last updated 29 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-5735: UI options grouping and filtering & Metrics hints Allow search/sort in the Options webUI Additional details, like the description is provided as well in a JScript lookup map. This helps reduce the need for the server to constantly recreate the entire page with the description details, as the client browser can fill in these details. Developers will be expected to update the description as old/new options are introduced or deprecated. System table options_val also carries the description, albeit a 'shorter' version.

closes #1279

  1. … 14 more files in changeset.
DRILL-6422: Replace guava imports with shaded ones

  1. … 970 more files in changeset.
DRILL-6671: Multi level lateral unnest join is throwing an exception during materializing the plan.

closes #1426

    • -4
    • +50
    ./visitor/LateralUnnestRowIDVisitor.java
  1. … 3 more files in changeset.
DRILL-6645: Transform TopN in Lateral Unnest pipeline to Sort and Limit.

closes #1417

    • -3
    • +3
    ./visitor/LateralUnnestRowIDVisitor.java
DRILL-6656: Disallow extra semicolons and multiple statements on the same line.

closes #1415

  1. … 136 more files in changeset.
DRILL-6617: Changing name of implicit RowId column from implicitColumn to implicitRIDColumn.

closes #1401

  1. … 7 more files in changeset.
DRILL-6636: Planner side changes to use PartitionLimitBatch in place of LimitBatch.

  1. … 1 more file in changeset.
DRILL-6617: Planner Side changed to propagate $drill_implicit_field$ information.

  1. … 22 more files in changeset.
DRILL-6475: Unnest: Null fieldId Pointer.

closes #1381

    • -0
    • +148
    ./visitor/AdjustOperatorsSchemaVisitor.java
    • -87
    • +0
    ./visitor/JoinPrelRenameVisitor.java
  1. … 2 more files in changeset.
DRILL-6545: Projection Push down into Lateral Join operator.

closes #1347

  1. … 12 more files in changeset.
DRILL-6502: Rename CorrelatePrel to LateralJoinPrel.

closes #1325

    • -0
    • +132
    ./LateralJoinPrel.java
    • -0
    • +55
    ./LateralJoinPrule.java
    • -10
    • +10
    ./explain/NumberingRelWriter.java
    • -3
    • +3
    ./visitor/ExcessiveExchangeIdentifier.java
  1. … 7 more files in changeset.
DRILL-6546: Allow unnest function with nested columns and complex expressions

Fix loss of projected names in right side of correlate when single field is projected

  1. … 7 more files in changeset.
DRILL-6491: Prevent merge join for full outer join at planning stage

closes #1320

  1. … 1 more file in changeset.
DRILL-6476: Generate explain plan which shows relation between Lateral and the corresponding Unnest.

closes #1308

    • -11
    • +58
    ./explain/NumberingRelWriter.java
  1. … 1 more file in changeset.
DRILL-6474: Don't use TopN when order by and offset are used without a limit specified.

closes #1313

  1. … 4 more files in changeset.
DRILL-6463 : Fix integer overflow in MockGroupScanPOP

closes #1303

  1. … 2 more files in changeset.
DRILL-6456: Planner shouldn't create any exchanges on the right side of Lateral Join.

This closes #1299

    • -7
    • +65
    ./visitor/ExcessiveExchangeIdentifier.java
  1. … 1 more file in changeset.
DRILL-6385: Support JPPD feature

    • -0
    • +56
    ./RuntimeFilterPrel.java
    • -0
    • +364
    ./visitor/RuntimeFilterVisitor.java
  1. … 60 more files in changeset.
DRILL-6431: Unnest operator requires table and a single column alias to be specified. Fixing the issues related to star column renaming, same field name renaming and also enforcing that an alias column is required for the unnest operator.

    • -9
    • +29
    ./visitor/JoinPrelRenameVisitor.java
    • -1
    • +14
    ./visitor/PrelVisualizerVisitor.java
  1. … 11 more files in changeset.
DRILL-6386: Remove unused imports and star imports.

  1. … 228 more files in changeset.
DRILL-6321: Lateral Join and Unnest - rules, options, logical plan supports

Included changes:

* Add planner.enable_unnest_lateral option. Default value set to false.

* Enable FilterCorrectRule

* Add support to logical plan

* Fix rebase errors for DRILL-6321 commits

  1. … 17 more files in changeset.
DRILL-6347: Change method names to "visitField".

Further change the method names to "visitField" with Vlad Rozov's comments.

closes #1236

DRILL-6422: Update guava to 23.0 and shade it

- Fix compilation errors for new version of Guava.

- Remove usage of deprecated API

- Shade guava and add dependencies to the shaded version

- Ban unshaded package

- Introduce drill-shaded module and move guava-shaded under it

- Add methods to convert shaded guava lists to the unshaded ones

- Add instruction for publishing artifacts to the Apache repository

  1. … 82 more files in changeset.
DRILL-6339: Add a new option to disable TopN (for testing)

This closes #1219

  1. … 2 more files in changeset.
DRILL-6320: Fixed license headers.

closes #1207

  1. … 2052 more files in changeset.
DRILL-6375 : Support for ANY_VALUE aggregate function

closes #1256

  1. … 36 more files in changeset.
DRILL-6321: Lateral Join and Unnest - initial implementation for parser and planning

    • -0
    • +89
    ./CorrelatePrel.java
    • -0
    • +56
    ./CorrelatePrule.java
  1. … 23 more files in changeset.
DRILL-6381: (Part 3) Planner and Execution implementation to support Secondary Indexes

  1. Index Planning Rules and Plan generators

    - DbScanToIndexScanRule: Top level physical planning rule that drives index planning for several relational algebra patterns.

- DbScanSortRemovalRule: Physical planning rule for index planning for Sort-based operations.

    - Plan Generators: Covering, Non-Covering and Intersect physical plan generators.

    - Support planning with functional indexes such as CAST functions.

    - Enhance PlannerSettings with several configuration options for indexes.

  2. Index Selection and Statistics

    - An IndexSelector that support cost-based index selection of covering and non-covering indexes using statistics and collation properties.

    - Costing of index intersection for comparison with single-index plans.

  3. Planning and execution operators

    - Support RangePartitioning physical operator during query planning and execution.

    - Support RowKeyJoin physical operator during query planning and execution.

    - HashTable and HashJoin changes to support RowKeyJoin and Index Intersection.

    - Enhance Materializer to keep track of subscan association with a particular rowkey join.

  4. Index Planning utilities

    - Utility classes to perform RexNode analysis, including conversion to and from SchemaPath.

    - Utility class to analyze filter condition and an input collation to determine output collation.

    - Helper classes to maintain index contexts for logical and physical planning phase.

    - IndexPlanUtils utility class for various helper methods.

  5. Miscellaneous

    - Separate physical rel for DirectScan.

    - Modify LimitExchangeTranspose rule to handle SingleMergeExchange.

- MD-3880: Return correct status from RangePartitionRecordBatch setupNewSchema

Co-authored-by: Aman Sinha <asinha@maprtech.com>

Co-authored-by: chunhui-shi <cshi@maprtech.com>

Co-authored-by: Gautam Parai <gparai@maprtech.com>

Co-authored-by: Padma Penumarthy <ppenumar97@yahoo.com>

Co-authored-by: Hanumath Rao Maduri <hmaduri@maprtech.com>

Conflicts:

exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/HashJoinPOP.java

exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScanBatch.java

exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashPartition.java

exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashTable.java

exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashTableTemplate.java

exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/HashJoinBatch.java

exec/java-exec/src/main/java/org/apache/drill/exec/planner/common/DrillRelOptUtil.java

exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/Materializer.java

exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillMergeProjectRule.java

exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillOptiq.java

exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillPushProjectIntoScanRule.java

exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillScanRel.java

exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/BroadcastExchangePrel.java

exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/DrillDistributionTrait.java

exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/HashJoinPrel.java

exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/PrelUtil.java

exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SystemOptionManager.java

exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetPushDownFilter.java

exec/java-exec/src/main/resources/drill-module.conf

logical/src/main/java/org/apache/drill/common/logical/StoragePluginConfig.java

Resolve merge comflicts and compilation issues.

    • -0
    • +44
    ./AbstractRangePartitionFunction.java
    • -0
    • +162
    ./DirectScanPrel.java
    • -0
    • +111
    ./LimitExchangeTransposeRule.java
    • -66
    • +0
    ./LimitUnionExchangeTransposeRule.java
    • -0
    • +52
    ./PrelFactories.java
    • -0
    • +130
    ./RangePartitionExchangePrel.java
  1. … 79 more files in changeset.
DRILL-6381: (Part 4) Enhance MapR-DB plugin to support querying secondary indexes

  1. Implementation of the index descriptor for MapR-DB.

2. MapR-DB specific costing for covering and non-covering indexes.

3. Discovery componenent to discover the indexes available for a MapR-DB table including CAST functional indexes.

4. Utility functions to build a canonical index descriptor.

5. Statistics: fetch and initialize statistcs from MapR-DB for a query condition. Maintain a query-scoped cache for the statistics. Utility functions to compute selectivity.

6. Range Partitioning: partitioning function that takes into account the tablet map to find out where a particular rowkey belongs.

7. Restricted Scan: support doing restricted (i.e skip) scan through lookups on the rowkey. Added a group-scan and record reader for this.

8. MD-3726: Simple Order by queries (without limit) when an index is used are showing regression.

9. MD-3995: Do not pushdown limit 0 past project with CONVERT_FROMJSON

10. MD-4259 : Account for limit during hashcode computation

Co-authored-by: Aman Sinha <asinha@maprtech.com>

Co-authored-by: chunhui-shi <cshi@maprtech.com>

Co-authored-by: Gautam Parai <gparai@maprtech.com>

Co-authored-by: Padma Penumarthy <ppenumar97@yahoo.com>

Co-authored-by: Hanumath Rao Maduri <hmaduri@maprtech.com>

Conflicts:

contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/MapRDBFormatMatcher.java

contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/MapRDBPushProjectIntoScan.java

contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/json/JsonTableGroupScan.java

exec/java-exec/src/main/java/org/apache/drill/exec/planner/index/rules/DbScanSortRemovalRule.java

exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/SortPrel.java

exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/TopNPrel.java

Fix additional compilation issues.

    • -0
    • +28
    ./visitor/ExcessiveExchangeIdentifier.java
  1. … 27 more files in changeset.
DRILL-6115: SingleMergeExchange is not scaling up when many minor fragments are allocated for a query.

DRILL-6115: Refactoring the existing code.

close apache/drill#1110

    • -0
    • +74
    ./OrderedMuxExchangePrel.java
    • -105
    • +10
    ./visitor/InsertLocalExchangeVisitor.java
  1. … 9 more files in changeset.