drill

Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-6115: SingleMergeExchange is not scaling up when many minor fragments are allocated for a query.

DRILL-6115: Refactoring the existing code.

close apache/drill#1110

  1. … 3 more files in changeset.
DRILL-6027: - Added memory claculator - Added unit tests and docs. - Fixed IOB caused by output vector allocation. - Don't double count records that were spilled in HashJoin

  1. … 41 more files in changeset.
add committer names to team page

update docs for DRILL-5964

DRILL-6381: (Part 2) MapRDB plugin update to 6.0.1

1. MD-3960: Update Drill to build with MapR-6.0.1 libraries

  2. MD-3995: Do not pushdown limit 0 past project with CONVERT_FROMJSON

3. MD-4054: Restricted scan limit is changed to dynamically read rows using the rowcount of the rightside instead of 4096.

4. MD-3688: Impersonating a view owner doesn't work with security disabled in 6.0

5. MD-4492: Missing limit pushdown changes in JsonTableGroupScan

Co-authored-by: chunhui-shi <cshi@maprtech.com>

Co-authored-by: Gautam Parai <gparai@maprtech.com>

Co-authored-by: Vlad Rozov <vrozov@mapr.com>

Conflicts:

contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/MapRDBFormatPlugin.java

contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/MapRDBSubScan.java

contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/binary/BinaryTableGroupScan.java

contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/json/CompareFunctionsProcessor.java

contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/json/JsonConditionBuilder.java

contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/json/JsonTableGroupScan.java

contrib/format-maprdb/src/main/java/org/apache/drill/exec/store/mapr/db/json/MaprDBJsonRecordReader.java

exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/PrelUtil.java

pom.xml

  1. … 48 more files in changeset.
DRILL-6381: (Part 1) Secondary Index framework

  1. Secondary Index planning interfaces and abstract classes like DBGroupScan, DbSubScan, IndexDecriptor etc.

  2. Statistics and Cost model interfaces/classes: PluginCost, Statistics, StatisticsPayload, AbstractIndexStatistics

  3. ScanBatch and RecordReader to support repeatable scan

  4. Secondary Index execution related interfaces: RangePartitionSender, RowKeyJoin, PartitionFunction

5. MD-3979: Query using cast index plan fails with NPE

Co-authored-by: Aman Sinha <asinha@maprtech.com>

Co-authored-by: chunhui-shi <cshi@maprtech.com>

Co-authored-by: Gautam Parai <gparai@maprtech.com>

Co-authored-by: Padma Penumarthy <ppenumar97@yahoo.com>

Co-authored-by: Hanumath Rao Maduri <hmaduri@maprtech.com>

Conflicts:

exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScanBatch.java

exec/java-exec/src/main/java/org/apache/drill/exec/planner/common/DrillRelOptUtil.java

exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillTable.java

protocol/src/main/java/org/apache/drill/exec/proto/UserBitShared.java

protocol/src/main/java/org/apache/drill/exec/proto/beans/CoreOperatorType.java

protocol/src/main/protobuf/UserBitShared.proto

  1. … 30 more files in changeset.
add kafka sp doc

add kafka sp doc

DILL-6148: TestSortSpillWithException is sometimes failing.

closes #1120

DRILL-6123: Limit batch size for Merge Join based on memory

closes #1107

DRILL-6144: Make direct memory amount configurable in tests.

This closes #1118

edits for JJ transform

  1. … 10 more files in changeset.
DRILL-6143: Made FragmentsRunner's rpc timeout larger to reduce random failures and made it configurable as a SystemOption.

closes #1119

DRILL-6324: Unnest - Initial Implementation

- Based on Flatten

- Implement unnestRecords in UnnestTemplate

- Remove unnecessary code inherited from Flatten/Project. Add schema change handling.

- Fix build failure after rebase since RecordBatchSizer used by UNNEST was relocated to a different package

- Add unit tests

- Handling of input row splitting across multiple batches. Also do not kill incoming in killIncoming.

- Schema change generated by Unnest

edits to put headings in order

    • -7
    • +7
    /_docs/tutorials/050-analyzing-highly-dynamic-datasets.md
add opentsdb SP doc, update other docs so titles have quotes

    • -6
    • +6
    /_docs/developer-information/rest-api/010-submitting-queries-from-the-rest-api-when-impersonation-is-enabled-and-authentication-is-disabled.md
    • -2
    • +2
    /_docs/install/050-starting-drill-in-distributed-mode.md
    • -2
    • +2
    /_docs/install/060-starting-the-web-ui.md
    • -2
    • +2
    /_docs/install/070-identifying-multiple-drill-versions-in-a-cluster.md
DRILL-6294: Changes to support Calcite 1.16.0 , and remove deprecated API usage

closes #1198

  1. … 25 more files in changeset.
DRILL-6140: Correctly list Operators in Profiles Page

Operators listed in Profiles Page don't always correspond with operator specified in Physical Plan.

This commit fixes that by using the PhysicalPlan as a reference, but reverts to the inferred names in the event of an Exchange-based operator

closes #1116

DRILL-6138: Move RecordBatchSizer to org.apache.drill.exec.record package

This closes #1115

DRILL-6114: Metadata revisions

Support for union vectors, list vectors, repeated list vectors. Refactored metadata classes.

closes #1112

  1. … 59 more files in changeset.
DRILL-5902: Queries encounter random failure due to RPC connection timed out

close apache/drill#1113

minor edit

add option exec.java.compiler.exp_in_method_size to docs

DRILL-6323: Lateral Join - Initial implementation

DRILL-6322: Lateral Join: Common changes - Add new iterOutcome, Operatortypes, MockRecordBatch for testing

Added new Iterator State EMIT, added operatos LATERA_JOIN & UNNEST in CoreOperatorType and added LateralContract interface

Implementation of MockRecordBatch to test operator behavior for different IterOutcomes. a) Creates new output container for schema change cases. b) Doesn't create new container for each next() call without schema change, since the operator in test expects the ValueVector object in it's incoming batch to be same unless a OK_NEW_SCHEMA case is hit. Since setup() method of operator in test will store the reference to value vector received in first batch

This closes #1211

DRILL-6137: Fixed join error when one file partition is empty

DRILL-6128: Wrong Result with Nested Loop Join

This closes #1109

DRILL-6321: Customize Drill's conformance. Allow support to APPLY keywords

close apache/drill#1224

Conflicts:

exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/SqlConverter.java

Fix checkstyle

DRILL-6130: Fix NPE during physical plan submission for various storage plugins

1. Fixed ser / de issues for Hive, Kafka, Hbase plugins.

2. Added physical plan submission unit test for all storage plugins in contrib module.

3. Refactoring.

closes #1108

  1. … 12 more files in changeset.
DRILL-6129: Fixed query failure due to nested column data type change

This closes #1106