drill

Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
MD-524: Drill API change broke the MapRDB Plugin

DRILL-4040: Build failure on master

This closes #244

DRILL-4032: Check if a new vector was created to determine if allocation is necessary in MapWriter closes #241

DRILL-4031: Log warning and ignore columns returned from JDBC source that are unsupported.

This closes #240

DRILL-4020: The not-equal operator returns incorrect results when used on the HBase row key

- Added a condition that checks if the filter to the scan specification doesn't have NOT_EQUAL operator

- Added testFilterPushDownRowKeyNotEqual() to TestHBaseFilterPushDown

This closes #309

DRILL-3171: handle concurrent Zookeeper node creation gracefully

DRILL-3765: Move partitioning pruning to HepPlanner to avoid the performance overhead for redundant rule execution.

Add fall back option in planner.

close apache/drill#255

DRILL-3423: Adding HTTPd Log Parsing functionality including full pushdown, type remapping and wildcard support. Pushed through the requested columns for push down to the parser. Added more tests to cover a few more use cases. Ensured that user query fields are now completely consistent with returned values.

DRILL-951: Add support for csv header row parsing

This closes #232

    • -0
    • +6
    /exec/java-exec/src/test/resources/store/text/data/cars.csvh
minor typo fix

DRILL-4025: Create FileSelection object with the appropriate file statuses

DRILL-3810: Add Avro schema support

Also updates the FileFormatPlugin to support providing schema.

DRILL-3956: Add support for MySQL text type

DRILL-4006: Reallocate offset vector in repeated vectors when index is beyond the current capacity

Author: Steven Phillips <smp@apache.org>

This closes #243, #242

DRILL-4000: Ensure storage plugins are not needlessly created. Add start and close capability to storage plugins.

This closes #227

Add a new configuration based map so any configurations that are passed to a node can leverage existing storage plugins.

Update FileSystemConfig to correctly implement hashcode()

Update StoragePlugin interface to extends Autocloseable and add start() method.

Update Mongo plugin to close clients when closing plugin.

DRILL-3994: make classpath scanning work on windows

DRILL-3634: Add number of partitions information in the plan for hive scans

closes #231

DRILL-4053: Reduce metadata cache file size. Save a merged schema instead of repeating for every row group. Save maxValue in a row group iff minVal equals maxVal and null otherwise. Maintain backward compatibility with older version. This closes #254

DRILL-3992: Add/fix support for JDBC schemas (tested against oracle and derby)

This closes #225

DRILL-2288: Fix ScanBatch violation of IterOutcome protocol and downstream chain of bugs.

Increments:

2288: Pt. 1 Core: Added unit test. [Drill2288GetColumnsMetadataWhenNoRowsTest, empty.json]

2288: Pt. 1 Core: Changed HBase test table #1's # of regions from 1 to 2. [HBaseTestsSuite]

Also added TODO(DRILL-3954) comment about # of regions.

2288: Pt. 2 Core: Documented IterOutcome much more clearly. [RecordBatch]

Also edited some related Javadoc.

2288: Pt. 2 Hyg.: Edited doc., added @Override, etc. [AbstractRecordBatch, RecordBatch]

Purged unused SetupOutcome.

Added @Override.

Edited comments.

Fix some comments to doc. comments.

2288: Pt. 3 Core&Hyg.: Added validation of IterOutcome sequence. [IteratorValidatorBatchIterator]

Also:

Renamed internal members for clarity.

Added comments.

2288: Pt. 4 Core: Fixed a NONE -> OK_NEW_SCHEMA in ScanBatch.next(). [ScanBatch]

(With nearby comments.)

2288: Pt. 4 Hyg.: Edited comments, reordered, whitespace. [ScanBatch]

Reordered

Added comments.

Aligned.

2288: Pt. 4 Core+: Fixed UnionAllRecordBatch to receive IterOutcome sequence right. (3659) [UnionAllRecordBatch]

2288: Pt. 5 Core: Fixed ScanBatch.Mutator.isNewSchema() to stop spurious "new schema" reports (fix short-circuit OR, to call resetting method right). [ScanBatch]

2288: Pt. 5 Hyg.: Renamed, edited comments, reordered. [ScanBatch, SchemaChangeCallBack, AbstractSingleRecordBatch]

Renamed getSchemaChange -> getSchemaChangedAndReset.

Renamed schemaChange -> schemaChanged.

Added doc. comments.

Aligned.

2288: Pt. 6 Core: Avoided dummy Null.IntVec. column in JsonReader when not needed (MapWriter.isEmptyMap()). [JsonReader, 3 vector files]

2288: Pt. 6 Hyg.: Edited comments, message. Fixed message formatting. [RecordReader, JSONFormatPlugin, JSONRecordReader, AbstractMapVector, JsonReader]

Fixed message formatting.

Edited comments.

Edited message.

Fixed spurious line break.

2288: Pt. 7 Core: Added column families in HBaseRecordReader* to avoid dummy Null.IntVec. clash. [HBaseRecordReader]

2288: Pt. 8 Core.1: Cleared recordCount in OrderedPartitionRecordBatch.innerNext(). [OrderedPartitionRecordBatch]

2288: Pt. 8 Core.2: Cleared recordCount in ProjectRecordBatch.innerNext. [ProjectRecordBatch]

2288: Pt. 8 Core.3: Cleared recordCount in TopNBatch.innerNext. [TopNBatch]

2288: Pt. 9 Core: Had UnorderedReceiverBatch reset RecordBatchLoader's record count. [UnorderedReceiverBatch, RecordBatchLoader]

2288: Pt. 9 Hyg.: Added comments. [RecordBatchLoader]

2288: Pt. 10 Core: Worked around mismatched map child vectors in MapVector.getObject(). [MapVector]

2288: Pt. 11 Core: Added OK_NEW_SCHEMA schema comparison for HashAgg. [HashAggTemplate]

2288: Pt. 12 Core: Fixed memory leak in BaseTestQuery's printing.

Fixed bad skipping of RecordBatchLoader.clear(...) and

QueryDataBatch.load(...) for zero-row batches in printResult(...).

Also, dropped suppression of call to

VectorUtil.showVectorAccessibleContent(...) (so zero-row batches are

as visible as others).

2288: Pt. 13 Core: Fixed test that used unhandled periods in column alias identifiers.

2288: Misc.: Added # of rows to showVectorAccessibleContent's output. [VectorUtil]

2288: Misc.: Added simple/partial toString() [VectorContainer, AbstractRecordReader, JSONRecordReader, BaseValueVector, FieldSelection, AbstractBaseWriter]

2288: Misc. Hyg.: Added doc. comments to VectorContainer. [VectorContainer]

2288: Misc. Hyg.: Edited comment. [DrillStringUtils]

2288: Misc. Hyg.: Clarified message for unhandled identifier containing period.

2288: Pt. 3 Core&Hyg. Upd.: Added schema comparison result to logging. [IteratorValidatorBatchIterator]

2288: Pt. 7 Core Upd.: Handled HBase columns too re NullableIntVectors. [HBaseRecordReader, TestTableGenerator, TestHBaseFilterPushDown]

Created map-child vectors for requested columns.

Added unit test method testDummyColumnsAreAvoided, adding new row to test table,

updated some row counts.

2288: Pt. 7 Hyg. Upd.: Edited comment. [HBaseRecordReader]

2288: Pt. 11 Core Upd.: REVERTED all of bad OK_NEW_SCHEMA schema comparison for HashAgg. [HashAggTemplate]

This reverts commit 0939660f4620c03da97f4e1bf25a27514e6d0b81.

2288: Pt. 6 Core Upd.: Added isEmptyMap override in new (just-rebased-in) PromotableWriter. [PromotableWriter]

Adjusted definition and default implementation of isEmptyMap (to handle MongoDB

storage plugin's use of JsonReader).

2288: Pt. 6 Hyg. Upd.: Purged old atLeastOneWrite flag. [JsonReader]

2288: Pt. 14: Disabled newly dying test testNestedFlatten().

  1. … 25 more files in changeset.
minor edit

typos, add direct to drillbit connection

add example of flatten

formatting

    • -0
    • +11
    /_docs/install/050-starting-drill-in-distributed-mode.md
DRILL-3983: Small test improvements improve error message when SQL parsing error add a simple test to Parquet writer make errors verbose by default in tests

This closes #221

Also includes:

- make the checkstyle conf not tied to the execution to allow running mvn checkstyle:check from the command line

- fix verbose output in tests

DRILL-3921: Initialize the underlying record reader lazily in HiveRecordReader

This closes #197

+ OperatorContextImpl has a lazily initialized executor service that is

a decorator around the worker pool. This service is used through

#runCallableAs to run a callable task as the given proxy user.

+ Unit tests use partition student table with chained impersonation to

exercise running multiple delegate callables (creates multiple readers,

a reader per partition).

+ Remove unused WorkManager#shutdownAndAwaitTermination method

DRILL-3937: Handle the case where min/max columns in metadata cache file are string or binary values.

Simplify serialization and check for nulls. Remove byte array comparison for min/max since it is not needed anymore.

close apache/drill#220

DRILL-3340: Part 2: Reverting 1a589ab and committing latest patch

Add operator metrics registry for metric definitions

+ Display metrics as a table within an operator profile panel

+ Rename FragmentStats#getOperatorStats to newOperatorStats

DRILL-3975: Make sure to strip scheme and authority from partition location.

This closes #217

Filter push-down support for JSON tables.

misc edits

DRILL-3941: Add timing instrumentation in different phases of partition pruning Add basic timing instrumentation while initializing FileSelection and reading the metadata cache file

closes #230

DRILL-3980: Upgrade hadoop dependency version to 2.7.x in mapr profile