Clone Tools
  • last updated 24 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-7314: Use TupleMetadata instead of concrete implementation

1. Add ser / de implementation for TupleMetadata interface based on types.

2. Replace TupleSchema usage where possible.

3. Move patcher classes into commons.

4. Upgrade some dependencies and general refactoring.

    • -0
    • +188
    ./ProtobufPatcher.java
  1. … 39 more files in changeset.
DRILL-7273: Introduce operators for handling metadata

closes #1886

  1. … 156 more files in changeset.
DRILL-7272: Drill Metastore Read / Write API and Drill Iceberg Metastore implementation

1. Drill Metastore Read / Write API.

2. Drill Iceberg Metastore implementation in iceberg-metastore module.

3. Patches Guava Preconditions class for Apache Iceberg.

4. General refactoring.

5. Unit tests.

6. Documentation.

  1. … 97 more files in changeset.
DRILL-7068: Support memory adjustment framework for resource management with Queues. closes #1677

    • -0
    • +39
    ./function/CheckedConsumer.java
  1. … 37 more files in changeset.
DRILL-6760: Retain original exception in Verbose Error Message

closes #1519

    • -0
    • +107
    ./DrillExceptionUtil.java
  1. … 3 more files in changeset.
DRILL-6762: Fix dynamic UDFs versioning issue

1. Added UndefinedVersionDelegatingStore to serve as versioned wrapper for those stores that do not support versioning.

2. Aligned remote and local function registries version type. Type will be represented as int since ZK version is returned as int.

3. Added NOT_AVAILABLE and UNDEFINED versions to DataChangeVersion holder to indicate proper registry state.

4. Added additional trace logging.

5. Minor refactoring and clean up.

closes #1484

    • -0
    • +63
    ./function/CheckedFunction.java
    • -0
    • +29
    ./function/CheckedSupplier.java
  1. … 20 more files in changeset.
DRILL-6422: Replace guava imports with shaded ones

  1. … 983 more files in changeset.
DRILL-6656: Disallow extra semicolons and multiple statements on the same line.

closes #1415

  1. … 144 more files in changeset.
DRILL-6421: Refactor DecimalUtility and CoreDecimalUtility classes

closes #1267

  1. … 13 more files in changeset.
DRILL-6422: Update guava to 23.0 and shade it

- Fix compilation errors for new version of Guava.

- Remove usage of deprecated API

- Shade guava and add dependencies to the shaded version

- Ban unshaded package

- Introduce drill-shaded module and move guava-shaded under it

- Add methods to convert shaded guava lists to the unshaded ones

- Add instruction for publishing artifacts to the Apache repository

  1. … 81 more files in changeset.
DRILL-6320: Fixed license headers.

closes #1207

    • -1
    • +1
    ./DecimalScalePrecisionAddFunction.java
    • -2
    • +1
    ./DecimalScalePrecisionDivideFunction.java
    • -2
    • +1
    ./DecimalScalePrecisionModFunction.java
    • -2
    • +1
    ./DecimalScalePrecisionMulFunction.java
  1. … 2054 more files in changeset.
DRILL-6094: Decimal data type enhancements

Add ExprVisitors for VARDECIMAL

Modify writers/readers to support VARDECIMAL

- Added usage of VarDecimal for parquet, hive, maprdb, jdbc;

- Added options to store decimals as int32 and int64 or fixed_len_byte_array or binary;

Add UDFs for VARDECIMAL data type

- modify type inference rules

- remove UDFs for obsolete DECIMAL types

Enable DECIMAL data type by default

Add unit tests for DECIMAL data type

Fix mapping for NLJ when literal with non-primitive type is used in join conditions

Refresh protobuf C++ source files

Changes in C++ files

Add support for decimal logical type in Avro.

Add support for date, time and timestamp logical types.

Update Avro version to 1.8.2.

    • -40
    • +0
    ./DecimalScalePrecisionAddFunction.java
    • -62
    • +0
    ./DecimalScalePrecisionDivideFunction.java
    • -43
    • +0
    ./DecimalScalePrecisionModFunction.java
    • -46
    • +0
    ./DecimalScalePrecisionMulFunction.java
    • -37
    • +0
    ./DrillBaseComputeScalePrecision.java
  1. … 196 more files in changeset.
DRILL-5783, DRILL-5841, DRILL-5894: Rationalize test temp directories

This change includes:

DRILL-5783:

- A unit test is created for the priority queue in the TopN operator.

- The code generation classes passed around a completely unused function registry reference in some places so it is removed.

- The priority queue had unused parameters for some of its methods so it is removed.

DRILL-5841:

- Created standardized temp directory classes DirTestWatcher, SubDirTestWatcher, and BaseDirTestWatcher. And updated all unit tests to use them.

DRILL-5894:

- Removed the dfs_test storage plugin for tests and replaced it with the already existing dfs storage plugin.

Misc:

- General code cleanup.

- Removed unnecessary use of String.format in the tests.

This closes #984

    • -0
    • +46
    ./DrillFileUtils.java
  1. … 363 more files in changeset.
DRILL-5819: Default value of security.admin.user_groups and security.admin.users is true

closes #983

  1. … 10 more files in changeset.
DRILL-5832: Change OperatorFixture to use system option manager

- Rename FixtureBuilder to ClusterFixtureBuilder

- Provide alternative way to reset system/session options

- Fix for DRILL-5833: random failure in TestParquetWriter

- Provide strict, but clear, errors for missing options

closes #970

  1. … 51 more files in changeset.
DRILL-4862: Fix binary_string to use an injected buffer as out buffer

+ Previously, binary_string used the input buffer as output buffer. So after calling binary_string, the original content was destroyed. Other expressions/ functions that need to access the original input buffer get wrong results.

+ This fix also sets readerIndex and writerIndex correctly for the output buffer, otherwise the consumer of the output buffer will hit issues.

closes #604

  1. … 3 more files in changeset.
DRILL-4237, DRILL-4478: Implement hash to use murmur3 and add correspondent unit tests

+ Avoid object or extra buffer creation

+ Clean up tests

closes #485

  1. … 13 more files in changeset.
DRILL-4184: Support variable length decimal fields in parquet

  1. … 68 more files in changeset.
DRILL-4203: Fix DrillVersionInfo to make it provide a valid version number even during the unit tests.

This is now a build-time generated class, rather than one that looks on the

classpath for META-INF files.

This pattern for file generation with parameters passed from the POM files

was borrowed from parquet-mr.

  1. … 2 more files in changeset.
DRILL-2618: handle queries over empty folders consistently so that they report table not found rather than failing. Refactor FileSelection to eliminate redundancy, make it more managable Fix WorkspaceSchemaFactory to handle empty folders. Introduce ParquetFileSelection, a sub-class of FileSelection that carries along metadata cache Fix MagicStringMatcher so that it operate on files only. Unit test file selection

  1. … 13 more files in changeset.
DRILL-3987: (MOVE) Move logical expressions and operators out of common. Move to new drill-logical model.

  1. … 212 more files in changeset.
DRILL-2288: Fix ScanBatch violation of IterOutcome protocol and downstream chain of bugs.

Increments:

2288: Pt. 1 Core: Added unit test. [Drill2288GetColumnsMetadataWhenNoRowsTest, empty.json]

2288: Pt. 1 Core: Changed HBase test table #1's # of regions from 1 to 2. [HBaseTestsSuite]

Also added TODO(DRILL-3954) comment about # of regions.

2288: Pt. 2 Core: Documented IterOutcome much more clearly. [RecordBatch]

Also edited some related Javadoc.

2288: Pt. 2 Hyg.: Edited doc., added @Override, etc. [AbstractRecordBatch, RecordBatch]

Purged unused SetupOutcome.

Added @Override.

Edited comments.

Fix some comments to doc. comments.

2288: Pt. 3 Core&Hyg.: Added validation of IterOutcome sequence. [IteratorValidatorBatchIterator]

Also:

Renamed internal members for clarity.

Added comments.

2288: Pt. 4 Core: Fixed a NONE -> OK_NEW_SCHEMA in ScanBatch.next(). [ScanBatch]

(With nearby comments.)

2288: Pt. 4 Hyg.: Edited comments, reordered, whitespace. [ScanBatch]

Reordered

Added comments.

Aligned.

2288: Pt. 4 Core+: Fixed UnionAllRecordBatch to receive IterOutcome sequence right. (3659) [UnionAllRecordBatch]

2288: Pt. 5 Core: Fixed ScanBatch.Mutator.isNewSchema() to stop spurious "new schema" reports (fix short-circuit OR, to call resetting method right). [ScanBatch]

2288: Pt. 5 Hyg.: Renamed, edited comments, reordered. [ScanBatch, SchemaChangeCallBack, AbstractSingleRecordBatch]

Renamed getSchemaChange -> getSchemaChangedAndReset.

Renamed schemaChange -> schemaChanged.

Added doc. comments.

Aligned.

2288: Pt. 6 Core: Avoided dummy Null.IntVec. column in JsonReader when not needed (MapWriter.isEmptyMap()). [JsonReader, 3 vector files]

2288: Pt. 6 Hyg.: Edited comments, message. Fixed message formatting. [RecordReader, JSONFormatPlugin, JSONRecordReader, AbstractMapVector, JsonReader]

Fixed message formatting.

Edited comments.

Edited message.

Fixed spurious line break.

2288: Pt. 7 Core: Added column families in HBaseRecordReader* to avoid dummy Null.IntVec. clash. [HBaseRecordReader]

2288: Pt. 8 Core.1: Cleared recordCount in OrderedPartitionRecordBatch.innerNext(). [OrderedPartitionRecordBatch]

2288: Pt. 8 Core.2: Cleared recordCount in ProjectRecordBatch.innerNext. [ProjectRecordBatch]

2288: Pt. 8 Core.3: Cleared recordCount in TopNBatch.innerNext. [TopNBatch]

2288: Pt. 9 Core: Had UnorderedReceiverBatch reset RecordBatchLoader's record count. [UnorderedReceiverBatch, RecordBatchLoader]

2288: Pt. 9 Hyg.: Added comments. [RecordBatchLoader]

2288: Pt. 10 Core: Worked around mismatched map child vectors in MapVector.getObject(). [MapVector]

2288: Pt. 11 Core: Added OK_NEW_SCHEMA schema comparison for HashAgg. [HashAggTemplate]

2288: Pt. 12 Core: Fixed memory leak in BaseTestQuery's printing.

Fixed bad skipping of RecordBatchLoader.clear(...) and

QueryDataBatch.load(...) for zero-row batches in printResult(...).

Also, dropped suppression of call to

VectorUtil.showVectorAccessibleContent(...) (so zero-row batches are

as visible as others).

2288: Pt. 13 Core: Fixed test that used unhandled periods in column alias identifiers.

2288: Misc.: Added # of rows to showVectorAccessibleContent's output. [VectorUtil]

2288: Misc.: Added simple/partial toString() [VectorContainer, AbstractRecordReader, JSONRecordReader, BaseValueVector, FieldSelection, AbstractBaseWriter]

2288: Misc. Hyg.: Added doc. comments to VectorContainer. [VectorContainer]

2288: Misc. Hyg.: Edited comment. [DrillStringUtils]

2288: Misc. Hyg.: Clarified message for unhandled identifier containing period.

2288: Pt. 3 Core&Hyg. Upd.: Added schema comparison result to logging. [IteratorValidatorBatchIterator]

2288: Pt. 7 Core Upd.: Handled HBase columns too re NullableIntVectors. [HBaseRecordReader, TestTableGenerator, TestHBaseFilterPushDown]

Created map-child vectors for requested columns.

Added unit test method testDummyColumnsAreAvoided, adding new row to test table,

updated some row counts.

2288: Pt. 7 Hyg. Upd.: Edited comment. [HBaseRecordReader]

2288: Pt. 11 Core Upd.: REVERTED all of bad OK_NEW_SCHEMA schema comparison for HashAgg. [HashAggTemplate]

This reverts commit 0939660f4620c03da97f4e1bf25a27514e6d0b81.

2288: Pt. 6 Core Upd.: Added isEmptyMap override in new (just-rebased-in) PromotableWriter. [PromotableWriter]

Adjusted definition and default implementation of isEmptyMap (to handle MongoDB

storage plugin's use of JsonReader).

2288: Pt. 6 Hyg. Upd.: Purged old atLeastOneWrite flag. [JsonReader]

2288: Pt. 14: Disabled newly dying test testNestedFlatten().

  1. … 39 more files in changeset.
DRILL-3742: Classpath scanning and build improvement

Makes the classpath scanning a build time class discovery

Makes the fmpp generation incremental

Removes some slowness in DrillBit closing

Reduces the build time by 30%

This closes #148

  1. … 143 more files in changeset.
DRILL-2726: Add Drill version, from Manifest, same as SqlLine in the sys.version.

This closes #198

    • -0
    • +58
    ./DrillVersionInfo.java
  1. … 1 more file in changeset.
DRILL-3822: Have PathScanner use own, not thread-context, class loader.

this closes #166

DRILL-3778: Add missed part of DRILL-3160 (making JDBC Javadoc available).

This closes #158

Main:

Configured Javadoc generation (title, package groups, version in headers).

Added link to JDBC page in Drill documentation site.

Edited/fixed some JDBC Javadoc comments.

Added explicit SQLConversionOverflowException to throws clauses for Javadoc

effect.

Added some imports for Javadoc references.

Misc.:

Fixed a couple Javadoc syntax errors.

Fixed POM indentation.

  1. … 11 more files in changeset.
DRILL-3589: Update JDBC driver to shade and minimize dependencies.

Update build process to use shading, stop using Proguard.

Add simple integration test that verifies that the JDBC driver works

correctly with a clean classpath.

  1. … 9 more files in changeset.
DRILL-3496: Updates from review comments

DRILL-3496: Update: Multiple log msgs. -> single, aligned multi-line message.

Main: Changed avoidance of original single hard-to-read list from

first draft's multiple single-item log calls (resulting in multiple log

calls/entries) to single log call with aligned multi-line message.

Also revised message wording, etc.

DRILL-3496: Update 2: Some DEBUG -> INFO; another single multi-line msg.

- logger.debug(...) -> logger.info(...) for config. file messages

- changed drill-module.conf to single multi-line message like

scanForImplementations cases

DRILL-3496: Update 3: Various review comment responses.

- Removed isTraceEnabled/etc. guards.

- Simplified logging line break/indentation code.

- Added some more "final".

- Undid logger cleanup. :-(

  1. … 7 more files in changeset.
DRILL-2815: Augment PathScanner logging; clean up code.

Added some DEBUG- and TRACE-level log calls.

Augmented/edited existing log call.

Misc. code hygiene:

- Added documentation of forResources(...).

- Renamed to clarify a number of local names.

- Made logger private.

- Added "final".

- Fixed indentation; wrapped some long lines; etc.

Conflicts:

common/src/main/java/org/apache/drill/common/util/PathScanner.java

DRILL-3483: Clarify CommonConstants' constants. [CommonConstants, DrillConfig, PathScanner]

Renamed constants.

Documented constants.

Removed extraneous "public static final" (redudant and abnormal since in interface).

Edit: Unwrapped a no-longer-long line.

closes #88

  1. … 2 more files in changeset.