Clone Tools
  • last updated 12 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-7436: Fix record count, vector structure issues in several operators

Adds additional vector checks to the BatchValidator.

Enables checking for the following operators:

* FilterRecordBatch

* PartitionLimitRecordBatch

* UnnestRecordBatch

* HashAggBatch

* RemovingRecordBatch

Fixes vector count issues for each of these.

Fixes empty-batch (record count = 0) handling in several of the

above operators. Added a method to VectorContainer to correctly

create an empty batch. (An empty batch, counter-intuitively,

needs vectors allocated to hold the 0 value in the first

position of each offset vector.)

Disables verbose logging for MongoDB tests. Details are written to

the log rather than the console.

Disables two invalid Mongo tests. See DRILL-7428.

Adjusts the expression tree materializer to not add the LATE type

to Union vectors. (See DRILL-7435.)

Ensures that Union vectors contain valid vectors for each subtype.

The present fix is a work-around, see DRILL-7434 for a better

long-term fix.

Cleans up code formatting and other minor issues in each file touched

during the fixes in this PR.

  1. … 36 more files in changeset.
DRILL-7045: Updates to address review comments

closes #7134

DRILL-7060: Support JsonParser Feature 'ALLOW_BACKSLASH_ESCAPING_ANY_CHARACTER' (#1663)

    • -0
    • +87
    ./complex/writer/TestJsonEscapeAnyChar.java
  1. … 7 more files in changeset.
DRILL-7045 UDF string_binary java.lang.IndexOutOfBoundsException

UDF string_binary was not reallocating the drillbuffer so it would fill

up and throw and out of bounds exception

    • -0
    • +16
    ./complex/writer/TestJsonNanInf.java
  1. … 1 more file in changeset.
DRILL-6810: Disable NULL_IF_NULL NullHandling for functions with ComplexWriter closes #1509

  1. … 16 more files in changeset.
DRILL-6422: Replace guava imports with shaded ones

  1. … 984 more files in changeset.
DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM

closes #1446

    • -3
    • +0
    ./complex/writer/TestExtendedTypes.java
  1. … 23 more files in changeset.
DRILL-6656: Disallow extra semicolons and multiple statements on the same line.

closes #1415

  1. … 143 more files in changeset.
DRILL-6629 BitVector split and transfer does not work correctly for transfer length < 8

closes #1395

  1. … 1 more file in changeset.
DRILL-6486: BitVector split and transfer does not work correctly for non byte-multiple transfer lengths

Fix for the bug in BitVector splitAndTransfer. The logic for handling copy of last-n bits was incorrect for none byte-multiple transfer lengths.

closes #1316

  1. … 2 more files in changeset.
DRILL-6438: Remove excess logging form the tests. - Removed usages of System.out and System.err from the test and replaced with loggers

closes #1284

    • -17
    • +14
    ./complex/writer/TestJsonReader.java
  1. … 89 more files in changeset.
DRILL-6386: Remove unused imports and star imports.

    • -1
    • +0
    ./complex/writer/TestComplexToJson.java
  1. … 230 more files in changeset.
DRILL-6363: Upgrade jmockit and mockito libs

  1. … 6 more files in changeset.
DRILL-6242 Use java.time.Local{Date|Time|DateTime} for Drill Date, Time, Timestamp types. (#3)

close apache/drill#1247

* DRILL-6242 - Use java.time.Local{Date|Time|DateTime} classes to hold values from corresponding Drill date, time, and timestamp types.

Conflicts:

exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/ExtendedJsonOutput.java

Fix merge conflicts and check style.

    • -1
    • +1
    ./complex/writer/TestExtendedTypes.java
  1. … 46 more files in changeset.
DRILL-6422: Update guava to 23.0 and shade it

- Fix compilation errors for new version of Guava.

- Remove usage of deprecated API

- Shade guava and add dependencies to the shaded version

- Ban unshaded package

- Introduce drill-shaded module and move guava-shaded under it

- Add methods to convert shaded guava lists to the unshaded ones

- Add instruction for publishing artifacts to the Apache repository

  1. … 82 more files in changeset.
DRILL-6320: Fixed license headers.

closes #1207

    • -1
    • +1
    ./complex/fn/TestJsonReaderWithSparseFiles.java
    • -1
    • +1
    ./complex/writer/TestComplexToJson.java
    • -1
    • +1
    ./complex/writer/TestExtendedTypes.java
    • -16
    • +16
    ./complex/writer/TestJsonNanInf.java
    • -3
    • +3
    ./complex/writer/TestPromotableWriter.java
  1. … 2055 more files in changeset.
DRILL-6217: NaN/Inf NestedLoopJoin processes NaN values incorrectly

- Changed loggic for equality functions to handle NaN values as the biggest ones

closes #1154

    • -7
    • +81
    ./complex/writer/TestJsonNanInf.java
  1. … 2 more files in changeset.
DRILL-6154: NaN, Infinity issues

- changed comparison rules for NaN, Infinity values. For now NaN is the biggest value, Infinity - second biggest value

- fixed min, max, trunc functions for NaN, Infinity values

- made drill use original sqrt function instead of power(x,0.5) substitution

close apache/drill#1123

    • -0
    • +57
    ./complex/writer/TestJsonNanInf.java
  1. … 6 more files in changeset.
DRILL-3993: Changes after review:

Moved checks for OOM into spillIfNeeded method to avoid excessive call of chooseAPartitionToFlush method.

Ignored unit tests until DRILL-6018 is fixed.

These failures appears because new Calcite tries to simplify expressions from the query when applying ReduceExpressionsRule and fails with NFE.

Similar problem, but with old version of Calcite was described in DRILL-6018, therefore these unit tests are marked as ignored until DRILL-6018 is fixed.

  1. … 2 more files in changeset.
DRILL-6020: Fix NullPointerException when querying JSON untyped path with Union setting on

closes #1068

    • -0
    • +28
    ./complex/writer/TestJsonReader.java
  1. … 1 more file in changeset.
DRILL-5919: Add non-numeric support for JSON processing

1. Added two session options store.json.reader.non_numeric_numbers and store.json.reader.non_numeric_numbers that allow to read/write NaN and Infinity as numbers. By default these options

are set to true.

2. Extended signature of convert_toJSON and convert_fromJSON functions by adding second optional parameter

that enables/disables read/write NaN and Infinity. By default it is set true.

3. Added unit tests with nan, infitity values for math and aggregate functions

4. Replaced JsonReader's constructors with builder.

This closes #1026

    • -0
    • +274
    ./complex/writer/TestJsonNanInf.java
  1. … 17 more files in changeset.
DRILL-5783, DRILL-5841, DRILL-5894: Rationalize test temp directories

This change includes:

DRILL-5783:

- A unit test is created for the priority queue in the TopN operator.

- The code generation classes passed around a completely unused function registry reference in some places so it is removed.

- The priority queue had unused parameters for some of its methods so it is removed.

DRILL-5841:

- Created standardized temp directory classes DirTestWatcher, SubDirTestWatcher, and BaseDirTestWatcher. And updated all unit tests to use them.

DRILL-5894:

- Removed the dfs_test storage plugin for tests and replaced it with the already existing dfs storage plugin.

Misc:

- General code cleanup.

- Removed unnecessary use of String.format in the tests.

This closes #984

    • -1
    • +1
    ./complex/fn/TestJsonReaderWithSparseFiles.java
    • -4
    • +4
    ./complex/writer/TestComplexToJson.java
    • -34
    • +25
    ./complex/writer/TestComplexTypeReader.java
    • -25
    • +21
    ./complex/writer/TestExtendedTypes.java
    • -145
    • +96
    ./complex/writer/TestJsonReader.java
  1. … 359 more files in changeset.
DRILL-5832: Change OperatorFixture to use system option manager

- Rename FixtureBuilder to ClusterFixtureBuilder

- Provide alternative way to reset system/session options

- Fix for DRILL-5833: random failure in TestParquetWriter

- Provide strict, but clear, errors for missing options

closes #970

    • -14
    • +4
    ./complex/writer/TestExtendedTypes.java
  1. … 51 more files in changeset.
DRILL-5757: CONVERT_TO_JSON function is failed while using non-existence field as a parameter.

close apache/drill#929

  1. … 1 more file in changeset.
DRILL-5752 this change includes:

1. Increased test parallelism and fixed associated bugs

2. Added test categories and categorized tests appropriately

- Don't exclude anything by default

- Increase test timeout

- Fixed flakey test

closes #940

  1. … 265 more files in changeset.
DRILL-5717: Let some test cases be Local or TimeZone independent.

closes #904

    • -9
    • +10
    ./complex/writer/TestExtendedTypes.java
  1. … 7 more files in changeset.
DRILL-5723: Added System Internal Options That can be Modified at Runtime Changes include:

1. Addition of internal options.

2. Refactoring of OptionManagers and OptionValidators.

3. Fixed ambiguity in the meaning of an option type, and changed its name to accessibleScopes.

4. Updated javadocs in the Option System classes.

5. Added RestClientFixture for testing the Rest API.

6. Fixed flakey test in TestExceptionInjection caused by race condition.

7. Fixed various tests which started zookeeper but failed to shut it down at the end of tests.

8. Added port hunting to the Drill Webserver for testing

9. Fixed various flaky tests

10. Fix compile issue

closes #923

    • -4
    • +7
    ./complex/writer/TestExtendedTypes.java
  1. … 85 more files in changeset.
DRILL-4264: Allow field names to include dots

    • -0
    • +29
    ./complex/writer/TestJsonReader.java
  1. … 98 more files in changeset.
DRILL-5523: Revert if condition in UnionAllRecordBatch changed in DRILL-5419

close apache/drill#842

    • -1
    • +20
    ./complex/writer/TestJsonReader.java
  1. … 1 more file in changeset.
DRILL-3562: Query fails when using flatten on JSON data where some documents have an empty array

closes #713

    • -2
    • +71
    ./complex/writer/TestJsonReader.java
  1. … 2 more files in changeset.