Clone Tools
  • last updated 12 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-7393: Revisit Drill tests to ensure that patching is executed before any test run

- Added BaseTest with patchers and extended all tests from it.

- Added a test to java-exec module to ensure that all tests there are inherited from BaseTest.

- Revised exception handling in the patchers, now it's individual for each patching method.

closes #1910

    • -1
    • +2
    ./common/TestNumericEquiDepthHistogram.java
    • -1
    • +2
    ./fragment/TestHardAffinityFragmentParallelizer.java
  1. … 136 more files in changeset.
DRILL-7441: Fix issues with fillEmpties, offset vectors

Fixes subtle issues with offset vectors and "fill empties"

logic.

Drill has an informal standard that if a batch has no rows, then

offset vectors within that batch should have zero size. Contrast

this with batches of size 1 that should have offset vectors of

size 2. Changed to enforce this rule throughout.

Nullable, repeated and variable-width vectors have "fill empties"

logic that is used in two places: when setting the value count and

when preparing to write a new value. The current logic is not

quite right for either case. Added tests and fixed the code to

properly handle each case.

Revised the batch validator to enforce the offset-vector length of 0 for

0-sized batches rule. The result was much simpler code.

Added tools to easily print a batch, restoring some code that

was recently lost when the RowSet classes were moved.

Code cleanup in all files touched.

Added logic to "dirty" allocated buffers when testing to ensure

logic is not sensitive to the "pristine" state of new buffers.

Added logic to the column writers to enforce the zero-size-batch rule

for offset vectors. Added unit tests for this case.

Fixed the column writers to set the "lastSet" mutator value for

nullable types since other code relies on this value.

Removed the "setCount" field in nullable vectors: turns out

it is not actually used.

closes #1896

  1. … 43 more files in changeset.
DRILL-7418: MetadataDirectGroupScan improvements

1. Replaced files listing with selection root information to reduce query plan size in MetadataDirectGroupScan.

2. Fixed MetadataDirectGroupScan ser / de issues.

3. Added PlanMatcher to QueryBuilder for more convenient plan matching.

4. Re-written TestConvertCountToDirectScan to use ClusterTest.

5. Refactoring and code clean up.

    • -191
    • +253
    ./logical/TestConvertCountToDirectScan.java
  1. … 12 more files in changeset.
DRILL-7242: Handle additional boundary cases and compute better estimates when popular values span multiple buckets.

Address review comments.

close apache/drill#1785

    • -0
    • +113
    ./common/TestNumericEquiDepthHistogram.java
  1. … 4 more files in changeset.
DRILL-7238: Fixed ConvertCountToDirectScan to handle non-existent columns

closes #1781

    • -0
    • +37
    ./logical/TestConvertCountToDirectScan.java
  1. … 1 more file in changeset.
DRILL-6988. Utility of the too long error message when syntax error

- Adding Drill wrapper around SqlparseException to customize produced by Calcite messages

- Fix Drill SQL parse exception formatter to calculate proper position for "^" character

closes #1753

  1. … 3 more files in changeset.
DRILL-6974: SET option command modification

- ALTER ... RESET ... and ALTER ... SET ... sub-parsers separated to 2

different SqlCall classes with same parent SqlSetOption

- parserImpls modified to handle new syntax of ALTER... SET...

expresion:

a) ALTER ... SET option.name - option.value - setting option value

b) ALTER ... SET option.name - display option value

- Handler for SqlSetOption separated to SetOptionHandler and

ResetOptionhandler for better representation of handled statements

- Base abstract class AbstractSqlSetHandler created to not repeat

shared implementation of same functions

- SetOptionHandler covered with unit tests for each statement

form.

Fix issues stated in the review

closes #1763

    • -0
    • +64
    ./sql/handlers/ResetOptionHandlerTest.java
    • -0
    • +90
    ./sql/handlers/SetOptionHandlerTest.java
  1. … 9 more files in changeset.
DRILL-7171: Create metadata directories cache file in the leaf level directories to support ConvertCountToDirectScan optimization. closes #1748

    • -16
    • +82
    ./logical/TestConvertCountToDirectScan.java
  1. … 1 more file in changeset.
DRILL-7166: Count query with wildcard should skip reading of metadata summary file

    • -0
    • +36
    ./logical/TestConvertCountToDirectScan.java
  1. … 1 more file in changeset.
DRILL-7146: Query failing with NPE when ZK queue is enabled.

  1. … 1 more file in changeset.
DRILL-7064: Leverage the summary metadata for plain COUNT aggregates.

Add unit test

Modify MetadataDirectGroupScan to track summary file information and use in unit test.

Conflicts:

exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/metadata/Metadata.java

exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/metadata/Metadata_V4.java

Fix NPE for DrillTable to account for non-eligible tables.

Fix bug with direct scan after directory pruning. Add unit test.

Address review comments.

closes #1736

    • -0
    • +75
    ./logical/TestConvertCountToDirectScan.java
  1. … 9 more files in changeset.
DRILL-7068: Support memory adjustment framework for resource management with Queues. closes #1677

    • -0
    • +227
    ./rm/TestMemoryCalculator.java
  1. … 37 more files in changeset.
DRILL-7041: CompileException happens if a nested coalesce function returns null - Made `NullExpression`s in `IfExpression` with nested `IfExpression`s to be rewritten to typed ones recursively if necessary closes #1668

    • -1
    • +21
    ./logical/TestCaseNullableTypes.java
  1. … 1 more file in changeset.
DRILL-6870: Upgrade to ANTLR4

closes #1554

    • -0
    • +56
    ./logical/DrillLogicalTestUtils.java
    • -70
    • +0
    ./logical/DrillLogicalTestutils.java
  1. … 19 more files in changeset.
DRILL-786: Allow CROSS JOIN syntax

1. Removed throw statement in UnsupportedOperatorsVisitor

2. Extended UnsupportedRelOperatorException's message

closes #1488

    • -0
    • +201
    ./sql/CrossJoinTest.java
  1. … 4 more files in changeset.
DRILL-6422: Replace guava imports with shaded ones

    • -2
    • +2
    ./fragment/TestHardAffinityFragmentParallelizer.java
  1. … 981 more files in changeset.
DRILL-6492: Ensure schema / workspace case insensitivity in Drill

1. StoragePluginsRegistryImpl was updated:

a. for backward compatibility at init to convert all existing storage plugins names to lower case, in case of duplicates, to log warning and skip the duplicate.

b. to wrap persistent plugins registry into case insensitive store wrapper (CaseInsensitivePersistentStore) to ensure all given keys are converted into lower case when performing insert, update, delete, search operations.

c. to load system storage plugins dynamically by @SystemStorage annotation.

2. StoragePlugins class was updated to stored storage plugins configs by name in case insensitive map.

3. SchemaUtilities.searchSchemaTree method was updated to convert all schema names into lower case to ensure that are they are matched case insensitively (all schemas are stored in Drill in lower case).

4. FileSystemConfig was updated to store workspaces by name in case insensitive hash map.

5. All plugins schema factories are now extend AbstractSchemaFactory to ensure that given schema name is converted to lower case.

6. New method areTableNamesAreCaseInsensitive was added to AbstractSchema to indicate if schema tables names are case insensitive. By default, false. Schema implementation is responsible for table names case insensitive search in case it supports one. Currently, information_schema, sys and hive do so.

7. System storage plugins (information_schema, sys) were refactored to ensure their schema, table names are case insensitive, also the annotation @SystemPlugin and additional constructor were added to allow dynamically load system plugins at storage plugin registry during init phase.

8. MetadataProvider was updated to concert all schema filter conditions into lower case to ensure schema would be matched case insensitively.

9. ShowSchemasHandler, ShowTablesHandler, DescribeTableHandler were updated to ensure schema / tables names (this depends if schema supports case insensitive table names) would be found case insensitively.

git closes #1439

  1. … 55 more files in changeset.
DRILL-6603: Set num_nulls for parquet statistics to -1 when actual number is not defined.

    • -36
    • +40
    ./logical/TestConvertCountToDirectScan.java
  1. … 8 more files in changeset.
DRILL-6431: Unnest operator requires table and a single column alias to be specified. Fixing the issues related to star column renaming, same field name renaming and also enforcing that an alias column is required for the unnest operator.

    • -0
    • +70
    ./logical/DrillLogicalTestutils.java
  1. … 17 more files in changeset.
DRILL-6386: Remove unused imports and star imports.

  1. … 231 more files in changeset.
DRILL-6242 Use java.time.Local{Date|Time|DateTime} for Drill Date, Time, Timestamp types. (#3)

close apache/drill#1247

* DRILL-6242 - Use java.time.Local{Date|Time|DateTime} classes to hold values from corresponding Drill date, time, and timestamp types.

Conflicts:

exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/ExtendedJsonOutput.java

Fix merge conflicts and check style.

  1. … 46 more files in changeset.
DRILL-6422: Update guava to 23.0 and shade it

- Fix compilation errors for new version of Guava.

- Remove usage of deprecated API

- Shade guava and add dependencies to the shaded version

- Ban unshaded package

- Introduce drill-shaded module and move guava-shaded under it

- Add methods to convert shaded guava lists to the unshaded ones

- Add instruction for publishing artifacts to the Apache repository

  1. … 82 more files in changeset.
DRILL-6320: Fixed license headers.

closes #1207

    • -3
    • +3
    ./fragment/TestHardAffinityFragmentParallelizer.java
    • -15
    • +16
    ./logical/TestCaseNullableTypes.java
  1. … 2062 more files in changeset.
DRILL-6094: Decimal data type enhancements

Add ExprVisitors for VARDECIMAL

Modify writers/readers to support VARDECIMAL

- Added usage of VarDecimal for parquet, hive, maprdb, jdbc;

- Added options to store decimals as int32 and int64 or fixed_len_byte_array or binary;

Add UDFs for VARDECIMAL data type

- modify type inference rules

- remove UDFs for obsolete DECIMAL types

Enable DECIMAL data type by default

Add unit tests for DECIMAL data type

Fix mapping for NLJ when literal with non-primitive type is used in join conditions

Refresh protobuf C++ source files

Changes in C++ files

Add support for decimal logical type in Avro.

Add support for date, time and timestamp logical types.

Update Avro version to 1.8.2.

    • -0
    • +18
    ./logical/TestCaseNullableTypes.java
  1. … 201 more files in changeset.
DRILL-5730: Mock testing improvements and interface improvements

closes #1045

    • -10
    • +6
    ./fragment/TestHardAffinityFragmentParallelizer.java
  1. … 223 more files in changeset.
DRILL-6054: don't try to split the filter when it is not AND

This closes #1078

  1. … 1 more file in changeset.
DRILL-5783, DRILL-5841, DRILL-5894: Rationalize test temp directories

This change includes:

DRILL-5783:

- A unit test is created for the priority queue in the TopN operator.

- The code generation classes passed around a completely unused function registry reference in some places so it is removed.

- The priority queue had unused parameters for some of its methods so it is removed.

DRILL-5841:

- Created standardized temp directory classes DirTestWatcher, SubDirTestWatcher, and BaseDirTestWatcher. And updated all unit tests to use them.

DRILL-5894:

- Removed the dfs_test storage plugin for tests and replaced it with the already existing dfs storage plugin.

Misc:

- General code cleanup.

- Removed unnecessary use of String.format in the tests.

This closes #984

    • -49
    • +44
    ./TestDirectoryExplorerUDFs.java
    • -2
    • +10
    ./logical/TestConvertCountToDirectScan.java
  1. … 362 more files in changeset.
DRILL-5752 this change includes:

1. Increased test parallelism and fixed associated bugs

2. Added test categories and categorized tests appropriately

- Don't exclude anything by default

- Increase test timeout

- Fixed flakey test

closes #940

    • -0
    • +3
    ./fragment/TestHardAffinityFragmentParallelizer.java
    • -0
    • +3
    ./logical/TestConvertCountToDirectScan.java
  1. … 258 more files in changeset.
DRILL-3993: Changes to support Calcite 1.13

- fixed all compiling errors (main changes were: Maven changes, chenges RelNode -> RelRoot, implementing some new methods from updated interfaces, chenges some literals, logger changes);

- fixed unexpected column errors, validation errors and assertion errors after Calcite update;

- fixed describe table/schema statement according to updated logic;

- added fixes with time-intervals;

- changed precision of BINARY to 65536 (was 1048576) according to updated logic (Calcite overrides bigger precision to own maxPrecision);

- ignored some incorrect tests with DRILL-3244;

- changed "Table not found" message to "Object not found within" according to new Calcite changes.

  1. … 70 more files in changeset.
DRILL-5751: Fix unit tests to use local file system even if it is not set by default

DRILL-5751: Changes after code review.

close apache/drill#927

    • -45
    • +34
    ./TestDirectoryExplorerUDFs.java
  1. … 18 more files in changeset.