Clone Tools
  • last updated 11 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-7436: Fix record count, vector structure issues in several operators

Adds additional vector checks to the BatchValidator.

Enables checking for the following operators:

* FilterRecordBatch

* PartitionLimitRecordBatch

* UnnestRecordBatch

* HashAggBatch

* RemovingRecordBatch

Fixes vector count issues for each of these.

Fixes empty-batch (record count = 0) handling in several of the

above operators. Added a method to VectorContainer to correctly

create an empty batch. (An empty batch, counter-intuitively,

needs vectors allocated to hold the 0 value in the first

position of each offset vector.)

Disables verbose logging for MongoDB tests. Details are written to

the log rather than the console.

Disables two invalid Mongo tests. See DRILL-7428.

Adjusts the expression tree materializer to not add the LATE type

to Union vectors. (See DRILL-7435.)

Ensures that Union vectors contain valid vectors for each subtype.

The present fix is a work-around, see DRILL-7434 for a better

long-term fix.

Cleans up code formatting and other minor issues in each file touched

during the fixes in this PR.

  1. … 33 more files in changeset.
DRILL-7424: Project operator fails to set the container row count

Enabled the "batch validator" for the Project operator. Ran tests.

Exceptions occurred because, in some paths, the Project operator

fails to set the container row count.

Fixes the project operator. Cleans up formatting issues in files

touched during the investigation. Cleaned up batch-related issues

in Project.

  1. … 8 more files in changeset.
DRILL-7196: Queries are still runnable on disabled plugins

- Storage client is not created anymore for disabled plugins

- GET "/storage/{name}.json" endpoint now working with

plugin configuration directly, without client instantination.

It have increased UI responsitivity.

- Hbase and mongo base test classes refactored to honor enabled

plugin attribute

- Fixed path contructor for mongo test datasets:

Now it is cross-platform

- Fixed test json files format which using plugin definitions

- Code cleanup

  1. … 100 more files in changeset.
DRILL-6422: Replace guava imports with shaded ones

  1. … 982 more files in changeset.
DRILL-6559: Travis timing out

* Excluding contrib module tests and all exec TPCH unit tests for travis full build.

* Introducing new TRAVIS profile

* Travis build is faster for 4-5 mins

closes #1364

  1. … 4 more files in changeset.
DRILL-6496: Added print methods for debugging tests, and fixed missing log statement in VectorUtils.

closes #1336

  1. … 33 more files in changeset.
DRILL-6386: Remove unused imports and star imports.

  1. … 230 more files in changeset.
DRILL-6380: Fix sporadic mongo db hangs.

closes #1249

DRILL-6341: Fixed failing tests for mongodb storage plugin by upgrading MongoDB version.

closes #1222

  1. … 1 more file in changeset.
DRILL-6320: Fixed license headers.

closes #1207

  1. … 2060 more files in changeset.
DRILL-6130: Fix NPE during physical plan submission for various storage plugins

1. Fixed ser / de issues for Hive, Kafka, Hbase plugins.

2. Added physical plan submission unit test for all storage plugins in contrib module.

3. Refactoring.

closes #1108

  1. … 26 more files in changeset.
DRILL-5783, DRILL-5841, DRILL-5894: Rationalize test temp directories

This change includes:

DRILL-5783:

- A unit test is created for the priority queue in the TopN operator.

- The code generation classes passed around a completely unused function registry reference in some places so it is removed.

- The priority queue had unused parameters for some of its methods so it is removed.

DRILL-5841:

- Created standardized temp directory classes DirTestWatcher, SubDirTestWatcher, and BaseDirTestWatcher. And updated all unit tests to use them.

DRILL-5894:

- Removed the dfs_test storage plugin for tests and replaced it with the already existing dfs storage plugin.

Misc:

- General code cleanup.

- Removed unnecessary use of String.format in the tests.

This closes #984

  1. … 365 more files in changeset.
DRILL-5895: Add logging mongod exception when failed to close all mongod processes during provided timeout

closes #1006

DRILL-5752 this change includes:

1. Increased test parallelism and fixed associated bugs

2. Added test categories and categorized tests appropriately

- Don't exclude anything by default

- Increase test timeout

- Fixed flakey test

closes #940

  1. … 262 more files in changeset.
DRILL-5723: Added System Internal Options That can be Modified at Runtime Changes include:

1. Addition of internal options.

2. Refactoring of OptionManagers and OptionValidators.

3. Fixed ambiguity in the meaning of an option type, and changed its name to accessibleScopes.

4. Updated javadocs in the Option System classes.

5. Added RestClientFixture for testing the Rest API.

6. Fixed flakey test in TestExceptionInjection caused by race condition.

7. Fixed various tests which started zookeeper but failed to shut it down at the end of tests.

8. Added port hunting to the Drill Webserver for testing

9. Fixed various flaky tests

10. Fix compile issue

closes #923

  1. … 84 more files in changeset.
DRILL-5196: Init MongoDB cluster when run a single test case directly through command line or IDE

Other fixes include:

+ Sync mongo-java-driver versions to newer 3.2.0

+ update flapdoodle package to latest accordingly

closes #741

  1. … 3 more files in changeset.
DRILL-4230: Fixing query failures when mongo collection is empty

  1. … 1 more file in changeset.
DRILL-4230: Fixing query failures when mongo collection is empty

  1. … 1 more file in changeset.
DRILL-4242: Updates to storage-mongo

+ Depends on the latest Mongo Java Driver (3.2.0)

+ Uses MongoDatabase rather than the deprecated DB class

+ Uses Document and Bson types rather than the legacy DBObject type

This closes #304.

  1. … 8 more files in changeset.
DRILL-3478: Implement BSON reader and use that for MongoDB by default

  1. … 10 more files in changeset.
DRILL-1752: Fixing query failures of unsharded db/collection in sharded cluster

- Add logback.xml in test/resources

- Add testcases and upgrading pom version of mongo and mongo test dependency

  1. … 4 more files in changeset.
DRILL-3742: Classpath scanning and build improvement

Makes the classpath scanning a build time class discovery

Makes the fmpp generation incremental

Removes some slowness in DrillBit closing

Reduces the build time by 30%

This closes #148

  1. … 143 more files in changeset.
DRILL-3773: Fix Mongo FieldSelection

Mongo plugin was previously rewriting a complex (multi-level) column reference as a simple selection of the top level field.

This changeset does not change this behavior in terms of the filter sent to mongo, but it add the original selected column to the list that will be read in by the JSON reader once that data is returned from mongo.

What this means is that we will be requesting more data from mongo that necessary (as we were previously), but this will be leveraging the existing functionality in the JSON reader to grab only the sub-selection actually requested in the query. This allows for difficult schema changes to be avoided by projecting only columns without schema changes.

This also fixes and adds unit tests for FieldSelection that cause wrong results when selecting a nested column and its parent.

  1. … 5 more files in changeset.
DRILL-2879: Enhancing extended json support for date in millies and binary with type info

Ignore project push down Mongo test until test completes correctly on Linux

  1. … 6 more files in changeset.
DRILL-1666: Add test code for mongo storage plugin

    • -0
    • +88
    ./org/apache/drill/exec/store/mongo/MongoTestBase.java
    • -0
    • +70
    ./org/apache/drill/exec/store/mongo/MongoTestConstants.java
    • -0
    • +245
    ./org/apache/drill/exec/store/mongo/MongoTestSuit.java
    • -0
    • +55
    ./org/apache/drill/exec/store/mongo/TestMongoFilterPushDown.java
    • -0
    • +55
    ./org/apache/drill/exec/store/mongo/TestMongoQueries.java
    • -0
    • +65
    ./org/apache/drill/exec/store/mongo/TestTableGenerator.java
  1. … 3 more files in changeset.
DRILL-98: MongoDB storage plugin

This commit disables MongoDB PStore due to changes to the PStore interface.

  1. … 29 more files in changeset.