Clone Tools
  • last updated 26 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-7764: Cleanup warning messages in GuavaPatcher class (#2093)

* DRILL-7764: Cleanup warning messages in GuavaPatcher class

DRILL-7711: Add data path, parameter filter pushdown to HTTP plugin

Adds an option to specify the path to data so the plugin will

ignore REST message "overhead" except the actual data.

Allows specifying HTTP URL parameters as filter push-downs from

SQL.

* Revised scan cost model to allow a scan more freedom

to define cost. Needed to ensure that filter push-down

is actually accepted by Calcite.

* Reduced size of HTTP scan plan by including on the one

needed connection config rather than all of them.

* Revised URL building so that the proxy sees the full

URL after adding parameters, etc.

* Many code refinements.

* Added more details to README

  1. … 34 more files in changeset.
DRILL-7702: Update shaded guava

    • -0
    • +72
    ./apache/drill/common/concurrent/CheckedFuture.java
  1. … 8 more files in changeset.
DRILL-7696: EVF v2 scan schema resolution

Provides the mechanism to resolve the scan schema from a

projection list, provided schema, early reader schema and

actual reader schema.

  1. … 75 more files in changeset.
DRILL-7694: Register drill.queries.* counter metrics on Drillbit startup

closes #2050

    • -0
    • +73
    ./apache/drill/exec/metrics/DrillCounters.java
  1. … 2 more files in changeset.
DRILL-7693: Updated protobuf version to 3.11.1

    • -13
    • +22
    ./apache/drill/common/util/ProtobufPatcher.java
  1. … 33 more files in changeset.
DRILL-7632: Improve user exception formatting

Adds a colon in the "getMessage()" format of a User

Exception between the context name and value:

My Context: value

closes #2017

DRILL-7637: Add an option to retrieve MapR SSL truststore/keystore credentials using MapR Web Security Manager

    • -0
    • +1
    ./apache/drill/common/config/DrillProperties.java
  1. … 9 more files in changeset.
DRILL-7615: UNION ALL query returns the wrong result for the decimal value

closes #2006

    • -8
    • +47
    ./apache/drill/common/types/Types.java
  1. … 4 more files in changeset.
DRILL-7590: Refactor plugin registry

Major cleanup of the plugin registry to split it into components

in preparation for a proper plugin API.

Better coordinates the named and ephemeral plugin caches.

Cleans up the registry API. Sharpens rules for modifying

plugin configs.

closes #1988

    • -0
    • +119
    ./apache/drill/common/PlanStringBuilder.java
    • -15
    • +8
    ./apache/drill/common/config/ConfigConstants.java
    • -1
    • +3
    ./apache/drill/common/scanner/BuildTimeScan.java
    • -5
    • +9
    ./apache/drill/exec/util/ActionOnFile.java
  1. … 153 more files in changeset.
DRILL-7634: Rollup of code cleanup changes

Collection of code cleanup changes. The most significant

is to create constants for function names.

closes #2020

    • -6
    • +2
    ./apache/drill/common/exceptions/DrillError.java
    • -5
    • +0
    ./apache/drill/common/types/Types.java
  1. … 123 more files in changeset.
DRILL-7576: Fail fast for operator errors

Converts operators to fail with a UserException rather than using

the STOP iterator status. The result is clearer error messages

and simpler code.

closes #1975

  1. … 66 more files in changeset.
DRILL-7549: Fix validation error when querying absent sub folder in embedded mode

closes #1963

  1. … 2 more files in changeset.
DRILL-7527: DROP METADATA doesn't work with table name starting with '/' inside workspace

closes #1958

    • -19
    • +18
    ./apache/drill/common/util/DrillStringUtils.java
  1. … 10 more files in changeset.
DRILL-7467: Jdbc plugin enhancements and fixes

1. Added logic to close data source when plugin is closed.

2. Added disabled jdbc plugin template to the bootstrap storage plugins.

3. Added new jdbc storage plugin configuration property sourceParameters which would allow setting data source parameters described in BasicDataSource Configuration Parameters.

4. Upgraded commons-dbcp2 version and added it to the dependency management section in common pom.xml.

closes #1956

    • -13
    • +7
    ./apache/drill/common/AutoCloseables.java
  1. … 17 more files in changeset.
DRILL-7506: Simplify code gen error handling

Pushes code gen error handling close to the code gen itself to

allow clearer error messages. Doing so avoids the need to bubble

code gen exceptions up the call stack, resulting in cleaner

operator code.

closes #1948

  1. … 40 more files in changeset.
DRILL-7502: Invalid codegen for typeof() with UNION

Also fixes DRILL-6362: typeof() reports NULL for primitive

columns with a NULL value.

typeof() is meant to return "NULL" if a UNION has a NULL

value, but the column type when known, such as for non-UNION

columns.

Also fixes DRILL-7499: sqltypeof() function with an array returns

"ARRAY", not type. This was due to treating REPEATED like LIST.

Handling of the Union vector in code gen is problematic

with about three special cases. Existing code handled two

of the cases. This change handles the third case.

Figuring out the change required poking around quite a bit

of unclear code. Added comments and restructuring to make

that code a bit more clear.

The fix modified code gen for the Union Holder. It can now

"go back in time" to add the union reader at the point we

need it.

closes #1945

    • -2
    • +5
    ./apache/drill/common/types/Types.java
  1. … 54 more files in changeset.
DRILL-7494: Unable to connect to Drill using JDBC driver when using custom authenticator

closes #1938

    • -3
    • +18
    ./apache/drill/common/util/GuavaPatcher.java
    • -5
    • +22
    ./apache/drill/common/util/ProtobufPatcher.java
  1. … 5 more files in changeset.
DRILL-6332: Allow to provide two-component Kerberos principals

closes #1931

    • -9
    • +18
    ./apache/drill/common/KerberosUtil.java
  1. … 1 more file in changeset.
DRILL-6832: Removes the old "unmanaged" external sort

When the "managed" external sort was implemented a couple

of years back, we retained the original "unmanaged" version

out of an abundance of caution. The new version is now

battle tested and it is time to retire the original one.

closes #1929

    • -0
    • +9
    ./apache/drill/common/AutoCloseables.java
  1. … 64 more files in changeset.
DRILL-7393: Revisit Drill tests to ensure that patching is executed before any test run

- Added BaseTest with patchers and extended all tests from it.

- Added a test to java-exec module to ensure that all tests there are inherited from BaseTest.

- Revised exception handling in the patchers, now it's individual for each patching method.

closes #1910

    • -115
    • +118
    ./apache/drill/common/util/GuavaPatcher.java
    • -78
    • +76
    ./apache/drill/common/util/ProtobufPatcher.java
  1. … 138 more files in changeset.
DRILL-7445: Create batch copier based on result set framework

The result set framework now provides both a reader and writer.

This PR provides a copier that copies batches using this

framework. Such a copier can:

- Copy selected records

- Copy all records, such as for an SV2 or SV4

The copier uses the result set loader to create uniformly-sized

output batches from input batches of any size. It does this

by merging or splitting input batches as needed.

Since the result set reader handles both SV2 and SV4s, the

copier can filter or reorder rows based on the SV associated

with the input batch.

This version assumes single stream of input batches, and handles

any schema changes in that input by creating output batches

that track the input schema. This would be used in, say, the

selection vector remover. A different design is needed for merging

such as in the merging receiver.

Adds a "copy" method to the column writers. Copy is implemented

by doing a direct memory copy from source to destination vectors.

A unit test verifies functionality for various use cases

and data types.

closes #1899

    • -8
    • +24
    ./apache/drill/common/types/Types.java
  1. … 39 more files in changeset.
DRILL-7439: Batch count fixes for six additional operators

Enables vector checks, and fixes batch count and vector issues for:

* StreamingAggBatch

* RuntimeFilterRecordBatch

* FlattenRecordBatch

* MergeJoinBatch

* NestedLoopJoinBatch

* LimitRecordBatch

Also fixes a zero-size batch validity issue for the CSV reader when

all files contain no data.

Includes code cleanup for files touched in this PR.

closes #1893

    • -5
    • +2
    ./apache/drill/exec/util/AssertionUtil.java
  1. … 21 more files in changeset.
DRILL-6540: Updated Hadoop and HBase libraries to the latest versions

Hadoop: 3.2.1

HBase: 2.2.2

closes #1895

    • -1
    • +16
    ./apache/drill/common/util/GuavaPatcher.java
  1. … 82 more files in changeset.
DRILL-7412: Minor unit test improvements

Many tests intentionally trigger errors. A debug-only log setting

sent those errors to stdout. The resulting stack dumps simply cluttered

the test output, so disabled error output to the console.

Drill can apply bounds checks to vectors. Tests run via Maven

enable bounds checking. Now, bounds checking is also enabled in

"debug mode" (when assertions are enabled, as in an IDE.)

Drill contains two test frameworks. The older BaseTestQuery was

marked as deprecated, but many tests still use it and are unlikely

to be changed soon. So, removed the deprecated marker to reduce the

number of spurious warnings.

Also includes a number of minor clean-ups.

closes #1876

  1. … 16 more files in changeset.
DRILL-7357: Expose Drill Metastore data through information_schema

1. Add additional columns to TABLES and COLUMNS tables.

2. Add PARTITIONS table.

3. General refactoring to adjust information_schema data retrieval from multiple sources.

closes #1860

    • -0
    • +38
    ./apache/drill/common/types/Types.java
  1. … 33 more files in changeset.
DRILL-7314: Use TupleMetadata instead of concrete implementation

1. Add ser / de implementation for TupleMetadata interface based on types.

2. Replace TupleSchema usage where possible.

3. Move patcher classes into commons.

4. Upgrade some dependencies and general refactoring.

    • -0
    • +91
    ./apache/drill/common/util/GuavaPatcher.java
    • -0
    • +188
    ./apache/drill/common/util/ProtobufPatcher.java
  1. … 39 more files in changeset.
DRILL-7315: Revise precision and scale order in the method arguments

    • -2
    • +2
    ./apache/drill/common/types/Types.java
  1. … 28 more files in changeset.
DRILL-7273: Introduce operators for handling metadata

closes #1886

    • -1
    • +1
    ./apache/drill/common/types/Types.java
  1. … 155 more files in changeset.
DRILL-7272: Drill Metastore Read / Write API and Drill Iceberg Metastore implementation

1. Drill Metastore Read / Write API.

2. Drill Iceberg Metastore implementation in iceberg-metastore module.

3. Patches Guava Preconditions class for Apache Iceberg.

4. General refactoring.

5. Unit tests.

6. Documentation.

    • -11
    • +20
    ./apache/drill/common/config/DrillConfig.java
    • -0
    • +86
    ./apache/drill/common/util/GuavaPatcher.java
  1. … 96 more files in changeset.