Clone Tools
  • last updated 28 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-7761: Drill fails with OOM for the case of large filter conditions

    • -1
    • +3
    ./store/parquet/FilePushDownFilter.java
  1. … 1 more file in changeset.
DRILL-7759: Code compilation exception for queries containing (untyped) NULL

  1. … 3 more files in changeset.
DRILL-7750: Drill fails to read KeyStore password from Credential provider (#2088)

DRILL-7744: Move Filters from HTTP Storage Plugin to Drill Core

    • -0
    • +384
    ./store/base/filter/ConstantHolder.java
    • -0
    • +234
    ./store/base/filter/ExprNode.java
    • -0
    • +149
    ./store/base/filter/FilterPushDownListener.java
    • -0
    • +323
    ./store/base/filter/FilterPushDownStrategy.java
    • -0
    • +322
    ./store/base/filter/FilterPushDownUtils.java
    • -0
    • +98
    ./store/base/filter/RelOp.java
  1. … 8 more files in changeset.
DRILL-7739: Allow implicit casts from required to nullable data type

closes #2080

    • -8
    • +32
    ./expr/ExpressionTreeMaterializer.java
  1. … 1 more file in changeset.
DRILL-7741: Columns are missing when using convert_from function

closes #2081

    • -0
    • +10
    ./physical/impl/project/ProjectRecordBatch.java
  1. … 1 more file in changeset.
DRILL-7707: Unable to analyze table metadata is it resides in non-writable workspace

  1. … 3 more files in changeset.
Revisions from review comments

DRILL-7734: Revise the result set reader

Revised into two forms: push (for streaming JSON results) and

pull (for one operator reading from another).

closes #2077

    • -0
    • +105
    ./physical/resultSet/PullResultSetReader.java
    • -0
    • +49
    ./physical/resultSet/PushResultSetReader.java
    • -13
    • +6
    ./physical/resultSet/ResultSetCopier.java
    • -99
    • +0
    ./physical/resultSet/ResultSetReader.java
    • -0
    • +134
    ./physical/resultSet/impl/PullResultSetReaderImpl.java
    • -0
    • +107
    ./physical/resultSet/impl/PushResultSetReaderImpl.java
  1. … 14 more files in changeset.
DRILL-7730: Improve web query efficiency

Implements a direct transfer of batches from Screen to web client.

Cleans up web client query processing to avoid duplicate schema

info.

Much related code cleanup.

    • -0
    • +1
    ./client/InvalidConnectionInfoException.java
    • -35
    • +20
    ./physical/impl/ScreenCreator.java
    • -0
    • +124
    ./physical/impl/materialize/QueryDataPackage.java
    • -6
    • +9
    ./record/AbstractSingleRecordBatch.java
  1. … 24 more files in changeset.
DRILL-7725: Updates to the EVF2 framework

* Supports internal implicit columns

* Better support for standard conversions

* Handle several reader corner cases

* Simplified file reader

closes #2073

    • -1
    • +1
    ./physical/impl/scan/ScanOperatorExec.java
    • -0
    • +208
    ./physical/impl/scan/v3/FixedReceiver.java
    • -1
    • +35
    ./physical/impl/scan/v3/ScanLifecycleBuilder.java
    • -0
    • +10
    ./physical/impl/scan/v3/SchemaNegotiator.java
    • -5
    • +97
    ./physical/impl/scan/v3/file/FileDescrip.java
    • -0
    • +99
    ./physical/impl/scan/v3/file/ImplicitColumnMarker.java
    • -0
    • +433
    ./physical/impl/scan/v3/file/ImplicitColumnResolver.java
  1. … 67 more files in changeset.
DRILL-7724: Refactor metadata controller batch

Also changed for (;;) infinite loops to

while (true) as preferred by IntelliJ.

    • -3
    • +0
    ./physical/config/StreamingAggregate.java
    • -1
    • +1
    ./physical/impl/scan/ScanOperatorExec.java
    • -1
    • +1
    ./physical/impl/xsort/ExternalSortBatch.java
    • -16
    • +18
    ./planner/physical/AggPrelBase.java
    • -1
    • +1
    ./store/easy/json/parser/ArrayParser.java
    • -1
    • +1
    ./store/easy/json/parser/DummyValueParser.java
    • -2
    • +2
    ./store/easy/json/parser/JsonValueParser.java
    • -2
    • +1
    ./store/easy/json/parser/ObjectParser.java
  1. … 11 more files in changeset.
DRILL-7717: Support Mongo extended types in V2 JSON loader

Adds support for Mongo extended types to the JSON loader.

Refactors the JSON code to make such extensions easier.

Added support for provided schema with extended types.

    • -0
    • +107
    ./store/easy/json/extended/BaseExtendedValueParser.java
    • -0
    • +282
    ./store/easy/json/extended/ExtendedTypeFieldFactory.java
    • -0
    • +51
    ./store/easy/json/extended/ExtendedTypeNames.java
    • -0
    • +150
    ./store/easy/json/extended/MongoBinaryValueParser.java
    • -0
    • +89
    ./store/easy/json/extended/MongoDateValueParser.java
    • -0
    • +48
    ./store/easy/json/extended/SimpleExtendedValueParser.java
    • -0
    • +142
    ./store/easy/json/extended/package-info.java
    • -0
    • +261
    ./store/easy/json/loader/BaseFieldFactory.java
    • -69
    • +0
    ./store/easy/json/loader/BigIntListener.java
    • -62
    • +0
    ./store/easy/json/loader/BooleanListener.java
    • -66
    • +0
    ./store/easy/json/loader/DoubleListener.java
    • -0
    • +66
    ./store/easy/json/loader/EmptyArrayFieldParser.java
  1. … 83 more files in changeset.
DRILL-7711: Add data path, parameter filter pushdown to HTTP plugin

Adds an option to specify the path to data so the plugin will

ignore REST message "overhead" except the actual data.

Allows specifying HTTP URL parameters as filter push-downs from

SQL.

* Revised scan cost model to allow a scan more freedom

to define cost. Needed to ensure that filter push-down

is actually accepted by Calcite.

* Reduced size of HTTP scan plan by including on the one

needed connection config rather than all of them.

* Revised URL building so that the proxy sees the full

URL after adding parameters, etc.

* Many code refinements.

* Added more details to README

    • -3
    • +9
    ./physical/resultSet/impl/ColumnState.java
    • -38
    • +36
    ./planner/logical/DrillScanRel.java
    • -0
    • +7
    ./store/easy/json/loader/JsonLoaderImpl.java
    • -0
    • +1
    ./store/easy/json/parser/MessageParser.java
    • -25
    • +114
    ./store/easy/json/parser/RootParser.java
  1. … 25 more files in changeset.
DRILL-7713: Upgrade misc libraries which outdated versions have reported vulnerabilities

1. Jackson

2. Retrofit

3. Commons-beanutils

4. Xalan

5. Xerdes

6. Commons-codec

7. Snakeyaml

8. Metadata-extractor

9. Protostuff

    • -6
    • +6
    ./store/pojo/DynamicPojoRecordReader.java
  1. … 12 more files in changeset.
DRILL-7703: Support for 3+D arrays in EVF JSON loader

Revises the EVF-based JSON loader to support nested

repeated lists.

    • -8
    • +9
    ./store/easy/json/loader/JsonLoaderImpl.java
    • -0
    • +2
    ./store/easy/json/loader/JsonLoaderOptions.java
    • -125
    • +245
    ./store/easy/json/loader/TupleListener.java
    • -0
    • +8
    ./store/easy/json/parser/TokenIterator.java
  1. … 7 more files in changeset.
DRILL-7701: EVF V2 Scan Framework

Revises the scan framework to use the revised schema resolution

introduced in DRILL-7696.

    • -16
    • +6
    ./physical/impl/scan/ReaderState.java
    • -33
    • +19
    ./physical/impl/scan/ScanOperatorExec.java
    • -0
    • +101
    ./physical/impl/scan/v3/ManagedReader.java
    • -0
    • +38
    ./physical/impl/scan/v3/ReaderFactory.java
    • -0
    • +271
    ./physical/impl/scan/v3/ScanLifecycleBuilder.java
    • -0
    • +217
    ./physical/impl/scan/v3/SchemaNegotiator.java
    • -0
    • +51
    ./physical/impl/scan/v3/file/FileReaderFactory.java
    • -0
    • +106
    ./physical/impl/scan/v3/file/FileScanLifecycle.java
    • -0
    • +84
    ./physical/impl/scan/v3/file/FileScanLifecycleBuilder.java
    • -0
    • +48
    ./physical/impl/scan/v3/file/FileSchemaNegotiator.java
    • -0
    • +98
    ./physical/impl/scan/v3/file/FileSchemaNegotiatorImpl.java
    • -0
    • +73
    ./physical/impl/scan/v3/file/ImplicitFileColumnsHandler.java
  1. … 43 more files in changeset.
DRILL-7702: Update shaded guava

  1. … 10 more files in changeset.
DRILL-7692: Add dropdown to select default storage

  1. … 1 more file in changeset.
DRILL-7702: Update ZooKeeper and Curator, exclude org.codehaus.jackson

  1. … 10 more files in changeset.
DRILL-7437: Storage Plugin for Generic HTTP REST API

    • -6
    • +30
    ./store/easy/json/JSONRecordReader.java
  1. … 33 more files in changeset.
DRILL-6168: Revise format plugin table functions

Allows table functions to inherit properties from a

defined format plugin.

Also DRILL-7612: enforces immutability for all format plugins.

    • -1
    • +1
    ./planner/logical/ConvertCountToDirectScanRule.java
    • -5
    • +5
    ./planner/logical/DynamicDrillTable.java
    • -3
    • +7
    ./store/dfs/FormatPluginOptionExtractor.java
    • -53
    • +202
    ./store/dfs/FormatPluginOptionsDescriptor.java
    • -17
    • +22
    ./store/dfs/NamedFormatPluginConfig.java
    • -59
    • +94
    ./store/dfs/WorkspaceSchemaFactory.java
  1. … 32 more files in changeset.
DRILL-7603 and DRILL-7604: Add schema, options to REST query

Update and revision of work originally done by dobesv.

DRILL-7603: Allow default schema to be set for HTTP queries

DRILL-7604: Allow session options to be set in HTTP queries

Merges the above two. Separates running a REST query from the

JSON representation. Allows setting all option types from

a string (as required by DRILL-7604).

Added default schema to query profile query editor.

Made the two query editors a bit more similar visually,

but see DRILL-7697 for more work needed.

Added a utility to run a server for UI teseting without

a full build.

    • -22
    • +5
    ./server/options/BaseOptionManager.java
    • -22
    • +76
    ./server/options/OptionValue.java
    • -1
    • +0
    ./server/options/SessionOptionManager.java
    • -160
    • +98
    ./server/rest/QueryWrapper.java
    • -0
    • +215
    ./server/rest/RestQueryRunner.java
  1. … 9 more files in changeset.
DRILL-7696: EVF v2 scan schema resolution

Provides the mechanism to resolve the scan schema from a

projection list, provided schema, early reader schema and

actual reader schema.

    • -0
    • +74
    ./physical/impl/scan/v3/file/FileDescrip.java
    • -0
    • +166
    ./physical/impl/scan/v3/schema/AbstractSchemaTracker.java
    • -0
    • +255
    ./physical/impl/scan/v3/schema/DynamicSchemaFilter.java
    • -0
    • +410
    ./physical/impl/scan/v3/schema/ImplicitColumnResolver.java
    • -0
    • +220
    ./physical/impl/scan/v3/schema/MutableTupleMetadata.java
    • -0
    • +220
    ./physical/impl/scan/v3/schema/MutableTupleSchema.java
    • -0
    • +190
    ./physical/impl/scan/v3/schema/ProjectedColumn.java
    • -0
    • +165
    ./physical/impl/scan/v3/schema/ProjectionSchemaTracker.java
    • -0
    • +158
    ./physical/impl/scan/v3/schema/ScanProjectionParser.java
    • -0
    • +100
    ./physical/impl/scan/v3/schema/ScanSchemaConfigBuilder.java
    • -0
    • +370
    ./physical/impl/scan/v3/schema/ScanSchemaResolver.java
    • -0
    • +511
    ./physical/impl/scan/v3/schema/ScanSchemaTracker.java
  1. … 61 more files in changeset.
DRILL-7694: Register drill.queries.* counter metrics on Drillbit startup

closes #2050

    • -21
    • +13
    ./work/foreman/QueryStateProcessor.java
  1. … 1 more file in changeset.
DRILL-7675: Work around for partitions sender memory use

Adds an ad-hoc system/session option to limit partition sender

memory use. See DRILL-7686 for the underlying issue.

Also includes code cleanup and diagnostic tools.

closes #2047

    • -4
    • +5
    ./physical/impl/join/HashJoinBatch.java
    • -0
    • +5
    ./server/options/BaseOptionManager.java
    • -0
    • +1
    ./server/options/SystemOptionManager.java
    • -13
    • +10
    ./work/foreman/QueryManager.java
  1. … 4 more files in changeset.
DRILL-7683: Add "message parsing" to new JSON loader

Adds the ability to parse "extra" JSON around the data payload,

as often needed for a REST API.

closes #2045

    • -7
    • +1
    ./store/easy/json/loader/JsonLoader.java
    • -17
    • +88
    ./store/easy/json/loader/JsonLoaderImpl.java
    • -0
    • +9
    ./store/easy/json/loader/JsonLoaderOptions.java
    • -1
    • +7
    ./store/easy/json/parser/ErrorFactory.java
    • -0
    • +43
    ./store/easy/json/parser/MessageParser.java
    • -0
    • +34
    ./store/easy/json/parser/RootParser.java
    • -0
    • +141
    ./store/easy/json/parser/SimpleMessageParser.java
  1. … 3 more files in changeset.
DRILL-7673: View set query fails with NPE for non-existing option

closes #2043

  1. … 1 more file in changeset.
DRILL-7672: Make metadata type required when reading from / writing into Drill Metastore

1. Upgraded Iceberg version and removed unneeded code for In / NotIn Expressions.

2. Updated Metastore Read / Modify interfaces to support required metadata types:

a. introduced abstract Read / Modify classes with boilerplate code;

b. added delete operation with filter and metadata type;

c. added metadata type validator which checks supported metadata types for each component;

d. made purge operation terminal;

e. made necessary changes in REAME.md files.

3. Added / updated unit tests.

closes #2042

    • -2
    • +3
    ./metastore/analyze/FileMetadataInfoCollector.java
    • -15
    • +16
    ./store/ischema/RecordCollector.java
  1. … 38 more files in changeset.
DRILL-7663: Code refactor to DefaultFunctionResolver

    • -7
    • +4
    ./resolver/DefaultFunctionResolver.java