Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-7330: Implement metadata usage for all format plugins

    • -2
    • +65
    ./AbstractGroupScanWithMetadata.java
  1. … 58 more files in changeset.
DRILL-7620: Fix plugin mutability issues

A recent commit made the plugin registry more strict about

the rule that, once a plugin is registered, it must be

immutable. A flaw enforcing that rule in the UI put the

registry in an inconsistent state.

Also

* Registry-specific errors

* Push more operations from UI layer into registry

* Clean up semantics of "resolve" for plugins

* Add more unit tests

* Better handling of "bad" plugins

* Force plugin names to lower case

* Fix comparison bugs in some format plugins

  1. … 101 more files in changeset.
DRILL-7590: Refactor plugin registry

Major cleanup of the plugin registry to split it into components

in preparation for a proper plugin API.

Better coordinates the named and ephemeral plugin caches.

Cleans up the registry API. Sharpens rules for modifying

plugin configs.

closes #1988

  1. … 163 more files in changeset.
DRILL-7634: Rollup of code cleanup changes

Collection of code cleanup changes. The most significant

is to create constants for function names.

closes #2020

  1. … 121 more files in changeset.
DRILL-7491: Incorrect count() returned for complex types in parquet

closes #1955

    • -12
    • +29
    ./AbstractGroupScanWithMetadata.java
  1. … 4 more files in changeset.
DRILL-7506: Simplify code gen error handling

Pushes code gen error handling close to the code gen itself to

allow clearer error messages. Doing so avoids the need to bubble

code gen exceptions up the call stack, resulting in cleaner

operator code.

closes #1948

  1. … 40 more files in changeset.
DRILL-7472: Fix ser / de for sys and information_schema schemas queries

closes #1925

  1. … 8 more files in changeset.
DRILL-7479: Partial fixes for metadata parameterized type issues

See DRILL-7479 and DRILL-7480 for an explanation. Adds generic

type parameters where needed to avoid the need to supporess

warnings. However, type parameters are probably not needed

at all and should be removed in the future for reasons explained

in DRILL-7480.

closes #1923

  1. … 37 more files in changeset.
DRILL-7450: Improve performance for ANALYZE command

- Implement two-phase aggregation for the lowest metadata aggregate to optimize performance

- Allow using complex functions with hash aggregate

- Use hash aggregation for PHASE_1of2 for ANALYZE to reduce memory usage and avoid sorting non-aggregated data

- Add sort above hash aggregation to fix correctness of merge exchange and stream aggregate

closes #1907

    • -14
    • +22
    ./AbstractGroupScanWithMetadata.java
  1. … 59 more files in changeset.
DRILL-7446: Fix Eclipse compilation issue in AbstractParquetGroupScan

Adds dummy parameter types to several files to avoid compilation errors

when loading Drill into Eclipse.

    • -43
    • +46
    ./AbstractGroupScanWithMetadata.java
  1. … 2 more files in changeset.
DRILL-7418: MetadataDirectGroupScan improvements

1. Replaced files listing with selection root information to reduce query plan size in MetadataDirectGroupScan.

2. Fixed MetadataDirectGroupScan ser / de issues.

3. Added PlanMatcher to QueryBuilder for more convenient plan matching.

4. Re-written TestConvertCountToDirectScan to use ClusterTest.

5. Refactoring and code clean up.

  1. … 10 more files in changeset.
DRILL-7314: Use TupleMetadata instead of concrete implementation

1. Add ser / de implementation for TupleMetadata interface based on types.

2. Replace TupleSchema usage where possible.

3. Move patcher classes into commons.

4. Upgrade some dependencies and general refactoring.

  1. … 40 more files in changeset.
DRILL-7273: Introduce operators for handling metadata

closes #1886

    • -23
    • +72
    ./AbstractGroupScanWithMetadata.java
  1. … 154 more files in changeset.
DRILL-7271: Refactor Metadata interfaces and classes to contain all needed information for the File based Metastore

    • -104
    • +264
    ./AbstractGroupScanWithMetadata.java
    • -79
    • +0
    ./FileSystemMetadataProviderManager.java
    • -40
    • +0
    ./ParquetTableMetadataProvider.java
    • -177
    • +0
    ./SimpleFileTableMetadataProvider.java
  1. … 112 more files in changeset.
DRILL-7199: Optimize population of metadata for non-interesting columns closes #1771

    • -8
    • +40
    ./AbstractGroupScanWithMetadata.java
    • -0
    • +6
    ./SimpleFileTableMetadataProvider.java
  1. … 6 more files in changeset.
DRILL-7098: File Metadata Metastore Plugin closes #1754

    • -1
    • +1
    ./SimpleFileTableMetadataProvider.java
    • -32
    • +0
    ./TableMetadataProviderBuilder.java
  1. … 54 more files in changeset.
DRILL-7062: Initial implementation of run-time rowgroup pruning closes #1738

    • -10
    • +33
    ./AbstractGroupScanWithMetadata.java
  1. … 24 more files in changeset.
DRILL-6965: Implement schema table function parameter

1. Added common schema table function parameter with can be used as single unit or with format plugin table function parameters.

2. Allowed creating schema without columns, in case if user needs only to indicate table properties.

3. Added unit tests.

closes #1777

    • -6
    • +6
    ./FileSystemMetadataProviderManager.java
    • -2
    • +3
    ./SimpleFileTableMetadataProvider.java
  1. … 28 more files in changeset.
DRILL-7089: Implement caching for TableMetadataProvider at query level and adapt statistics to use Drill metastore API

closes #1728

    • -8
    • +21
    ./AbstractGroupScanWithMetadata.java
    • -0
    • +79
    ./FileSystemMetadataProviderManager.java
    • -0
    • +59
    ./MetadataProviderManager.java
    • -0
    • +170
    ./SimpleFileTableMetadataProvider.java
    • -0
    • +32
    ./TableMetadataProviderBuilder.java
  1. … 42 more files in changeset.
DRILL-6852: Adapt current Parquet Metadata cache implementation to use Drill Metastore API

Co-authored-by: Volodymyr Vysotskyi <vvovyk@gmail.com>

Co-authored-by: Vitalii Diravka <vitalii@apache.org>

close apache/drill#1646

    • -0
    • +716
    ./AbstractGroupScanWithMetadata.java
    • -0
    • +69
    ./ParquetMetadataProvider.java
    • -0
    • +40
    ./ParquetTableMetadataProvider.java
    • -0
    • +88
    ./TableMetadataProvider.java
  1. … 58 more files in changeset.
DRILL-7068: Support memory adjustment framework for resource management with Queues. closes #1677

  1. … 34 more files in changeset.
DRILL-6952: Host compliant text reader on the row set framework

The result set loader allows controlling batch sizes. The new scan framework

built on top of that framework handles projection, implicit columns, null

columns and more. This commit converts the "new" ("compliant") text reader

to use the new framework. Options select the use of the V2 ("new") or V3

(row-set based) versions. Unit tests demonstrate V3 functionality.

closes #1683

  1. … 56 more files in changeset.
DRILL-5603: Replace String file paths to Hadoop Path - replaced all String path representation with org.apache.hadoop.fs.Path - added PathSerDe.Se JSON serializer - refactoring of DFSPartitionLocation code by leveraging existing listPartitionValues() functionality

closes #1657

  1. … 81 more files in changeset.
Drill 6735: Implement Semi-Join for the Hash-Join operator (#1522)

  1. … 17 more files in changeset.
DRILL-1328: Support table statistics

  1. … 51 more files in changeset.
DRILL-6381: Address review comments (part 2): fix formatting issues and add javadoc.

  1. … 29 more files in changeset.
DRILL-6773: The renamed schema with aliases is not shown for queries on empty directories

closes #1492

  1. … 17 more files in changeset.
DRILL-6381: (Part 5) Update Javadoc for a few interfaces.

  1. … 3 more files in changeset.
DRILL-6721: Fix SchemalessScan plan serialization / deserialization

  1. … 1 more file in changeset.
DRILL-6422: Replace guava imports with shaded ones

  1. … 975 more files in changeset.