Clone Tools
  • last updated 20 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-7724: Refactor metadata controller batch

Also changed for (;;) infinite loops to

while (true) as preferred by IntelliJ.

  1. … 25 more files in changeset.
DRILL-6604: Upgrade Drill Hive client to Hive3.1 version

closes #2038

    • -18
    • +21
    ./org/apache/drill/exec/store/hive/HiveUtilities.java
    • -0
    • +33
    ./org/apache/logging/log4j/util/Strings.java
  1. … 14 more files in changeset.
DRILL-7330: Implement metadata usage for all format plugins

  1. … 59 more files in changeset.
DRILL-7620: Fix plugin mutability issues

A recent commit made the plugin registry more strict about

the rule that, once a plugin is registered, it must be

immutable. A flaw enforcing that rule in the UI put the

registry in an inconsistent state.

Also

* Registry-specific errors

* Push more operations from UI layer into registry

* Clean up semantics of "resolve" for plugins

* Add more unit tests

* Better handling of "bad" plugins

* Force plugin names to lower case

* Fix comparison bugs in some format plugins

  1. … 95 more files in changeset.
DRILL-7590: Refactor plugin registry

Major cleanup of the plugin registry to split it into components

in preparation for a proper plugin API.

Better coordinates the named and ephemeral plugin caches.

Cleans up the registry API. Sharpens rules for modifying

plugin configs.

closes #1988

  1. … 163 more files in changeset.
DRILL-7634: Rollup of code cleanup changes

Collection of code cleanup changes. The most significant

is to create constants for function names.

closes #2020

  1. … 128 more files in changeset.
DRILL-7530: Fix class names in loggers

1. Fix incorrect class names for loggers.

2. Minor code cleanup.

closes #1957

  1. … 52 more files in changeset.
DRILL-7406: Update Calcite to 1.21.0

1. DRILL-7386 - added tests to TestHiveStructs.

2. DRILL-4527 - the DrillAvgVarianceConvertlet can't be removed without test failures.

3. DRILL-6215 - switched to prepared statement in JdbcRecordReader.

4. DRILL-6905 - added test into TestExampleQueries.

5. DRILL-7415 - Fixed jdbc show tables when 2 tables with same name are present in different schemas.

6. DRILL-7340 - Fixed jdbc filter pushdown when few jdbc datasources enabled.

7. Split SqlConverter into multiple source files.

8. Minor refactorings for jdbc and other places.

closes #1940

  1. … 54 more files in changeset.
DRILL-7359: Add support for DICT type in RowSet Framework

closes #1870

  1. … 82 more files in changeset.
DRILL-7254: Read Hive union w/o nulls

  1. … 18 more files in changeset.
DRILL-7252: Read Hive map using Dict<K,V> vector

  1. … 13 more files in changeset.
DRILL-7337: Add vararg UDFs support

  1. … 35 more files in changeset.
DRILL-7314: Use TupleMetadata instead of concrete implementation

1. Add ser / de implementation for TupleMetadata interface based on types.

2. Replace TupleSchema usage where possible.

3. Move patcher classes into commons.

4. Upgrade some dependencies and general refactoring.

  1. … 39 more files in changeset.
DRILL-7315: Revise precision and scale order in the method arguments

  1. … 28 more files in changeset.
DRILL-7307: casthigh for decimal type can lead to the issues with VarDecimalHolder

- Fixed code-gen for VarDecimal type

- Fixed code-gen issue with nullable holders for simple cast functions

with passed constants as arguments.

- Code-gen now honnoring DataType.Optional type defined by UDF for

NULL-IF-NULL functions.

  1. … 9 more files in changeset.
DRILL-7313: Use Hive schema for MaprDB native reader when field was empty

- Added all_text_mode option for hive maprDB Json

- Improved logic to convert Hive's schema into Drill's one

- Added unit tests for schema conversion

    • -0
    • +100
    ./org/apache/drill/exec/store/hive/HiveUtilities.java
  1. … 26 more files in changeset.
DRILL-7271: Refactor Metadata interfaces and classes to contain all needed information for the File based Metastore

  1. … 119 more files in changeset.
DRILL-7253: Read Hive struct w/o nulls

  1. … 15 more files in changeset.
DRILL-7268: Read Hive array with parquet native reader

1. Fixed preserving of group originalType for projected schema

in DrillParquetReader

2. Added reading of LIST logical type to DrillParquetGroupConverter.

Intermediate noop converter used to skip writing for next nested

repeated field after recognition of parent field as LIST. For this

skipRepeated 'true' passed to child converter's constructor.

close apache/drill#1805

  1. … 6 more files in changeset.
DRILL-7251: Read Hive array w/o nulls

1. HiveFieldConverter replaced by Hive writers for primitives

2. Created HiveValueWriterFactory and HiveListWriter to implement arrays support

4. Readers generation replaced by HiveDefaultRecordReader and HiveTextRecordReader

5. Few reader initializers replaced by one

6. Added method to repeated vardecimal writer

7. Minor fix for array column in View

    • -0
    • +38
    ./org/apache/drill/exec/store/hive/writers/HiveValueWriter.java
  1. … 39 more files in changeset.
DRILL-7196: Queries are still runnable on disabled plugins

- Storage client is not created anymore for disabled plugins

- GET "/storage/{name}.json" endpoint now working with

plugin configuration directly, without client instantination.

It have increased UI responsitivity.

- Hbase and mongo base test classes refactored to honor enabled

plugin attribute

- Fixed path contructor for mongo test datasets:

Now it is cross-platform

- Fixed test json files format which using plugin definitions

- Code cleanup

  1. … 106 more files in changeset.
DRILL-7062: Initial implementation of run-time rowgroup pruning closes #1738

  1. … 23 more files in changeset.
DRILL-7115: Improve Hive schema show tables performance

1. To make SHOW TABLES for Hive schema work much faster, additional Drill

feature of showing only accesible tables when Storage-Based authorization

is enabled was sacrificed. Now the behaviour matches to Hive/Beeline, all

tables will be shown despite of accessibility. For details about previous

show tables results, check description of DRILL-540.

2. In HiveDatabaseSchema implemented faster getTableNamesAndTypes() method

and removed bulk related code.

3. Deprecated bulk related options and removed bulk code from AbstractSchema,

DrillHiveMetastoreClient.

4. For 8000 Hive tables query returned in 1.8 seconds, for combination of

4000 tables and 8000 views query returned in 2.3 seconds. Note, that

after first query table names will be cached and next queries will perform

in less than 1 sec.

5. Refactored WorkspaceSchemaFactory's getTableNamesAndTypes()

method to reuse existing getViews() method.

6. DrillHiveMetastoreClient was refactored. Classes were unnested and enclosed

within client package with restricted visibility. Also was updated cache

values type to avoid unnecessarry List to Set back and forth conversions.

Client creation methods moved to separate class. So the new package

exposes only factory and client class.

closes #1706

    • -0
    • +72
    ./org/apache/drill/exec/store/hive/client/TableName.java
  1. … 6 more files in changeset.
DRILL-7089: Implement caching for TableMetadataProvider at query level and adapt statistics to use Drill metastore API

closes #1728

  1. … 49 more files in changeset.
DRILL-6852: Adapt current Parquet Metadata cache implementation to use Drill Metastore API

Co-authored-by: Volodymyr Vysotskyi <vvovyk@gmail.com>

Co-authored-by: Vitalii Diravka <vitalii@apache.org>

close apache/drill#1646

  1. … 65 more files in changeset.
DRILL-6927: Avoid double conversion from impala timestamp when hive native parquet reader is used closes #1655

  1. … 1 more file in changeset.
DRILL-5603: Replace String file paths to Hadoop Path - replaced all String path representation with org.apache.hadoop.fs.Path - added PathSerDe.Se JSON serializer - refactoring of DFSPartitionLocation code by leveraging existing listPartitionValues() functionality

closes #1657

  1. … 78 more files in changeset.
DRILL-7019: Add check for redundant imports

close apache/drill#1629

  1. … 23 more files in changeset.
DRILL-6944: UnsupportedOperationException thrown for view over MapR-DB binary table

1. Added persistence of MAP key and value types in Drill views (affects .view.drill file) for avoiding cast problems in future.

2. Preserved backward compatibility of older view files by treating untyped maps as ANY.

closes #1602

  1. … 5 more files in changeset.
DRILL-6929: Exclude maprfs jar for default profile closes #1586

  1. … 6 more files in changeset.