Clone Tools
  • last updated 12 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-7673: View set query fails with NPE for non-existing option

closes #2043

  1. … 1 more file in changeset.
DRILL-7672: Make metadata type required when reading from / writing into Drill Metastore

1. Upgraded Iceberg version and removed unneeded code for In / NotIn Expressions.

2. Updated Metastore Read / Modify interfaces to support required metadata types:

a. introduced abstract Read / Modify classes with boilerplate code;

b. added delete operation with filter and metadata type;

c. added metadata type validator which checks supported metadata types for each component;

d. made purge operation terminal;

e. made necessary changes in REAME.md files.

3. Added / updated unit tests.

closes #2042

    • -2
    • +7
    ./MetastoreDropTableMetadataHandler.java
  1. … 41 more files in changeset.
DRILL-7643: Fix issues with using columns with the same name as a reserved keyword

closes #2028

  1. … 7 more files in changeset.
DRILL-7620: Fix plugin mutability issues

A recent commit made the plugin registry more strict about

the rule that, once a plugin is registered, it must be

immutable. A flaw enforcing that rule in the UI put the

registry in an inconsistent state.

Also

* Registry-specific errors

* Push more operations from UI layer into registry

* Clean up semantics of "resolve" for plugins

* Add more unit tests

* Better handling of "bad" plugins

* Force plugin names to lower case

* Fix comparison bugs in some format plugins

  1. … 100 more files in changeset.
DRILL-7477: Allow passing table function parameters into ANALYZE statement

- Fix logical dir pruning when table function is used

closes #2005

    • -0
    • +171
    ./DrillTableInfo.java
    • -41
    • +14
    ./MetastoreAnalyzeTableHandler.java
  1. … 18 more files in changeset.
DRILL-7594: Remove unused DrillStoreRel

closes #1990

  1. … 2 more files in changeset.
DRILL-7565: ANALYZE TABLE ... REFRESH METADATA does not work for empty Parquet files

- Fixed ConvertMetadataAggregateToDirectScanRule rule to distinguish array columns correctly and proceed using other parquet metadata if such columns are found.

- Added new implicit column which signalizes whether the empty result is obtained during collecting metadata and helps to distinguish real data results from metadata results.

- Updated scan to return row with metadata if the above implicit column is present.

- Added unit tests for checking the correctness of both optional and required columns from empty files.

closes #1985

  1. … 13 more files in changeset.
DRILL-7590: Refactor plugin registry

Major cleanup of the plugin registry to split it into components

in preparation for a proper plugin API.

Better coordinates the named and ephemeral plugin caches.

Cleans up the registry API. Sharpens rules for modifying

plugin configs.

closes #1988

  1. … 161 more files in changeset.
DRILL-7634: Rollup of code cleanup changes

Collection of code cleanup changes. The most significant

is to create constants for function names.

closes #2020

  1. … 127 more files in changeset.
DRILL-7527: DROP METADATA doesn't work with table name starting with '/' inside workspace

closes #1958

  1. … 8 more files in changeset.
DRILL-7530: Fix class names in loggers

1. Fix incorrect class names for loggers.

2. Minor code cleanup.

closes #1957

  1. … 55 more files in changeset.
DRILL-7502: Invalid codegen for typeof() with UNION

Also fixes DRILL-6362: typeof() reports NULL for primitive

columns with a NULL value.

typeof() is meant to return "NULL" if a UNION has a NULL

value, but the column type when known, such as for non-UNION

columns.

Also fixes DRILL-7499: sqltypeof() function with an array returns

"ARRAY", not type. This was due to treating REPEATED like LIST.

Handling of the Union vector in code gen is problematic

with about three special cases. Existing code handled two

of the cases. This change handles the third case.

Figuring out the change required poking around quite a bit

of unclear code. Added comments and restructuring to make

that code a bit more clear.

The fix modified code gen for the Union Holder. It can now

"go back in time" to add the union reader at the point we

need it.

closes #1945

  1. … 54 more files in changeset.
DRILL-7450: Improve performance for ANALYZE command

- Implement two-phase aggregation for the lowest metadata aggregate to optimize performance

- Allow using complex functions with hash aggregate

- Use hash aggregation for PHASE_1of2 for ANALYZE to reduce memory usage and avoid sorting non-aggregated data

- Add sort above hash aggregation to fix correctness of merge exchange and stream aggregate

closes #1907

    • -19
    • +23
    ./MetastoreAnalyzeTableHandler.java
  1. … 59 more files in changeset.
DRILL-7406: Update Calcite to 1.21.0

1. DRILL-7386 - added tests to TestHiveStructs.

2. DRILL-4527 - the DrillAvgVarianceConvertlet can't be removed without test failures.

3. DRILL-6215 - switched to prepared statement in JdbcRecordReader.

4. DRILL-6905 - added test into TestExampleQueries.

5. DRILL-7415 - Fixed jdbc show tables when 2 tables with same name are present in different schemas.

6. DRILL-7340 - Fixed jdbc filter pushdown when few jdbc datasources enabled.

7. Split SqlConverter into multiple source files.

8. Minor refactorings for jdbc and other places.

closes #1940

  1. … 52 more files in changeset.
DRILL-7168: Implement ALTER SCHEMA ADD / REMOVE commands

  1. … 15 more files in changeset.
DRILL-7273: Introduce operators for handling metadata

closes #1886

    • -0
    • +536
    ./MetastoreAnalyzeTableHandler.java
    • -0
    • +97
    ./MetastoreDropTableMetadataHandler.java
  1. … 153 more files in changeset.
DRILL-7253: Read Hive struct w/o nulls

  1. … 17 more files in changeset.
DRILL-7183: TPCDS query 10, 35, 69 take longer with sf 1000 when Statistics are disabled. This commit reverts the changes done for DRILL-6997.

  1. … 5 more files in changeset.
DRILL-6974: SET option command modification

- ALTER ... RESET ... and ALTER ... SET ... sub-parsers separated to 2

different SqlCall classes with same parent SqlSetOption

- parserImpls modified to handle new syntax of ALTER... SET...

expresion:

a) ALTER ... SET option.name - option.value - setting option value

b) ALTER ... SET option.name - display option value

- Handler for SqlSetOption separated to SetOptionHandler and

ResetOptionhandler for better representation of handled statements

- Base abstract class AbstractSqlSetHandler created to not repeat

shared implementation of same functions

- SetOptionHandler covered with unit tests for each statement

form.

Fix issues stated in the review

closes #1763

    • -0
    • +86
    ./AbstractSqlSetHandler.java
    • -0
    • +73
    ./ResetOptionHandler.java
  1. … 8 more files in changeset.
DRILL-7159: Fix typeString method to return correct name for MAP (aka STRUCT) closes #1741

  1. … 2 more files in changeset.
DRILL-7157: Wrap SchemaParsingException into UserException when creating schema

closes #1740

DRILL-7143: Support default value for empty columns

Modifies the prior work to add default values for columns. The prior work added defaults

when the entire column is missing from a reader (the old Nullable Int column). The Row

Set mechanism now will also "fill empty" slots with the default value.

Added default support for the column writers. The writers automatically obtain the

default value from the column schema. The default can also be set explicitly on

the column writer.

Updated the null column mechanism to use this feature rather than the ad-hoc

implemention in the prior commit.

Semantics changed a bit. Only Required columns take a default. The default value

is ignored or nullable columns since nullable columns already have a file default: NULL.

Other changes:

* Updated the CSV-with-schema tests to illustrate the new behavior.

* Made multiple fixes for Boolean and Decimal columns and added unit tests.

* Upgraded Fremarker to version 2.3.28 to allow use of the continue statement.

* Reimplemented the Bit column reader and writer to use the BitVector directly since this vector is rather special.

* Added get/set Boolean methods for column accessors

* Moved the BooleanType class to the common package

* Added more CSV unit tests to explore decimal types, booleans, and defaults

* Add special handling for blank fields in from-string conversions

* Added options to the conversion factory to specify blank-handling behavior.

CSV uses a mapping of blanks to null (nullable) or default value (non-nullable)

closes #1726

  1. … 72 more files in changeset.
DRILL-7138: Implement command to describe schema for table

closes #1719

  1. … 6 more files in changeset.
DRILL-7110: Skip writing profile when an ALTER SESSION is executed (#1703)

Allows (by default) for `ALTER SESSION SET <option>=<value>` queries to NOT be writen to the profile store. This would avoid the risk of potentially adding up to a lot of profiles being written unnecessarily, since those changes are also reflected on the queries that follow.

  1. … 5 more files in changeset.
DRILL-7125: REFRESH TABLE METADATA fails after upgrade from Drill 1.13.0 to Drill 1.15.0

  1. … 1 more file in changeset.
DRILL-7124: Fix logger for ShowFilesHandler

closes #1705

DRILL-6965: Implement schema table function parameter

1. Added common schema table function parameter with can be used as single unit or with format plugin table function parameters.

2. Allowed creating schema without columns, in case if user needs only to indicate table properties.

3. Added unit tests.

closes #1777

  1. … 30 more files in changeset.
DRILL-7089: Implement caching for TableMetadataProvider at query level and adapt statistics to use Drill metastore API

closes #1728

  1. … 49 more files in changeset.
DRILL-7085: Fix table-path check in AnalyzeTableHandler

closes #1685

DRILL-7063: Seperate metadata cache file into summary, file metadata

closes #1723

  1. … 18 more files in changeset.