Clone Tools
  • last updated 14 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-7168: Implement ALTER SCHEMA ADD / REMOVE commands

    • -0
    • +2
    ./parser/CompoundIdentifierConverter.java
  1. … 13 more files in changeset.
DRILL-7337: Add vararg UDFs support

    • -3
    • +5
    ./DrillSqlAggOperatorWithoutInference.java
    • -5
    • +3
    ./DrillSqlOperatorWithoutInference.java
    • -0
    • +59
    ./VarArgOperandTypeChecker.java
  1. … 32 more files in changeset.
DRILL-7316: Move classes from org.apache.drill.metastore into org.apache.drill.exec.metastore package in java-exec module

  1. … 32 more files in changeset.
DRILL-7315: Revise precision and scale order in the method arguments

  1. … 28 more files in changeset.
DRILL-7313: Use Hive schema for MaprDB native reader when field was empty

- Added all_text_mode option for hive maprDB Json

- Improved logic to convert Hive's schema into Drill's one

- Added unit tests for schema conversion

  1. … 27 more files in changeset.
DRILL-7271: Refactor Metadata interfaces and classes to contain all needed information for the File based Metastore

  1. … 119 more files in changeset.
DRILL-7253: Read Hive struct w/o nulls

    • -1
    • +12
    ./DrillCalciteSqlOperatorWrapper.java
  1. … 15 more files in changeset.
DRILL-7250: Query with CTE fails when its name matches to the table name without access

  1. … 3 more files in changeset.
DRILL-7050: RexNode convert exception in sub-query closes #1770

  1. … 3 more files in changeset.
DRILL-7183: TPCDS query 10, 35, 69 take longer with sf 1000 when Statistics are disabled. This commit reverts the changes done for DRILL-6997.

  1. … 5 more files in changeset.
DRILL-6988. Utility of the too long error message when syntax error

- Adding Drill wrapper around SqlparseException to customize produced by Calcite messages

- Fix Drill SQL parse exception formatter to calculate proper position for "^" character

closes #1753

    • -0
    • +105
    ./parser/impl/DrillSqlParseException.java
  1. … 1 more file in changeset.
DRILL-6974: SET option command modification

- ALTER ... RESET ... and ALTER ... SET ... sub-parsers separated to 2

different SqlCall classes with same parent SqlSetOption

- parserImpls modified to handle new syntax of ALTER... SET...

expresion:

a) ALTER ... SET option.name - option.value - setting option value

b) ALTER ... SET option.name - display option value

- Handler for SqlSetOption separated to SetOptionHandler and

ResetOptionhandler for better representation of handled statements

- Base abstract class AbstractSqlSetHandler created to not repeat

shared implementation of same functions

- SetOptionHandler covered with unit tests for each statement

form.

Fix issues stated in the review

closes #1763

    • -0
    • +86
    ./handlers/AbstractSqlSetHandler.java
    • -0
    • +73
    ./handlers/ResetOptionHandler.java
    • -73
    • +46
    ./handlers/SetOptionHandler.java
    • -2
    • +2
    ./parser/CompoundIdentifierConverter.java
    • -0
    • +103
    ./parser/DrillSqlResetOption.java
    • -0
    • +75
    ./parser/DrillSqlSetOption.java
  1. … 4 more files in changeset.
DRILL-7160: e.q.max_rows QUERY-level option shown even if not set

The fix is to force setting to zero IFF autoLimit was intended to be set originally but is inapplicable; such as 'SHOW DATABASES'. If autolimit was not intended to be applied, setting the value to zero is not required.

WebUI is also patched to not show the zero value set in such scenarios.

  1. … 1 more file in changeset.
DRILL-7159: Fix typeString method to return correct name for MAP (aka STRUCT) closes #1741

  1. … 2 more files in changeset.
DRILL-7157: Wrap SchemaParsingException into UserException when creating schema

closes #1740

DRILL-7048: Implement JDBC Statement.setMaxRows() with System Option

This introduces support for JDBC's Statement.setMaxRows(int) API, which can help Drill execute a query much faster if it knows that not ALL the records in the resultset will be consumed upfront.

This Commit introduces the core changes to support the feature within Drill's execution engine

Protobuf Changes

1. RunQuery: Added "autolimit_rowcount"

2. QueryProfile: Added "autoLimit"

3. Regenerated Java and C++ client files

REST API support

1. Support for REST server to interpret a submitted query and also for rendering this information for an executed query

2. Updates to the Freemarker templates (for WebUI)

3. Safety check within Javascript (for WebUI)

JDBC API support

1. Introduces backend execution of 'ALTER SESSION' to apply the auto-limiting of resultset size

2. Added Unit Tests for PreparedStatement and Statement objects

3. Added getter setter methods to be skipped in testing for org.apache.drill.jdbc.test.Drill2489CallsAfterCloseThrowExceptionsTest.testclosedPreparedStmtOfOpenConnMethodsThrowRight()

Updates based on review comments

Additional Updates

Test Cleanup

1. Revert Drill2489 hack

2. Formatting in *StatementTest

3. Removal f redundant `statement.close()`

4. Manage new Exception thrown when setting invalid maxRow values

Final updates

1. Test changes

2. Trim trailing spaces in auto-limit value (Javascript)

3. Before & After annotations to synchronize changes to system values for MaxRows(auto-limit)

Reorganized tests due to synchronized locking

Removed conflicting JsonCreator in QueryWrapper

Additional test cleanup

closes #1714

  1. … 34 more files in changeset.
DRILL-7143: Support default value for empty columns

Modifies the prior work to add default values for columns. The prior work added defaults

when the entire column is missing from a reader (the old Nullable Int column). The Row

Set mechanism now will also "fill empty" slots with the default value.

Added default support for the column writers. The writers automatically obtain the

default value from the column schema. The default can also be set explicitly on

the column writer.

Updated the null column mechanism to use this feature rather than the ad-hoc

implemention in the prior commit.

Semantics changed a bit. Only Required columns take a default. The default value

is ignored or nullable columns since nullable columns already have a file default: NULL.

Other changes:

* Updated the CSV-with-schema tests to illustrate the new behavior.

* Made multiple fixes for Boolean and Decimal columns and added unit tests.

* Upgraded Fremarker to version 2.3.28 to allow use of the continue statement.

* Reimplemented the Bit column reader and writer to use the BitVector directly since this vector is rather special.

* Added get/set Boolean methods for column accessors

* Moved the BooleanType class to the common package

* Added more CSV unit tests to explore decimal types, booleans, and defaults

* Add special handling for blank fields in from-string conversions

* Added options to the conversion factory to specify blank-handling behavior.

CSV uses a mapping of blanks to null (nullable) or default value (non-nullable)

closes #1726

  1. … 72 more files in changeset.
DRILL-7138: Implement command to describe schema for table

closes #1719

    • -0
    • +1
    ./parser/CompoundIdentifierConverter.java
  1. … 3 more files in changeset.
DRILL-7110: Skip writing profile when an ALTER SESSION is executed (#1703)

Allows (by default) for `ALTER SESSION SET <option>=<value>` queries to NOT be writen to the profile store. This would avoid the risk of potentially adding up to a lot of profiles being written unnecessarily, since those changes are also reflected on the queries that follow.

  1. … 5 more files in changeset.
DRILL-7125: REFRESH TABLE METADATA fails after upgrade from Drill 1.13.0 to Drill 1.15.0

    • -15
    • +18
    ./handlers/RefreshMetadataHandler.java
  1. … 1 more file in changeset.
DRILL-7124: Fix logger for ShowFilesHandler

closes #1705

DRILL-7115: Improve Hive schema show tables performance

1. To make SHOW TABLES for Hive schema work much faster, additional Drill

feature of showing only accesible tables when Storage-Based authorization

is enabled was sacrificed. Now the behaviour matches to Hive/Beeline, all

tables will be shown despite of accessibility. For details about previous

show tables results, check description of DRILL-540.

2. In HiveDatabaseSchema implemented faster getTableNamesAndTypes() method

and removed bulk related code.

3. Deprecated bulk related options and removed bulk code from AbstractSchema,

DrillHiveMetastoreClient.

4. For 8000 Hive tables query returned in 1.8 seconds, for combination of

4000 tables and 8000 views query returned in 2.3 seconds. Note, that

after first query table names will be cached and next queries will perform

in less than 1 sec.

5. Refactored WorkspaceSchemaFactory's getTableNamesAndTypes()

method to reuse existing getViews() method.

6. DrillHiveMetastoreClient was refactored. Classes were unnested and enclosed

within client package with restricted visibility. Also was updated cache

values type to avoid unnecessarry List to Set back and forth conversions.

Client creation methods moved to separate class. So the new package

exposes only factory and client class.

closes #1706

  1. … 20 more files in changeset.
DRILL-6965: Implement schema table function parameter

1. Added common schema table function parameter with can be used as single unit or with format plugin table function parameters.

2. Allowed creating schema without columns, in case if user needs only to indicate table properties.

3. Added unit tests.

closes #1777

  1. … 28 more files in changeset.
DRILL-7089: Implement caching for TableMetadataProvider at query level and adapt statistics to use Drill metastore API

closes #1728

  1. … 48 more files in changeset.
DRILL-7085: Fix table-path check in AnalyzeTableHandler

closes #1685

DRILL-7063: Seperate metadata cache file into summary, file metadata

closes #1723

    • -1
    • +1
    ./handlers/RefreshMetadataHandler.java
  1. … 18 more files in changeset.
DRILL-7200: Update Calcite to 1.19.0 / 1.20.0

    • -33
    • +33
    ./parser/UnsupportedOperatorsVisitor.java
  1. … 41 more files in changeset.
DRILL-5603: Replace String file paths to Hadoop Path - replaced all String path representation with org.apache.hadoop.fs.Path - added PathSerDe.Se JSON serializer - refactoring of DFSPartitionLocation code by leveraging existing listPartitionValues() functionality

closes #1657

    • -19
    • +9
    ./handlers/AnalyzeTableHandler.java
    • -2
    • +2
    ./handlers/RefreshMetadataHandler.java
  1. … 82 more files in changeset.
DRILL-7058: Refresh command to support subset of columns closes #1666

    • -1
    • +35
    ./handlers/RefreshMetadataHandler.java
    • -1
    • +1
    ./parser/CompoundIdentifierConverter.java
  1. … 4 more files in changeset.
DRILL-6997: Semijoin is changing the join ordering for some tpcds queries.

close apache/drill#1620

  1. … 6 more files in changeset.