Clone Tools
  • last updated 25 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-7380: Query of a field inside of an array of structs returns null

1. Fixed parquet reader projection for Logical lists (DrillParquetReader.java)

2. Fixed projection pushdown for RexFieldAccess (ProjectFieldsVisitor.java)

3. DrillParquetReader.getProjection(...) splitted into few methods

4. Added javadocs for PathSegment and SchemaPath

  1. … 6 more files in changeset.
DRILL-7373: Fix problems involving reading from DICT type

- Fixed FieldIdUtil to resolve reading from DICT for some complex cases;

- optimized reading from DICT given a key by passing an appropriate Object type to DictReader#find(...) and DictReader#read(...) methods when schema is known (e.g. when reading from Hive tables) instead of generating it on fly based on int or String path and key type;

- fixed error when accessing value by not existing key value in Avro table.

  1. … 10 more files in changeset.
DRILL-7168: Implement ALTER SCHEMA ADD / REMOVE commands

    • -6
    • +176
    ./sql/handlers/SchemaHandler.java
  1. … 13 more files in changeset.
DRILL-7337: Add vararg UDFs support

    • -3
    • +5
    ./sql/DrillSqlAggOperatorWithoutInference.java
    • -5
    • +3
    ./sql/DrillSqlOperatorWithoutInference.java
    • -0
    • +59
    ./sql/VarArgOperandTypeChecker.java
  1. … 31 more files in changeset.
DRILL-7317: Close ClassLoaders used for udf jars uploading when closing FunctionImplementationRegistry

- Fix issue with caching DrillMergeProjectRule and FunctionImplementationRegistry when different drillbits are started within the same JVM

  1. … 3 more files in changeset.
DRILL-7316: Move classes from org.apache.drill.metastore into org.apache.drill.exec.metastore package in java-exec module

  1. … 30 more files in changeset.
DRILL-7315: Revise precision and scale order in the method arguments

  1. … 27 more files in changeset.
DRILL-7313: Use Hive schema for MaprDB native reader when field was empty

- Added all_text_mode option for hive maprDB Json

- Improved logic to convert Hive's schema into Drill's one

- Added unit tests for schema conversion

  1. … 27 more files in changeset.
DRILL-7271: Refactor Metadata interfaces and classes to contain all needed information for the File based Metastore

    • -1
    • +4
    ./common/NumericEquiDepthHistogram.java
    • -5
    • +5
    ./cost/DrillRelMdDistinctRowCount.java
    • -12
    • +12
    ./cost/DrillRelMdSelectivity.java
    • -3
    • +3
    ./logical/ConvertCountToDirectScanRule.java
    • -4
    • +4
    ./physical/ConvertCountToDirectScanPrule.java
  1. … 106 more files in changeset.
DRILL-7253: Read Hive struct w/o nulls

    • -22
    • +20
    ./physical/visitor/RexVisitorComplexExprSplitter.java
    • -63
    • +66
    ./physical/visitor/SplitUpComplexExpressions.java
    • -1
    • +12
    ./sql/DrillCalciteSqlOperatorWrapper.java
  1. … 12 more files in changeset.
DRILL-7272: Drill Metastore Read / Write API and Drill Iceberg Metastore implementation

1. Drill Metastore Read / Write API.

2. Drill Iceberg Metastore implementation in iceberg-metastore module.

3. Patches Guava Preconditions class for Apache Iceberg.

4. General refactoring.

5. Unit tests.

6. Documentation.

    • -1
    • +1
    ./cost/DrillRelMdDistinctRowCount.java
  1. … 97 more files in changeset.
DRILL-7250: Query with CTE fails when its name matches to the table name without access

  1. … 3 more files in changeset.
DRILL-7242: Handle additional boundary cases and compute better estimates when popular values span multiple buckets.

Address review comments.

close apache/drill#1785

    • -38
    • +87
    ./common/NumericEquiDepthHistogram.java
  1. … 2 more files in changeset.
DRILL-7245: Cap NDV at row count after applying filters

closes #1786

    • -5
    • +6
    ./cost/DrillRelMdDistinctRowCount.java
DRILL-7238: Fixed ConvertCountToDirectScan to handle non-existent columns

closes #1781

    • -1
    • +5
    ./logical/ConvertCountToDirectScanRule.java
  1. … 1 more file in changeset.
DRILL-7227: Fix predicate check in DrillRelOptUtil.analyzeSimpleEquiJoin

closes #1775

    • -20
    • +44
    ./cost/DrillRelMdDistinctRowCount.java
  1. … 1 more file in changeset.
DRILL-7228: Upgrade to a newer version of t-digest to address inaccuracies in histogram buckets. closes #1774

    • -2
    • +2
    ./common/NumericEquiDepthHistogram.java
  1. … 3 more files in changeset.
DRILL-7050: RexNode convert exception in sub-query closes #1770

  1. … 3 more files in changeset.
DRILL-7187: Improve selectivity estimation of BETWEEN predicates and arbitrary combination of range predicates.

Address review comments.

Modify unit test expected rowcount after rebasing.

close apache/drill#1772

    • -122
    • +178
    ./common/NumericEquiDepthHistogram.java
  1. … 2 more files in changeset.
DRILL-7183: TPCDS query 10, 35, 69 take longer with sf 1000 when Statistics are disabled. This commit reverts the changes done for DRILL-6997.

    • -183
    • +0
    ./logical/DrillSemiJoinRule.java
  1. … 1 more file in changeset.
DRILL-7098: File Metadata Metastore Plugin closes #1754

    • -2
    • +2
    ./logical/ConvertCountToDirectScanRule.java
    • -1
    • +2
    ./physical/ConvertCountToDirectScanPrule.java
  1. … 57 more files in changeset.
DRILL-6988. Utility of the too long error message when syntax error

- Adding Drill wrapper around SqlparseException to customize produced by Calcite messages

- Fix Drill SQL parse exception formatter to calculate proper position for "^" character

closes #1753

    • -0
    • +105
    ./sql/parser/impl/DrillSqlParseException.java
  1. … 1 more file in changeset.
DRILL-6974: SET option command modification

- ALTER ... RESET ... and ALTER ... SET ... sub-parsers separated to 2

different SqlCall classes with same parent SqlSetOption

- parserImpls modified to handle new syntax of ALTER... SET...

expresion:

a) ALTER ... SET option.name - option.value - setting option value

b) ALTER ... SET option.name - display option value

- Handler for SqlSetOption separated to SetOptionHandler and

ResetOptionhandler for better representation of handled statements

- Base abstract class AbstractSqlSetHandler created to not repeat

shared implementation of same functions

- SetOptionHandler covered with unit tests for each statement

form.

Fix issues stated in the review

closes #1763

    • -0
    • +86
    ./sql/handlers/AbstractSqlSetHandler.java
    • -0
    • +73
    ./sql/handlers/ResetOptionHandler.java
    • -73
    • +46
    ./sql/handlers/SetOptionHandler.java
    • -0
    • +103
    ./sql/parser/DrillSqlResetOption.java
    • -0
    • +75
    ./sql/parser/DrillSqlSetOption.java
  1. … 4 more files in changeset.
DRILL-7182: Incorrect access specifier for Join DrillRelMdDistinctRowCount

    • -5
    • +5
    ./cost/DrillRelMdDistinctRowCount.java
DRILL-7166: Count query with wildcard should skip reading of metadata summary file

    • -1
    • +7
    ./logical/ConvertCountToDirectScanRule.java
  1. … 1 more file in changeset.
DRILL-7160: e.q.max_rows QUERY-level option shown even if not set

The fix is to force setting to zero IFF autoLimit was intended to be set originally but is inapplicable; such as 'SHOW DATABASES'. If autolimit was not intended to be applied, setting the value to zero is not required.

WebUI is also patched to not show the zero value set in such scenarios.

  1. … 1 more file in changeset.
DRILL-7159: Fix typeString method to return correct name for MAP (aka STRUCT) closes #1741

  1. … 2 more files in changeset.
DRILL-7157: Wrap SchemaParsingException into UserException when creating schema

closes #1740

DRILL-7154: TPCH query 4, 17 and 18 take longer with sf 1000 when Statistics are disabled.

closes #1737

    • -1
    • +1
    ./fragment/DefaultQueryParallelizer.java
DRILL-7048: Implement JDBC Statement.setMaxRows() with System Option

This introduces support for JDBC's Statement.setMaxRows(int) API, which can help Drill execute a query much faster if it knows that not ALL the records in the resultset will be consumed upfront.

This Commit introduces the core changes to support the feature within Drill's execution engine

Protobuf Changes

1. RunQuery: Added "autolimit_rowcount"

2. QueryProfile: Added "autoLimit"

3. Regenerated Java and C++ client files

REST API support

1. Support for REST server to interpret a submitted query and also for rendering this information for an executed query

2. Updates to the Freemarker templates (for WebUI)

3. Safety check within Javascript (for WebUI)

JDBC API support

1. Introduces backend execution of 'ALTER SESSION' to apply the auto-limiting of resultset size

2. Added Unit Tests for PreparedStatement and Statement objects

3. Added getter setter methods to be skipped in testing for org.apache.drill.jdbc.test.Drill2489CallsAfterCloseThrowExceptionsTest.testclosedPreparedStmtOfOpenConnMethodsThrowRight()

Updates based on review comments

Additional Updates

Test Cleanup

1. Revert Drill2489 hack

2. Formatting in *StatementTest

3. Removal f redundant `statement.close()`

4. Manage new Exception thrown when setting invalid maxRow values

Final updates

1. Test changes

2. Trim trailing spaces in auto-limit value (Javascript)

3. Before & After annotations to synchronize changes to system values for MaxRows(auto-limit)

Reorganized tests due to synchronized locking

Removed conflicting JsonCreator in QueryWrapper

Additional test cleanup

closes #1714

  1. … 34 more files in changeset.