Clone
 

bohdan kazydub <bohdan.kazydub@gmail.com> in drill

DRILL-7759: Code compilation exception for queries containing (untyped) NULL

DRILL-7694: Register drill.queries.* counter metrics on Drillbit startup

closes #2050

DRILL-7504: Upgrade Parquet library to 1.11.0

closes #1970

DRILL-7509: Incorrect TupleSchema is created for DICT column when querying Parquet files

DRILL-7473: Parquet reader failed to get field of repeated map

closes #1933

DRILL-7453: Update joda-time to 2.10.5 to have correct time zone info

DRILL-7448: Fix warnings when running Drill memory tests

closes #1902

    • -25
    • +22
    /common/src/test/resources/logback-test.xml
DRILL-7440: Failure during loading of RepeatedCount functions

closes #1894

DRILL-7397: Fix logback errors when building the project

DRILL-7359: Add support for DICT type in RowSet Framework

closes #1870

  1. … 68 more files in changeset.
DRILL-7373: Fix problems involving reading from DICT type

- Fixed FieldIdUtil to resolve reading from DICT for some complex cases;

- optimized reading from DICT given a key by passing an appropriate Object type to DictReader#find(...) and DictReader#read(...) methods when schema is known (e.g. when reading from Hive tables) instead of generating it on fly based on int or String path and key type;

- fixed error when accessing value by not existing key value in Avro table.

DRILL-7096: Develop vector for canonical Map<K,V>

- Added new type DICT;

- Created value vectors for the type for single and repeated modes;

- Implemented corresponding FieldReaders and FieldWriters;

- Made changes in EvaluationVisitor to be able to read values from the map by key;

- Made changes to DrillParquetGroupConverter to be able to read Parquet's MAP type;

- Added an option `store.parquet.reader.enable_map_support` to disable reading MAP type as DICT from Parquet files;

- Updated AvroRecordReader to use new DICT type for Avro's MAP;

- Added support of the new type to ParquetRecordWriter.

  1. … 94 more files in changeset.
DRILL-7079: Drill can't query views from the S3 storage when plain authentication is enabled

closes #1712

DRILL-7041: CompileException happens if a nested coalesce function returns null - Made `NullExpression`s in `IfExpression` with nested `IfExpression`s to be rewritten to typed ones recursively if necessary closes #1668

DRILL-7200: Update Calcite to 1.19.0 / 1.20.0

  1. … 32 more files in changeset.
DRILL-4858: REPEATED_COUNT on an array of maps and an array of arrays is not implemented

- Implemented 'repeated_count' function for repeated MAP and repeated LIST;

- Updated RepeatedListReader and RepeatedMapReader implementations to return correct value from size() method

- Moved repeated_count to freemarker template and added support for more repeated types for the function

closes #1641

DRILL-7038: Queries on partitioned columns scan the entire datasets

- Added new optimizer rule which checks if query references directory columns only and has DISTINCT or GROUP BY operation. If the condition holds, instead of scanning full file set the following will be performed:

1) if there is cache metadata file, these directories will be read from it,

2) otherwise directories will be gathered from selection object (PartitionLocation).

In the end Scan node will be transformed to DrillValuesRel (containing constant literals) with gathered values so no scan will be performed.

closes #1640

DRILL-6928: Update description for exec.query.return_result_set_for_ddl option to reflect it affects JDBC connections only closes #1601

DRILL-6962: Function coalesce returns an Error when none of the columns in coalesce exist in a parquet file

- Updated UntypedNullVector to hold value count when vector is allocated and transfered to another one;

- Updated RecordBatchLoader and DrillCursor to handle case when only UntypedNull values are present in RecordBatch (special case when data buffer is null but actual values are present);

- Added functions to cast UntypedNull value to other types for use in UDFs;

- Moved UntypedReader, UntypedHolderReaderImpl and UntypedReaderImpl from org.apache.drill.exec.vector.complex.impl to org.apache.drill.exec.vector package.

closes #1614

    • -0
    • +61
    /exec/java-exec/src/main/codegen/templates/CastUntypedNull.java
DRILL-6894: CTAS and CTTAS are not working on S3 storage when cache is disabled

- provided JsonRecordWriter, ParquetRecordWriter and DrillTextRecordWriter with file system configuration

closes #1576

DRILL-6874: Close input stream after AsyncPageReaderTask is completed

close apache/drill#1565

DRILL-6863: Drop table is not working if path within workspace starts with "/"

- Made workspace to be honored when table/view name starts with "/" for DROP TABLE, DROP VIEW, CREATE VIEW and SELECT from view queries;

- Made "/{name}" and "{name}" to be equivalent names (the leading "/" is removed) when creating temporary tables so that SELECT ... FROM "/{name}" ... and SELECT ... FROM "{name}" ... produce the same results and behave as regular tables in the context.

closes #1557

DRILL-6834: Introduce option to disable result set for DDL queries for JDBC connection

- Added session-scoped option `drill.exec.fetch_resultset_for_ddl` to control whether update count or result set should be returned for JDBC connection session. By default the option is set to `true` which ensures that result set is returned;

- Updated Drill JDBC: `DrillCursor` and `DrillStatement` to achieve desired behaviour.

closes #1549

  1. … 12 more files in changeset.
DRILL-6810: Disable NULL_IF_NULL NullHandling for functions with ComplexWriter closes #1509

DRILL-6793: FragmentExecutor cannot send its final state for the case when RootExec root wasn't initialized

closes #1506

DRILL-6783: CAST string literal as INTERVAL MONTH/YEAR works inconsistently when selecting from a table with multiple rows

close apache/drill#1496

DRILL-6768: Improve to_date, to_time and to_timestamp and corresponding cast functions to handle empty string when option is enabled closes #1494

  1. … 10 more files in changeset.
DRILL-6724: Dump operator context to logs when error occurs during query execution

closes #1455

  1. … 88 more files in changeset.
DRILL-6689: Include query user information to drillbit.log

closes #1432

DRILL-6662: Access AWS access key ID and secret access key using Credential Provider API for S3 storage plugin

closes #1419