Clone Tools
  • last updated 10 mins ago
Constraints: committers
Constraints: files
Constraints: dates
DRILL-4482: Fix Avro nested field selection regression

Update some of the Avro tests to properly verify their results,

others still need to be fixed. These will be addressed in DRILL-4110.

Closes #419

DRILL-4474: Ensure that ConvertCountToDirectScan does not push through project when nullable input of count is not RexInputRef This closes #416

DRILL-4479: For empty fields under all_text_mode enabled (a) use varchar for the default columns and (b) ensure we create fields corresponding to all columns.

close apache/drill#420

DRILL-4487: add unit test for DRILL-4449

MD-771: Query with nested functions within a sub-query on MapR-DB JSON Tables fail with ClassCastException

Fix for failing test cases.

DRILL-4485 - MapR profile - switch to MapR 5.1.0, and improve compatibility with maprfs storage format and MapR DB storage plugin. This closes #417

DRILL-4483: Fix text plan regression in query profiles

DRILL-4457: Difference in results returned by window function over BIGINT data

this closes #410

    • -0
    • +3
DRILL-4281: Support authorized proxy users to impersonate other users

closes #400

DRILL-4486: Fix expression serialization escaping

Closes #412

Refactoring code for better organization.

+ Adding skeleton streams plugin.

  1. … 33 more files in changeset.
DRILL-4476: Allow UnionAllRecordBatch to manager situations where left input side or both sides come(s) from empty source(s).

close apache/drill#407

DRILL-4474: Ensure that ConvertCountToDirectScan only pushes through project when project is trivial. This closes #406

    • binary
DRILL-4467: Fix field ordering issue in PrelUtil

DRILL-4372: Expose the functions return type to Drill

- Drill-Calite version update:

This commit needs to have Calcite's patch (CALCITE-1062) to plugin customized SqlOperator.

- FunctionTemplate

Add FunctionArgumentNumber annotation. This annotation element tells if the number of argument(s) is fixed or arbitrary (e.g., String concatenation function).

Due to this modification, there are some minor changes in DrillFuncHolder, DrillFunctionRegistry and FunctionAttributes.

- Checker

Add a new Checker (which Calcite uses to validate the legitimacy of the number of argument(s) for a function) to allow functions with arbitrary arguments to pass Caclite's validation

- Type conversion between Drill and Calcite

DrillConstExector is given a static method getDrillTypeFromCalcite() to convert Calcite types to Drill's.

- Extract function's return type inference

Unlike other functions, Extract function's return type can be determined solely based on the first argument. A logic is added in to allow this inference to happen

- DrillCalcite wrapper:

From the aspects of return type inference and argument type checks, Calcite's mechanism is very different from Drill's. In addition, currently, there is no straightforward way for Drill to plug-in customized mechanisms to Calcite. Thus, wrappers are provided to serve the objective.

Except for the mechanisms of type inference and argument type checks, these wrappers just forward any method calls to the wrapped SqlOpertor, SqlFuncion or SqlAggFunction to respond.

A interface DrillCalciteSqlWrapper is also added for the callers of the three wrappers to get the wrapped objects easier.

Due to these wrappers, UnsupportedOperatorsVisitor is modified in a minor manner.

- Calcite's SqlOpertor, SqlFuncion or SqlAggFunction are wrapped in DrillOperatorTable

Instead of returning Caclite's native SqlOpertor, SqlFuncion or SqlAggFunction, return the wrapped ones to ensure customized behaviors can be adopted.

- Type inference mechanism

This mechanism is used across all SqlOpertor, SqlFuncion or SqlAggFunction. Thus, it is factored out as its own method in TypeInferenceUtils

- Upgrade Drill-Calcite

Bump version number to 1.4.0-drill-test-r16

- Implement two argument version of lpad, rpad

- Implement one argument version of ltrim, rtrim, btrim

  1. … 23 more files in changeset.
DRILL-4372: (continued) Type inference for HiveUDFs

MD-726: Add support for `read_numbers_as_double` and `all_text_mode` options

+ MD-773: Add support to push-down filters on Date, Time and Timestamp

+ Fix the offsets in Date and Time data types in returned values

+ Updated support for DATE, TIME, and TIMESTAMP types

+ Modified unit-tests to check for operation push-down

DRILL-4446: Support mandatory work assignment to endpoint requirements of operators

  1. … 9 more files in changeset.
update per DRILL-4461

DRILL-4437: Operator unit test framework

Closes #394

DRILL-4442: Move getSV2 and getSV4 methods to VectorAccessible

Up one level from previous location RecordBatch, most implementations

already implement the method as they implement RecordBatch rather than

VectorAccessible itself. Add unsupported operation exception to others.

DRILL-4443: MIN/MAX on VARCHAR throw a NullPointerException

    • -0
    • +2
DRILL-4441: Fix varchar data read out of Avro filtering incorrectly due to metadata bug

The precision of the Varchar datatype was not being set causing inconsistent

truncation of values to the default length of 1. Fixed the same issue with varbinary.

The test framework was previously taking a string as the baseline for a binary value,

which cannot express all possible values. Fixed the test to intstead use a byte array.

Thie required updating the hive tests that were using the old method of specifying

baselines with a String.

Fix cast to varbinary when reading from a data source with schema needed for writing

a test.

Updated patch to remove varchar lengths from table creation.

This issue was fixed more generally by DRILL-4465, which provides a default

type length for varchar and varbinary during the setup of calcite. This update now

just provides tests to verify the fix in this case.

Closes #393

DRILL-4465: Simplify Calcite parsing & planning integration

- Canonicalize Planning phases with PlannerPhase enumeration

- Canonicalize PlannerType transforms

- Remove dependency on Calcite's Frameworks.Planner since Drill need stop heavily customize interactions

- Update AbstractStoragePlugin to implement a phase-aware planning rule injection behavior.

- Avoid (or at least reduce) duplicated registerSchemas() invocations

This closes #401.

  1. … 12 more files in changeset.
DRILL-4589: Reduce planning time for file system partition pruning by reducing filter evaluation overhead

MD-711: Enable support for UNION types in Drill-MapRDB plugin

MD-741: Cannot select array elements by index in MapR-DB JSON Tables

DRILL-4448: Clean up deserialization of oderings in sorts

Fix sort operator deserialization and validation to respect existing

contract specified in the tests.

DRILL-4449: Wrong results when metadata cache is used with specific set of queries

close apache/drill#392

DRILL-4434: Deprecate GroupScan.enforceWidth API