Clone Tools
  • last updated a few minutes ago
Constraints: committers
Constraints: files
Constraints: dates
DRILL-4459: Resolve SchemaChangeException while querying hive json table

- Replace drill var16char to varchar datatype for hive string datatype

- Change testGenericUDF() and testUDF() to use VarChar instead of Var16Char

- Add unit test for hive GET_JSON_OBJECT UDF

closes #431

    • -0
    • +19
DRILL-4484: NPE when querying empty directory

DRILL-4490: Ensure the count generated by ConvertCountToDirectScan is non-nullable

DRILL-4376: Wrong results when doing a count(*) on part of directories with metadata cache

DRILL-4493 - Fixed issues in various POMs with MapR profile This closes #421

DRILL-4372: (continued) Add option to disable/enable function output type inference

MD-572: Column names in MapR-DB JSON tables are case-sensitive

Disable pushdown of both filter and projects (by default set to false).

This will allow Drill to handle both of these operators in a case-insensitive way.

DRILL-4482: Fix Avro nested field selection regression

Update some of the Avro tests to properly verify their results,

others still need to be fixed. These will be addressed in DRILL-4110.

Closes #419

DRILL-4474: Ensure that ConvertCountToDirectScan does not push through project when nullable input of count is not RexInputRef This closes #416

DRILL-4479: For empty fields under all_text_mode enabled (a) use varchar for the default columns and (b) ensure we create fields corresponding to all columns.

close apache/drill#420

DRILL-4487: add unit test for DRILL-4449

MD-771: Query with nested functions within a sub-query on MapR-DB JSON Tables fail with ClassCastException

Fix for failing test cases.

DRILL-4485 - MapR profile - switch to MapR 5.1.0, and improve compatibility with maprfs storage format and MapR DB storage plugin. This closes #417

DRILL-4483: Fix text plan regression in query profiles

DRILL-4457: Difference in results returned by window function over BIGINT data

this closes #410

    • -0
    • +3
DRILL-4281: Support authorized proxy users to impersonate other users

closes #400

DRILL-4486: Fix expression serialization escaping

Closes #412

Refactoring code for better organization.

+ Adding skeleton streams plugin.

  1. … 33 more files in changeset.
DRILL-4476: Allow UnionAllRecordBatch to manager situations where left input side or both sides come(s) from empty source(s).

close apache/drill#407

DRILL-4474: Ensure that ConvertCountToDirectScan only pushes through project when project is trivial. This closes #406

    • binary
DRILL-4467: Fix field ordering issue in PrelUtil

DRILL-4372: Expose the functions return type to Drill

- Drill-Calite version update:

This commit needs to have Calcite's patch (CALCITE-1062) to plugin customized SqlOperator.

- FunctionTemplate

Add FunctionArgumentNumber annotation. This annotation element tells if the number of argument(s) is fixed or arbitrary (e.g., String concatenation function).

Due to this modification, there are some minor changes in DrillFuncHolder, DrillFunctionRegistry and FunctionAttributes.

- Checker

Add a new Checker (which Calcite uses to validate the legitimacy of the number of argument(s) for a function) to allow functions with arbitrary arguments to pass Caclite's validation

- Type conversion between Drill and Calcite

DrillConstExector is given a static method getDrillTypeFromCalcite() to convert Calcite types to Drill's.

- Extract function's return type inference

Unlike other functions, Extract function's return type can be determined solely based on the first argument. A logic is added in to allow this inference to happen

- DrillCalcite wrapper:

From the aspects of return type inference and argument type checks, Calcite's mechanism is very different from Drill's. In addition, currently, there is no straightforward way for Drill to plug-in customized mechanisms to Calcite. Thus, wrappers are provided to serve the objective.

Except for the mechanisms of type inference and argument type checks, these wrappers just forward any method calls to the wrapped SqlOpertor, SqlFuncion or SqlAggFunction to respond.

A interface DrillCalciteSqlWrapper is also added for the callers of the three wrappers to get the wrapped objects easier.

Due to these wrappers, UnsupportedOperatorsVisitor is modified in a minor manner.

- Calcite's SqlOpertor, SqlFuncion or SqlAggFunction are wrapped in DrillOperatorTable

Instead of returning Caclite's native SqlOpertor, SqlFuncion or SqlAggFunction, return the wrapped ones to ensure customized behaviors can be adopted.

- Type inference mechanism

This mechanism is used across all SqlOpertor, SqlFuncion or SqlAggFunction. Thus, it is factored out as its own method in TypeInferenceUtils

- Upgrade Drill-Calcite

Bump version number to 1.4.0-drill-test-r16

- Implement two argument version of lpad, rpad

- Implement one argument version of ltrim, rtrim, btrim

  1. … 23 more files in changeset.
DRILL-4372: (continued) Type inference for HiveUDFs

MD-726: Add support for `read_numbers_as_double` and `all_text_mode` options

+ MD-773: Add support to push-down filters on Date, Time and Timestamp

+ Fix the offsets in Date and Time data types in returned values

+ Updated support for DATE, TIME, and TIMESTAMP types

+ Modified unit-tests to check for operation push-down

DRILL-4446: Support mandatory work assignment to endpoint requirements of operators

  1. … 9 more files in changeset.
update per DRILL-4461

DRILL-4437: Operator unit test framework

Closes #394

DRILL-4442: Move getSV2 and getSV4 methods to VectorAccessible

Up one level from previous location RecordBatch, most implementations

already implement the method as they implement RecordBatch rather than

VectorAccessible itself. Add unsupported operation exception to others.

DRILL-4443: MIN/MAX on VARCHAR throw a NullPointerException

    • -0
    • +2
DRILL-4441: Fix varchar data read out of Avro filtering incorrectly due to metadata bug

The precision of the Varchar datatype was not being set causing inconsistent

truncation of values to the default length of 1. Fixed the same issue with varbinary.

The test framework was previously taking a string as the baseline for a binary value,

which cannot express all possible values. Fixed the test to intstead use a byte array.

Thie required updating the hive tests that were using the old method of specifying

baselines with a String.

Fix cast to varbinary when reading from a data source with schema needed for writing

a test.

Updated patch to remove varchar lengths from table creation.

This issue was fixed more generally by DRILL-4465, which provides a default

type length for varchar and varbinary during the setup of calcite. This update now

just provides tests to verify the fix in this case.

Closes #393