drill

Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-2441: For outer-join, if there is any inequality condition, Cartesian-Join exception will be thrown

DRILL-2128.1: Preparatory changes: Labeled result cols.; formatted SQL. [MetaImpl]

DRILL-2128.2: Fixed DatabaseMetaData.getColumns's DATA_TYPE, TYPE_NAME.

- Created basic test for DATA_TYPE and TYPE_NAME.

- Fixed DATA_TYPE: Added mapping from type name/descriptor strings from

INFORMATION_SCHEMA.COLUMNS.DATA_TYPE to java.sql.Types.* integer type codes

for DatabaseMetaData.getColumns's DATA_TYPE.

- Fixed TYPE_NAME: Added TYPE_NAME returning type name/descriptor strings from

INFORMATION_SCHEMA.COLUMNS.DATA_TYPE

- Added FIXMEs for some missing/misnamed/wrong fields. (See DRILL-2420.)

DRILL-2397, new data types doc, misc other fixes

    • -16
    • +23
    /_docs/connect/002-plugin-conf.md
    • -20
    • +11
    /_docs/data-sources/001-hive-types.md
    • -6
    • +6
    /_docs/data-sources/003-parquet-ref.md
    • binary
    /_docs/img/connect-plugin.png
    • -60
    • +103
    /_docs/sql-ref/001-data-types.md
    • -12
    • +16
    /_docs/sql-ref/002-lexical-structure.md
    • -0
    • +77
    /_docs/sql-ref/data-types/002-disparate-data-types.md
DRILL-2225: Fix missing PartitionSenderRootExec stats.

DRILL-2413: FileSystemPlugin refactoring: avoid sharing DrillFileSystem across schemas

  1. … 19 more files in changeset.
DRILL-2414: Give proper error message if Union-All is applied on schema-less tables

DRILL-1833: Avoid storing view names in PStore cache

...always rely on view files in schema location for listing views.

DRILL-2060: Constant folding rule

2060 update - Constant folding work completed.

Fix issue with date, time and timestamp literal creation.

Fix literal creation during expression interpretation to match nullability of incoming expression.

Fix decimal literals in interpreted expression eval.

Disable test with an exposed planning bug when the project instance of the constant folding rule is enabled. The rule is not actually influencing the final plan when the rule is firing and making expression reductions. This is due to our current cost model fro project which just counts the number of expressions and does not consider expression complexity. The issues have been logged in DRILL-2218 for further investigation, they do not need to be solved to merge the other constant folding rules and all of the interpreted expression work that has been done.

Get rid of clutter in RuleSets, explanation has been moved to the 2218 JIRA.

Belongs with 2060, fix constant expression executor to use the new constant expression interpreter interface that returns a ValueHolder instead a ValueVector with a single value filled in.

2060 update - change test baseline due to new column ordering (no functional or performance impacting changes to plan)

2060 - address Aman's comments.

add test ignore - DRILL-2218

Baseline update for project pushdown test (only column ordering on a scan, no functional or performance impacting plan changes)

Turn back on project instance.

Small casting bug in constant executor.

Don't fold hive UDFs.

Modify DrillBuf to allow a BufferManager to be the owning context for a DrilllBuf.

TODO - refactor to remove remaining common code from OperatorContext and FragmentContext,

have them both use the new BufferManager.

Add system option for disabling constant folding.

2060 update - test option to disable constant folding.

Update RuleSets to actually allow turning the constant folding rules on and off as well as establish general pattern for turning logical rules on an off, similar to how some physical rules can be already.

Change the estimated row count in EasyGroupScan to report a number of files in the case where the file size indicates an estimated total count of 0 records. Allows very small files to be pruned.

Fix folding expressions that result in null after refactoring the interpreted expression evaluation to return a ValueHolder in the case of a constant expression. Previously a value vector was returned in the same manner as the interpreter can still do when given an input VectorAccessible and an expression that may contain fild references. Calling getObject on the output vector previously gracefully handled nulls as they were passed into the Calciate API to create literals. This process has to be a bit more manual now.

Address Jinfeng's review comments.

A few more review comments.

Disable cost calculation change, complete fix will come in 2553.

Throw a runtime exception of there is an error materializing the expression, as the same materialization will take place at query execution time we should fail early.

Add a test that does prune appropriately, still have a test for the outstanding issue tracked in DRILL-2553.

Small fix for test to properly set session option and set it back after completion.

Fixing comment that was garbled somehow.

small fix for case where expression returns a null result during constant folding.

Add a little defensive code to give a good error message if a type that does not appear in the mapping from Drill to Calcite types attempts to be folded into a null value.

  1. … 7 more files in changeset.
DRILL-2406: part 2 - Allow interpreted expression evaluation at planning time.

Changes needed after rebase to expose function determinism to calcite appropriately.

Address Jacques review comments.

Address chris' review comments.

Make things work now that BufferManager is AutoClosable.

Fixes tests that were creating plan fragments directly to create their own query start time,

as this information is now passed along from QueryContext during standard query initialization

(this enables the query start time and timezone to be available to planning time expression

evaluation).

Fix docs in BufferManger.

Update UDF interface to track determinism rather than randomness.

DRILL-412: FoodMart data (account.json) cause JsonParseException

DRILL-367: FoodMart data (category.json) packaged with Drill does not conform with JSON specification

Obsolete Pentaho repo

DRILL-2695: Add Support for large in conditions through the use of the Values operator. Update JSON reader to support reading Extended JSON. Update JSON writer to support writing extended JSON data. Update JSON reader to automatically unwrap a file that includes a single top-level array (used by values). Update Options manager to use getOption(<Type>Validator) to directly retrieve typed value. Remove JSON rewinding Add support for CONVERT_TO( [], 'SIMPLEJSON') to disable extended types as part of udf use.

  1. … 51 more files in changeset.
DRILL-2275: Added support to get information about current cluster memory and threads

+ SystemRecordReader reads a SystemRecord e.g. MemoryRecord

+ Added generic data type for static tables

+ GroupScan can enforce width to be maximum width on ExcessiveExchangeRemover

+ GroupScan has minimum width for SimpleParallelizer

  1. … 8 more files in changeset.
DRILL-2010: MergeJoin: Store/restore the right batch state when existing join loop due to output batch full.

DRILL-2381 lexical structure plus fixes

    • -59
    • +41
    /_docs/data-sources/003-parquet-ref.md
    • -209
    • +201
    /_docs/data-sources/004-json-ref.md
    • -0
    • +141
    /_docs/sql-ref/002-lexical-structure.md
    • -0
    • +70
    /_docs/sql-ref/003-operators.md
    • -0
    • +186
    /_docs/sql-ref/004-functions.md
    • -10
    • +0
    /_docs/sql-ref/004-nest-functions.md
    • -0
    • +10
    /_docs/sql-ref/005-nest-functions.md
    • -0
    • +9
    /_docs/sql-ref/006-cmd-summary.md
    • -0
    • +16
    /_docs/sql-ref/007-reserved-wds.md
DRILL-2358: Ensure DrillScanRel differentiates skip-all, scan-all & scan-some in a backward compatible fashion

DRILL-2220: Complex reader unable to read FIXED_LEN_BYTE_ARRAY types in parquet file

DRILL-2402: Update hash functions to use seed strategy as opposed to xor strategy.

Also: Simplify and consolidate expression materialization.

DRILL-1897: Fix return type of functions to be nullable during planning phase

fold in review changes

    • -2
    • +2
    /_docs/data-sources/003-parquet-ref.md
    • -110
    • +189
    /_docs/data-sources/004-json-ref.md
DRILL-2372: Fix current_date to convert input to UTC before storing in the vector

DRILL-2118: inform user with a user friendly error message if kvgen fails due to heterogenous types

DRILL-2406: part 1 - Remove interpreter generation, add new reflection based expression interpretation.

Changed interpreted evaluation to run the setup method after the input parameters have been set using reflection

so they are available in the case where the inputs are constant and are used in the setup method.

Changes that were originally committed in later patches for 2060 and 2173, as they are needed for this to run on its own:

- Change to DrillSimpleFuncHolder that belongs with the interpreter refactoring.

- ValueHolderHelper changes needed for interpreter refactoring.

Updates after review comments from Jinfeng:

Change the DrillSimpleFunc creation in the new interpreter to use a direct reference to the class type of the Function rather than the class name.

Add test case for 'like' function evaluated in the interpreter (has a meaningful setup method that uses one of the inputs to initialize a pattern matcher)

Update from Aman's review:

Add a test case for using a cast in interpreted expression evaluation.

  1. … 8 more files in changeset.
DRILL-2143: Part 2 - fix interpreter and add QueryDateTimeInfo injectable to fill in holes from removing RecordBatch from UDFs - Move list of accepted injectable types into the UdfUtilities interface for easier maintenance.

Fix one more new function to remove RecordBatch from setup method.

Add back UdfUtilities interface to FragmentContext lost in rebasing.

Remove unneeded testcase in TestExampleQueries, remove commented out code in InterpreterGenerator

previously used to handle DrillBuf injectables, now being handled by reflection to set the DrillBuf

rather than the previously generated code that would use the direct reference to the incoming

RecordBatch to get a buffer.

Fix docs on UdfUtilities and package docs for drill/exec/expr, belongs with 2143 part 2 patch.

Fix belongs with 2143, change interpreter to output a ValueHolder instead of a ValueVector in the case of a constant expression.

2143 update - Use reflection to remove boilerplate for adding new injectable types for UDFs.

fix links in gh-pages-master

  1. … 54 more files in changeset.
DRILL-2323: Added parquet metadata to logs (+filename to JSON reader)

DRILL-1692: Fixing Mongo join issue when * is selected

DRILL-2336 plugin updates

    • -27
    • +16
    /_docs/connect/001-plugin-reg.md
    • -0
    • +123
    /_docs/connect/002-plugin-conf.md
    • -0
    • +74
    /_docs/connect/003-workspaces.md
    • -0
    • +64
    /_docs/connect/004-reg-fs.md
    • -0
    • +34
    /_docs/connect/005-reg-hbase.md
    • -0
    • +83
    /_docs/connect/006-reg-hive.md
    • -0
    • +60
    /_docs/connect/007-default-frmt.md
    • -167
    • +0
    /_docs/connect/007-mongo-plugin.md
    • -31
    • +0
    /_docs/connect/008-mapr-db-plugin.md
  1. … 16 more files in changeset.
DRILL-2359: ClassPathFileSystem.getFileStatus() should throw FileNotFoundException when path doesn't exist

DRILL-1875: getTransferClone should ignore given wrappers