Clone Tools
  • last updated 19 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-7337: Add vararg UDFs support

  1. … 37 more files in changeset.
DRILL-7315: Revise precision and scale order in the method arguments

  1. … 28 more files in changeset.
DRILL-6533: Allow using literal values in functions which expect FieldReader instead of ValueHolder

closes #1617

  1. … 2 more files in changeset.
DRILL-6763: Codegen optimization of SQL functions with constant values(#1481)

closes #1481

  1. … 19 more files in changeset.
DRILL-6422: Replace guava imports with shaded ones

  1. … 984 more files in changeset.
DRILL-6320: Fixed license headers.

closes #1207

  1. … 2066 more files in changeset.
DRILL-6094: Decimal data type enhancements

Add ExprVisitors for VARDECIMAL

Modify writers/readers to support VARDECIMAL

- Added usage of VarDecimal for parquet, hive, maprdb, jdbc;

- Added options to store decimals as int32 and int64 or fixed_len_byte_array or binary;

Add UDFs for VARDECIMAL data type

- modify type inference rules

- remove UDFs for obsolete DECIMAL types

Enable DECIMAL data type by default

Add unit tests for DECIMAL data type

Fix mapping for NLJ when literal with non-primitive type is used in join conditions

Refresh protobuf C++ source files

Changes in C++ files

Add support for decimal logical type in Avro.

Add support for date, time and timestamp logical types.

Update Avro version to 1.8.2.

  1. … 201 more files in changeset.
DRILL-1950: Parquet rowgroup level filter pushdown in query planning time.

Implement Parquet rowgroup level filter pushdown. The filter pushdown is performed in

in Drill physical planning phase.

Only a local filter, which refers to columns in a single table, is qualified for filter pushdown.

A filter may be qualified if it is a simple comparison filter, or a compound "and/or" filter consists of

simple comparison filter. Data types allowed in comparison filter are int, bigint, float, double, date,

timestamp, time. Comparison operators are =, !=, <, <=, >, >=. Operands have to be a column of the above

data types, or an explicit cast or implicit cast function, or a constant expressions.

This closes #637

  1. … 43 more files in changeset.
DRILL-4823: Fix OOM while trying to prune partitions with reasonable data size

closes #560

  1. … 3 more files in changeset.
DRILL-4327: Fix rawtypes warnings in drill codebase

Fixing most rawtypes warning issues in drill modules.

Closes #347

  1. … 77 more files in changeset.
DRILL-3666: Replace assert check in InterpreterEvaluator.

DRILL-2060: Constant folding rule

2060 update - Constant folding work completed.

Fix issue with date, time and timestamp literal creation.

Fix literal creation during expression interpretation to match nullability of incoming expression.

Fix decimal literals in interpreted expression eval.

Disable test with an exposed planning bug when the project instance of the constant folding rule is enabled. The rule is not actually influencing the final plan when the rule is firing and making expression reductions. This is due to our current cost model fro project which just counts the number of expressions and does not consider expression complexity. The issues have been logged in DRILL-2218 for further investigation, they do not need to be solved to merge the other constant folding rules and all of the interpreted expression work that has been done.

Get rid of clutter in RuleSets, explanation has been moved to the 2218 JIRA.

Belongs with 2060, fix constant expression executor to use the new constant expression interpreter interface that returns a ValueHolder instead a ValueVector with a single value filled in.

2060 update - change test baseline due to new column ordering (no functional or performance impacting changes to plan)

2060 - address Aman's comments.

add test ignore - DRILL-2218

Baseline update for project pushdown test (only column ordering on a scan, no functional or performance impacting plan changes)

Turn back on project instance.

Small casting bug in constant executor.

Don't fold hive UDFs.

Modify DrillBuf to allow a BufferManager to be the owning context for a DrilllBuf.

TODO - refactor to remove remaining common code from OperatorContext and FragmentContext,

have them both use the new BufferManager.

Add system option for disabling constant folding.

2060 update - test option to disable constant folding.

Update RuleSets to actually allow turning the constant folding rules on and off as well as establish general pattern for turning logical rules on an off, similar to how some physical rules can be already.

Change the estimated row count in EasyGroupScan to report a number of files in the case where the file size indicates an estimated total count of 0 records. Allows very small files to be pruned.

Fix folding expressions that result in null after refactoring the interpreted expression evaluation to return a ValueHolder in the case of a constant expression. Previously a value vector was returned in the same manner as the interpreter can still do when given an input VectorAccessible and an expression that may contain fild references. Calling getObject on the output vector previously gracefully handled nulls as they were passed into the Calciate API to create literals. This process has to be a bit more manual now.

Address Jinfeng's review comments.

A few more review comments.

Disable cost calculation change, complete fix will come in 2553.

Throw a runtime exception of there is an error materializing the expression, as the same materialization will take place at query execution time we should fail early.

Add a test that does prune appropriately, still have a test for the outstanding issue tracked in DRILL-2553.

Small fix for test to properly set session option and set it back after completion.

Fixing comment that was garbled somehow.

small fix for case where expression returns a null result during constant folding.

Add a little defensive code to give a good error message if a type that does not appear in the mapping from Drill to Calcite types attempts to be folded into a null value.

  1. … 21 more files in changeset.
DRILL-2406: part 1 - Remove interpreter generation, add new reflection based expression interpretation.

Changed interpreted evaluation to run the setup method after the input parameters have been set using reflection

so they are available in the case where the inputs are constant and are used in the setup method.

Changes that were originally committed in later patches for 2060 and 2173, as they are needed for this to run on its own:

- Change to DrillSimpleFuncHolder that belongs with the interpreter refactoring.

- ValueHolderHelper changes needed for interpreter refactoring.

Updates after review comments from Jinfeng:

Change the DrillSimpleFunc creation in the new interpreter to use a direct reference to the class type of the Function rather than the class name.

Add test case for 'like' function evaluated in the interpreter (has a meaningful setup method that uses one of the inputs to initialize a pattern matcher)

Update from Aman's review:

Add a test case for using a cast in interpreted expression evaluation.

  1. … 18 more files in changeset.
DRILL-2143: Part 2 - fix interpreter and add QueryDateTimeInfo injectable to fill in holes from removing RecordBatch from UDFs - Move list of accepted injectable types into the UdfUtilities interface for easier maintenance.

Fix one more new function to remove RecordBatch from setup method.

Add back UdfUtilities interface to FragmentContext lost in rebasing.

Remove unneeded testcase in TestExampleQueries, remove commented out code in InterpreterGenerator

previously used to handle DrillBuf injectables, now being handled by reflection to set the DrillBuf

rather than the previously generated code that would use the direct reference to the incoming

RecordBatch to get a buffer.

Fix docs on UdfUtilities and package docs for drill/exec/expr, belongs with 2143 part 2 patch.

Fix belongs with 2143, change interpreter to output a ValueHolder instead of a ValueVector in the case of a constant expression.

2143 update - Use reflection to remove boilerplate for adding new injectable types for UDFs.

  1. … 10 more files in changeset.
DRILL-2353: Add interpreter based partition pruning.

Integrate Jacques's interpreter based partition pruning with Jason's interpreter refactoring that removed interpreter module and added that functionality in the exec module.

Ensure boolean operators are correctly handled when traversing expression tree to find partition filters.

Resolve merge conflicts after rebasing to master branch.

Additional fixes for handling OR conditions.

  1. … 9 more files in changeset.
DRILL-2173: Partition queries to drive dynamic pruning

Adds new interface on the QueryContext as well as individual schemas for exploring partitions of tables.

Adds injectable type for partition explorer for use in UDFs. This is hooked into both to expression

materialization and interpreted evaluation. The FragmentContext throws an exception to tell users to turn on

constant folding if a UDF that uses the PartitionExplorer makes it past planning.

2173 update -Address Chris' review comments.

Change the PartitionExplorer to return an Iterable<String> instead of String[]

Add interface level description to PartitionExplorer and StoragePluginPartitionExplorer.

New inner class in FileSystemPlugin to fulfill the new Iterable interface for partitions.

Formatting/cleanup fixes

Clean up error reporting code in MaxDir UDF. Remove method to get a string from a DrillBuf, as it was already defined in StringFunctionHelpers. Add new utility method to specifically convert a VarCharHolder to a string to remove some boilerplate.

Fixed an errant copy paste in a comment and removed unused imports.

Fix docs in FileSystemPlugin, belongs with the 2173 changes.

Fix references in Javadoc to properly use @link instead of @see.

2173 fixes, correctly return an empty list of sub-partitions if the path requested in the partition explorer interface is a file. Fix a few docs.

More 2173, finishing Chris' comments

2173 update - Add validation for PartitionExplorer injectable in UdfUtiltiers.

small change to fix refactored unit tests.

cleanup for 2173

Fix maxdir UDF so it can compile in runtime generated code as well as the interpreted expression system (needed to fully qualify classes and interfaces). It still fails to execute, as we prevent requesting a schema from a non-root fragment. We do not expect these types of functions to ever be used without constant folding so this should not be an issue.

Update error message in the case where the partition explorer is being used outside of planning.

Adding free marker generated maxdir, imaxdir, mindir and imindir

remove import that violates build checks, fix typo in new test class name

Separate out SubDirectoryList from FileSystemSchemaFactory.

Fix unit test to correctly test all four functions.

Update partition explorer to take List instead of Collection. As the lists are used in parallel it should be explicit that these are expected to be ordered (which Collections do not guarantee).

Drop the extra file generated due to the header in the free marker template and fix a typo and remove an unused import.

  1. … 19 more files in changeset.
DRILL-1383: Support interpreted execution for Drill expression tree.

Fix OOM while running InterpreterBuilder

    • -0
    • +26
    ./DrillFuncInterpreter.java
    • -0
    • +30
    ./DrillSimpleFuncInterpreter.java
    • -0
    • +69
    ./InterpreterBuilder.java
    • -0
    • +334
    ./InterpreterEvaluator.java
    • -0
    • +143
    ./InterpreterGenerator.java
  1. … 10 more files in changeset.