Clone Tools
  • last updated 29 mins ago
Constraints: committers
Constraints: files
Constraints: dates
DRILL-7337: Add vararg UDFs support

    • -0
    • +10
    • -38
    • +38
  1. … 36 more files in changeset.
DRILL-7011: Support schema in scan framework

* Adds schema support to the row set-based scan framework and to the "V3" text reader based on that framework.

* Adding the schema made clear that passing options as a long list of constructor arguments was not sustainable. Refactored code to use a builder pattern instead.

* Added support for default values in the "null column loader", which required adding a "setValue" method to the column accessors.

* Added unit tests for all new or changed functionality. See TestCsvWithSchema for the overall test of the entire integrated mechanism.

* Added tests for explicit projection with schema

* Better handling of date/time in column accessors

* Converted recent column metadata work from Java 8 date/time to Joda.

* Added more CSV-with-schema unit tests

* Removed the ID fields from "resolved columns", used "instanceof" instead.

* Added wildcard projection with an output schema. Handles both "lenient" and "strict" schemas.

* Tagged projection columns with their output schema, when available.

* Scan projection added modes for wildcard with an output schema. The reader projection added support for merging reader and output schemas.

* Includes refactoring of scan operator tests (the test file grew too large.)

* Renamed some classes to avoid confusing reader schemas with output schemas.

* Added unit tests for the new functionality.

* Added "lenient" wildcard with schema test for CSV

* Added more type conversions: string-to-bit, many-to-string

* Fixed bug in column writer for VarDecimal

* Added missing unit tests, and fixed bugs, in Bit column reader/writer

* Cleaned up a number of unneded "SuppressWarnings"

closes #1711

  1. … 223 more files in changeset.
DRILL-2326: Fix scalar replacement for the case when static method which does not return values is called

- Fix check for return function value to handle the case when created object is returned without assigning it to the local variable

closes #1687

    • -11
    • +13
  1. … 2 more files in changeset.
DRILL-6524: Prevent incorrect scalar replacement for the case of assigning references inside if block

    • -28
    • +110
    • -1
    • +30
    • -3
    • +3
  1. … 3 more files in changeset.
DRILL-6715: Update descriptions for System Options table

With introduction of DRILL-5735 , the descriptions for about half the system options still remain missing. This commit collects descriptions review by @bbevens

1. Update options for HashAgg/Join (@Ben-Zvi )

2. Update options for Parquet Reader/Writer (@sachouche )

3. Update options for Planners (@HanumathRao , @vdiravka , @KazydubB )

4. Update options for BatchSizing (@bitblender )

5. Update options for Planner Optimizations (@arina-ielchiieva )

6. Update options for Security & Kafka (Krystal Nguyen)

7. Update options for Misc entries (@arina-ielchiieva , @vvysotskyi )

In additional, there is a patch for `org.apache.drill.exec.compile.ClassTransformer.scalar_replacement` , which appears to have replaced `exec.compile.scalar_replacement`. References to the latter have been removed to avoid confusion.

Additional changes include moving the `ClassTransformer` validator to ``

Adding support for internal options' descriptions

Removed mention of {{Will be removed in 1.15.0}}. (Refer DRILL-6527)

  1. … 6 more files in changeset.
DRILL-6868: Upgrade Janino compiler to 3.0.11

- Remove workaround where removing adjacent ALOAD-POP instruction pairs

- Remove ModifiedUnparser and use DeepCopier for modifying methods instead of modifying it with custom Unparser implementation

closes #1553

    • -8
    • +2
  1. … 3 more files in changeset.
DRILL-5735: UI options grouping and filtering & Metrics hints Allow search/sort in the Options webUI Additional details, like the description is provided as well in a JScript lookup map. This helps reduce the need for the server to constantly recreate the entire page with the description details, as the client browser can fill in these details. Developers will be expected to update the description as old/new options are introduced or deprecated. System table options_val also carries the description, albeit a 'shorter' version.

closes #1279

  1. … 13 more files in changeset.
DRILL-6422: Replace guava imports with shaded ones

    • -1
    • +1
  1. … 970 more files in changeset.
DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM

closes #1446

  1. … 22 more files in changeset.
DRILL-6656: Disallow extra semicolons and multiple statements on the same line.

closes #1415

  1. … 144 more files in changeset.
DRILL-6386: Remove unused imports and star imports.

  1. … 229 more files in changeset.
DRILL-6320: Fixed license headers.

closes #1207

  1. … 2052 more files in changeset.
DRILL-6094: Decimal data type enhancements

Add ExprVisitors for VARDECIMAL

Modify writers/readers to support VARDECIMAL

- Added usage of VarDecimal for parquet, hive, maprdb, jdbc;

- Added options to store decimals as int32 and int64 or fixed_len_byte_array or binary;

Add UDFs for VARDECIMAL data type

- modify type inference rules

- remove UDFs for obsolete DECIMAL types

Enable DECIMAL data type by default

Add unit tests for DECIMAL data type

Fix mapping for NLJ when literal with non-primitive type is used in join conditions

Refresh protobuf C++ source files

Changes in C++ files

Add support for decimal logical type in Avro.

Add support for date, time and timestamp logical types.

Update Avro version to 1.8.2.

    • -0
    • +4
    • -0
    • +6
  1. … 200 more files in changeset.
DRILL-6375 : Support for ANY_VALUE aggregate function

closes #1256

    • -0
    • +7
  1. … 36 more files in changeset.
DRILL-6259: Support parquet filter push down for complex types

close apache/drill#1173

    • -0
    • +6
  1. … 24 more files in changeset.
DRILL-6028: Allow splitting generated code in ChainedHashTable into blocks to avoid "code too large" error

1. Added new parameter seedValue to getHashBuild and getHashProbe methods in HashTableTemplate.

2. Generate logical expression for each key so its can be split into blocks if number of expressions in method exceeds upper limit.

3. ParameterExpression was added to generate reference to method parameter during code generation.

closes #1071

    • -1
    • +6
  1. … 12 more files in changeset.
DRILL-5783, DRILL-5841, DRILL-5894: Rationalize test temp directories

This change includes:


- A unit test is created for the priority queue in the TopN operator.

- The code generation classes passed around a completely unused function registry reference in some places so it is removed.

- The priority queue had unused parameters for some of its methods so it is removed.


- Created standardized temp directory classes DirTestWatcher, SubDirTestWatcher, and BaseDirTestWatcher. And updated all unit tests to use them.


- Removed the dfs_test storage plugin for tests and replaced it with the already existing dfs storage plugin.


- General code cleanup.

- Removed unnecessary use of String.format in the tests.

This closes #984

  1. … 364 more files in changeset.
DRILL-5723: Added System Internal Options That can be Modified at Runtime Changes include:

1. Addition of internal options.

2. Refactoring of OptionManagers and OptionValidators.

3. Fixed ambiguity in the meaning of an option type, and changed its name to accessibleScopes.

4. Updated javadocs in the Option System classes.

5. Added RestClientFixture for testing the Rest API.

6. Fixed flakey test in TestExceptionInjection caused by race condition.

7. Fixed various tests which started zookeeper but failed to shut it down at the end of tests.

8. Added port hunting to the Drill Webserver for testing

9. Fixed various flaky tests

10. Fix compile issue

closes #923

  1. … 85 more files in changeset.
DRILL-5547: Linking config options with system option manager

closes #868

  1. … 38 more files in changeset.
DRILL-5140: Fix CompileException in run-time generated code when record batch has large number of fields.

- Changed estimation of max index value and added comments.

close #818

  1. … 2 more files in changeset.
DRILL-5319: Refactor "contexts" for unit testing closes #787

This PR is purely a refactoring: no functionality is added or changed.

The refactoring splits various context and related classes into a set

of new interfaces with needed for operator-level unit tests. The other,

Drillbit-related methods are left in the original interfaces. Most code

need not change.

The changes here allow operator-level unit tests to mock up the

exec-time methods so we can use them without firing up a Drillbit (or

using mocking libraries).

A later PR will provide the sub-operator test framework that uses this


Changes include:

* The OptionManager is split, with read-only methods moving to a new

OptionSet interface.

* The FragmentContext is split, with an exec-only FragmentExecContext

proving low-level methods.

* OperatorStats is split, with a new OperatorStatReceiver class

providing write-only support to operators.

* Several places that accepted an OperatorContext or FragmentContext,

but needed only an allocator, are changed to accept the allocator


Includes fixes for code review comments

Adds more comments. Postpones the suggested rename until all affected

code is in master, else it will be difficult to synchronize the rename

across multiple branches.

  1. … 25 more files in changeset.
DRILL-5116: Enable generated code debugging in each Drill operator

DRILL-5052 added the ability to debug generated code. The reviewer suggested

permitting the technique to be used for all Drill operators. This PR provides

the required fixes. Most were small changes, others dealt with the rather

clever way that the existing byte-code merge converted static nested classes

to non-static inner classes, with the way that constructors were inserted

at the byte-code level and so on. See the JIRA for the details.

This code passed the unit tests twice: once with the traditional byte-code

manipulations, a second time using "plain-old Java" code compilation.

Plain-old Java is turned off by default, but can be turned on for all

operators with a single config change: see the JIRA for info. Consider

the plain-old Java option to be experimental: very handy for debugging,

perhaps not quite tested enough for production use.

close apache/drill#716

  1. … 56 more files in changeset.
DRILL-5052: Option to debug generated Java code using an IDE

Provides a second compilation path for generated code: “plan old Java”

in which generated code inherit from their templates. Such code can be

compiled directly, allowing easy debugging of generated code.

Also show to generate two classes in the External Sort Batch as “plain

old Java” to enable IDE debugging of that generated code. Required

minor clean-up of the templates.

Fixes some broken toString( ) methods in code generation classes

Fixes a variety of small compilation warnings

Adds Java doc to a few classes

Includes clean-up from code review comments.

close apache/drill#660

    • -0
    • +69
    • -0
    • +189
    • -0
    • +146
  1. … 19 more files in changeset.
DRILL-4743: Allow new options to control filter selectivity (min/max bounds)

Addressed review comments 2

Addressed review comments 3

close apache/drill#534

  1. … 13 more files in changeset.
DRILL-4437: Operator unit test framework

Closes #394

  1. … 10 more files in changeset.
DRILL-4275: create TransientStore for short-lived objects; refactor PersistentStore to introduce pagination mechanism

  1. … 100 more files in changeset.
DRILL-3581: Upgrade HPPC to 0.7.1

  1. … 13 more files in changeset.
DRILL-3742: Classpath scanning and build improvement

Makes the classpath scanning a build time class discovery

Makes the fmpp generation incremental

Removes some slowness in DrillBit closing

Reduces the build time by 30%

This closes #148

  1. … 143 more files in changeset.
DRILL-3589: Update JDBC driver to shade and minimize dependencies.

Update build process to use shading, stop using Proguard.

Add simple integration test that verifies that the JDBC driver works

correctly with a clean classpath.

  1. … 9 more files in changeset.
DRILL-3497: Throw UserException#validationError instead of SetOptionException and ExpressionParsingException for options

This closes #98

+ Fixed bug in PositiveLongValidator (0 is not positive)

+ Added UserExceptionMatcher for unit tests

DRILL-3497: Addressing Jason's review comments (1)

  1. … 12 more files in changeset.