drill

Clone Tools
  • last updated 21 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-6373:

- Adds code to return the proper vector type given the actual vector, adjusting metadata as needed.

- Refactor result set loader

- Revised projection & vector cache

closes #1244

  1. … 25 more files in changeset.
DRILL-6321: Lateral Join and Unnest - rules, options, logical plan supports

Included changes:

* Add planner.enable_unnest_lateral option. Default value set to false.

* Enable FilterCorrectRule

* Add support to logical plan

* Fix rebase errors for DRILL-6321 commits

  1. … 4 more files in changeset.
DRILL-6281: Introduce Collectors class for internal iterators

closes #1238

DRILL-6347: Change method names to "visitField".

Further change the method names to "visitField" with Vlad Rozov's comments.

closes #1236

DRILL-6361: Revised typeOf() function versions

Added more unit tests.

Updated to handle VARDECIMAL

The VARDECIMAL type was recently added to Drill. Added support for this type. The sqlTypeOf() function now returns DECIMAL(p, s) for precision p, scale s.

closes #1242

DRILL-6027: - Added fallback option for HashJoin. - No copy of incoming for single partition, and avoid HT resize. - Fix memory leak when cancelling while spill file is read - get correct schema when probe side is empty - Re-create the HashJoinProbe

  1. … 28 more files in changeset.
DRILL-6364: Handle Cluster Info in WebUI when existing/new bits restart

As a follow up to DRILL-6289, the following improvements have been done:

1. When loading the page for the first time, the WebUI enables the shutdown button without actually checking the state of the Drillbits.

The ideal behaviour should be to disable the button till the state is verified. [Done]

If a Drillbit is confirmed down (i.e. not in `/state` response), it is marked as OFFLINE and button is disabled.

2. When shutting down the current Drillbit, the WebUI no more has access to the cluster state.

The ideal behaviour here should be to fetch the state from any of the other Drillbits to update the status. [Done]

With the current Drillbit down, the other bits are requested for cluster state info and update accordingly.

3. When a new, previously unseen Drillbit comes up, the WebUI will never render it because the table is statically generated during the first page load.

The idea behaviour should be to append to the table on discovery of a new node. [Done]

The new Drillbit info is injected and a prompt appears to refresh the page to re-populate any missing info. This also works with feature (2) mentioned above.

The only Java code change was to have the state response carry the address and http-port as a tuple, instead of the user-port (which is never used).

Updates based on review

1. Added descriptive comments to functions

2. Handled possible race condition from multiple httpRequests for cluster state

3. Eliminated unused stateKey variable

4. Best practice of using local (let) instead of global (var) objects, and substituting currentRow variable instead of the jQuery search.

Additional changes based on review

1. Random selection of Drillbits to query for state when primary Drillbit is down (limited to 3; with a timeout of 3 sec)

2. Indicate when an Offline Drillbit is de-registered, versus just Offline as per Zookeeper

3. Hide shutdown column when Authentication is enabled, but the user is NOT an Admin.

When the column is visible, remote bits are disabled because

4. Metrics will be shown all the time (except HTTPS), because the information is available and a non-admin user would anyway not have actionable capabilities

5. Basic clean up on unused variables

Hide shutdown buttons for HTTPS scenarios

Fixed check for 'https'

Since, `location.protocol` returns a trailing ':' as well

Handle metrics lookup on local Drillbit for HTTPS

If the webpage is accessed via IP, the certificate is tied to the IP and not the FQDN, which is what is used for fetching the metrics.

close apache/drill#1241

DRILL-6345: DRILL Query fails on Function LOG10

- Added log10 function implementation

closes #1230

DRILL-143: CGroup Support for Drill-on-YARN

Initial patch

1. Minor fix up

2. Updated variable name for pid file

Revert changes to yarn-drillbit.sh

Based on discussion for PR #1239 (Ref: https://github.com/apache/drill/pull/1239)

closes #1239

    • -15
    • +15
    /distribution/src/resources/drillbit.sh
DRILL-6422: Update guava to 23.0 and shade it

- Fix compilation errors for new version of Guava.

- Remove usage of deprecated API

- Shade guava and add dependencies to the shaded version

- Ban unshaded package

- Introduce drill-shaded module and move guava-shaded under it

- Add methods to convert shaded guava lists to the unshaded ones

- Add instruction for publishing artifacts to the Apache repository

    • -0
    • +62
    /docs/dev/UpgradeGuava.md
    • -0
    • +72
    /drill-shaded/drill-shaded-guava/pom.xml
    • -0
    • +74
    /drill-shaded/pom.xml
  1. … 68 more files in changeset.
DRILL-6348: Received batches are now owned by the receive operators instead of the parent

closes #1237

DRILL-6281: Refactor TimedRunnable

DRILL-6281: Refactor TimedRunnable (rename TimedRunnable to TimedCallable)

DRILL-5927: Fixed memory leak in TestBsonRecordReader, and sped up the test.

closes #1234

DRILL-6333: Fixed Quotation marks

Initial step to making the source-code ready for Javadoc generation

This closes #1229

DRILL-6342: Fix schema path unIndexed method to return correct value

closes #1231

doc edits

DRILL-6343: bit vector copyFromSafe is not doing realloc

DRILL-3855: Enable FilterSetOpTransposeRule, DrillProjectSetOpTransposeRule

closes #1226

Drill 1.13 doc updates and other edits

    • -44
    • +1
    /_docs/install/050-starting-drill-in-distributed-mode.md
    • -2
    • +2
    /_docs/install/060-starting-the-web-ui.md
    • -0
    • +53
    /_docs/install/061-stopping-drill.md
DRILL-6323: Fix license headers

DRILL-5846: Improve parquet performance for Flat Data Types

closes #1060

  1. … 12 more files in changeset.
DRILL-6328: Adding unit testing docs.

closes #1220

    • -0
    • +247
    /docs/dev/ClusterFixture.md
    • -0
    • +75
    /docs/dev/LegacyTestingFrameworks.md
    • -0
    • +40
    /docs/dev/MockRecordReader.md
    • -0
    • +33
    /docs/dev/OperatorFixture.md
    • -0
    • +105
    /docs/dev/TestLogging.md
    • -0
    • +72
    /docs/dev/Testing.md
DRILL-6328: Consolidated developer docs in the docs folder.

closes #1220

    • -0
    • +46
    /docs/dev/Environment.md
DRILL-6339: Add a new option to disable TopN (for testing)

This closes #1219

DRILL-6323: Lateral Join - Review feedback changes with License header fixes

This closes #1212

DRILL-6341: Fixed failing tests for mongodb storage plugin by upgrading MongoDB version.

closes #1222

DRILL-6338: Do not skip license maven plugin when formatting generated protobuf files.

This closes #1215

DRILL-6173: Support transitive closure during filter push down and partition pruning

closes #1216

  1. … 21 more files in changeset.
DRILL-6335: Column accessor refactoring

closes #1218

  1. … 29 more files in changeset.