drill

Clone Tools
  • last updated 13 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
edit spnego and impersonation docs

DRILL-6327: Update unary operators to handle IterOutcome.EMIT Note: Handles for Non-Blocking Unary operators (like Filter/Project/etc) with EMIT Iter.Outcome

closes #1240

edit to spnego doc

DRILL-6375 : Support for ANY_VALUE aggregate function

closes #1256

  1. … 22 more files in changeset.
minor edit

edits

Add SPNEGO content

    • -0
    • +148
    /_docs/configure-drill/securing-drill/091-configuring-drill-to-use-spnego-for-http-authentication.md
    • -2
    • +4
    /_docs/install/060-starting-the-web-ui.md
DRILL-143: Support CGROUPs resource management

Introduces the DRILLBIT_CGROUP option in drill-env.sh.

The startup script checks if the specified CGroup (ver 2) is available and tries to apply it to the launched Drillbit JVM.

This would benefit not just Drill-on-YARN usecases, but any setup that would like CGroups for enforcement of (cpu) resources management.

(Also introduced SYS_CGROUP_DIR to account for possible non default locations of CGroup).

e.g when Drillbit is configured to use `drillcpu` cgroup

```

[root@maprlabs ~]# /opt/mapr/drill/apache-drill-1.14.0-SNAPSHOT/bin/drillbit.sh restart

Stopping drillbit

..

Starting drillbit, logging to /var/log/drill/drillbit.out

WARN: Drillbit's CPU resource usage will be managed under the CGroup : drillcpu (up to 4.00 cores allowed)

```

e.g. Non-existent CGroup `droolcpu` is used

```

[root@kk127 ~]# /opt/mapr/drill/apache-drill-1.14.0-SNAPSHOT/bin/drillbit.sh restart

Stopping drillbit

..

Starting drillbit, logging to /var/log/drill/drillbit.out

ERROR: cgroup droolcpu does not found. Ensure that daemon is running and cgroup exists

```

closes #1200

    • -0
    • +10
    /distribution/src/resources/drill-env.sh
    • -0
    • +39
    /distribution/src/resources/drillbit.sh
DRILL-6296: Add operator metrics for batch sizing for merge join

closes #1181

DRILL-6286: Fixed incorrect reference to shutdown in drillbit.log

closes #1196

DRILL-6273: Removed dependency licensed under Category X

closes #1195

    • -0
    • +24
    /tools/fmpp/src/main/java/bsh/EvalError.java
    • -0
    • +24
    /tools/fmpp/src/main/java/bsh/package-info.java
DRILL-6300: Refresh protobuf C++ source files

close apache/drill#1194

DRILL-6299: Fixed a filter pushed down issue when a column doesn't have stats

close apache/drill#1192

DRILL-6270: Added general option --jvm, which can be used to set the remote debug jvm options.

The option is not needed for drillbit startup, you can set the jvm properties directly after the start command.

Usage:

bin/drill-embedded --jvm "[JVM options]"

bin/sqlline -u "jdbc:drill:zk=zkaddr:port" --jvm "[JVM options]"

bin/drillbit.sh start [JVM options]

sqlline.bat -u "jdbc:drill:zk=local" --jvm "[JVM options]"

closes #1210

    • -5
    • +13
    /distribution/src/resources/sqlline.bat
DRILL-5937: drill-module.conf : Changed timeout to 30 seconds, ExecConstant.java : Changed comment

DRILL-5937: ExecConstant.java : removed comment

DRILL-5937: CheckStyle fix

close apache/drill#1190

DRILL-6103: lsb_release: command not found

close apache/drill#1191

Thanks to Sanel Zukan for providing a small patch that checks for /etc/fedora-release path. This is more common, than lsb_release command on Linux distros.

DRILL-6321: Lateral Join and Unnest - initial implementation for parser and planning

  1. … 12 more files in changeset.
edit

edit

doc edit

1.13 doc updates and edits

DRILL-6256: Remove references to java 7 from readme and other files

close apache/drill#1172

    • -1
    • +1
    /exec/java-exec/src/main/sh/drill-config.sh
DRILL-6303: Provide a button to copy the Drillbit's JStack shown in /threads

Provides a button in the web UI to copy the thread stack to the user's clipboard.

closes #1199

DRILL-6279: Indicate operators that spilled in-memory data to disk on Web UI

As part of this coomit added Bootstrap's Glyphicons.

closes #1197

DRILL-6289: Cluster view should show more relevant information

Protobuf change to carry HTTP port info

Allow CORS for access to remote Drillbit metrics

Cross-origin resource sharing (CORS) is required to ensure that the WebServer is able serve REST calls for status pages.

Materialize relevant metrics

1. Heap memory (incl usage)

2. Heap memory (incl usage)

3. Average System Load (last 1 min)

4. Option to view from other nodes (pop out)

5. Added Glyphicons

Update System Table and related tests

1. Updated System Table to show HTTP port

2. Updated unit tests

Skip updating remote bit info when HTTPS (SSL) or Authentication is enabled.

Default CpuGaugeSet is public; Added Gauges

* CPU Utiization by Drill

* Uptime

Show ALL Buttons, but do HTTPS Check

Reduce power button to icon

Allowing CORS for /status/metrics only

Accounting for situations when JVM does not report Process CPU Load

i.e. returned value is negative.

See https://docs.oracle.com/javase/7/docs/jre/api/management/extension/com/sun/management/OperatingSystemMXBean.html#getProcessCpuLoad()

Addressed shutdown security conditions

Added C++ Client Protobuf

Added steps for Protobuf generation to protocol/readme.txt

This closes #1203

    • -0
    • +1
    /protocol/src/main/protobuf/Coordination.proto
DRILL-6271: Updated copyright range in NOTICE

closes #1188

DRILL-6290: Refactor TestInfoSchemaFilterPushDown tests to use PlanTestBase utility methods

closes #1186

DRILL-6224: Publish metrics gauge values correctly

The `metrics.ftl` page had gauges incorrectly set to near zero values. The commit for metrics.ftl fixes that, and also provides an estimate of the current direct memory actively in use (based on the `drill.allocator.root.used` value reported by the Drillbit)

closes #1160

DRILL-6288: Upgrade org.javassist:javassist and org.reflections:reflections

closes #1185

1.13 doc updates - cgroups memory updates

    • -0
    • +94
    /_docs/configure-drill/121-configuring-cgroups-to-control-cpu-usage.md
    • -94
    • +0
    /_docs/drill-on-yarn/094-appendix-e-using-cgroups-to-control-cpu-usage.md