profile

With statistics in place, it is useful to have the estimated rowcount along side the actual rowcount query profile's operator overview. A toggle button allows this with the estimated rows hidden by default

We can extract this from the Physical Plan section of the profile.

Added a toggle-ready table-column header

closes #1779

This bug arises because of the `toggleWarning()` javascript function that shows the warning if the number of slow major fragments matches the number of major fragments.

For failed queries, the number of major fragments = 0; and that matches the number of tags (which is also zero for a non-running query). Hence the warning shows.

This introduces support for JDBC's Statement.setMaxRows(int) API, which can help Drill execute a query much faster if it knows that not ALL the records in the resultset will be consumed upfront.

This Commit introduces the core changes to support the feature within Drill's execution engine

Protobuf Changes

1. RunQuery: Added "autolimit_rowcount"

2. QueryProfile: Added "autoLimit"

3. Regenerated Java and C++ client files

REST API support

1. Support for REST server to interpret a submitted query and also for rendering this information for an executed query

2. Updates to the Freemarker templates (for WebUI)

3. Safety check within Javascript (for WebUI)

JDBC API support

1. Introduces backend execution of 'ALTER SESSION' to apply the auto-limiting of resultset size

2. Added Unit Tests for PreparedStatement and Statement objects

3. Added getter setter methods to be skipped in testing for org.apache.drill.jdbc.test.Drill2489CallsAfterCloseThrowExceptionsTest.testclosedPreparedStmtOfOpenConnMethodsThrowRight()

Updates based on review comments

Additional Updates

Test Cleanup

1. Revert Drill2489 hack

2. Formatting in *StatementTest

3. Removal f redundant `statement.close()`

4. Manage new Exception thrown when setting invalid maxRow values

Final updates

1. Test changes

2. Trim trailing spaces in auto-limit value (Javascript)

3. Before & After annotations to synchronize changes to system values for MaxRows(auto-limit)

Reorganized tests due to synchronized locking

Removed conflicting JsonCreator in QueryWrapper

Additional test cleanup

closes #1714

The errorMessage.ftl has been moved to root dir, and unused `error.ftl` was removed

This provides an option to order the list of query profiles based on any of the displayed fields, including total duration. This way, a user can easily identify long running queries.

In addition, the number of profiles listed per page for both, completed and running list of queries, has been made configurable with the parameter: `drill.exec.http.profiles_per_page` (default is 10,25,50,100)

closes #1594

On query submission, a modal popup blocks further interaction until the result of the query are available.

Since the Query ID is not available at this point, the only way to cancel a running query is to navigate to it via the `/profiles` tab. For this, the modal allows a button to pop out to this tab without closing the current window which is waiting for the result-set.

[Update] Shared running query modal introduced

closes #1592

1. Rename return result option name to `exec.query.return_result_set_for_ddl`.

2. Add check if option value is the same as current value in DrillSqlWorker before setting result set option on query level.

3. Separate Session and Query options on Web UI.

closes #1584

1. Introduced warning for non-progressive fragments. Based on a threshold (`drill.exec.http.profile.warning.progress.threshold`), if all fragments have not made progress within that time, a warning is issued. The default is 5 minutes (300 sec)

2. Introduced a warning if any of the buffered operators spill to disk.

3. Introduced a warning for operators where the longest running fragment runs beyond a minimum threshold (drill.exec.http.profile.warning.time.skew.min), and runs atleast 2 times longer than the average (drill.exec.http.profile.warning.time.skew.ratio.process). The clock symbol with a tooltip indicates the extent of the skew. For wait times, the ratio is defined by `drill.exec.http.profile.warning.time.skew.ratio.wait`

3. Introduced a warning for operators where the average wait time of a scan operator exceeds its processing time, for a minimum threshold (drill.exec.http.profile.warning.scan.wait.min). The turtle symbol with a tooltip indicates which scan operator spent more time waiting than processing.

4. TableBuilder Refactored

a. Using attribute map instead of String arguments, eg. for 'title'

b. Removed APIs that pass a hyperlink since that is never used.

closes #1572

The Freemarker template missed a check for a query's Running state to display the [Cancel] button.

The fix is trivial and only needs the check.

DRILL-5571 ( PR #1531 ) accidentally introduced a bug that permanently keeps the "Duration" panel closed instead of opening it. This trivial patch fixes that.

A PR for DRILL-6084 broke the auto-complete library required for Edit Query page, with the removal of the static javascript library.

https://github.com/apache/drill/pull/1491/files#diff-2bb3c04f1a9eead1faf57212cce5d2a9

The dynamically loaded script is not defined in the Freemarker template page.

The fix is trivial.

1. BugFix on parent commit: Ensure query submission is done with user name when impersonation is enabled.

2. Support non-Mac browsers

3. Support keyboard submission for profile pages with Edit Query tab.

Inject a small space to improve readability of the pagination links (CSS)

closes #1369

* DRILL-6432: Show Button to print visualized query plan

Having the capability to only print the visualized query plan means the user can specifyi the zoom level (in the printer settings) when printing large complex plans that might span multiple pages.

* Update based on review comment

Padded button by 15px to avoid sticking to pane

closes #1278

When running a query in Drill, the `/profiles` page will show the search (and pagination) capabilities only for the top most visible table (i.e. Running Queries ).

The Completed Queries table will show the search feature only when there are no running queries. This is because the backend uses a generalized freemarker macro to define the seach capabilities for the tables being rendered. With running queries, both, running and completed queries tables have the same element ID, resulting in the search capability only being applied to the first table.

This modifies the Freemarker macro to take an additional argument for distinguishing between Running and Completed Queries table.

Introduced change to the Protobuf to inject the text-equivalent of the QueryID into the profile. This way, the profile's file name can be changed, but restored back based on this new field.

The Profile UI also shows the Query ID, though this is not inferred from this new field, for sake of backward compatibility with older profiles.

closes #1265

closes #1207

As part of this coomit added Bootstrap's Glyphicons.

closes #1197

Based on the commit for DRILL-5981 (PR #1043), this commit further leverages the Ace JavaScript library with customizations specific to Drill.

This commit introduces the following to the Query Editor (including the Edit Query tab within an existing profile to rerunning the query).

1. Syntax highlighting (This is supported for submitted query profiles

2. Autocomplete supported in editors

3. Specifying Drill specific keywords and functions in visible autocomplete

4. Key snippets (template SQLs) allowing for rapid writing of syntax:

i. Query System Tables

ii. CView, CTAS and CTempTAS

iii. Alter Session

iv. Explain and Select * queries

NOTE: The lists for #3 and #4 are not exhaustive. As more features are added to Drill, these lists can be expanded.

Updates based on review comments

1. Disabled warning message

2. Extended reserved keyword list

3. Fixed bugs

List:

Update 1: Bug fix for when Impersonation is enabled

Update 2: Remove the duplicate editor that might get injected

Update 3: Removed trailing whitespaces in Javascripts

close apache/drill#1084

closes #1061

Leverage existing DataTables libraries to paginate a long pre-fetched list of profiles for listing.

Added benefit of querying through that list (search field) is also available for a user

Minor change made to the display text for prefetching of profiles (DRILL-5259) so that it is not confused with what this commit adds to the UI.

This closes #1029

closes #1021

Using the DataTables jQuery library, we can now sort tables (fragments and operators) to group like values or sort on a column.

In addition, additional tooltips have been provided for these columns

closes #969

Added an additional field in the UI allowing a user to specify the max number of profiles to load via Refresh

closes #963

Minor update to restore the fragment Gantt chart

https://issues.apache.org/jira/browse/DRILL-5766?focusedCommentId=16176955&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16176955

closes #956