Clone Tools
  • last updated 17 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-7530: Fix class names in loggers

1. Fix incorrect class names for loggers.

2. Minor code cleanup.

closes #1957

  1. … 55 more files in changeset.
DRILL-7417: Add user logged in/out event in info level logs

  1. … 1 more file in changeset.
DRILL-7351: Added tokens to Web forms to prevent CSRF attacks

    • -0
    • +60
    ./CsrfTokenInjectFilter.java
    • -0
    • +61
    ./CsrfTokenValidateFilter.java
  1. … 16 more files in changeset.
DRILL-7367: Remove Server details from response headers

closes #1851

  1. … 1 more file in changeset.
DRILL-7222: Visualize estimated and actual row counts for a query

With statistics in place, it is useful to have the estimated rowcount along side the actual rowcount query profile's operator overview. A toggle button allows this with the estimated rows hidden by default

We can extract this from the Physical Plan section of the profile.

Added a toggle-ready table-column header

closes #1779

  1. … 3 more files in changeset.
DRILL-7338: REST API calls to Drill fail due to insufficient heap memory

This PR allows for the 85% threshold to be customizable with a value of 0 meant for disabling.

closes #1837

  1. … 2 more files in changeset.
DRILL-7204: Add proper validation when creating plugin

- Added validation for an empty plugin name.

- Added an URL encoding for pluing name, so plugins with special characters can be accessed without issues.

- Replaced alerts with modal windows.

- Added a confirmation dialog when disabling a plugin on Update page.

  1. … 3 more files in changeset.
DRILL-7196: Queries are still runnable on disabled plugins

- Storage client is not created anymore for disabled plugins

- GET "/storage/{name}.json" endpoint now working with

plugin configuration directly, without client instantination.

It have increased UI responsitivity.

- Hbase and mongo base test classes refactored to honor enabled

plugin attribute

- Fixed path contructor for mongo test datasets:

Now it is cross-platform

- Fixed test json files format which using plugin definitions

- Code cleanup

  1. … 106 more files in changeset.
DRILL-7190: Missing backward compatibility for REST API with DRILL-6562

  1. … 1 more file in changeset.
DRILL-7186: Add back storage.json REST endpoint (regression after DRILL-6562)

DRILL-7160: e.q.max_rows QUERY-level option shown even if not set

The fix is to force setting to zero IFF autoLimit was intended to be set originally but is inapplicable; such as 'SHOW DATABASES'. If autolimit was not intended to be applied, setting the value to zero is not required.

WebUI is also patched to not show the zero value set in such scenarios.

  1. … 1 more file in changeset.
DRILL-7048: Implement JDBC Statement.setMaxRows() with System Option

This introduces support for JDBC's Statement.setMaxRows(int) API, which can help Drill execute a query much faster if it knows that not ALL the records in the resultset will be consumed upfront.

This Commit introduces the core changes to support the feature within Drill's execution engine

Protobuf Changes

1. RunQuery: Added "autolimit_rowcount"

2. QueryProfile: Added "autoLimit"

3. Regenerated Java and C++ client files

REST API support

1. Support for REST server to interpret a submitted query and also for rendering this information for an executed query

2. Updates to the Freemarker templates (for WebUI)

3. Safety check within Javascript (for WebUI)

JDBC API support

1. Introduces backend execution of 'ALTER SESSION' to apply the auto-limiting of resultset size

2. Added Unit Tests for PreparedStatement and Statement objects

3. Added getter setter methods to be skipped in testing for org.apache.drill.jdbc.test.Drill2489CallsAfterCloseThrowExceptionsTest.testclosedPreparedStmtOfOpenConnMethodsThrowRight()

Updates based on review comments

Additional Updates

Test Cleanup

1. Revert Drill2489 hack

2. Formatting in *StatementTest

3. Removal f redundant `statement.close()`

4. Manage new Exception thrown when setting invalid maxRow values

Final updates

1. Test changes

2. Trim trailing spaces in auto-limit value (Javascript)

3. Before & After annotations to synchronize changes to system values for MaxRows(auto-limit)

Reorganized tests due to synchronized locking

Removed conflicting JsonCreator in QueryWrapper

Additional test cleanup

closes #1714

  1. … 31 more files in changeset.
DRILL-7145: Exceptions happened during retrieving values from ValueVector are not being displayed at the Drill Web UI closes #1727

  1. … 1 more file in changeset.
DRILL-7011: Support schema in scan framework

* Adds schema support to the row set-based scan framework and to the "V3" text reader based on that framework.

* Adding the schema made clear that passing options as a long list of constructor arguments was not sustainable. Refactored code to use a builder pattern instead.

* Added support for default values in the "null column loader", which required adding a "setValue" method to the column accessors.

* Added unit tests for all new or changed functionality. See TestCsvWithSchema for the overall test of the entire integrated mechanism.

* Added tests for explicit projection with schema

* Better handling of date/time in column accessors

* Converted recent column metadata work from Java 8 date/time to Joda.

* Added more CSV-with-schema unit tests

* Removed the ID fields from "resolved columns", used "instanceof" instead.

* Added wildcard projection with an output schema. Handles both "lenient" and "strict" schemas.

* Tagged projection columns with their output schema, when available.

* Scan projection added modes for wildcard with an output schema. The reader projection added support for merging reader and output schemas.

* Includes refactoring of scan operator tests (the test file grew too large.)

* Renamed some classes to avoid confusing reader schemas with output schemas.

* Added unit tests for the new functionality.

* Added "lenient" wildcard with schema test for CSV

* Added more type conversions: string-to-bit, many-to-string

* Fixed bug in column writer for VarDecimal

* Added missing unit tests, and fixed bugs, in Bit column reader/writer

* Cleaned up a number of unneded "SuppressWarnings"

closes #1711

  1. … 219 more files in changeset.
DRILL-7051: Upgrade jetty - upgrade Jetty dependencies to 9.3 version - adaptation to the new Jetty API (SessionHandler, LoginService, AbstractLoginService) - add JavaDocs and code refactoring

closes #1681

  1. … 5 more files in changeset.
DRILL-7056: Drill fails with NPE when starting in distributed mode & 31010 port is used closes #1656

  1. … 1 more file in changeset.
DRILL-7052: Relative path for URL redirection

  1. … 5 more files in changeset.
DRILL-7036: Improve UI for alert and error messages closes #1644 This PR standardizes error and alert messages to a cleaner interface by leveraging Bootstraps UX elements for publishing the messages in a presentable format. Exceptions reported back to the browser and rendered in a neat tabular format (using Panels) All errors can be redirected to errorMessage.ftl which will render it in a neat format. Alerts are replaced with modals. Interactions (pages) affected by Alert modals 1. Missing Query submission 2. profile Query Rerun 3. invalid Profile Listing Fetch 4. invalid Option Value for update 5. Missing username/password submission

The errorMessage.ftl has been moved to root dir, and unused `error.ftl` was removed

  1. … 9 more files in changeset.
DRILL-6971: Display query state in query result page

  1. … 2 more files in changeset.
DRILL-6050: Provide a limit to number of rows fetched for a query in UI

Currently, the WebServer side needs to process the entire set of results and stream it back to the WebClient.

Since the WebUI does paginate results, we can load a larger set for pagination on the browser client and relieve pressure off the WebServer to host all the data (most of which will never be streamed to the browser).

e.g. Fetching all rows from a 1Billion records table is impractical and can be capped at (say) 1K. Currently, the user has to explicitly specify LIMIT in the submitted query.

An option is provided in the field to allow for this entry, and can be set to selected by default for the Web UI.

The submitted query indicates that an auto-limiting wrapper was applied.

[Update #1] Updated as per comments

1. Limit Wrapping Unchecked by default

2. Full List configuration of results

[Update #2] Minor update

[Update #3] Followup

closes #1593

  1. … 5 more files in changeset.
DRILL-6942: Provide ability to sort list of profiles on Drill Web UI

This provides an option to order the list of query profiles based on any of the displayed fields, including total duration. This way, a user can easily identify long running queries.

In addition, the number of profiles listed per page for both, completed and running list of queries, has been made configurable with the parameter: `drill.exec.http.profiles_per_page` (default is 10,25,50,100)

closes #1594

  1. … 6 more files in changeset.
DRILL-6933: Fix ctrl+enter when Impersonation is disabled

Without impersonation, the key combination did not work because a conditional block of the FTLs skipped the Javascript functions required for submitting with the key combination.

This commit fixes that by unifying the functions that are common to both approaches of submitting a query (with and without impersoination)

[Update] Changes made to isOnlyImpersonationEnabled method and onlyImpersonationEnabled variable

[Update] userName is a global var in the script

closes #1591

  1. … 3 more files in changeset.
DRILL-6921: Add Clear button for /options filter

This commit adds the following search enhancements:

1. Addition of a clear ('x') button in the search field

2. If a filter term has been entered in the search box, on clicking the Update or Reset-to-Default button of any option, the reloaded page will re-apply the last entered filter term in the search box.

3. If the search box is empty, on clicking the Update or Reset-to-Default button of any option, the reloaded page will automatically filter out everything except the updated/reset option to show the changed value.

4. Customization of the quick search terms. (using defaults in drill-module.conf)

closes #1588

  1. … 3 more files in changeset.
DRILL-6922: Do not set return result set option on query level if it is the same as current value

1. Rename return result option name to `exec.query.return_result_set_for_ddl`.

2. Add check if option value is the same as current value in DrillSqlWorker before setting result set option on query level.

3. Separate Session and Query options on Web UI.

closes #1584

  1. … 5 more files in changeset.
DRILL-6879: Show warnings for potential performance issues

1. Introduced warning for non-progressive fragments. Based on a threshold (`drill.exec.http.profile.warning.progress.threshold`), if all fragments have not made progress within that time, a warning is issued. The default is 5 minutes (300 sec)

2. Introduced a warning if any of the buffered operators spill to disk.

3. Introduced a warning for operators where the longest running fragment runs beyond a minimum threshold (drill.exec.http.profile.warning.time.skew.min), and runs atleast 2 times longer than the average (drill.exec.http.profile.warning.time.skew.ratio.process). The clock symbol with a tooltip indicates the extent of the skew. For wait times, the ratio is defined by `drill.exec.http.profile.warning.time.skew.ratio.wait`

3. Introduced a warning for operators where the average wait time of a scan operator exceeds its processing time, for a minimum threshold (drill.exec.http.profile.warning.scan.wait.min). The turtle symbol with a tooltip indicates which scan operator spent more time waiting than processing.

4. TableBuilder Refactored

a. Using attribute map instead of String arguments, eg. for 'title'

b. Removed APIs that pass a hyperlink since that is never used.

closes #1572

    • -0
    • +37
    ./profile/HtmlAttribute.java
  1. … 4 more files in changeset.
DRILL-6847: Add Query Metadata to RESTful Interface

closes #1539

DRILL-6668: In Web UI, highlight options that are not default values

This commit introduces a new button on the options page that allows a user to reset an option to its system default value.

To simplify things, a tooltip is shown when the mouse hovers over the button. If the option value is already default, the button is disabled.

Currently, the Update button redirects to /option/optionName . This change reuses what we already are using to set the default (using AJAX) and auto-refreshing

Switch [Default] label to [Reset]

Patch To Pass StatusResourcesTest

closes #1543

  1. … 1 more file in changeset.
DRILL-6715: Update descriptions for System Options table

With introduction of DRILL-5735 , the descriptions for about half the system options still remain missing. This commit collects descriptions review by @bbevens

1. Update options for HashAgg/Join (@Ben-Zvi )

2. Update options for Parquet Reader/Writer (@sachouche )

3. Update options for Planners (@HanumathRao , @vdiravka , @KazydubB )

4. Update options for BatchSizing (@bitblender )

5. Update options for Planner Optimizations (@arina-ielchiieva )

6. Update options for Security & Kafka (Krystal Nguyen)

7. Update options for Misc entries (@arina-ielchiieva , @vvysotskyi )

In additional, there is a patch for `org.apache.drill.exec.compile.ClassTransformer.scalar_replacement` , which appears to have replaced `exec.compile.scalar_replacement`. References to the latter have been removed to avoid confusion.

Additional changes include moving the `ClassTransformer` validator to `ExecConstants.java`

Adding support for internal options' descriptions

Removed mention of {{Will be removed in 1.15.0}}. (Refer DRILL-6527)

  1. … 6 more files in changeset.
DRILL-6084: Show Drill functions in WebUI for autocomplete

Building on top of DRILL-3988 and leveraging DRILL-5868, this allows support for Drill functions to be now available in the WebUI.

If users wants UDFs to show up, they should place the UDF jars in the `$DRILL_HOME/jars/3rdparty` directory so that this can be loaded during the Drillbit's startup.

Concept of internal Drill functions are introduced. With this, internal Drill functions like `ConvertToNullableXYZ` has been marked as internal.

The WebUI will not show these functions. However, they are still visible in `sys.functions` table with an additional column indicating that it is an internal function.

Tests have been added as a part of this commit to verify the internal functions concept.

  1. … 13 more files in changeset.
DRILL-6775: The schema for empty output is not shown in Drill Web UI

Removed an excess check for the result emptiness that prevented retrieval of the column names.

close apache/drill#1498