Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-7357: Expose Drill Metastore data through information_schema

1. Add additional columns to TABLES and COLUMNS tables.

2. Add PARTITIONS table.

3. General refactoring to adjust information_schema data retrieval from multiple sources.

closes #1860

  1. … 32 more files in changeset.
DRILL-7084: ResultSet getObject method throws not implemented exception if the column type is NULL

closes #1825

    • binary
    ./resources/testGetObjectNull.parquet
  1. … 1 more file in changeset.
DRILL-7048: Implement JDBC Statement.setMaxRows() with System Option

This introduces support for JDBC's Statement.setMaxRows(int) API, which can help Drill execute a query much faster if it knows that not ALL the records in the resultset will be consumed upfront.

This Commit introduces the core changes to support the feature within Drill's execution engine

Protobuf Changes

1. RunQuery: Added "autolimit_rowcount"

2. QueryProfile: Added "autoLimit"

3. Regenerated Java and C++ client files

REST API support

1. Support for REST server to interpret a submitted query and also for rendering this information for an executed query

2. Updates to the Freemarker templates (for WebUI)

3. Safety check within Javascript (for WebUI)

JDBC API support

1. Introduces backend execution of 'ALTER SESSION' to apply the auto-limiting of resultset size

2. Added Unit Tests for PreparedStatement and Statement objects

3. Added getter setter methods to be skipped in testing for org.apache.drill.jdbc.test.Drill2489CallsAfterCloseThrowExceptionsTest.testclosedPreparedStmtOfOpenConnMethodsThrowRight()

Updates based on review comments

Additional Updates

Test Cleanup

1. Revert Drill2489 hack

2. Formatting in *StatementTest

3. Removal f redundant `statement.close()`

4. Manage new Exception thrown when setting invalid maxRow values

Final updates

1. Test changes

2. Trim trailing spaces in auto-limit value (Javascript)

3. Before & After annotations to synchronize changes to system values for MaxRows(auto-limit)

Reorganized tests due to synchronized locking

Removed conflicting JsonCreator in QueryWrapper

Additional test cleanup

closes #1714

    • -0
    • +261
    ./java/org/apache/drill/jdbc/PreparedStatementMaxRowsTest.java
    • -0
    • +248
    ./java/org/apache/drill/jdbc/StatementMaxRowsTest.java
    • -21
    • +19
    ./java/org/apache/drill/jdbc/StatementTest.java
  1. … 31 more files in changeset.
DRILL-6946: Implement java.sql.Connection setSchema and getSchema methods in DrillConnectionImpl closes #1596

  1. … 9 more files in changeset.
DRILL-6834: Introduce option to disable result set for DDL queries for JDBC connection

- Added session-scoped option `drill.exec.fetch_resultset_for_ddl` to control whether update count or result set should be returned for JDBC connection session. By default the option is set to `true` which ensures that result set is returned;

- Updated Drill JDBC: `DrillCursor` and `DrillStatement` to achieve desired behaviour.

closes #1549

  1. … 26 more files in changeset.
DRILL-6422: Replace guava imports with shaded ones

    • -3
    • +3
    ./java/org/apache/drill/jdbc/ConnectionInfo.java
    • -1
    • +1
    ./java/org/apache/drill/jdbc/DriverTest.java
    • -3
    • +3
    ./java/org/apache/drill/jdbc/JdbcTestBase.java
  1. … 972 more files in changeset.
DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM

closes #1446

  1. … 23 more files in changeset.
DRILL-6656: Disallow extra semicolons and multiple statements on the same line.

closes #1415

    • -4
    • +4
    ./java/org/apache/drill/jdbc/StatementTest.java
  1. … 143 more files in changeset.
DRILL-6647: Update Calcite version to 1.17.0

closes #1425

    • -104
    • +25
    ./java/org/apache/drill/jdbc/JdbcTestBase.java
  1. … 15 more files in changeset.
DRILL-6438: Remove excess logging form the tests. - Removed usages of System.out and System.err from the test and replaced with loggers

closes #1284

    • -1
    • +1
    ./java/org/apache/drill/jdbc/JdbcTestBase.java
  1. … 82 more files in changeset.
DRILL-6386: Remove unused imports and star imports.

    • -1
    • +3
    ./java/org/apache/drill/jdbc/ConnectionTest.java
    • -6
    • +2
    ./java/org/apache/drill/jdbc/StatementTest.java
  1. … 219 more files in changeset.
DRILL-6320: Fixed license headers.

closes #1207

    • -1
    • +1
    ./java/org/apache/drill/jdbc/ConnectionInfo.java
    • -1
    • +1
    ./java/org/apache/drill/jdbc/StatementTest.java
  1. … 2052 more files in changeset.
DRILL-6094: Decimal data type enhancements

Add ExprVisitors for VARDECIMAL

Modify writers/readers to support VARDECIMAL

- Added usage of VarDecimal for parquet, hive, maprdb, jdbc;

- Added options to store decimals as int32 and int64 or fixed_len_byte_array or binary;

Add UDFs for VARDECIMAL data type

- modify type inference rules

- remove UDFs for obsolete DECIMAL types

Enable DECIMAL data type by default

Add unit tests for DECIMAL data type

Fix mapping for NLJ when literal with non-primitive type is used in join conditions

Refresh protobuf C++ source files

Changes in C++ files

Add support for decimal logical type in Avro.

Add support for date, time and timestamp logical types.

Update Avro version to 1.8.2.

  1. … 201 more files in changeset.
DRILL-6106: Use valueOf method instead of constructor since valueOf has a higher performance by caching frequently requested values.

closes #1099

  1. … 10 more files in changeset.
DRILL-3640: Support JDBC Statement.setQueryTimeout(int)

Allow for queries to be cancelled if they don't complete within the stipulated time.

This is done by having Drill[Prepared]StatementImpl create a Stopwatch timer to track elapsed time.

* DrillCursor uses this to detect timeouts.

* DrillResultSetImpl uses this to detech timeout from the client side (e.g. a slow client, when all batches have been processed by DrillCursor)

* Tests added to test these and other query timeout scenarios.

* Dependent on DRILL-5973 for enabling server-triggered timeout tests

NOTE: PreparedStatementTest.testServerTriggeredQueryTimeout is disabled

closes #1024

    • -33
    • +164
    ./java/org/apache/drill/jdbc/StatementTest.java
  1. … 6 more files in changeset.
DRILL-5783, DRILL-5841, DRILL-5894: Rationalize test temp directories

This change includes:

DRILL-5783:

- A unit test is created for the priority queue in the TopN operator.

- The code generation classes passed around a completely unused function registry reference in some places so it is removed.

- The priority queue had unused parameters for some of its methods so it is removed.

DRILL-5841:

- Created standardized temp directory classes DirTestWatcher, SubDirTestWatcher, and BaseDirTestWatcher. And updated all unit tests to use them.

DRILL-5894:

- Removed the dfs_test storage plugin for tests and replaced it with the already existing dfs storage plugin.

Misc:

- General code cleanup.

- Removed unnecessary use of String.format in the tests.

This closes #984

    • -4
    • +330
    ./java/org/apache/drill/jdbc/JdbcTestBase.java
  1. … 351 more files in changeset.
DRILL-3993: Fix unit test failures connected with support Calcite 1.13

- Use root schema as default for describe table statement.

Fix TestOpenTSDBPlugin.testDescribe() and TestInfoSchemaOnHiveStorage.varCharMaxLengthAndDecimalPrecisionInInfoSchema() unit tests.

- Modify expected results for tests:

TestPreparedStatementProvider.invalidQueryValidationError();

TestProjectPushDown.testTPCH1();

TestProjectPushDown.testTPCH3();

TestStorageBasedHiveAuthorization.selectUser1_db_u0_only();

TestStorageBasedHiveAuthorization.selectUser0_db_u1g1_only()

- Fix TestCTAS.whenTableQueryColumnHasStarAndTableFiledListIsSpecified(), TestViewSupport.createViewWhenViewQueryColumnHasStarAndViewFiledListIsSpecified(), TestInbuiltHiveUDFs.testIf(), testDisableUtf8SupportInQueryString unit tests.

- Fix UnsupportedOperationException and NPE for jdbc tests.

- Fix AssertionError: Conversion to relational algebra failed to preserve datatypes

*DrillCompoundIdentifier:

According to the changes, made in [CALCITE-546], star Identifier is replaced by empty string during parsing the query. Since Drill uses its own DrillCompoundIdentifier, it should also replace star by empty string before creating SqlIdentifier instance to avoid further errors connected with star column. see SqlIdentifier.isStar() method.

*SqlConverter:

In [CALCITE-1417] added simplification of expressions which should be projected every time when a new project rel node is created using RelBuilder. It causes assertion errors connected with types nullability. This hook was set to false to avoid project expressions simplification. See usage of this hook and RelBuilder.project() method.

In Drill the type nullability of the function depends on only the nullability of its arguments. In some cases, a function may return null value even if it had non-nullable arguments. When Calice simplifies expressions, it checks that the type of the result is the same as the type of the expression. Otherwise, makeCast() method is called. But when a function returns null literal, this cast does nothing, even when the function has a non-nullable type. So to avoid this issue, method makeCast() was overridden.

*DrillAvgVarianceConvertlet:

Problem with sum0 and specific changes in old Calcite (it is CALCITE-777). (see HistogramShuttle.visitCall method) Changes were made to avoid changes in Calcite.

*SqlConverter, DescribeTableHandler, ShowTablesHandler:

New Calcite tries to combine both default and specified workspaces during the query validation. In some cases, for example, when describe table statement is used, Calcite tries to find INFORMATION_SCHEMA in the schema used as default. When it does not find the schema, it tries to find a table with such name. For some storage plugins, such as opentsdb and hbase, when a table was not found, the error is thrown, and the query fails. To avoid this issue, default schema was changed to root schema for validation stage for describe table and show tables queries.

  1. … 16 more files in changeset.
DRILL-5792: CONVERT_FROM_JSON on an empty file throws runtime exception

closes #961

  1. … 2 more files in changeset.
DRILL-5761: Disable Lilith ClassicMultiplexSocketAppender by default. Unify logback files.

  1. … 13 more files in changeset.
DRILL-5752 this change includes:

1. Increased test parallelism and fixed associated bugs

2. Added test categories and categorized tests appropriately

- Don't exclude anything by default

- Increase test timeout

- Fixed flakey test

closes #940

    • -3
    • +8
    ./java/org/apache/drill/jdbc/ConnectionTest.java
    • -1
    • +3
    ./java/org/apache/drill/jdbc/DriverTest.java
    • -0
    • +3
    ./java/org/apache/drill/jdbc/JdbcTestBase.java
  1. … 251 more files in changeset.
DRILL-3993: Changes to support Calcite 1.13

- fixed all compiling errors (main changes were: Maven changes, chenges RelNode -> RelRoot, implementing some new methods from updated interfaces, chenges some literals, logger changes);

- fixed unexpected column errors, validation errors and assertion errors after Calcite update;

- fixed describe table/schema statement according to updated logic;

- added fixes with time-intervals;

- changed precision of BINARY to 65536 (was 1048576) according to updated logic (Calcite overrides bigger precision to own maxPrecision);

- ignored some incorrect tests with DRILL-3244;

- changed "Table not found" message to "Object not found within" according to new Calcite changes.

  1. … 69 more files in changeset.
DRILL-1051: Casting timestamp as date gives wrong result for dates earlier than 1883

- Fix DateAccessor's, TimestampAccessor's and TimeAccessor's converting joda time to java.sql

close apache/drill#915

  1. … 1 more file in changeset.
DRILL-4264: Allow field names to include dots

  1. … 98 more files in changeset.
DRILL-4755: Fix IOBE for convert_from/convert_to functions with incorrect encoding type

closes #867

  1. … 2 more files in changeset.
DRILL-5547: Linking config options with system option manager

closes #868

  1. … 38 more files in changeset.
DRILL-5419: Calculate return string length for literals & some string functions

1. Revisited calculation logic for string literals and some string functions

(cast, upper, lower, initcap, reverse, concat, concat operator, rpad, lpad, case statement,

coalesce, first_value, last_value, lag, lead).

Synchronized return type length calculation logic between limit 0 and regular queries.

2. Deprecated width and changed it to precision for string types in MajorType.

3. Revisited FunctionScope and splitted it into FunctionScope and ReturnType.

FunctionScope will indicate only function usage in term of number of in / out rows, (n -> 1, 1 -> 1, 1->n).

New annotation in UDFs ReturnType will indicate which return type strategy should be used.

4. Changed MAX_VARCHAR_LENGTH from 65536 to 65535.

5. Updated calculation of precision and display size for INTERVALYEAR & INTERVALDAY.

6. Refactored part of function code-gen logic (ValueReference, WorkspaceReference, FunctionAttributes, DrillFuncHolder).

This closes #819

  1. … 76 more files in changeset.
DRILL-5413: DrillConnectionImpl.isReadOnly() throws NullPointerException

change is in CALCITE-843.

update drill's calcite version to 1.4.0-drill-r21

close #806

    • -2
    • +8
    ./java/org/apache/drill/jdbc/ConnectionTest.java
  1. … 2 more files in changeset.
DRILL-4280: CORE (unit tests)

+ Modify existing tests to use new authentication configuration

+ Add TestUserBitKerberos and TestBitBitKerberos using Apache Kerby library

  1. … 14 more files in changeset.
DRILL-5091: Handle new Java 8 JDBC functions and add missing test config parameter

* JDBC unit test fails to set up test storage plugin config on Java 8

* Cleans up compiler warnings seen in Eclipse.

* Added TODO's based on code reviews

closes #676

    • -3
    • +4
    ./java/org/apache/drill/jdbc/JdbcTestBase.java
  1. … 6 more files in changeset.
DRILL-5098: Improving fault tolerance for connection between client and foreman node.

Adding tries config option in connection string. Improving fault tolerance in Drill client when trying to make first connection with foreman. The client will try to connect to min(tries, num_drillbits) unique drillbits unless a successfull connection is established.

HYGIENE: Refactoring BasicClient::close to call RemoteConnection::close

close apache/drill#679

  1. … 5 more files in changeset.