Clone Tools
  • last updated 17 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-6850: Force setting DRILL_LOGICAL Convention for DrillRelFactories and DrillFilterRel

- Fix workspace case insensitivity for JDBC storage plugin

  1. … 13 more files in changeset.
DRILL-6862: Update Calcite to 1.18.0

1. Moved Calcite dependency from profile hadoop-default to general dependency managment

2. Updated Calcite version to 1.18.0-drill-r0 and Avatica version to 1.13.0

3. Hook.REL_BUILDER_SIMPLIFY moved to static block, cause now it can't be removed (fixes DRILL-6830)

4. Removed WrappedAccessor, since it was workaround fixed in CALCITE-1408

5. Fixed setting of multiple options in TestBuilder

6. Timstampadd type inference aligned with CALCITE-2699

7. Dependency update caused 417 kB increase of jdb-all jar size, so the maxsize limit was

increased from 39.5 to 40 MB

8. Added test into TestDrillParquetReader to ensure that DRILL-6856 was

fixed by Calcite update

close apache/drill#1631

    • -447
    • +0
    ./apache/drill/jdbc/impl/WrappedAccessor.java
  1. … 10 more files in changeset.
DRILL-6422: Replace guava imports with shaded ones

    • -2
    • +2
    ./apache/drill/jdbc/impl/DrillCursor.java
    • -3
    • +3
    ./apache/drill/jdbc/impl/DrillMetaImpl.java
  1. … 979 more files in changeset.
DRILL-6656: Disallow extra semicolons and multiple statements on the same line.

closes #1415

    • -2
    • +1
    ./apache/drill/jdbc/proxy/ProxiesManager.java
  1. … 143 more files in changeset.
DRILL-6647: Update Calcite version to 1.17.0

closes #1425

    • -299
    • +101
    ./apache/drill/jdbc/impl/DrillConnectionImpl.java
    • -1055
    • +439
    ./apache/drill/jdbc/impl/DrillResultSetImpl.java
    • -281
    • +46
    ./apache/drill/jdbc/impl/DrillStatementImpl.java
  1. … 10 more files in changeset.
DRILL-6386: Remove unused imports and star imports.

    • -1
    • +0
    ./apache/drill/jdbc/DrillDatabaseMetaData.java
    • -2
    • +0
    ./apache/drill/jdbc/DrillPreparedStatement.java
    • -7
    • +0
    ./apache/drill/jdbc/JdbcApiSqlException.java
    • -1
    • +0
    ./apache/drill/jdbc/impl/DrillAccessorList.java
    • -2
    • +0
    ./apache/drill/jdbc/impl/DrillCursor.java
  1. … 218 more files in changeset.
DRILL-6422: Update guava to 23.0 and shade it

- Fix compilation errors for new version of Guava.

- Remove usage of deprecated API

- Shade guava and add dependencies to the shaded version

- Ban unshaded package

- Introduce drill-shaded module and move guava-shaded under it

- Add methods to convert shaded guava lists to the unshaded ones

- Add instruction for publishing artifacts to the Apache repository

  1. … 81 more files in changeset.
DRILL-6320: Fixed license headers.

closes #1207

    • -1
    • +1
    ./apache/drill/jdbc/DrillConnection.java
    • -1
    • +1
    ./apache/drill/jdbc/DrillConnectionConfig.java
    • -6
    • +7
    ./apache/drill/jdbc/DrillDatabaseMetaData.java
    • -7
    • +8
    ./apache/drill/jdbc/DrillPreparedStatement.java
    • -1
    • +1
    ./apache/drill/jdbc/DrillResultSet.java
    • -7
    • +8
    ./apache/drill/jdbc/DrillStatement.java
    • -1
    • +1
    ./apache/drill/jdbc/SQLConversionException.java
    • -1
    • +1
    ./apache/drill/jdbc/SchemaChangeListener.java
    • -1
    • +1
    ./apache/drill/jdbc/SqlTimeoutException.java
    • -1
    • +1
    ./apache/drill/jdbc/impl/BasicList.java
    • -1
    • +1
    ./apache/drill/jdbc/impl/DrillAccessorList.java
    • -1
    • +1
    ./apache/drill/jdbc/impl/DrillCursor.java
  1. … 2052 more files in changeset.
DRILL-6094: Decimal data type enhancements

Add ExprVisitors for VARDECIMAL

Modify writers/readers to support VARDECIMAL

- Added usage of VarDecimal for parquet, hive, maprdb, jdbc;

- Added options to store decimals as int32 and int64 or fixed_len_byte_array or binary;

Add UDFs for VARDECIMAL data type

- modify type inference rules

- remove UDFs for obsolete DECIMAL types

Enable DECIMAL data type by default

Add unit tests for DECIMAL data type

Fix mapping for NLJ when literal with non-primitive type is used in join conditions

Refresh protobuf C++ source files

Changes in C++ files

Add support for decimal logical type in Avro.

Add support for date, time and timestamp logical types.

Update Avro version to 1.8.2.

    • -0
    • +2
    ./apache/drill/jdbc/impl/DrillMetaImpl.java
  1. … 200 more files in changeset.
DRILL-6202: Deprecate usage of IndexOutOfBoundsException to re-alloc vectors

closes #1144

  1. … 9 more files in changeset.
DRILL-6294: Changes to support Calcite 1.16.0 , and remove deprecated API usage

closes #1198

    • -0
    • +2
    ./apache/drill/jdbc/impl/DrillMetaImpl.java
  1. … 38 more files in changeset.
DRILL-6090: While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of "Curator-Framework-0" threads are created if connection to drill-bit is not successful(no drill-bits are up/reachable)

I am using Drill JDBC driver 1.12.0 to connect to MapR-DB. I am finding the available drill-bits using Zookeepers. When drill-bits are not up or not reachable, the connection is failed with exception: "Failure in connecting to Drill: oadd.org.apache.drill.exec.rpc.RpcException: Failure setting up ZK for client", which is expected, but number of threads created by ZKClusterCoordinator just keeps on increasing.

Steps to reproduce the issue

Setup a connection with a drill-bit using Apache Drill JDBC driver 1.12.0 through Zookeeper hosts(port 5181)

Now stop the drill-bit services or block the drill-bit IPs using iptable rules

Truncate catalina logs

Try to connect to the drill-bit/hit a code path that requires connection to drill-bits.

Take thread dump using kill -QUIT <java process id>

grep -c "Curator-Framework-0" catalina.out

Observe that the curator framework thread just keep on accumulating

RCA:

ZKClusterCoordinator creates curator threads in the constructor

ZKClusterCoordinator is instantiated by DrillClient.connect

DrillClient.connect is called in DrillConnectionImpl constructor

Fix:

Call DrillConnectionImpl .cleanup() from all the catch blocks in the DrillConnectionImpl constructor.

close apache/drill#1094

DRILL-3640: Support JDBC Statement.setQueryTimeout(int)

Allow for queries to be cancelled if they don't complete within the stipulated time.

This is done by having Drill[Prepared]StatementImpl create a Stopwatch timer to track elapsed time.

* DrillCursor uses this to detect timeouts.

* DrillResultSetImpl uses this to detech timeout from the client side (e.g. a slow client, when all batches have been processed by DrillCursor)

* Tests added to test these and other query timeout scenarios.

* Dependent on DRILL-5973 for enabling server-triggered timeout tests

NOTE: PreparedStatementTest.testServerTriggeredQueryTimeout is disabled

closes #1024

    • -15
    • +9
    ./apache/drill/jdbc/DrillStatement.java
    • -5
    • +9
    ./apache/drill/jdbc/SqlTimeoutException.java
    • -8
    • +53
    ./apache/drill/jdbc/impl/DrillCursor.java
  1. … 3 more files in changeset.
DRILL-5783, DRILL-5841, DRILL-5894: Rationalize test temp directories

This change includes:

DRILL-5783:

- A unit test is created for the priority queue in the TopN operator.

- The code generation classes passed around a completely unused function registry reference in some places so it is removed.

- The priority queue had unused parameters for some of its methods so it is removed.

DRILL-5841:

- Created standardized temp directory classes DirTestWatcher, SubDirTestWatcher, and BaseDirTestWatcher. And updated all unit tests to use them.

DRILL-5894:

- Removed the dfs_test storage plugin for tests and replaced it with the already existing dfs storage plugin.

Misc:

- General code cleanup.

- Removed unnecessary use of String.format in the tests.

This closes #984

  1. … 365 more files in changeset.
DRILL-3993: Fix unit test failures connected with support Calcite 1.13

- Use root schema as default for describe table statement.

Fix TestOpenTSDBPlugin.testDescribe() and TestInfoSchemaOnHiveStorage.varCharMaxLengthAndDecimalPrecisionInInfoSchema() unit tests.

- Modify expected results for tests:

TestPreparedStatementProvider.invalidQueryValidationError();

TestProjectPushDown.testTPCH1();

TestProjectPushDown.testTPCH3();

TestStorageBasedHiveAuthorization.selectUser1_db_u0_only();

TestStorageBasedHiveAuthorization.selectUser0_db_u1g1_only()

- Fix TestCTAS.whenTableQueryColumnHasStarAndTableFiledListIsSpecified(), TestViewSupport.createViewWhenViewQueryColumnHasStarAndViewFiledListIsSpecified(), TestInbuiltHiveUDFs.testIf(), testDisableUtf8SupportInQueryString unit tests.

- Fix UnsupportedOperationException and NPE for jdbc tests.

- Fix AssertionError: Conversion to relational algebra failed to preserve datatypes

*DrillCompoundIdentifier:

According to the changes, made in [CALCITE-546], star Identifier is replaced by empty string during parsing the query. Since Drill uses its own DrillCompoundIdentifier, it should also replace star by empty string before creating SqlIdentifier instance to avoid further errors connected with star column. see SqlIdentifier.isStar() method.

*SqlConverter:

In [CALCITE-1417] added simplification of expressions which should be projected every time when a new project rel node is created using RelBuilder. It causes assertion errors connected with types nullability. This hook was set to false to avoid project expressions simplification. See usage of this hook and RelBuilder.project() method.

In Drill the type nullability of the function depends on only the nullability of its arguments. In some cases, a function may return null value even if it had non-nullable arguments. When Calice simplifies expressions, it checks that the type of the result is the same as the type of the expression. Otherwise, makeCast() method is called. But when a function returns null literal, this cast does nothing, even when the function has a non-nullable type. So to avoid this issue, method makeCast() was overridden.

*DrillAvgVarianceConvertlet:

Problem with sum0 and specific changes in old Calcite (it is CALCITE-777). (see HistogramShuttle.visitCall method) Changes were made to avoid changes in Calcite.

*SqlConverter, DescribeTableHandler, ShowTablesHandler:

New Calcite tries to combine both default and specified workspaces during the query validation. In some cases, for example, when describe table statement is used, Calcite tries to find INFORMATION_SCHEMA in the schema used as default. When it does not find the schema, it tries to find a table with such name. For some storage plugins, such as opentsdb and hbase, when a table was not found, the error is thrown, and the query fails. To avoid this issue, default schema was changed to root schema for validation stage for describe table and show tables queries.

    • -5
    • +9
    ./apache/drill/jdbc/impl/DrillMetaImpl.java
  1. … 16 more files in changeset.
DRILL-3993: Changes to support Calcite 1.13

- fixed all compiling errors (main changes were: Maven changes, chenges RelNode -> RelRoot, implementing some new methods from updated interfaces, chenges some literals, logger changes);

- fixed unexpected column errors, validation errors and assertion errors after Calcite update;

- fixed describe table/schema statement according to updated logic;

- added fixes with time-intervals;

- changed precision of BINARY to 65536 (was 1048576) according to updated logic (Calcite overrides bigger precision to own maxPrecision);

- ignored some incorrect tests with DRILL-3244;

- changed "Table not found" message to "Object not found within" according to new Calcite changes.

    • -10
    • +63
    ./apache/drill/jdbc/impl/DrillMetaImpl.java
  1. … 66 more files in changeset.
DRILL-4264: Allow field names to include dots

  1. … 98 more files in changeset.
DRILL-5431: SSL Support (Java) - Java client server SSL implementation

Also enable OpenSSL support

Also fix exclusions and java-exec pom file to eliminate netty-tcnative as a transitive dependency on all projects

  1. … 30 more files in changeset.
DRILL-5568: Include hadoop-common jars inside drill-jdbc-all.jar 1) Introduce a new file inside exec/jdbc-all resource package which contains the property key/value pair for prefix 2) At build time based upon the profile choosen this property value is set inside the file 3) It is later consumed by SecurityConfiguration class to rename classpath for classes inside hadoop package.

DRILL-5568: Code review changes

close apache/drill#849

    • -3
    • +3
    ./apache/drill/jdbc/impl/DrillFactory.java
  1. … 5 more files in changeset.
DRILL-5413: DrillConnectionImpl.isReadOnly() throws NullPointerException

change is in CALCITE-843.

update drill's calcite version to 1.4.0-drill-r21

close #806

  1. … 2 more files in changeset.
DRILL-5326: Unit tests failures related to the SERVER_METADTA

- adding of the sql type name for the "GENERIC_OBJECT";

- changing "NullCollation" in the "ServerMetaProvider" to the correct default value;

- changing RpcType to GET_SERVER_META in the appropriate ServerMethod

close #775

  1. … 3 more files in changeset.
DRILL-5301: Server metadata API

Add a Server metadata API to the User protocol, to query server support

of various SQL features.

Add support to the client (DrillClient) to query this information.

Add support to the JDBC driver to query this information, if the server supports

the new API, or fallback to the previous behaviour (rely on Avatica defaults) otherwise.

close #764

    • -3
    • +2
    ./apache/drill/jdbc/DrillConnectionConfig.java
    • -4
    • +4
    ./apache/drill/jdbc/impl/DrillMetaImpl.java
  1. … 32 more files in changeset.
DRILL-4280: CORE (user to bit authentication, Java)

+ Add logic for authentication in UserClient and UserServer with

backward compatibility in both directions

+ Add abstract extension to ServerConnection and ClientConnection

+ Add concrete extensions to abstract connections:

BitToUserConnection and UserToBitConnection

+ Add ConnectionConfig interface with abstract and concrete

implementations to encapsulate configuration for server-side

connections

+ Encapsulate all requests handled by UserServer in

UserServerRequestHandler

+ Clear UserSession when connection is closed either by user or

bit

+ Add DrillProperties to encapsulate all connection properties

used during connection time

  1. … 15 more files in changeset.
DRILL-5241: JDBC proxy driver: Do not put null value in map

This closes #724

    • -1
    • +1
    ./apache/drill/jdbc/proxy/ProxiesManager.java
DRILL-5015: Randomly select the drillbit from the list provided by user in connection string Note: Improved the connection string validation and error handling during parsing. Added unit test for the new parsing mechanism.

close apache/drill#648

  1. … 3 more files in changeset.
DRILL-4994: Add back JDBC prepared statement for older servers

When the JDBC client is connected to an older Drill server, it always

attempted to use server-side prepared statement with no fallback.

With this change, client will check server version and will fallback to the

previous client-side prepared statement (which is still limited to only execute

queries and does not provide metadata).

close #613

    • -0
    • +4
    ./apache/drill/jdbc/DrillConnectionConfig.java
    • -1
    • +10
    ./apache/drill/jdbc/impl/DrillCursor.java
  1. … 19 more files in changeset.
DRILL-4730: Update JDBC DatabaseMetaData implementation to use new Metadata APIs

Update JDBC driver to use Metadata APIs instead of executing SQL queries

close #613

    • -1
    • +5
    ./apache/drill/jdbc/DrillConnectionConfig.java
    • -14
    • +649
    ./apache/drill/jdbc/impl/DrillMetaImpl.java
    • -0
    • +448
    ./apache/drill/jdbc/impl/WrappedAccessor.java
  1. … 13 more files in changeset.
DRILL-4994: Refactor DrillCursor

Refactor DrillCursor to be more self-contained.

    • -22
    • +301
    ./apache/drill/jdbc/impl/DrillCursor.java
    • -288
    • +14
    ./apache/drill/jdbc/impl/DrillResultSetImpl.java
DRILL-4969: Basic implementation of display size for column metadata

Add a simple implementation of display size, based on the ODBC table

available at: https://msdn.microsoft.com/en-us/library/ms713974(v=vs.85).aspx

closes #632

  1. … 5 more files in changeset.
DRILL-4369: Exchange name and version infos during handshake

There's no name and version exchanged between client and server over the User RPC

channel.

On client side, having access to the server name and version is useful to expose it

to the user (through JDBC or ODBC api like DatabaseMetadata#getDatabaseProductVersion()),

or to implement fallback strategy when some recent API are not available (like

metadata API).

On the server side, having access to the client version might be useful for audit

purposes and eventually to implement fallback strategy if it doesn't require a RPC

version change.

this closes #622

    • -1
    • +2
    ./apache/drill/jdbc/impl/DriverImpl.java
  1. … 19 more files in changeset.