Clone Tools
  • last updated 14 mins ago
Constraints: committers
Constraints: files
Constraints: dates
DRILL-7494: Unable to connect to Drill using JDBC driver when using custom authenticator

closes #1938

  1. … 6 more files in changeset.
DRILL-7143: Support default value for empty columns

Modifies the prior work to add default values for columns. The prior work added defaults

when the entire column is missing from a reader (the old Nullable Int column). The Row

Set mechanism now will also "fill empty" slots with the default value.

Added default support for the column writers. The writers automatically obtain the

default value from the column schema. The default can also be set explicitly on

the column writer.

Updated the null column mechanism to use this feature rather than the ad-hoc

implemention in the prior commit.

Semantics changed a bit. Only Required columns take a default. The default value

is ignored or nullable columns since nullable columns already have a file default: NULL.

Other changes:

* Updated the CSV-with-schema tests to illustrate the new behavior.

* Made multiple fixes for Boolean and Decimal columns and added unit tests.

* Upgraded Fremarker to version 2.3.28 to allow use of the continue statement.

* Reimplemented the Bit column reader and writer to use the BitVector directly since this vector is rather special.

* Added get/set Boolean methods for column accessors

* Moved the BooleanType class to the common package

* Added more CSV unit tests to explore decimal types, booleans, and defaults

* Add special handling for blank fields in from-string conversions

* Added options to the conversion factory to specify blank-handling behavior.

CSV uses a mapping of blanks to null (nullable) or default value (non-nullable)

closes #1726

  1. … 72 more files in changeset.
DRILL-6422: Replace guava imports with shaded ones

    • -3
    • +3
  1. … 972 more files in changeset.
DRILL-6656: Disallow extra semicolons and multiple statements on the same line.

closes #1415

  1. … 144 more files in changeset.
DRILL-6386: Remove unused imports and star imports.

  1. … 231 more files in changeset.
DRILL-6255: Drillbit while sending control message to itself creates a connection instead of submitting locally

closes #1253

  1. … 15 more files in changeset.
DRILL-6389: Fixed building javadocs - Added documentation about how to build javadocs - Fixed some of the javadoc warnings

closes #1276

  1. … 64 more files in changeset.
DRILL-6320: Fixed license headers.

closes #1207

    • -1
    • +1
  1. … 2052 more files in changeset.
DRILL-6187: Exception in RPC communication between DataClient/ControlClient and respective servers when bit-to-bit security is on

This closes #1145

    • -0
    • +300
    • -0
    • +80
    • -0
    • +70
  1. … 17 more files in changeset.
DRILL-5881:Java Client: [Threat Modeling] Drillbit may be spoofed by an attacker and this may lead to data being written to the attacker's target instead of Drillbit Also set connection to null after its closed

This closes #999

  1. … 2 more files in changeset.
DRILL-5749: solve deadlock occured between foreman and netty threads

closes #943

DRILL-5425: Support HTTP Kerberos auth using SPNEGO

closes #1040

  1. … 25 more files in changeset.
DRILL-5701: Fix drill.connections.rpc.<user/control/data>.<encrypted/unencrypted> metric behavior

closes #894

  1. … 10 more files in changeset.
DRILL-5599: Notify StatusHandler that batch sending has failed even if channel is still open

close #857

DRILL-5457: Spill implementation for Hash Aggregate

closes #822

  1. … 35 more files in changeset.
DRILL-5431: SSL Support (Java) - Java client server SSL implementation

Also enable OpenSSL support

Also fix exclusions and java-exec pom file to eliminate netty-tcnative as a transitive dependency on all projects

    • -0
    • +235
  1. … 26 more files in changeset.
DRILL-4280: CORE (revert DRILL-3242)

+ DRILL-3242 aims to provide offloading request handling to a secondary thread, but this feature is disabled by default due to concurrency issues

+ One of the implications of the feature was to ignore exceptions that were not of UserRpcException type. But exceptions must not be ignored, they should be handled properly, specially in the context of security

DRILL-4335: Apache Drill should support network encryption.

NOTE: This pull request provides support for on-wire encryption using SASL framework. The communication channel that are covered are:

1) Between Drill JDBC client and Drillbit.

2) Between Drillbit to Drillbit i.e. control/data channels.

3) It has UI change to view encryption is enabled on which network channel and number of encrypted/unencrypted connections for

user/control/data connections.

close apache/drill#773

    • -0
    • +99
    • -0
    • +37
    • -0
    • +95
    • -0
    • +160
    • -0
    • +177
  1. … 46 more files in changeset.

+ Extract RemoteConnection interface, and add AbstractRemoteConnection

+ Add ServerConnection and ClientConnection interfaces

+ Add RequestHandler interface to decouple connections from how requests are handled

+ Add NonTransientRpcException

+ Remove unused classes and methods

+ Code style changes

    • -0
    • +227
    • -0
    • +30
    • -29
    • +0
    • -0
    • +33
    • -0
    • +50
    • -0
    • +37
  1. … 3 more files in changeset.
DRILL-5098: Improving fault tolerance for connection between client and foreman node.

Adding tries config option in connection string. Improving fault tolerance in Drill client when trying to make first connection with foreman. The client will try to connect to min(tries, num_drillbits) unique drillbits unless a successfull connection is established.

HYGIENE: Refactoring BasicClient::close to call RemoteConnection::close

close apache/drill#679

  1. … 4 more files in changeset.
DRILL-5015: Randomly select the drillbit from the list provided by user in connection string Note: Improved the connection string validation and error handling during parsing. Added unit test for the new parsing mechanism.

close apache/drill#648

    • -0
    • +29
  1. … 3 more files in changeset.
DRILL-4956: Temporary tables support

close apache/drill#666

  1. … 40 more files in changeset.
DRILL-4654: Add new metrics to the MetricRegistry

+ New metrics:

- drill.queries.enqueued

number of queries that have been submitted to the drillbit but have

not started

- drill.queries.running

number of running queries for which this drillbit is the foreman

- drill.queries.completed

number of completed queries (or cancelled or failed) for which this

drillbit was the foreman

- drill.fragments.running

number of query fragments that are running in the drillbit

- drill.allocator.root.used

amount of memory used in bytes by the internal memory allocator

- drill.allocator.root.peak

peak amount of memory used in bytes by the internal memory allocator

- fd.usage

ratio of used to total file descriptors (on *nix systems)

+ Rename "current" to "used" for RPC allocator current memory usage to

follow convention

+ Borrow SystemPropertyUtil class from Netty

+ Configure DrillMetrics through system properties

+ Remove unused methods and imports

closes #495

  1. … 13 more files in changeset.
DRILL-4623: Disable epoll transport by default

closes #486

  1. … 1 more file in changeset.
DRILL-3714: Avoid cascading disconnection when a single connection is broken.

- Move the coordination id management to be connection level instead of RpcBus level

- Rename CoordinationQueue to a more appropriate name: RequestIdMap

- Simplify locking and memory overhead of RequestIdMap. It used to be that this would accessed by a large number of threads concurrently. We modified the behavior so that it is only accessed by two threads at most. Rather than have memory overhead of ConcurrentHashMap, switch to simple locking approach since contention should be low.

- Update all methods associated with coordination to improve names as well as add javadocs. Move these methods to the RemoteConnection.

- Consolidate the two different close handlers into a single, ordered close handler managed inside the connection.

- Add better javadoc around the close method of RemoteConnection

- Add some preconditions checks.

- Update the HPPC version in the base memory module since it conflicts with the one in the java-exec module.

This closes #463.

    • -0
    • +192
  1. … 1 more file in changeset.
DRILL-4327: Fix rawtypes warnings in drill codebase

Fixing most rawtypes warning issues in drill modules.

Closes #347

  1. … 76 more files in changeset.
DRILL-4278: Heap memory leak issues

- Fix issue where WorkspaceConfig was not returning consistent hashCode()s for equal objects.

- Fix issue where we were misusing recycler causing object reference leaks

This closes #331.

  1. … 5 more files in changeset.
Fix issue where FutureBitComment didn't return buffer as part of future.

DRILL-4134: Allocator Improvements

- make Allocator mostly lockless

- change BaseAllocator maps to direct references

- add documentation around memory management model

- move transfer and ownership methods to DrillBuf

- Improve debug messaging.

- Fix/revert sort changes

- Remove unused fragment limit flag

- Add time to HistoricalLog events

- Remove reservation amount from RootAllocator constructor (since not allowed)

- Fix concurrency issue where allocator is closing at same moment as incoming batch transfer, causing leaked memory and/or query failure.

- Add new AutoCloseables.close(Iterable<AutoCloseable>)

- Remove extraneous DataResponseHandler and Impl (and update TestBitRpc to use smarter mock of FragmentManager)

- Remove the concept of poison pill record batches, using instead FragmentContext.isOverMemoryLimit()

- Update incoming data batches so that they are transferred under protection of a close lock

- Improve field names in IncomingBuffers and move synchronization to collectors as opposed to IncomingBuffers (also change decrementing to decrementToZero rather than two part check).

This closes #238.

  1. … 119 more files in changeset.
DRILL-3987: (MOVE) Extract RPC, memory-base and memory-impl as separate modules.

    • -0
    • +69
    • -0
    • +41
    • -0
    • +316
    • -0
    • +76
    • -0
    • +237
    • -0
    • +39
    • -0
    • +25
    • -0
    • +22
    • -0
    • +160
    • -0
    • +28
    • -0
    • +80
    • -0
    • +79
    • -0
    • +66
    • -0
    • +70
  1. … 127 more files in changeset.