apache

closes #943

closes #1040

closes #894

close #857

closes #822

Also enable OpenSSL support

Also fix exclusions and java-exec pom file to eliminate netty-tcnative as a transitive dependency on all projects

+ DRILL-3242 aims to provide offloading request handling to a secondary thread, but this feature is disabled by default due to concurrency issues

+ One of the implications of the feature was to ignore exceptions that were not of UserRpcException type. But exceptions must not be ignored, they should be handled properly, specially in the context of security

NOTE: This pull request provides support for on-wire encryption using SASL framework. The communication channel that are covered are:

1) Between Drill JDBC client and Drillbit.

2) Between Drillbit to Drillbit i.e. control/data channels.

3) It has UI change to view encryption is enabled on which network channel and number of encrypted/unencrypted connections for

user/control/data connections.

close apache/drill#773

+ Extract RemoteConnection interface, and add AbstractRemoteConnection

+ Add ServerConnection and ClientConnection interfaces

+ Add RequestHandler interface to decouple connections from how requests are handled

+ Add NonTransientRpcException

+ Remove unused classes and methods

+ Code style changes

Adding tries config option in connection string. Improving fault tolerance in Drill client when trying to make first connection with foreman. The client will try to connect to min(tries, num_drillbits) unique drillbits unless a successfull connection is established.

HYGIENE: Refactoring BasicClient::close to call RemoteConnection::close

close apache/drill#679

close apache/drill#648

close apache/drill#666

+ New metrics:

- drill.queries.enqueued

number of queries that have been submitted to the drillbit but have

not started

- drill.queries.running

number of running queries for which this drillbit is the foreman

- drill.queries.completed

number of completed queries (or cancelled or failed) for which this

drillbit was the foreman

- drill.fragments.running

number of query fragments that are running in the drillbit

- drill.allocator.root.used

amount of memory used in bytes by the internal memory allocator

- drill.allocator.root.peak

peak amount of memory used in bytes by the internal memory allocator

- fd.usage

ratio of used to total file descriptors (on *nix systems)

+ Rename "current" to "used" for RPC allocator current memory usage to

follow convention

+ Borrow SystemPropertyUtil class from Netty

+ Configure DrillMetrics through system properties

+ Remove unused methods and imports

closes #495

closes #486

- Move the coordination id management to be connection level instead of RpcBus level

- Rename CoordinationQueue to a more appropriate name: RequestIdMap

- Simplify locking and memory overhead of RequestIdMap. It used to be that this would accessed by a large number of threads concurrently. We modified the behavior so that it is only accessed by two threads at most. Rather than have memory overhead of ConcurrentHashMap, switch to simple locking approach since contention should be low.

- Update all methods associated with coordination to improve names as well as add javadocs. Move these methods to the RemoteConnection.

- Consolidate the two different close handlers into a single, ordered close handler managed inside the connection.

- Add better javadoc around the close method of RemoteConnection

- Add some preconditions checks.

- Update the HPPC version in the base memory module since it conflicts with the one in the java-exec module.

This closes #463.

Fixing most rawtypes warning issues in drill modules.

Closes #347

- Fix issue where WorkspaceConfig was not returning consistent hashCode()s for equal objects.

- Fix issue where we were misusing recycler causing object reference leaks

This closes #331.

- make Allocator mostly lockless

- change BaseAllocator maps to direct references

- add documentation around memory management model

- move transfer and ownership methods to DrillBuf

- Improve debug messaging.

- Fix/revert sort changes

- Remove unused fragment limit flag

- Add time to HistoricalLog events

- Remove reservation amount from RootAllocator constructor (since not allowed)

- Fix concurrency issue where allocator is closing at same moment as incoming batch transfer, causing leaked memory and/or query failure.

- Add new AutoCloseables.close(Iterable<AutoCloseable>)

- Remove extraneous DataResponseHandler and Impl (and update TestBitRpc to use smarter mock of FragmentManager)

- Remove the concept of poison pill record batches, using instead FragmentContext.isOverMemoryLimit()

- Update incoming data batches so that they are transferred under protection of a close lock

- Improve field names in IncomingBuffers and move synchronization to collectors as opposed to IncomingBuffers (also change decrementing to decrementToZero rather than two part check).

This closes #238.

- Replace Stopwatch constructors with .createStarted() or .createUnstarted()

- Stop using InputSupplier and Closeables.closeQuietly

- Clean up quiet closes to log or (preferably) propagate.

- Add log4j to enforcer exclusions.

- Update HBaseTestSuite to add patching of Closeables.closeQuietly() and Stopwatch legacy methods. Only needed when running HBaseMiniCluster.

- Remove log4j from HBase's pom to provide exception logging.

- Remove log4j from Hive's shaded pom.

- Update Catastrophic failures to use the same pattern to ensure reporting.

- Update test framework to avoid trying IPv6 resolution. (This removes 90s pause from HBase startup in my tests)

This closes #361.

This closes #157.