Clone Tools
  • last updated 23 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-4956: Temporary tables support

close apache/drill#666

    • -1
    • +8
    ./drill/exec/rpc/RemoteConnection.java
  1. … 40 more files in changeset.
DRILL-4654: Add new metrics to the MetricRegistry

+ New metrics:

- drill.queries.enqueued

number of queries that have been submitted to the drillbit but have

not started

- drill.queries.running

number of running queries for which this drillbit is the foreman

- drill.queries.completed

number of completed queries (or cancelled or failed) for which this

drillbit was the foreman

- drill.fragments.running

number of query fragments that are running in the drillbit

- drill.allocator.root.used

amount of memory used in bytes by the internal memory allocator

- drill.allocator.root.peak

peak amount of memory used in bytes by the internal memory allocator

- fd.usage

ratio of used to total file descriptors (on *nix systems)

+ Rename "current" to "used" for RPC allocator current memory usage to

follow convention

+ Borrow SystemPropertyUtil class from Netty

+ Configure DrillMetrics through system properties

+ Remove unused methods and imports

closes #495

  1. … 13 more files in changeset.
DRILL-4623: Disable epoll transport by default

closes #486

  1. … 1 more file in changeset.
DRILL-3714: Avoid cascading disconnection when a single connection is broken.

- Move the coordination id management to be connection level instead of RpcBus level

- Rename CoordinationQueue to a more appropriate name: RequestIdMap

- Simplify locking and memory overhead of RequestIdMap. It used to be that this would accessed by a large number of threads concurrently. We modified the behavior so that it is only accessed by two threads at most. Rather than have memory overhead of ConcurrentHashMap, switch to simple locking approach since contention should be low.

- Update all methods associated with coordination to improve names as well as add javadocs. Move these methods to the RemoteConnection.

- Consolidate the two different close handlers into a single, ordered close handler managed inside the connection.

- Add better javadoc around the close method of RemoteConnection

- Add some preconditions checks.

- Update the HPPC version in the base memory module since it conflicts with the one in the java-exec module.

This closes #463.

    • -160
    • +0
    ./drill/exec/rpc/CoordinationQueue.java
    • -39
    • +0
    ./drill/exec/rpc/PositiveAtomicInteger.java
    • -30
    • +81
    ./drill/exec/rpc/RemoteConnection.java
    • -0
    • +192
    ./drill/exec/rpc/RequestIdMap.java
  1. … 1 more file in changeset.
DRILL-4327: Fix rawtypes warnings in drill codebase

Fixing most rawtypes warning issues in drill modules.

Closes #347

  1. … 76 more files in changeset.
DRILL-4278: Heap memory leak issues

- Fix issue where WorkspaceConfig was not returning consistent hashCode()s for equal objects.

- Fix issue where we were misusing recycler causing object reference leaks

This closes #331.

    • -21
    • +2
    ./drill/common/SerializedExecutor.java
  1. … 4 more files in changeset.
Fix issue where FutureBitComment didn't return buffer as part of future.

    • -0
    • +1
    ./drill/exec/rpc/FutureBitCommand.java
    • -3
    • +6
    ./drill/exec/rpc/RpcCheckedFuture.java
DRILL-4134: Allocator Improvements

- make Allocator mostly lockless

- change BaseAllocator maps to direct references

- add documentation around memory management model

- move transfer and ownership methods to DrillBuf

- Improve debug messaging.

- Fix/revert sort changes

- Remove unused fragment limit flag

- Add time to HistoricalLog events

- Remove reservation amount from RootAllocator constructor (since not allowed)

- Fix concurrency issue where allocator is closing at same moment as incoming batch transfer, causing leaked memory and/or query failure.

- Add new AutoCloseables.close(Iterable<AutoCloseable>)

- Remove extraneous DataResponseHandler and Impl (and update TestBitRpc to use smarter mock of FragmentManager)

- Remove the concept of poison pill record batches, using instead FragmentContext.isOverMemoryLimit()

- Update incoming data batches so that they are transferred under protection of a close lock

- Improve field names in IncomingBuffers and move synchronization to collectors as opposed to IncomingBuffers (also change decrementing to decrementToZero rather than two part check).

This closes #238.

  1. … 119 more files in changeset.
DRILL-3987: (MOVE) Move logical expressions and operators out of common. Move to new drill-logical model.

    • -0
    • +144
    ./drill/common/SerializedExecutor.java
  1. … 213 more files in changeset.
DRILL-3987: (MOVE) Extract RPC, memory-base and memory-impl as separate modules.

    • -0
    • +69
    ./drill/exec/rpc/AbstractHandshakeHandler.java
    • -0
    • +27
    ./drill/exec/rpc/Acks.java
    • -0
    • +41
    ./drill/exec/rpc/BaseRpcOutcomeListener.java
    • -0
    • +316
    ./drill/exec/rpc/BasicClient.java
    • -0
    • +76
    ./drill/exec/rpc/BasicClientWithConnection.java
    • -0
    • +237
    ./drill/exec/rpc/BasicServer.java
    • -0
    • +39
    ./drill/exec/rpc/ChannelClosedException.java
    • -0
    • +25
    ./drill/exec/rpc/ChannelListenerWithCoordinationId.java
    • -0
    • +22
    ./drill/exec/rpc/ConnectionThrottle.java
    • -0
    • +160
    ./drill/exec/rpc/CoordinationQueue.java
    • -0
    • +28
    ./drill/exec/rpc/DrillRpcFuture.java
    • -0
    • +80
    ./drill/exec/rpc/DrillRpcFutureImpl.java
    • -0
    • +79
    ./drill/exec/rpc/FutureBitCommand.java
    • -0
    • +66
    ./drill/exec/rpc/InboundRpcMessage.java
    • -0
    • +70
    ./drill/exec/rpc/ListeningCommand.java
  1. … 127 more files in changeset.
DRILL-4085: Disable RPC thread offload until concurrency bug is found.

DRILL-3581: Upgrade to Guava 18.0

- Replace Stopwatch constructors with .createStarted() or .createUnstarted()

- Stop using InputSupplier and Closeables.closeQuietly

- Clean up quiet closes to log or (preferably) propagate.

- Add log4j to enforcer exclusions.

- Update HBaseTestSuite to add patching of Closeables.closeQuietly() and Stopwatch legacy methods. Only needed when running HBaseMiniCluster.

- Remove log4j from HBase's pom to provide exception logging.

- Remove log4j from Hive's shaded pom.

- Update Catastrophic failures to use the same pattern to ensure reporting.

- Update test framework to avoid trying IPv6 resolution. (This removes 90s pause from HBase startup in my tests)

This closes #361.

This closes #157.

  1. … 64 more files in changeset.