BitData.proto

Clone Tools
  • last updated 19 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-6792: Find the right probe side fragment wrapper & fix DrillBuf reference count bugs & tune the execution flow & support left deep tree

closes #1504

  1. … 28 more files in changeset.
DRILL-6731: Move the BFs aggregating work from the Foreman to the RuntimeFilter

  1. … 17 more files in changeset.
DRILL-6385: Support JPPD feature

  1. … 63 more files in changeset.
DRILL-4280: CORE (Java protocol)

+ Define SaslStatus and SaslMessage messages in protocol

+ Add "authenticationMechanisms" field to all handshakes

+ Add "saslSupport” field to UserToBitHandshake

  1. … 20 more files in changeset.
DRILL-4134: Allocator Improvements

- make Allocator mostly lockless

- change BaseAllocator maps to direct references

- add documentation around memory management model

- move transfer and ownership methods to DrillBuf

- Improve debug messaging.

- Fix/revert sort changes

- Remove unused fragment limit flag

- Add time to HistoricalLog events

- Remove reservation amount from RootAllocator constructor (since not allowed)

- Fix concurrency issue where allocator is closing at same moment as incoming batch transfer, causing leaked memory and/or query failure.

- Add new AutoCloseables.close(Iterable<AutoCloseable>)

- Remove extraneous DataResponseHandler and Impl (and update TestBitRpc to use smarter mock of FragmentManager)

- Remove the concept of poison pill record batches, using instead FragmentContext.isOverMemoryLimit()

- Update incoming data batches so that they are transferred under protection of a close lock

- Improve field names in IncomingBuffers and move synchronization to collectors as opposed to IncomingBuffers (also change decrementing to decrementToZero rather than two part check).

This closes #238.

  1. … 119 more files in changeset.
DRILL-2187: Single Broadcast Sender

Also includes:

1. Fix merge join planning issue (1c5c810 by jinfengni)

2. ExternalSort: Check the memory available for in-memory sorting or not in making decision to spill or not (36f9dd1)

3. Cleanup in ExternalSortBatch and its helper classes (36f9dd1)

4. MergeJoinBatch: Limit the outgoing record batch size to 2^15 (37dfeb8)

5. StreamingAggBatch: Limit outgoing record batch size to 2^15 (7d8a2e4)

  1. … 57 more files in changeset.
DRILL-620: Memory consumption fixes

accounting fixes

trim buffers

switch to using setSafe and copySafe methods only

adaptive allocation

operator based allocator wip

handle OOM

Operator Context

  1. … 158 more files in changeset.
DRILL-334: Subdivide Drillbit control and data messages. Add support for socket backpressure. Add TopLevel and Child memory allocator with debug mode to capture memory leaks. Various memory leak fixes to get build to complete.

Also includes fixes from reviews by Tim.

  1. … 212 more files in changeset.