Clone
Sorabh Hamirwasia <shamirwasia@maprtech.com>
committed
on 09 May 18
DRILL-6446: Support for EMIT outcome in TopN - Added comments for TopNBatch and PriorityQueueTemplate - Adding support for SchemaChange ac… Show more
DRILL-6446: Support for EMIT outcome in TopN    - Added comments for TopNBatch and PriorityQueueTemplate        - Adding support for SchemaChange across next() call with HyperVector in incoming container. This is achieved by adding a new method in HyperVectorWrapper which just updates the vector[] array          holding multiple vectors with provided input ValueVector array. And also modifying RemovingRecordBatch GenericSV4Copier to hold reference to VectorWrapper instead of ValueVector[] for each column in incoming batch         - Handling empty batches. Two cases like empty batches in the begining with EMIT outcome and empty batches between consecutive EMIT outcome but after receiving some batches with data and EMIT outcome.                Note: In first case of empty batch it was only returning EMIT outcome without properly creating the output container and SV4 vector. Because of that there could be a case where let's say first batch with EMIT outcome                is empty then TopN will return an empty batch with SV mode NONE and if later batch comes with some records and EMIT outcome, that will generate output batch with OK_NEW_SCHEMA (since TopN always              generate first output batch with records with OK_NEW_SCHEMA as it returns output with SV4 mode). Also let's consider both batch with EMIT outcome were produced after processing first 2 rows of an input batch.                This is a problem as this is simulating schema change across rows of same incoming batch which will never be the case.

                Note: In second case of empty batches priority queue will not be null and will be uninitialized. Also optimize to send EMIT outcome with output batch which has all the data to return for current iteration

               rather than sending it with OK followed by empty batch with EMIT outcome.

closes #1293

Show less

master + 3 more