dbarclay <>
on 28 Oct 15
DRILL-2288: Fix ScanBatch violation of IterOutcome protocol and downstream chain of bugs.

2288: Pt. 1 Core: Added unit test. … Show more
DRILL-2288: Fix ScanBatch violation of IterOutcome protocol and downstream chain of bugs.


2288:  Pt. 1 Core:  Added unit test.  [Drill2288GetColumnsMetadataWhenNoRowsTest, empty.json]

2288:  Pt. 1 Core:  Changed HBase test table #1's # of regions from 1 to 2.  [HBaseTestsSuite]

Also added TODO(DRILL-3954) comment about # of regions.

2288:  Pt. 2 Core:  Documented IterOutcome much more clearly.  [RecordBatch]

Also edited some related Javadoc.

2288:  Pt. 2 Hyg.:  Edited doc., added @Override, etc.  [AbstractRecordBatch, RecordBatch]

Purged unused SetupOutcome.

Added @Override.

Edited comments.

Fix some comments to doc. comments.

2288:  Pt. 3 Core&Hyg.:  Added validation of IterOutcome sequence.  [IteratorValidatorBatchIterator]


Renamed internal members for clarity.

Added comments.

2288:  Pt. 4 Core:  Fixed a NONE -> OK_NEW_SCHEMA in  [ScanBatch]

(With nearby comments.)

2288:  Pt. 4 Hyg.:  Edited comments, reordered, whitespace.  [ScanBatch]


Added comments.


2288:  Pt. 4 Core+:  Fixed UnionAllRecordBatch to receive IterOutcome sequence right.  (3659)  [UnionAllRecordBatch]

2288:  Pt. 5 Core:  Fixed ScanBatch.Mutator.isNewSchema() to stop spurious "new schema" reports (fix short-circuit OR, to call resetting method right).  [ScanBatch]

2288:  Pt. 5 Hyg.:  Renamed, edited comments, reordered.  [ScanBatch, SchemaChangeCallBack, AbstractSingleRecordBatch]

Renamed getSchemaChange -> getSchemaChangedAndReset.

Renamed schemaChange -> schemaChanged.

Added doc. comments.


2288:  Pt. 6 Core:  Avoided dummy Null.IntVec. column in JsonReader when not needed (MapWriter.isEmptyMap()).  [JsonReader, 3 vector files]

2288:  Pt. 6 Hyg.:  Edited comments, message.  Fixed message formatting.  [RecordReader, JSONFormatPlugin, JSONRecordReader, AbstractMapVector, JsonReader]

Fixed message formatting.

Edited comments.

Edited message.

Fixed spurious line break.

2288:  Pt. 7 Core:  Added column families in HBaseRecordReader* to avoid dummy Null.IntVec. clash.  [HBaseRecordReader]

2288:  Pt. 8 Core.1:  Cleared recordCount in OrderedPartitionRecordBatch.innerNext().  [OrderedPartitionRecordBatch]

2288:  Pt. 8 Core.2:  Cleared recordCount in ProjectRecordBatch.innerNext.  [ProjectRecordBatch]

2288:  Pt. 8 Core.3:  Cleared recordCount in TopNBatch.innerNext.  [TopNBatch]

2288:  Pt. 9 Core:  Had UnorderedReceiverBatch reset RecordBatchLoader's record count.  [UnorderedReceiverBatch, RecordBatchLoader]

2288:  Pt. 9 Hyg.:  Added comments.  [RecordBatchLoader]

2288:  Pt. 10 Core:  Worked around mismatched map child vectors in MapVector.getObject().  [MapVector]

2288:  Pt. 11 Core:  Added OK_NEW_SCHEMA schema comparison for HashAgg.  [HashAggTemplate]

2288:  Pt. 12 Core:  Fixed memory leak in BaseTestQuery's printing.

Fixed bad skipping of RecordBatchLoader.clear(...) and

QueryDataBatch.load(...) for zero-row batches in printResult(...).

Also, dropped suppression of call to

VectorUtil.showVectorAccessibleContent(...) (so zero-row batches are

as visible as others).

2288:  Pt. 13 Core:  Fixed test that used unhandled periods in column alias identifiers.

2288:  Misc.:  Added # of rows to showVectorAccessibleContent's output.  [VectorUtil]

2288:  Misc.:  Added simple/partial toString() [VectorContainer, AbstractRecordReader, JSONRecordReader, BaseValueVector, FieldSelection, AbstractBaseWriter]

2288:  Misc. Hyg.:  Added doc. comments to VectorContainer.  [VectorContainer]

2288:  Misc. Hyg.:  Edited comment.  [DrillStringUtils]

2288:  Misc. Hyg.:  Clarified message for unhandled identifier containing period.

2288:  Pt. 3 Core&Hyg. Upd.:  Added schema comparison result to logging.  [IteratorValidatorBatchIterator]

2288:  Pt. 7 Core Upd.:  Handled HBase columns too re NullableIntVectors.  [HBaseRecordReader, TestTableGenerator, TestHBaseFilterPushDown]

Created map-child vectors for requested columns.

Added unit test method testDummyColumnsAreAvoided, adding new row to test table,

updated some row counts.

2288:  Pt. 7 Hyg. Upd.:  Edited comment.  [HBaseRecordReader]

2288:  Pt. 11 Core Upd.:  REVERTED all of bad OK_NEW_SCHEMA schema comparison for HashAgg.  [HashAggTemplate]

This reverts commit 0939660f4620c03da97f4e1bf25a27514e6d0b81.

2288:  Pt. 6 Core Upd.:  Added isEmptyMap override in new (just-rebased-in) PromotableWriter.  [PromotableWriter]

Adjusted definition and default implementation of isEmptyMap (to handle MongoDB

storage plugin's use of JsonReader).

2288:  Pt. 6 Hyg. Upd.:  Purged old atLeastOneWrite flag.  [JsonReader]

2288:  Pt. 14:  Disabled newly dying test testNestedFlatten().

Show less

master + 14 more