Clone Tools
  • last updated 13 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-7441: Fix issues with fillEmpties, offset vectors

Fixes subtle issues with offset vectors and "fill empties"

logic.

Drill has an informal standard that if a batch has no rows, then

offset vectors within that batch should have zero size. Contrast

this with batches of size 1 that should have offset vectors of

size 2. Changed to enforce this rule throughout.

Nullable, repeated and variable-width vectors have "fill empties"

logic that is used in two places: when setting the value count and

when preparing to write a new value. The current logic is not

quite right for either case. Added tests and fixed the code to

properly handle each case.

Revised the batch validator to enforce the offset-vector length of 0 for

0-sized batches rule. The result was much simpler code.

Added tools to easily print a batch, restoring some code that

was recently lost when the RowSet classes were moved.

Code cleanup in all files touched.

Added logic to "dirty" allocated buffers when testing to ensure

logic is not sensitive to the "pristine" state of new buffers.

Added logic to the column writers to enforce the zero-size-batch rule

for offset vectors. Added unit tests for this case.

Fixed the column writers to set the "lastSet" mutator value for

nullable types since other code relies on this value.

Removed the "setCount" field in nullable vectors: turns out

it is not actually used.

closes #1896

    • -22
    • +22
    ./TestJsonReaderWithSparseFiles.java
  1. … 43 more files in changeset.
DRILL-6320: Fixed license headers.

closes #1207

  1. … 2066 more files in changeset.
DRILL-5783, DRILL-5841, DRILL-5894: Rationalize test temp directories

This change includes:

DRILL-5783:

- A unit test is created for the priority queue in the TopN operator.

- The code generation classes passed around a completely unused function registry reference in some places so it is removed.

- The priority queue had unused parameters for some of its methods so it is removed.

DRILL-5841:

- Created standardized temp directory classes DirTestWatcher, SubDirTestWatcher, and BaseDirTestWatcher. And updated all unit tests to use them.

DRILL-5894:

- Removed the dfs_test storage plugin for tests and replaced it with the already existing dfs storage plugin.

Misc:

- General code cleanup.

- Removed unnecessary use of String.format in the tests.

This closes #984

  1. … 365 more files in changeset.
DRILL-2997: Remove references to groupCount from SerializedField

  1. … 17 more files in changeset.
DRILL-2923: Ensure all unit tests pass without assertions enabled Modified a number of tests not to use assert, but to instead use one of junit's assertTrue(), assertFalse(), or some other form. Modified test support code that used asserts to throw IllegalStateExceptions instead.

    • -30
    • +23
    ./TestJsonReaderWithSparseFiles.java
  1. … 19 more files in changeset.
DRILL-2498: Separate QueryResult into two messages QueryResult and QueryData

  1. … 71 more files in changeset.
DRILL-1885: fix a problem regarding ordinal to vector mapping that report incorrect result or fails a query & refactor code, eliminiate redundancy support case insensitive vector lookup yet case sensitive result reporting rely on fields in the order that they show up in the schema while copying vectors

  1. … 17 more files in changeset.
DRILL-1781: Fast Complex Schema

  1. … 44 more files in changeset.
DRILL-1660: return accurate buffers size when repeated map vector has no underlying vectors

    • -65
    • +77
    ./TestJsonReaderWithSparseFiles.java
  1. … 1 more file in changeset.
DRILL-1660: adding a test case to cover an array of empty and null valued objects. i) remove redundant guard statement from JsonRecordReader2 ii) rename unit test suite

    • -0
    • +127
    ./TestJsonReaderWithSparseFiles.java
  1. … 2 more files in changeset.
DRILL-1547: enforce writers to explicitly check for buffer bounds to avoid IndexOutOfBounds errors; make writer hierarchy to stop immediately in case of a write error

    • -0
    • +127
    ./JsonReaderTests.java
  1. … 15 more files in changeset.