Clone
Paul Rogers <progers@cloudera.com>
committed
on 12 Oct 18
DRILL-6791: Scan projection framework
The "schema projection" mechanism:

* Handles none (SELECT COUNT\(*)), some (SELECT a, b, x) and all (… Show more
DRILL-6791: Scan projection framework

The "schema projection" mechanism:

* Handles none (SELECT COUNT\(*)), some (SELECT a, b, x) and all (SELECT *) projection.

* Handles null columns (for projection a column "x" that does not exist in the base table.)

* Handles constant columns as used for file metadata (AKA "implicit" columns).

* Handle schema persistence: the need to reuse the same vectors across different scanners

* Provides a framework for consuming externally-supplied metadata

* Since we don't yet have a way to provide "real" metadata, obtains metadata hints from

 previous batches and from the projection list (a.b implies that "a" is a map, c[0]

 implies that "c" is an array, etc.)

* Handles merging the set of data source columns and null columns to create the final output batch.

* Running tests found a failure due to an uninialized "bits" vector. Added code to explicitly fill

 the bits vectors with zeros in the "result set loader."

Show less

master + 2 more