Clone
Ali Alsuliman <ali.al.solaiman@gmail.com>
committed
on 24 Apr
[ASTERIXDB-2713][EXT] CSV & TSV support for external dataset p4
- user model changes: no
- storage format changes: no
- interface changes: y… Show more
[ASTERIXDB-2713][EXT] CSV & TSV support for external dataset p4

- user model changes: no

- storage format changes: no

- interface changes: yes

        - IValueParser

Details:

- added parameter "null" to allow user to specifiy what string represents

 a null value. Only nullable fields whose values match this string will

 produce NULL. Otherwise, no NULLs will be produced.

- empty fields:

 - for string fields, produce empty string "".

 - for non-string fields, issue a warning an ignore the record.

- changed IValueParser to return boolean to allow some implementations

 to return true or false instread of throwing an exception.

- added parameter "redact-warnings" to allow user to specifiy if parser

 warnings should exclude information like file name.

- changed the Integer and Long parsers to consider spaces properly and

 also to handle overflow and underflow.

- changed the boolean parser to align with other parsers that ignore

 leading and trailing spaces.

- added test cases.

Change-Id: Ib6aed0095a472510b69cc29a3e444e7de5a2c1ae

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5963

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Show less