Clone
 

ali alsuliman <ali.al.solaiman@gmail.com> in asterixdb

[NO ISSUE] Feed operator doc updates

- user model changes: no

- storage format changes: no

- interface changes: no

Change-Id: Ic3ffbcd691b7a864861272282dcbf093e6999e03

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/7463

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Michael Blow <mblow@apache.org>

[NO ISSUE][RT] Add Support For Feed Ingestion Without Message

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Add an option to specify whether FeedMetaStoreNodePushable

should process a message as part of the feed data or not.

Change-Id: I804fb4ae884020906dc09be188fa976867662d40

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/7404

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Michael Blow <mblow@apache.org>

[NO ISSUE][EXT] Copy JSON object data to buffer only when reading JSON object has started

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Copy JSON object data to buffer only when reading JSON object has started since the JSON

object could be a nested object (i.e. leading comma character and spaces should not be copied over)

Change-Id: Iddac6d5c3926367770ffd31714c54361d99b4268

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/7043

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Hussain Towaileb <hussainht@gmail.com>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

[ASTERIXDB-2726][EXT] Report line number in errors of JSON parser

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Report line number in errors of JSON parser.

Change-Id: I9481cfd4a9244591d22963781497390216df67bb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6423

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Hussain Towaileb <hussainht@gmail.com>

    • -0
    • +7
    /asterixdb/asterix-app/data/json/malformed-json-2.json
    • -0
    • +4
    /asterixdb/asterix-app/data/jsonl/malformed-jsonl-1.json
    • -0
    • +14
    /asterixdb/asterix-app/data/jsonl/malformed-jsonl-2.json
  1. … 4 more files in changeset.
Merge commit '594e7eca21' from 'gerrit/mad-hatter'

Change-Id: I975228925366adc94f136bbbb014cb6817bf180e

[NO ISSUE][EXT] Minor refactoring of external datasets details validation

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Minor refactoring of external datasets details validation.

- Added a missing error message in Hyracks.

Change-Id: I975228925366adc94f136bbbb014cb6817bf180e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6325

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

[ASTERIXDB-2735][EXT] Support boolean value for properties of external data

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

The configuration properties of the external data in the DDL statements are

supplied as key-value pairs of strings. The user should be able to supply

normal boolean values for boolean properties such as "header" which tells

whether the data source has a header or not (for delimited data such as CSV).

Change-Id: I0470f7f0bbf7476e1fbd6b9f124355cf8b00c17c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6343

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

[ASTERIXDB-2731][API] GET is not supported for the query service servlet

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Remove the restriction of only supporting POST requests in the query service servlet

while maintaining that GET requests cannot change the state of the system

(i.e. no modifying statements can be evaluated when using the GET method).

Change-Id: I989ff489d863697808cdc26898a52b867a9621d6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6303

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Till Westmann <tillw@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[ASTERIXDB-2726][EXT] Report line number instead of record number in messages of parsers

- user model changes: no

- storage format changes: no

- interface changes: yes

IRecordReader:

added getLineNumber() to provide line number for parsers and converters.

IRecordConverter:

added configure() to pass the line number supplier to the record converter.

IRecordDataParser:

pass line number supplier from the Reader to the Parser.

Details:

Report line number instead of record number in messages of parsers.

- added getPreviousStreamName() to allow readers to report errors happening on

the previous stream when the underlying stream has already switched to a new one.

- changed the test executor to compare actual warnings issues by a test case with

the expected warnigns properly.

Change-Id: I00508d8eeca4d9bae95f55ab51ecfb0ce2ced6b0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6245

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Hussain Towaileb <hussainht@gmail.com>

    • -0
    • +3
    /asterixdb/asterix-app/data/csv/error1_line_num.csv
    • -0
    • +5
    /asterixdb/asterix-app/data/csv/error2_line_num.csv
  1. … 26 more files in changeset.
[ASTERIXDB-2720][EXT] Support arrays of objects for external dataset with json format

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Currently, the json format that the external dataset support is

the extended version of jsonl. The format is consecutive json objects

(normally one object per line). Arrays of objects should

be supported as valid values as well.

Change-Id: I59462c274ec3186cf35ba4ce222a9ffc4056e00a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6063

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

[NO ISSUE][EXT] Fix error reporting when processing external datasets records

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Fix error reporting when the record size retrieved from external datasets

exceeds the limit instead of reporting internal error.

Change-Id: I0f0973356c727d7d7a28252163aba2591c4205db

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6083

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Till Westmann <tillw@apache.org>

[ASTERIXDB-2713][EXT] Change the name of the new adapter parameter for escaping

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Change the name of the new parameter "quote-escape" to just "escape".

Change-Id: I3b51fa317bf327fbad18b32f0fd45e47e853ca50

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6023

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

[ASTERIXDB-2713][EXT] CSV & TSV support for external dataset p4

- user model changes: no

- storage format changes: no

- interface changes: yes

- IValueParser

Details:

- added parameter "null" to allow user to specifiy what string represents

a null value. Only nullable fields whose values match this string will

produce NULL. Otherwise, no NULLs will be produced.

- empty fields:

- for string fields, produce empty string "".

- for non-string fields, issue a warning an ignore the record.

- changed IValueParser to return boolean to allow some implementations

to return true or false instread of throwing an exception.

- added parameter "redact-warnings" to allow user to specifiy if parser

warnings should exclude information like file name.

- changed the Integer and Long parsers to consider spaces properly and

also to handle overflow and underflow.

- changed the boolean parser to align with other parsers that ignore

leading and trailing spaces.

- added test cases.

Change-Id: Ib6aed0095a472510b69cc29a3e444e7de5a2c1ae

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5963

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +13
    /asterixdb/asterix-app/data/csv/header/h_invalid_values.csv
  1. … 60 more files in changeset.
[ASTERIXDB-2713][EXT] CSV & TSV support for external dataset p3

- user model changes: no

- storage format changes: no

- interface changes: yes

IRecordDataParser, IRecordReader, IRecordConverter

Details:

- record parser:

- delimited-data (CSV/TSV) parser: ignore and warn for invalid records.

- other parses: continue to use their existing behaviour.

- stream parser:

continue to use their existing behaviour.

- fixes:

- fixed S3 stream read() to properly advance to next files and also

to notify consumers to handle properties like header properly.

- fixed localfs stream read() when reached end of current file

and notifying of a new file source.

- extracted the read() of both streams since now they are identical.

- report file, record number and field number in warnings of parser

- propagate stream name to parsers that need report stream name

- add test cases

Change-Id: Ie1ba545d753d8afef9cef4e290e058019a465201

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5926

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -5
    • +0
    /asterixdb/asterix-app/data/csv/empty.csv
    • -0
    • +5
    /asterixdb/asterix-app/data/csv/empty_lines.csv
    • -0
    • +4
    /asterixdb/asterix-app/data/csv/header/h_mul_rec.csv
    • -0
    • +4
    /asterixdb/asterix-app/data/csv/header/h_mul_rec_with_ln.csv
    • -0
    • +2
    /asterixdb/asterix-app/data/csv/header/h_one_rec.csv
    • -0
    • +2
    /asterixdb/asterix-app/data/csv/header/h_one_rec_with_ln.csv
    • -0
    • +1
    /asterixdb/asterix-app/data/csv/header/h_only.csv
    • -0
    • +0
    /asterixdb/asterix-app/data/tsv/empty.tsv
  1. … 130 more files in changeset.
[ASTERIXDB-2713][EXT] CSV & TSV support for external dataset p2

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- validate adapter configuration

- ignore empty lines in CSV/TSV files

- require "header" parameter for CSV/TSV formats

- make some parameters case-insensitive

- few fixes and clean-ups

Change-Id: I2f523de0d482a358ada0c27236ff24616ad0d7da

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5848

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

    • -0
    • +5
    /asterixdb/asterix-app/data/csv/empty.csv
    • -0
    • +1
    /asterixdb/asterix-app/data/csv/sample_12.csv
    • -0
    • +11
    /asterixdb/asterix-app/data/csv/sample_13.csv
  1. … 23 more files in changeset.
[ASTERIXDB-2713][EXT] Add CSV & TSV support for external dataset

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Add CSV support for external dataset.

- support S3

- add boolean parser to Hyracks

Change-Id: Id1790fa73461e9f4a5fb443c51c1905ac588cee6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5743

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

    • -0
    • +3
    /asterixdb/asterix-app/data/csv/01.csv
    • -0
    • +3
    /asterixdb/asterix-app/data/csv/02.csv
    • -0
    • +17
    /asterixdb/asterix-app/data/csv/sample_09.csv
    • -0
    • +39
    /asterixdb/asterix-app/data/csv/sample_10.csv
    • -0
    • +4
    /asterixdb/asterix-app/data/csv/sample_11.csv
    • -0
    • +15
    /asterixdb/asterix-app/data/csv/sample_12.csv
    • -0
    • +3
    /asterixdb/asterix-app/data/tsv/01.tsv
    • -0
    • +3
    /asterixdb/asterix-app/data/tsv/02.tsv
    • -0
    • +28
    /asterixdb/asterix-app/data/tsv/sample_01.tsv
  1. … 34 more files in changeset.
[NO ISSUE] Minor refactoring

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Minor refactoring

Change-Id: I154a312cfe2182b76d74a7c34543263b3345139d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5247

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

[ASTERIXDB-2700][*DB][RT] Fix invalid plan caused by order-by operator in a subquery

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Instead of removing a redundant sort operator from the plan, replace it with

an empty assign operator to avoid potential connection of a sort-merge connector

to another connector.

- modified RemoveUnusedAssignAndAggregateRule to excluded necessary empty

assign operators from being removed.

Change-Id: I8bc11fa046cb15fab04057086817bd400b7809c0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5103

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

[ASTERIXDB-2689][FUN] Make arithmetic functions return NULL on invalid arguments

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Make arithmetic functions return NULL on invalid arguments:

Functions are for: +, -, *, /, DIV, %, ^

Change-Id: I6a822e59c91d06506505597927fab93e403385d8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4723

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

[ASTERIXDB-2689][FUN] Make numeric functions return NULL on invalid arguments p2

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Make numeric functions return NULL on invalid arguments.

round(num), round(num, num), trunc(num, num), round-half-to-even(num, num).

- added warning to round() function

- extracted evaluators of trunc() and round-half-to-even() to a common evaluator

- renamed checkWarnOrSetInteger() to setInteger()

Change-Id: I1ccef0740d64cce3ada8b60d3a699748e7ef8776

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4704

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 21 more files in changeset.
[ASTERIXDB-2689][FUN] Make numeric functions return NULL on invalid arguments

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Make numeric functions return NULL on invalid arguments.

Change-Id: Id26229064058c3ecbaa54ece3001d3b8980f8d8b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4583

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 9 more files in changeset.
[ASTERIXDB-2688][HYR] Fix use of a Hyracks task across join stages

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

In hash join, a task from the build stage is being used in the probe stage.

This is a problem since such tasks have already finished and notified the CC they are done.

One observed issue is related to issuing a warning in the probe phase where some warnings are

not reported because they are issued to tasks that have finished (the way this happened is that

a comparator was created in the build phase using the build-phase task. Then, this comparator

was used in the probe phase and issued a warning).

- make IHyracksJobletContext extend IHyracksCommonContext so that it is also a frame manager context

- make activites of join operators use the joblet context instead of the task context for acquiring buffers

- create the probe-to-build comparator in the probe phase so that the right task is used in the comparator

Change-Id: I38a4a779b9620494f15606162f0f1e9487fd0984

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4563

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Michael Blow <mblow@apache.org>

[ASTERIXDB-2687][RT] Issue a warning when incomparable types are compared

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Issue a warning when NULL was returned as the result of a comparison

of 2 incomparable types.

- added error/warning code 49

- refactored dot format plans generator

- removed some AQL tests that are already covered in SQL++ tests

Change-Id: Iad3127e4db775315817a7819feea6aa12e4de2ea

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4464

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 25 more files in changeset.
[ASTERIXDB-2686][RT] Delete run files eagerly

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Delete run files eagerly.

Change-Id: Idc780b44b0554bf36e2378700aae1b38b8e57353

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4403

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

[ASTERIXDB-2686][RT] Run files are not removed on time

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Remove run files on time.

Change-Id: Iacff5cef38cee147712bc1ff947ffa40efb0d162

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4423

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

[NO ISSUE][COMP] Extension to InjectTypeCastForFunctionArgumentsRule

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Extend InjectTypeCastForFunctionArgumentsRule for functions

that need arguments casting.

Change-Id: I68c264e7885e4f7d51a90fc615891a832a69e785

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4163

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

[ASTERIXDB-2634][FUN] String functions to return null on data/type errors

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- substring(string, int, int)

- codepoint-to-string([int])

- string-concat([string])

Change-Id: I17905ad9e767b43822d28a46123bc1390a159cc7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4083

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Hussain Towaileb <hussainht@gmail.com>

  1. … 5 more files in changeset.
[ASTERIXDB-2671][FUN] Fix handling nullable inputs in OBJECT_*()

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

The input record to OBJECT_*() functions can be nullable/missable

record.

- fixed the type inferrer to take care of that when inferring

the input record type.

Change-Id: Ic7e86598eda03f5aeb3c0e4457127c99bdbed0e4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4123

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 61 more files in changeset.
[ASTERIXDB-2672][API] Change the valid values for "format" request parameter

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Allowed values for "format" request parameter: json, csv, adm.

- Recognizable format values in "Accept":

application/x-adm

application/json

application/json;lossless=true/false

text/csv

text/csv;header=present/absent

Test framework changes:

- ResultExtractor: if the OutputFormat is json/lossless-json, print the "result"

field of the response similar to how adm would be printed, one json value

per line (and using same spacing).

- Changed some queries that use "EXPLAIN SELECT..." and specify OutputFormat as JSON.

The queries extension is ".plans.sqlpp". "param optimized-logical-plan:string=true"

is specified in those queries to print the logical plan in the "plans" field of

the response.

- added "// compareunorderedarray=true" for test quries that use .regexjson to compare

one json value against another where the order of elements in a json array is not

deterministic.

- TestExecutor: OutputFormat.LOSSLESS_JSON & OutputFormat.CSV_HEADER formats are set

in the "Accept". Otherwise, the desired format is set in the "format" request

parameter as usual.

- TestHelper: changed "equalJson()" to allow comparing json array in two modes.

- Removed some test cases that used to set mime types in the "format" request parameter

since now it's not allowed to do so.

Change-Id: Ie3c7a35446322c2d97679e7e724b9778e2a4ba83

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4043

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 18 more files in changeset.
[ASTERIXDB-2659][API] validate request parameters

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Return 400 BAD request for invalid request parameters.

- Changes in specifying the output format:

HEADER "Accept": application/json;lossless=<BOOLEAN>, text/csv;header=absent/present, application/x-adm

Request parameter "format": csv, json (and above mime types)

- when format is supplied in "Accept", only validate value of attribute if the mime is one of the above

and the attribute name is one of the above (header & lossless).

Change-Id: Ia89fa3179cc36a73fe950f41eab1f0a9d17a541c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/3767

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Till Westmann <tillw@apache.org>

Contrib: Till Westmann <tillw@apache.org>

  1. … 66 more files in changeset.