Clone
 

chris hillery <chillery@lambda.nu> in asterixdb

Add "select *" perf test case

Change-Id: I392d959b50465db6873a5d8e513ef5bbb44bdf59

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1187

Reviewed-by: Ian Maxon <imaxon@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Depend on asterix-opt-bom with same version as AsterixDB itself

Change-Id: Ifd5b95bc052c87419d0e925e46f7741b97cc1132

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1137

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Use more accurate name 'NCService' for HyracksVirtualCluster

Change-Id: I2c68c8a7abde3906df2961b214f046bdd9bfd426

Reviewed-on: https://asterix-gerrit.ics.uci.edu/982

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

ASTERIXDB-1482: Added NCServiceExecutionIT, HyracksVirtualCluster.

NCServiceExecutionIT runs all execution tests against a local cluster

managed by the NCService deployment framework.

HyracksVirtualCluster offers programmatic NCService deployment

control along with improved HyracksNCProcess/HyracksCCProcess.

Further fixes and improvements:

1. Fix handling of iodevices/storagedir (ASTERIXDB-1482)

2. Proper handling of [nc] default section in all cases

3. Ensure asterixnc, etc. scripts are executable

4. Consolidate Ini handling

5. Pruned some dead code, including VirtualClusterDriver

6. A bit of refactoring and extended commenting

Change-Id: If3eb450782a595cf85d04a2c2e9cc732564e65e6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/958

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 7 more files in changeset.
Move maven-failsafe-plugin to asterixdb top level

Also improved logging for AsterixLifecycleIT

Change-Id: I605a9e55a3504dd9942dbdcbc7468ffda90d8a93

Reviewed-on: https://asterix-gerrit.ics.uci.edu/914

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -28
    • +0
    /asterixdb/asterix-installer/pom.xml
Improve creating log directory

Change-Id: I1d3df0dbdb46df30b2e6467896e3a224cb7911a6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/890

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Fix first-time test failure

Change-Id: Id96760b94afcfadcf884c52bb7c00d82b1b35ef4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/888

Reviewed-by: Ian Maxon <imaxon@apache.org>

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

ASTERIXDB-1428: Config-management improvements for AsterixDB

1. Added ability for NC Service to log NC output to file

2. Improve config communication from CC to NC

3. Allow overriding NC command name from CC config (and removed said

option from NC Service config as that's the wrong place to manage it)

4. Improve finding NC command (don't depend on PATH)

5. Create asterixncservice.bat script for Windows

6. Fixed NCServiceIT accordingly

Change-Id: I0f2ad32d489ffc27adbb06aebcc1f22a9fcf784e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/887

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Fixes and tweaks for running Asterix atop new config-management framework.

Change-Id: Ie3027c8c839f25ea858790bd3340187f4b11f213

Reviewed-on: https://asterix-gerrit.ics.uci.edu/874

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Overhaul of Hyracks configuration management.

Includes Asterix changes to make use of new conf management as a

Hyracks application.

Change-Id: Ie3027c8c839f25ea858790bd3340187f4b11f212

Reviewed-on: https://asterix-gerrit.ics.uci.edu/336

Tested-by: Chris Hillery <ceej@lambda.nu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 22 more files in changeset.
Allow for optional modules in build.

Change-Id: Ib99bfb3f92dfaa66dc2299db31b734adb5cceeca

Reviewed-on: https://asterix-gerrit.ics.uci.edu/768

Reviewed-by: Chris Hillery <ceej@lambda.nu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

ASTERIXDB-1361: Meaningful error when missing output-record-type

Change-Id: Id5ca7d634b2be5164af82ad128ff577c0bdbd457

Reviewed-on: https://asterix-gerrit.ics.uci.edu/741

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

ASTERIXDB-1165: Use clean JSON consistently

When printing nullable fields and unions, the clean JSON PrinterFactories

were using lossless JSON printers. This is now fixed.

Change-Id: I50442edc3187d003987bc4119559eda676c9b2eb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/493

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

ASTERIXDB-1148: Selectable array-wrapping of results

Introduce "wrapper-array" parameter to HTTP API which selects (for ADM and

JSON) whether to wrap the result sequence in a generated outer array. For

JSON this defaults to "true" as before. For ADM this defaults to false,

resulting in a large number of expected-results changes.

Also introduce ability to have AQL tests which provide HTTP parameters.

Change-Id: I3122f136ff9ca8a2c2268238c57bb5eddab7b27e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/473

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Chris Hillery <ceej@lambda.nu>

  1. … 1426 more files in changeset.
ASTERIXDB-976: CSV support for all basic types

Implements date/time, spatial, UUID, and hexbinary types for CSV output.

Spatial output is "best we can do" - a double-quoted string containing the

"clean JSON" representation of the value. Interval remains unsupported.

Also fixes a few import errors in the clean JSON implementation.

Change-Id: I7a515efd2bbf25895537413b45eb0992484c7412

Reviewed-on: https://asterix-gerrit.ics.uci.edu/444

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 10 more files in changeset.
ASTERIXDB-1120: Fix sub-project builds.

The maven-assembly-plugin configuration for creating source tarballs was

interfering with building in a subdirectory. Since we only need tarballs

at release time, move this plugin to a new 'asterix-release' profile.

(Use same name as asterix project in preparation for merging of the projects.)

Change-Id: Ia42ccd27a9882ac8a27ce1d17c9d71c9ad5505b6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/432

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Tested-by: Chris Hillery <ceej@lambda.nu>

ASTERIXDB-1120: Fix sub-project builds.

The maven-assembly-plugin configuration for creating source tarballs was

interfering with building in a subdirectory. Since we only need tarballs

at release time, move this plugin to a new 'asterix-release' profile.

Change-Id: I457c1700459741158a245f20a9ab2adfc79731d2

Reviewed-on: https://asterix-gerrit.ics.uci.edu/431

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Tested-by: Chris Hillery <ceej@lambda.nu>

Corrected JSON output of date/time/duration types.

Change-Id: I5c37d3b5aad7f286eba1cb8cb5f05ad456a6521e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/363

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Chris Hillery <ceej@lambda.nu>

  1. … 4 more files in changeset.
Almost full support for clean JSON, plus some lossless JSON fixes

Clean JSON correctly supports all numeric and spatial types, plus hex and uuid.

Lossless JSON correctly supports uuid and hex, and "circle" is correct now.

Date/time types remain unfixed for both JSON formats.

Change-Id: I5c37d3b5aad7f286eba1cb8cb5f05ad456a6521d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/362

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Chris Hillery <ceej@lambda.nu>

    • -0
    • +25
    /asterix-app/data/nontagged/allData.adm
    • -17
    • +0
    /asterix-app/data/nontagged/allData.json
  1. … 33 more files in changeset.
Adding 'cleanjson' output format, along with implementation of lists.

Also includes test-framework update to allow testing of cleanjson.

Change-Id: I5c37d3b5aad7f286eba1cb8cb5f05ad456a6521c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/361

Reviewed-by: Ian Maxon <imaxon@apache.org>

Tested-by: Ian Maxon <imaxon@apache.org>

  1. … 168 more files in changeset.
Move ADM-specific printers and related classes into .adm packages.

Change-Id: I5c37d3b5aad7f286eba1cb8cb5f05ad456a6521b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/360

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 125 more files in changeset.
Issue 867: Test cases for CSV with headers and various line separators.

Change-Id: I046aa30824d14cbc894a28719c6c6560f46d0133

Reviewed-on: https://asterix-gerrit.ics.uci.edu/247

Reviewed-by: Ian Maxon <imaxon@uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Preston Carman <ecarm002@ucr.edu>

    • -9
    • +0
    /asterix-app/data/csv/sample_08_header.csv
    • -0
    • +1
    /asterix-app/data/csv/sample_08_header.csv.cr
    • -0
    • +9
    /asterix-app/data/csv/sample_08_header.csv.crlf
    • -0
    • +9
    /asterix-app/data/csv/sample_08_header.csv.lf
  1. … 5 more files in changeset.
Issue 867: Handle delimited files using CR-only line separators

Also simplify record- and field-counting logic.

Change-Id: Ie28abda93fc9e5996008fac8b60aaf906df49cb7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/246

Reviewed-by: Ian Maxon <imaxon@uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Preston Carman <ecarm002@ucr.edu>

Allow retries on IPCSystem.getHandle().

NC will retry indefinitely to connect CC.

Change-Id: I0f4c15cacd265c3fbe85307af9f5c33577035447

Reviewed-on: https://asterix-gerrit.ics.uci.edu/250

Reviewed-by: Chris Hillery <ceej@lambda.nu>

Tested-by: Chris Hillery <ceej@lambda.nu>

Several CSV, API, HTTP API, and Web interface improvements.

- APIFramework: internal refactoring to consolidate output PrintWriter,

OutputFormat, and all output flags into SessionConfig

- APIFramework: "HTML" is now a flag, rather than an OutputFormat

- HTTP API: Output format can be select via query parameter in

addition to HTTP Accept header

- CSV: default output is now without header, to improve roundtripping

- CSV: header can be requested via Accept header or "header" query

parameter

- Web interface: Added ability to select output format (JSON, CSV or ADM)

Change-Id: I91398bd30dbd6f3b1f69eb51fbf201010d0e5d93

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/242

Reviewed-by: Chris Hillery <ceej@lambda.nu>

Tested-by: Chris Hillery <ceej@lambda.nu>

    • -18
    • +36
    /asterix-doc/src/site/markdown/csv.md
Doc warning about CSV files with headers on HDFS

Change-Id: I8ca0a12ed09a7a067302b376558998b069c2ef78

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/233

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

Make testre work right

Change-Id: I8dbfdcc8842efbfadc6bb764ae5f82483cebfc59

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/232

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

Add test case and documentation for CSV parsing with headers.

Also fix a minor CSV parsing bug with line counting.

Change-Id: Ib875d60aa2465d4a50ee50c5c0e9356185f35c73

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/228

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Taewoo Kim <wangsaeu@gmail.com>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

    • -0
    • +9
    /asterix-app/data/csv/sample_08_header.csv
    • -3
    • +13
    /asterix-doc/src/site/markdown/csv.md
Introduce ("header"="true") option for CSV parsing

Change-Id: I473bc7b66d0c0d1355b5fb3e392cb9ece151e50e

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/226

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

Tested-by: Chris Hillery <ceej@lambda.nu>

Command-line option -metadata-port to specify fixed port for metadata RMI

Change-Id: Ibf039b481a23ef82c9e80950e3b8daa1e9afb07a

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/212

Reviewed-by: Ian Maxon <imaxon@uci.edu>

Tested-by: Chris Hillery <ceej@lambda.nu>