Clone Tools
  • last updated 18 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[ASTERIXDB-2397][*DB] Fix sample cluster on Java 10

(cherry picked from commit 4dad013297f033e7aaa83fbf72fa11eaada7f927)

Change-Id: I3a03eba1c970d1f8c7d27f93dc4e7336a24800ea

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2701

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

[ASTERIXDB-2397][*DB] Fix sample cluster on Java 10

Change-Id: Ie40107e1f402e977413127783f51dc3a91dad2d2

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2695

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

Relax JAVA_HOME check for sample cluster

Just require that the default configured java is

version 1.8 and not that the environment variable

is explicitly set.

Change-Id: I6b9421b9b480eeea94644a227faf614c2ccf2fb2

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2638

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Taewoo Kim <wangsaeu@gmail.com>

    • -6
    • +49
    ./local/bin/start-sample-cluster.sh
[NO ISSUE] Remove instance.name, storage.subdir config properties

Change-Id: Ic530559e96eb4c744499f46a5595e3a4c59f7683

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2585

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Integration-Tests: Murtadha Hubail <mhubail@apache.org>

Tested-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 14 more files in changeset.
[ASTERIXDB-1723] Delete managix

- user model changes: yes

- storage format changes: no

- interface changes: yes

Details:

- Delete managix installer method and all dependencies

- Remove xml-based configuration

Change-Id: I5c710b6ec870bedd49dd46b74ed96a3665670e9b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2155

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 1378 more files in changeset.
[NO ISSUE][API] Add NC Storage API

- user model changes: no

- storage format changes: no

- interface changes: yes

Add IStorageSubsystem to track storage partitions

replicas.

Details:

- Add NC API port.

- Add storage API to NCs.

- Add StorageSubsystem to track storage partitions

replicas.

Change-Id: I120d9892bc9fe5a73395cd5a2ddc30b51b73ced2

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2190

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 25 more files in changeset.
[ASTERIXDB-2032] Restore use of REST shutdown API

This reverts commit 122a73693e4a6a2484936ed86967e843d4dfa4e1.

Change-Id: Ie73cc5dab3b9ad06120ca634f5aee9dfeef6d07b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1958

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

[TEST] Add performance test for AsterixDB

1. Fix the issue that external var in ansible is truncked by space.

2. Add performance test comparison code for SparkSQL using small

instances.

Change-Id: Ic9ff8efa0be71bde67190ba6f9fbd647c7799084

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1879

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 41 more files in changeset.
[ASTERIXDB-2032]Let stop-sample-cluster.sh to kill processes.

Change-Id: I44d75e518cca604273790aa1d862ca5764fa6395

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1933

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Use JRE for AWS installation.

Change-Id: I25ca959f53c49c7aa1e2841227d27d7c254578be

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1776

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Reviewed-by: Michael Blow <mblow@apache.org>

    • -0
    • +26
    ./aws/yaml/install_jre.yml
Handle Spaces in Path For Sample Cluster

Fixes ASTERIXDB-1842- start-sample-cluster.sh with user name ending with

space.

Change-Id: Id4b17698762e3cb2959152e6f957dad175d0f99b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1747

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -11
    • +11
    ./local/bin/start-sample-cluster.sh
  1. … 1 more file in changeset.
Enable HTTP API processing on NCs

- Query/Status/Result are answered by NC nodes

- other HTTP requests are proxied to the CC node

- SessionConfig refactoring – split into config and output (SessionOutput)

- TestExecutor now can send http requests do multiple nodes (round robin)

Change-Id: I19414a23e163fc4deef9805c8f9089609f1ebe07

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1709

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 41 more files in changeset.
Enable debug option in Ansible/AWS scripts.

When the debug option is set to be true, for each NCDriver process:

- a debugger port is open;

- a Yourkit agent is attached and a Yourkit remote port is open.

Add -Xmx for NCDriver if the user-provided jvm.args does not contain it.

Change-Id: Idae06144ea39a942d30d6482d96c8d8f258ec783

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1700

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +11
    ./ansible/conf/instance_settings.yml
    • -0
    • +24
    ./ansible/yaml/download_yourkit.yml
  1. … 1 more file in changeset.
Add install/uninstall UDF to Ansible

1. Add udf.sh for handling install/uninstall UDF packages to the cluster.

2. Remove unnecessary path check in TestLibrarian.

Change-Id: If7ea9640a06e2b691c19a8a819307d84b55a679e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1682

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +72
    ./ansible/bin/udf.sh
    • -0
    • +30
    ./ansible/yaml/deploy_udf.yml
    • -0
    • +33
    ./ansible/yaml/destroy_udf.yml
  1. … 2 more files in changeset.
Support stopping and restarting for a AWS-based cluster instance.

Change-Id: I77fff4a8a5f6e5d8ebee52eb66d86cc18f32cd12

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1643

Reviewed-by: Michael Blow <mblow@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +28
    ./aws/bin/terminate.sh
    • -0
    • +37
    ./aws/yaml/aws_terminate.yml
    • -0
    • +41
    ./aws/yaml/instance_init.yml
  1. … 1 more file in changeset.
AsterixDB Rat Execution Audit & Fixes

Change-Id: I3d771c5b63eaef4b6a0b4da769039ed5bca17736

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1609

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 492 more files in changeset.
Make aws scripts work with all latest instance types that are equiped with instance-stores.

Change-Id: I16efff22a029e316c7a7dc0402c0131e0c0bdea4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1603

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

    • -0
    • +41
    ./aws/bin/gen_volumes.sh
Remove Extraneous Quotes in Start Sample Batchfile

e.g.

"--------------------------"

"Mon 03/20/2017 19:31:01.81"

"--------------------------"

Change-Id: I24581eb36e193628b93cb5be52903beb206c603d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1605

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Minor fix for the aws_start script.

Change-Id: I7446868484a1ffd7ca84c3652276e69db5208498

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1599

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Auto-config SSDs for the AWS installtion.

Change-Id: I584cc0a9e7420b2b9144635fee107494cb3fa487

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1598

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +37
    ./aws/conf/aws_types.yml
    • -0
    • +28
    ./aws/yaml/mount.yml
Remove Usages Of Jps To Enable Running With JRE

Fixes ASTERIXDB-1832 - Cluster scripts shouldn't require JDK

Change-Id: Iddb52d2a2696c8c43af92fda9b6806d9798bf881

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1563

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 40 more files in changeset.
Fix ERROR When Running Stop Sample Bat On Windows

When running stop-sample-cluster.bat from a non-interactive shell on

Windows, the following error has been observed:

ERROR: Input redirection is not supported, exiting the process immediately

Implemented a sleep function in client helper, to workaround the

limitations of the Windows TIMEOUT command.

Change-Id: Id2a68ec8217690a79b6c49fa670728cf67e81654

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1565

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 3 more files in changeset.
Update [app] to [common] in configs.

Change-Id: Ieb81dbcb9ac592227133f3e80adcc032d7232f25

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1557

Reviewed-by: Michael Blow <mblow@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Allow spaces in sample script folder for Windows

Change-Id: I74da9e032ccf9d5f13f6352b6f5d48650b9ab63b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1545

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -13
    • +13
    ./local/bin/start-sample-cluster.bat
Configuration Revamp

- Ini section of node / cc details now returns ini param names instead of

managix option names

- Normalized command line -vs- ini file configuration parameter names

- Eliminated unused parameters

- Ini validation

- Migrate *DB parameters out of [app] and into nc / cc sections as

appropriate

- Eliminate [app] section. Cluster-wide configuration lives in [common]

- Sort properties alphabetically when returned by HTTP api

Change-Id: I95b7e0bd4538ef42817c8826e76412150074b754

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1487

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

  1. … 171 more files in changeset.
Fix log redirection in Ansible start cluster scripts.

Change-Id: Ia62a9f5d313c5d7ad0629af95199c2f7b4538435

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1553

Reviewed-by: Michael Blow <mblow@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Fix local cluster stop and erase scripts.

- Use ps to list processes

- Stop the instance first in bin/erase.sh

- Copy cc.conf to {{ binarydir }} so that it can

be erased

Change-Id: Ib62483bfbb08ebd6f3ed97d1b64645541ce19d35

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1549

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Ansible scripts for local cluster installation.

This change includes the following parts:

- Refactored aws scripts and share

the common part with non-aws cluster installation;

- Fixed aws_start.yml and aws_stop.yml to count

based on the user-provided tag.

Change-Id: I310f6f198c9ce41d7d7667dc4fbc21b3e34a77ab

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1525

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -0
    • +31
    ./ansible/bin/deploy.sh
    • -0
    • +32
    ./ansible/bin/erase.sh
    • -0
    • +31
    ./ansible/bin/start.sh
    • -0
    • +32
    ./ansible/bin/stop.sh
    • -0
    • +24
    ./ansible/conf/cc.conf
    • -0
    • +45
    ./ansible/conf/instance_settings.yml
    • -0
    • +27
    ./ansible/conf/inventory
    • -0
    • +29
    ./ansible/yaml/deploy.yml
    • -0
    • +24
    ./ansible/yaml/deploy_all.yml
    • -0
    • +23
    ./ansible/yaml/erase.yml
    • -0
    • +24
    ./ansible/yaml/erase_all.yml
    • -0
    • +37
    ./ansible/yaml/gen_conf.yml
    • -0
    • +52
    ./ansible/yaml/instance_start.yml
    • -0
    • +46
    ./ansible/yaml/instance_stop.yml
  1. … 15 more files in changeset.
Various fixes for Windows NCService

- Fix RESOURCE_PATH

- Use windows platform in appassembler instead of booter-windows for client-helper

- Fix a few redirection bugs in sample start/stop batch files

Change-Id: If7af80292bddd5191d83bac18f0a518621d7db71

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1518

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Ian Maxon <imaxon@apache.org>

Tested-by: Ian Maxon <imaxon@apache.org>

    • -15
    • +1
    ./local/bin/start-sample-cluster.bat
    • -14
    • +3
    ./local/bin/stop-sample-cluster.bat
  1. … 3 more files in changeset.
Add cluster performance testing scripts.

- Wait SSH instead of sleep in the AWS installation script;

- Remove Java code and generate config files in Ansible;

- TPC-H is one benchmark, and more benchmarks could be added into the framework.

Change-Id: Ie17f20dc87694a865aebc561d8eded064c66f5bb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1504

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 83 more files in changeset.