Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Make Default Dir a Command Line Option

- Avoid using static field to hold default directory, in favor of

an option.

- Improve output on failed regexadm match (indicate where actual

and expected diverge)

Change-Id: I7855c8f344eea9c9b6a394d85413a062a3ddb609

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1751

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

  1. … 15 more files in changeset.
Remove Plan Plotter

Change-Id: Ib6a77070344a40c3ae5bb9ff20b8593506d3c746

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1587

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 15 more files in changeset.
Add Missing Descriptions For Config Options

Also, expose knob for enabling transaction commit profile logs as config

option

Change-Id: I04826591133e7e8283b7288e14324659fd831954

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1589

Tested-by: Michael Blow <mblow@apache.org>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 11 more files in changeset.
Config Cleanup, Update Docs

- added some missing descriptions

- updated ncservice.md with updated parameter names / sections

- minor cleanup

- removed some unused properties

- renamed feed properties to active

Change-Id: I9edc9ced4c2b99db239a2075903f9d8b256c26e4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1555

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 28 more files in changeset.
Configuration Revamp

- Ini section of node / cc details now returns ini param names instead of

managix option names

- Normalized command line -vs- ini file configuration parameter names

- Eliminated unused parameters

- Ini validation

- Migrate *DB parameters out of [app] and into nc / cc sections as

appropriate

- Eliminate [app] section. Cluster-wide configuration lives in [common]

- Sort properties alphabetically when returned by HTTP api

Change-Id: I95b7e0bd4538ef42817c8826e76412150074b754

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1487

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

  1. … 174 more files in changeset.
Add AWS automation scripts to asterix-server.

- Allows users to customize an AWS-based instance by editing

conf/aws_settings.yml and conf/instance_settings.yml.

- Starts an AWS cluster, install JDK and AsterixDB automatically.

- Let the default value of storage.metadata.memorycomponent.numpages

adjust to the available JVM heap size.

Change-Id: If4061501e3561a649c3a2bb3068dc257f03c092d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1475

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 16 more files in changeset.
Let log level be default (WARN) in tests

Change-Id: I880e6bf015986dea9235360a859de2f49bfc6e3a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1433

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 6 more files in changeset.
ASTERIXDB-1714: Eliminate dependency on org.json

Change-Id: Ie9c5400fd134ae75d43385255af7794e968b1c7e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1392

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 133 more files in changeset.
ASTERIXDB-1556, ASTERIXDB-1733: Hash Group By and Hash Join conform to the memory budget

- External Hash Group By and Hash Join now conform to the memory budget (compiler.groupmemory and compiler.joinmemory)

- For Optimzed Hybrid Hash Join, we calculate the expected hash table size when the build phase is done and

try to spill one or more partitions if the freespace can't afford the hash table size.

- For External Hash Group By, the number of hash entries (hash table size) is calculated based on

an estimation of the aggregated tuple size and possible hash values for the given field size in that tuple.

- Garbage Collection feature has been added to SerializableHashTable. For external hash group-by,

whenever we spill a data partition to the disk, we also check the ratio of garbage in the hash table.

If it's greater than the given threshold, we conduct a GC on Hash Table.

Change-Id: I2b323e9a2141b4c1dd1652a360d2d9354d3bc3f5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1056

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 41 more files in changeset.
Supports flexible degree of parallelism.

Detailed list of changes include:

- Add the IClusterInfoCollector interface and feed that to the compiler;

- Add the number of cores into the heartbeat;

- Add partition calculation in APIFramework;

- Fix partitioning property requirements in binary operators such as Join and UnionAll;

- Fix PushSubplanIntoGroupByRule for type propagation;

- Fix InlineLeftNtsInSubplanJoinFlatteningVisitor for the returned top join operator;

- Add unit test for partition calculation in APIFrameworkTest

- Add integration test SqlppExecutionFullParallelismIT, SqlppExecutionLessParallelismIT,

AqlExecutionFullParallelismIT, and AqlExecutionLessParallelismIT for different

parallelism settings;

- Refactor AQLExecutionTest and SqlppExecutionTest;

- Separate one cluster state test out from AqlExecutionTest.

Change-Id: If70faf52bed995c7098cc343f2fabd1b6c8d96e7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1398

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 115 more files in changeset.
ASTERIXDB-1570: "Unable to find free page in buffer cache after 1000 cycles"

Increase test buffer cache size to reliably accomodate 20M values

Change-Id: I620de9ade875f7f7de26aa7495d9fd873824ac83

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1346

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 3 more files in changeset.
Reorganize Replication Properties For HTTP API

Replication properties are available via separate cluster endpoint,

http://<host>:19002/admin/cluster/replication

Change-Id: I3cff78800afaa2f4271e8df192431a687263ca9d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1319

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 8 more files in changeset.
Add Diagnostics Endpoint

Add an endpoint (http://<host>:19001/admin/diagnostics) to gather common

diagnostics information on the cluster for cc and each ncs:

- threaddump

- config

- stats

Change-Id: I664b713f6614acd5ce4f8bf8ce6f8f71345cab06

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1310

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 11 more files in changeset.
Omit Managix [nc|cc].java.opts From HTTP Api, Ini > Env

- Managix options for cc / nc java opts are not used for NC service

clusters, omit these from the cluster HTTP api.

- Ini jvm.args should take precedence over $JAVA_OPTS in environment

when launching NCDriver

Change-Id: If52da9f5cc2a66b7c327f39a6e4ea51d4a8ccf89

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1301

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

Integration-Tests: Michael Blow <mblow@apache.org>

  1. … 2 more files in changeset.
Wait For Metadata Registration Before Active, Etc.

Also:

- metadata.port moved to [app]; honored

- += metadata.node to [app] to optionally specify metadata node

- += metadata.callback.port to [app]

- Decrease timeout for metadata registration from 7 days to default of

one minute, configurable with property

- Log swallowed exception in SynchronizableWork

- Add missing properties (metadata) to cluster state http api

- Make AsterixPropertiesAccessor, to ensure consistent values when

accessed in virtual cluster

Change-Id: I48d7c10b3e43181ec307f7d890ba721f61bc2ab0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1247

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 43 more files in changeset.
HTTP API++, NCService Termination After Dead NCService

- include PID of NC service in NC config API

- shutdown NCService even when NC is unresponsive (e.g. dead) when

shutdown?all=true

- include PIDs of NC service & NC in shutdown response document

- interrogate cluster state from client helper

- advertise shutdown, version cluster HTTP APIs

- fix possible bad return code from stop-sample-cluster in case of very

fast CCDriver exit

- cleanup transposed JUnit assert args in SampleLocalClusterIT

Change-Id: Ia48eb16696e48444f610fd5e8d7d4666d0257a38

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1193

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 23 more files in changeset.
Log ResultStateSweeper As Finer, Test Default Level->INFO

Change-Id: Ie8e9408590734694ce1fa5840783adee8d7ce695

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1176

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 2 more files in changeset.
HTTP API++: Threaddumps / CC / INI, Misc

- Thread dump returned in JSON format.

e.g. http://localhost:19002/admin/cluster/node/asterix_nc2/threaddump

- Add CC config, stats, threaddumps

- Add CC / NC ini settings under config endpoints

- Wrap doGet implementations for cluster APIs to fix SonarQube issues

- Avoid boxing arrays & lists for JSONObject.put()

Change-Id: I536606a1fbc394c6c70bb8ce14791cc411691617

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1159

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 33 more files in changeset.
Extend Cluster API Servlet, += Config / Stats

- New APIs to return NC stats & configs, discoverable URIs returned as

part of /admin/cluster.

- Tests for constant info (still need non-exact match tests for config /

stats endpoints)

Change-Id: Ia7549f2bb0b6621886356d50df800d447928aa2c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1147

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 22 more files in changeset.
Expose Asterix Configuration on Cluster Servlet

Also, a few test executor improvements

Change-Id: I828d6a61afe615f7826079ede4b1d638bbd7ac5d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1153

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +58
    ./cluster_state_1.1.adm
  1. … 14 more files in changeset.