Clone Tools
  • last updated 20 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[NO ISSUE][*DB][CLUS] Ensure registering node is authorized

Change-Id: I1c47e8e7934b87b758b32e186b613b33d852066d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4863

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Michael Blow <mblow@apache.org>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 2 more files in changeset.
[NO ISSUE][*DB][CLUS] Add support for a rebalance required cluster state

Allow extensions to mandate that a rebalance is required in order for

the cluster to go active

Change-Id: I863f8bf3fe1ce8d59522c9a28a1283006ffa414c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3394

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 2 more files in changeset.
[NO ISSUE][*DB][CLUS] Add API to return all known nodes atomically

Change-Id: Ie6bb64e331666ef44b07034d430ba2ee563bba82

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3340

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 1 more file in changeset.
[NO ISSUE][*DB][CLUS] Add failed nodes API to ClusterStateManager

Change-Id: I1b56b29c38f0bbfe54b30ee5781770e90404c528

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3335

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 1 more file in changeset.
[NO ISSUE][CLUS] Include NC Local Counters in Startup Message

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- Include NC local counters in the NC startup tasks

completion completion message.

- Ensure no NC is marked as active without its local

counters reported.

- Remove the need for individual NC local counters

message.

- Clean up ITxnIdFactory and IResourceIdManager APIs.

Change-Id: Ief8b9d43783ea22810c6fdb29947a1284e32daee

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2767

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 17 more files in changeset.
[NO ISSUE] Add support for NodeGroup upsert, etc.

Also, enable resource id generation when only pending removal nodes are absent

from the cluster

Change-Id: I15cfb74bc345680102cedafa99f7ff4f144860bc

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2389

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Integration-Tests: Murtadha Hubail <mhubail@apache.org>

Tested-by: Murtadha Hubail <mhubail@apache.org>

  1. … 12 more files in changeset.
[NO ISSUE][CLUS] Add Metadata Cluster Partition

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- Add a cluster partition reference to the cluster

partition in which metadata is stored. This allows

the initial metadata node to be removed from the

cluster and another metadata node to be assigned

to that metadata cluster partition. Initially,

it is assigned to the first partition of the first

metadata node.

- Use metadata cluster partition in defining metadata

datasets file splits instead of the assumption of the

first partition on the initial metadata node.

Change-Id: I2ac99252cacba92b4c4484c0d34cdc77fee307e8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2348

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 3 more files in changeset.
[ASTERIXDB-1723] Delete managix

- user model changes: yes

- storage format changes: no

- interface changes: yes

Details:

- Delete managix installer method and all dependencies

- Remove xml-based configuration

Change-Id: I5c710b6ec870bedd49dd46b74ed96a3665670e9b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2155

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 1378 more files in changeset.
[ASTERIXDB-2158] Remove AsterixException (step 1)

- user model changes: no

- storage format changes: no

- interface changes: replace AsterixException on signatures

Details:

This change is the first step towards removing AsterixException from

all interfaces.

Change-Id: I3e858576f39f671cbf5eb14adc7a22aab1335faf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2135

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Murtadha Hubail <mhubail@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 49 more files in changeset.
[NO ISSUE][CLUS] Prevent Possible Deadlock in GlobalRecoveryManager

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Perform global recovery on a different thread to prevent

possible deadlocks in ClusterStateManager.

- Shutdown the system in case of global recovery failure

since the system will be in inconsistent state.

Change-Id: I3b0491a84fb24b428d5ce98f392adafc6dfacff9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2072

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 2 more files in changeset.
[ASTERIXDB-2107][CLUS] Prevent Invalid UNUSABLE State in Dynamic Topology

- user model changes: no

- storage format changes: no

- interface changes: yes

Renamed IClusterStateManager add/Remove NCConfig methods

to notifyNode join/failure.

Details:

- Mark node as participant when it completes its startup and

not when it joins the cluster.

- Allow partitions to be added with pending activation state.

- Remove the use of static MetadataProperties for reporting number of nodes.

- Add test cases.

Change-Id: I7a0db2d66cf44650dcc673b3f2de537816cb84c7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2029

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 9 more files in changeset.
[NO ISSUE][*DB] Explicitly create ClusterStateManager

- user model changes: no

- storage format changes: no

- interface changes: no

details:

- Previously, we create the cluster state manager using the

singleton pattern.

- After this change, cluster state manager is created

as part of the cc application start.

- To access the cluster state manager after this change,

the cc application context is used.

Change-Id: Id6532245033ac4c6f6aa9f193539944eecb832f7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1944

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 40 more files in changeset.
[ASTERIXDB-2019][CLUS] Update cluster state on partitions changes

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Set the cluster to UNUSABLE when no partitions are registered

- Update cluster state after partitions register/de-register

- Reject unregistered nodes queries on CC

- Avoid NPE when trying to send to a node that was de-registered

Change-Id: I7d11733a1dcd86136e157d80517bff4abcfc776b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1918

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 3 more files in changeset.
[ASTERIXDB-1992][ING] Suspend/Resume active entities

- user model changes: wait for completion only returns

when the entity becomes inactive.

- storage format changes: no

- interface changes:

01) Introduce:

IMetadataLockManager for entity locking.

IActiveNotificationHandler for handling active events.

IRetryPolicy for recovery of failed active jobs.

IActiveEntityController for controlling active entities.

02) IJobLifecycleListener.notifyJobFinish now passes the

JobStatus and Exceptions.

03) IActiveEntityEventsListener.isActive() returns true,

if entity is active, false otherwise.

04) IActiveEntityEventsListener.unregister() removes the

listener upon entity deletion.

05) IActiveEntityEventsListener.getJobFailure() returns

the job failure if the entity is in a failed state.

06) IStatementExecutor.getComponentProvider() returns

the storage component provider.

07) IStatementExecutor.getApplicationContext() returns

the application context.

08) IMetadataManager.upsertEntity to perfrom metadata

entities' upsert operations.

09) IMetadataNode.upsertEntity to perfrom metadata

entities' upsert operations

10) ICcApplicationContext.getMetadataLockManager() returns

the lock manger.

11) ICcApplicationContext.getClusterStateManager() returns

the cluster state manager.

details:

- Starting and stopping of active entities now go through

their listeners rather than having the listener indicating

active entity and not having it indicating inactive entity.

This facilitates suspend/resume operations of long running

active jobs for the sake of DDL operations or topology

changes.

- Unit tests for the vast majority of code paths and

different possible scenarios have been added.

Change-Id: Ifeac8c73e6bad39a13663b84a52121356e3c6b40

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1875

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 140 more files in changeset.
Enable Adding Nodes to Running *DB Cluster

Also ability to configure unique partition ids without having access to

complete cluster topology

Change-Id: If978442a95687c00ef78c89ed1b4440f5e308b99

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1785

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 23 more files in changeset.
Remove static cc application context instance

Change-Id: Ia2e250405967ec880e7af6387aa981f39b3392c0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1606

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 181 more files in changeset.
Wait For ClusterState ACTIVE Before Notifying ZK

Defer notifying ZooKeeper that the cluster is up until it transitions to

ACTIVE

Change-Id: Ieaaeb2876edad9cfa3f23c2cbe00e058bdc1c8cc

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1678

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 4 more files in changeset.
Configuration Revamp

- Ini section of node / cc details now returns ini param names instead of

managix option names

- Normalized command line -vs- ini file configuration parameter names

- Eliminated unused parameters

- Ini validation

- Migrate *DB parameters out of [app] and into nc / cc sections as

appropriate

- Eliminate [app] section. Cluster-wide configuration lives in [common]

- Sort properties alphabetically when returned by HTTP api

Change-Id: I95b7e0bd4538ef42817c8826e76412150074b754

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1487

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

  1. … 174 more files in changeset.
Introduce Strategy Based Replication and Fault-Tolerance

This change includes the following:

- Introduce new APIs for Replication and Fault-Tolerance Strategies.

- Add configuration in cluster description file for high-availability.

- Add built-in replication strategies (Metadata_Only, Chained_Declustering)

- Add built-in fault-tolerance strategies (Auto, Metadata_Node)

- Remove none cluster state functionality from ClusterStateManager.

- Add customizable NC startup sequence

Change-Id: I1d1012f5541ce786f127866efefb9f3db434fedd

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1405

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -0
    • +85
    ./IClusterStateManager.java
  1. … 111 more files in changeset.
Introduce IStorageComponentProvider

Change-Id: If86750cdb2436c713f6598e54d4aaaf23d9f7bbf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1451

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +29
    ./IGlobalRecoveryManager.java
  1. … 426 more files in changeset.
Wait For Metadata Registration Before Active, Etc.

Also:

- metadata.port moved to [app]; honored

- += metadata.node to [app] to optionally specify metadata node

- += metadata.callback.port to [app]

- Decrease timeout for metadata registration from 7 days to default of

one minute, configurable with property

- Log swallowed exception in SynchronizableWork

- Add missing properties (metadata) to cluster state http api

- Make AsterixPropertiesAccessor, to ensure consistent values when

accessed in virtual cluster

Change-Id: I48d7c10b3e43181ec307f7d890ba721f61bc2ab0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1247

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 43 more files in changeset.