Clone Tools
  • last updated 28 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[NO ISSUE][OTH] Remove Unused Imports

- user model changes: no

- storage format changes: no

- interface changes: no

Change-Id: Iafff39073d0fedaff74a26ef7e3260008a79ff0c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2893

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

  1. … 67 more files in changeset.
[NO ISSUE][CLUS] Include NC Local Counters in Startup Message

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- Include NC local counters in the NC startup tasks

completion completion message.

- Ensure no NC is marked as active without its local

counters reported.

- Remove the need for individual NC local counters

message.

- Clean up ITxnIdFactory and IResourceIdManager APIs.

Change-Id: Ief8b9d43783ea22810c6fdb29947a1284e32daee

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2767

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

    • -40
    • +0
    ./ReportLocalCountersRequestMessage.java
  1. … 15 more files in changeset.
[NO ISSUE][TXN] Fix txnid leak (off by one)

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Fix leak of one id at start of each allocated block

- Increase block size from 100 -> 512

Change-Id: Ia97d8472f30ef0f68c51830b4cbb4660184d7393

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2465

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 1 more file in changeset.
[NO ISSUE] Add support for NodeGroup upsert, etc.

Also, enable resource id generation when only pending removal nodes are absent

from the cluster

Change-Id: I15cfb74bc345680102cedafa99f7ff4f144860bc

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2389

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Integration-Tests: Murtadha Hubail <mhubail@apache.org>

Tested-by: Murtadha Hubail <mhubail@apache.org>

  1. … 12 more files in changeset.
[ASTERIXDB-2281][RT] Consider reserved txn ids when determining max

Change-Id: I88f14fb351976db239ed752693e59882da62d588

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2368

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

    • -0
    • +99
    ./TxnIdBlockRequest.java
    • -0
    • +52
    ./TxnIdBlockResponse.java
  1. … 18 more files in changeset.
[NO ISSUE] More multi-CC support, ConfigManager updates

- add ability for OptionTypes to natively parse JsonNodes

- allow all options to be overridden at the NC level, not just NC options (i.e. common, cc)

- accept controller id from the CC, avoid configuring this on NCs

- update all CCs with metadata bootstrap, not just the primary CC

- remove TxnIdFactory static singleton, management by metadata node

- remove unused build-properties style test configs

- cleanup test iodevices

Change-Id: Iff60887bf71ce3f3ed7201afd9499612bfc83485

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2344

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 67 more files in changeset.
[ASTERIXDB-2110] Introduce Cluster Controller Id

Change-Id: Iec1b01444bfbd923e38f5c162c5244e17c4d5f03

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2323

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -2
    • +4
    ./ReportLocalCountersRequestMessage.java
  1. … 79 more files in changeset.
[ASTERIXDB-2195][REPL] Replace Static Replication

- user model changes: no

- storage format changes: no

- interface changes: yes

- Redesigned all replication interfaces

Details:

- Replace static replication and fault tolerance by

dynamic storage API.

- Remove static based fault tolerance strategies.

- Redesign replication APIs and classes to smaller

maintainable parts.

- Clean up replication properties.

- Unify logic for checkpoints when replication is

enabled.

- Remove static replication test cases.

- Add replication runtime test cases for:

- Bulkload component replication.

- Memory component recovery.

- Flushed component replication.

- Add replication integration test for:

- Resync failed replica.

Change-Id: Ic5c4b0ac199a4530c807e558c8aebb1eb1284048

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2252

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 252 more files in changeset.
[ASTERIXDB-1564][CONF] Consistently Use Log4j2

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Replace java.util.logging by Log4j2.

- Excluded classes due to their tests:

- IoUtil

- Tracer

Change-Id: Ic137571292f45de1f1994c61d328b97185012197

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2226

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 390 more files in changeset.
[ASTERIXDB-1723] Delete managix

- user model changes: yes

- storage format changes: no

- interface changes: yes

Details:

- Delete managix installer method and all dependencies

- Remove xml-based configuration

Change-Id: I5c710b6ec870bedd49dd46b74ed96a3665670e9b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2155

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 1378 more files in changeset.
[NO ISSUE][TX] Ensure TxnIdFactory Value is Initialized

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Report local max txn id after node registration.

- Add node status BOOTING.

- Distinguish between node first time registration and

registration after restarting by using NodeStatus

BOOTING to respond with the proper node post

registration tasks.

- Rename node status ALIVE -> ACTIVE.

- Rename StartupTask* to RegistrationTasks*

Change-Id: I6899c9e7d6e744ca92d0108556e086a23639d78b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2151

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +74
    ./ReportLocalCountersMessage.java
    • -0
    • +38
    ./ReportLocalCountersRequestMessage.java
    • -38
    • +0
    ./ReportMaxResourceIdRequestMessage.java
  1. … 18 more files in changeset.
[NO ISSUE][HYR][*DB][CLUS] Startup lifecycle fixes

- Ensure thread factory is configured before using it

- Don't mark cluster state ACTIVE until after global recovery has

completed

- Failure of global recovery causes CC to shutdown

- Don't mark cluster state ACTIVE until max resource id has been

reported by all nodes

Change-Id: Id30415325047008c013e305ca11ccbb76bc7d8d8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2004

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 15 more files in changeset.
[NO ISSUE][*DB] Explicitly create ClusterStateManager

- user model changes: no

- storage format changes: no

- interface changes: no

details:

- Previously, we create the cluster state manager using the

singleton pattern.

- After this change, cluster state manager is created

as part of the cc application start.

- To access the cluster state manager after this change,

the cc application context is used.

Change-Id: Id6532245033ac4c6f6aa9f193539944eecb832f7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1944

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 40 more files in changeset.
Remove static cc application context instance

Change-Id: Ia2e250405967ec880e7af6387aa981f39b3392c0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1606

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

    • -5
    • +5
    ./ReportMaxResourceIdRequestMessage.java
    • -6
    • +4
    ./ResourceIdRequestResponseMessage.java
  1. … 177 more files in changeset.
Unify structure of Cluster Controller and Node Controller

Change-Id: Ife3a002371a07ee9cdd32e1ffd50cc775bf1d453

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1514

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -5
    • +2
    ./ResourceIdRequestResponseMessage.java
  1. … 157 more files in changeset.
Cleanup and bug fixes in Feeds pipeline

The bug fix is for the MessagingFrameTupleAppender. It used

to consume one extra byte per frame.

Cleanups are for feeds. These include:

1. Remove the use of feed dataflow marker. Feeds which

need to send progress can and should do that without

a marker thread.

2. Lock the memory component for feed commit to be able

to add information to the memory component's metadata

page safely.

In addition, this change introduces a frame level callback

for index operations.

Change-Id: Ie97b2133ebecb7380cf0ba336e60ed714d06f8ee

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1523

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 71 more files in changeset.
Introduce Strategy Based Replication and Fault-Tolerance

This change includes the following:

- Introduce new APIs for Replication and Fault-Tolerance Strategies.

- Add configuration in cluster description file for high-availability.

- Add built-in replication strategies (Metadata_Only, Chained_Declustering)

- Add built-in fault-tolerance strategies (Auto, Metadata_Node)

- Remove none cluster state functionality from ClusterStateManager.

- Add customizable NC startup sequence

Change-Id: I1d1012f5541ce786f127866efefb9f3db434fedd

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1405

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -92
    • +0
    ./CompleteFailbackRequestMessage.java
    • -54
    • +0
    ./CompleteFailbackResponseMessage.java
    • -119
    • +0
    ./PreparePartitionsFailbackRequestMessage.java
    • -50
    • +0
    ./PreparePartitionsFailbackResponseMessage.java
    • -69
    • +0
    ./TakeoverMetadataNodeRequestMessage.java
    • -48
    • +0
    ./TakeoverMetadataNodeResponseMessage.java
    • -107
    • +0
    ./TakeoverPartitionsRequestMessage.java
    • -60
    • +0
    ./TakeoverPartitionsResponseMessage.java
  1. … 102 more files in changeset.
Introduce IStorageComponentProvider

Change-Id: If86750cdb2436c713f6598e54d4aaaf23d9f7bbf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1451

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -1
    • +1
    ./CompleteFailbackResponseMessage.java
    • -1
    • +1
    ./PreparePartitionsFailbackResponseMessage.java
    • -1
    • +1
    ./TakeoverMetadataNodeResponseMessage.java
    • -1
    • +1
    ./TakeoverPartitionsResponseMessage.java
  1. … 422 more files in changeset.
ASTERIXDB-1711: remove more mentions of Asterix

Change-Id: I21666fd54075ac56de8aab76bb7d5cdbec020121

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1387

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Steven Jacobs <sjaco002@ucr.edu>

    • -3
    • +3
    ./PreparePartitionsFailbackRequestMessage.java
    • -3
    • +3
    ./ResourceIdRequestResponseMessage.java
    • -3
    • +3
    ./TakeoverMetadataNodeRequestMessage.java
    • -3
    • +3
    ./TakeoverPartitionsRequestMessage.java
  1. … 563 more files in changeset.
Cleanup FileSplit and FileReference

This change gives FileSplit and FileReference specific meaning to

avoid confusion of an absolute vs relative, local vs global, inside

an IO device vs outside IO devices.

In addition, it enables better abstraction of global partitions and

delegate the responsibility of choosing which partition goes to which

IO device to the IO Manager through the introduction of FileDeviceComputer

In details:

Previously, the LocalResource in Hyracks had partition (storage partition)

and there is no such thing in Hyracks. This scope leak is bad. In addition

The local resource had a name and a path. they were always the same and so

the name was removed.

The storage partition was instead moved to asterixdb implementation of the

serialized object in the local resource.

With all of these changes, the cluster controller (compiler) only needs to

know about partitions and relative paths. It doesn't need to worry about

heterogenous Node setups and different io device configurations. For File

assignment to IO devices, a new interface (IFileDeviceComputer) was

introduced which can be overriden by applications to have their own

strategy for distributing files among IO devices.

Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1352

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 287 more files in changeset.
Wait For Metadata Registration Before Active, Etc.

Also:

- metadata.port moved to [app]; honored

- += metadata.node to [app] to optionally specify metadata node

- += metadata.callback.port to [app]

- Decrease timeout for metadata registration from 7 days to default of

one minute, configurable with property

- Log swallowed exception in SynchronizableWork

- Add missing properties (metadata) to cluster state http api

- Make AsterixPropertiesAccessor, to ensure consistent values when

accessed in virtual cluster

Change-Id: I48d7c10b3e43181ec307f7d890ba721f61bc2ab0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1247

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -1
    • +1
    ./CompleteFailbackResponseMessage.java
    • -1
    • +1
    ./PreparePartitionsFailbackRequestMessage.java
    • -1
    • +1
    ./PreparePartitionsFailbackResponseMessage.java
    • -1
    • +1
    ./ReportMaxResourceIdRequestMessage.java
    • -1
    • +1
    ./TakeoverMetadataNodeRequestMessage.java
    • -1
    • +1
    ./TakeoverMetadataNodeResponseMessage.java
    • -1
    • +1
    ./TakeoverPartitionsRequestMessage.java
    • -1
    • +1
    ./TakeoverPartitionsResponseMessage.java
  1. … 32 more files in changeset.
Separate static cluster properties from runtime state

This change includes the following:

- Remove cluster static properties from AsterixClusterProperties.

- Add new ClusterProperties class for static cluster properties.

- Rename AsterixClusterProperties to ClusterStateManager.

- Refactor cluster static properties.

Change-Id: I3da72f639476f081ba8a0ef21ec176d32ff8bdec

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1183

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -2
    • +2
    ./CompleteFailbackResponseMessage.java
    • -2
    • +2
    ./PreparePartitionsFailbackResponseMessage.java
    • -2
    • +2
    ./TakeoverMetadataNodeResponseMessage.java
    • -2
    • +2
    ./TakeoverPartitionsResponseMessage.java
  1. … 34 more files in changeset.
Small Cleanup Towards Fixing LifeCycle Issues

Before this change, dataset lifecycle manager was providing a set

of functionalities that are loosly related to management of datasets

and indexes. However, it was not clear what the possible states of

a dataset or an index and what is the responsibility of each object.

This change takes the first step towards fixing this area. Indexes

of a dataset are now grouped together under a single lifecycle class

A resource aka dataset must be created outside the lifecycle manager

and registered with it before it can be assigned resources (memory)

and before it can be used by any operation. This is still not the

case.

Change-Id: I84005a33837725f41ae63297a3711215dccce1d8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1148

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -6
    • +2
    ./CompleteFailbackResponseMessage.java
    • -7
    • +3
    ./PreparePartitionsFailbackRequestMessage.java
    • -2
    • +2
    ./PreparePartitionsFailbackResponseMessage.java
    • -4
    • +4
    ./ReportMaxResourceIdRequestMessage.java
    • -7
    • +12
    ./ResourceIdRequestResponseMessage.java
    • -5
    • +5
    ./TakeoverMetadataNodeRequestMessage.java
    • -4
    • +4
    ./TakeoverMetadataNodeResponseMessage.java
    • -9
    • +5
    ./TakeoverPartitionsRequestMessage.java
    • -4
    • +4
    ./TakeoverPartitionsResponseMessage.java
  1. … 57 more files in changeset.
Refactor Messaging

In this refactoring, each message implementation includes

a handle method. This avoids bloating of message brokers

and enable better extensibility for messaging.

Change-Id: I7c918bf504058c98ecf89f5b019503278e9aa01f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1128

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -0
    • +41
    ./AbstractFailbackPlanMessage.java
    • -0
    • +96
    ./CompleteFailbackRequestMessage.java
    • -0
    • +58
    ./CompleteFailbackResponseMessage.java
    • -0
    • +206
    ./NodeFailbackPlan.java
    • -0
    • +123
    ./PreparePartitionsFailbackRequestMessage.java
    • -0
    • +50
    ./PreparePartitionsFailbackResponseMessage.java
    • -0
    • +72
    ./ReplicaEventMessage.java
    • -0
    • +76
    ./ReportMaxResourceIdMessage.java
    • -0
    • +38
    ./ReportMaxResourceIdRequestMessage.java
    • -0
    • +81
    ./ResourceIdRequestMessage.java
    • -0
    • +57
    ./ResourceIdRequestResponseMessage.java
    • -0
    • +69
    ./TakeoverMetadataNodeRequestMessage.java
    • -0
    • +48
    ./TakeoverMetadataNodeResponseMessage.java
    • -0
    • +111
    ./TakeoverPartitionsRequestMessage.java
    • -0
    • +60
    ./TakeoverPartitionsResponseMessage.java
  1. … 112 more files in changeset.