Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Fix indefinite wait time for replication Job ACK

Change-Id: I88d2d61270522c766441e16fd996ac975935594b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1372

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -2
    • +11
    ./management/ReplicationManager.java
Cleanup FileSplit and FileReference

This change gives FileSplit and FileReference specific meaning to

avoid confusion of an absolute vs relative, local vs global, inside

an IO device vs outside IO devices.

In addition, it enables better abstraction of global partitions and

delegate the responsibility of choosing which partition goes to which

IO device to the IO Manager through the introduction of FileDeviceComputer

In details:

Previously, the LocalResource in Hyracks had partition (storage partition)

and there is no such thing in Hyracks. This scope leak is bad. In addition

The local resource had a name and a path. they were always the same and so

the name was removed.

The storage partition was instead moved to asterixdb implementation of the

serialized object in the local resource.

With all of these changes, the cluster controller (compiler) only needs to

know about partitions and relative paths. It doesn't need to worry about

heterogenous Node setups and different io device configurations. For File

assignment to IO devices, a new interface (IFileDeviceComputer) was

introduced which can be overriden by applications to have their own

strategy for distributing files among IO devices.

Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1352

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -1
    • +1
    ./storage/ReplicaResourcesManager.java
  1. … 287 more files in changeset.
Separate static cluster properties from runtime state

This change includes the following:

- Remove cluster static properties from AsterixClusterProperties.

- Add new ClusterProperties class for static cluster properties.

- Rename AsterixClusterProperties to ClusterStateManager.

- Refactor cluster static properties.

Change-Id: I3da72f639476f081ba8a0ef21ec176d32ff8bdec

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1183

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -3
    • +3
    ./storage/ReplicaResourcesManager.java
  1. … 36 more files in changeset.
Small Cleanup Towards Fixing LifeCycle Issues

Before this change, dataset lifecycle manager was providing a set

of functionalities that are loosly related to management of datasets

and indexes. However, it was not clear what the possible states of

a dataset or an index and what is the responsibility of each object.

This change takes the first step towards fixing this area. Indexes

of a dataset are now grouped together under a single lifecycle class

A resource aka dataset must be created outside the lifecycle manager

and registered with it before it can be assigned resources (memory)

and before it can be used by any operation. This is still not the

case.

Change-Id: I84005a33837725f41ae63297a3711215dccce1d8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1148

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 70 more files in changeset.
Refactor Messaging

In this refactoring, each message implementation includes

a handle method. This avoids bloating of message brokers

and enable better extensibility for messaging.

Change-Id: I7c918bf504058c98ecf89f5b019503278e9aa01f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1128

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -2
    • +2
    ./storage/ReplicaResourcesManager.java
  1. … 123 more files in changeset.
Misc Cleanup, SonarQube Fixes

Change-Id: If87126cdd435067a50087e339522a36021fbc2c0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1108

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 10 more files in changeset.
Add Test NodeController, Test Data Generator, and Marker Logs

This test enable creating a node controller for unit test purposes.

The Node controller is identical to the regular node controller

except that it doesn't communicate with a cluster controller at all.

In this change, Test Data Generator is introduced which should

facilitate writing unit test cases which requires data generation.

The change also includes enabling feeds to send progress data. progress

information can then be sent through the pipeline and persisted in the

transaction logs and primary index component. A Unit test case has

been created to test adding progress markers to logs and index

components and then reading them.

The last part of this change is the addition of marker logs and their

callbacks. They enable components to create arbitrary logs and get a

callback when they are written to the transaction logs. Initial set of

unit tests were added for marker logs.

Change-Id: I3b9aa8de758b7d26ca34868b16e5ce693e0c0243

Reviewed-on: https://asterix-gerrit.ics.uci.edu/962

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -12
    • +15
    ./management/ReplicationChannel.java
  1. … 82 more files in changeset.
Txn Log Replication Optimizations

- Add configurations for txn log replication.

- Establish handshake for continuous log replication.

- Replicate logs in batches instead of one by one.

- Eliminate HashMap iterator object creation per logs batch.

- Fix synchronization for jobs' ACKs from remote replicas.

- Bug fixes for big objects replication.

- Some SONAR fixes for old code.

Change-Id: I25b5b84eba0cd41ac8e87e71368072879fcf8582

Reviewed-on: https://asterix-gerrit.ics.uci.edu/883

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -54
    • +52
    ./logging/ReplicationLogBuffer.java
    • -107
    • +0
    ./logging/ReplicationLogFlusher.java
    • -0
    • +108
    ./logging/TxnLogReplicator.java
    • -103
    • +144
    ./management/ReplicationChannel.java
    • -158
    • +279
    ./management/ReplicationManager.java
    • -9
    • +12
    ./recovery/RemoteRecoveryManager.java
  1. … 10 more files in changeset.
Remove Log-Based Remote Recovery

This change removes the code that supports log-based remote recovery.

Remote recovery was replaced by the failback process which depends on

copying LSM disk component instead of logs.

Change-Id: I86e3b5832b52207e36c8409a072ccbda564d78b5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/871

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -113
    • +49
    ./functions/ReplicationProtocol.java
    • -101
    • +4
    ./management/ReplicationChannel.java
    • -88
    • +0
    ./management/ReplicationManager.java
    • -94
    • +0
    ./recovery/RemoteRecoveryManager.java
  1. … 6 more files in changeset.
ASTERIXDB-1436: Big Object Support For Storage

Preliminary submit for big object storage support (values only).

Enable hyracks frames up to Integer.MAX_VALUE

Change-Id: Ie0a3cb855768dfd0cd1da4a5fc9f62aedebcbc5f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/840

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -1
    • +17
    ./management/ReplicationManager.java
  1. … 96 more files in changeset.
Merge branch 'master' into hyracks-merge2

  1. … 484 more files in changeset.