Clone Tools
  • last updated 24 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[NO ISSUE] Support multipart dataverse names

- user model changes: yes

- storage format changes: no

- interface changes: yes

Details:

- Support dataverse names that consist of multiple parts

- Introduce DataverseName class and adopt it throughout the system

- Add testcases for multipart dataverse names

Change-Id: I1313b2dc0e8df6a9b0ded48dea122afc3ba5dbe7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4004

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 284 more files in changeset.
[NO ISSUE][OTH] Remove Unnecessary Null Checks

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Remove unnecessary null checks before instanceof.

Change-Id: I870df76623f5f983a90b5437abf7e3993a54c744

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2895

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 15 more files in changeset.
[NO ISSUE][OTH] Remove Unused Imports

- user model changes: no

- storage format changes: no

- interface changes: no

Change-Id: Iafff39073d0fedaff74a26ef7e3260008a79ff0c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2893

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

  1. … 67 more files in changeset.
[ASTERIXDB-1992][ING] Suspend/Resume active entities

- user model changes: wait for completion only returns

when the entity becomes inactive.

- storage format changes: no

- interface changes:

01) Introduce:

IMetadataLockManager for entity locking.

IActiveNotificationHandler for handling active events.

IRetryPolicy for recovery of failed active jobs.

IActiveEntityController for controlling active entities.

02) IJobLifecycleListener.notifyJobFinish now passes the

JobStatus and Exceptions.

03) IActiveEntityEventsListener.isActive() returns true,

if entity is active, false otherwise.

04) IActiveEntityEventsListener.unregister() removes the

listener upon entity deletion.

05) IActiveEntityEventsListener.getJobFailure() returns

the job failure if the entity is in a failed state.

06) IStatementExecutor.getComponentProvider() returns

the storage component provider.

07) IStatementExecutor.getApplicationContext() returns

the application context.

08) IMetadataManager.upsertEntity to perfrom metadata

entities' upsert operations.

09) IMetadataNode.upsertEntity to perfrom metadata

entities' upsert operations

10) ICcApplicationContext.getMetadataLockManager() returns

the lock manger.

11) ICcApplicationContext.getClusterStateManager() returns

the cluster state manager.

details:

- Starting and stopping of active entities now go through

their listeners rather than having the listener indicating

active entity and not having it indicating inactive entity.

This facilitates suspend/resume operations of long running

active jobs for the sake of DDL operations or topology

changes.

- Unit tests for the vast majority of code paths and

different possible scenarios have been added.

Change-Id: Ifeac8c73e6bad39a13663b84a52121356e3c6b40

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1875

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

    • -248
    • +0
    ./ActiveEntityEventsListener.java
  1. … 140 more files in changeset.
[ASTERIXDB-1950][ING][API] Add IT for ActiveEntity stats

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

1. Added one IT for feed stats request API.

2. Minor tweak to the API to produce proper JSON.

Change-Id: Iecbe00bcd93dcfd4ef9b800ce5719e85ae2cfd42

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1862

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 13 more files in changeset.
[ASTERIXDB-1970][ING] Fix Active Stats Test

- user model changes: no

- storage format changes: no

- interface changes: no

details:

- Active Stats Test fails because it issues many async

calls and doesn't wait until the calls complete. In

addition, it bypass the active event inbox incorrectly.

Change-Id: I518a6b1f7d8e86703ee5537869d207e609a7c293

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1865

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Xikui Wang <xkkwww@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 6 more files in changeset.
[ASTERIXDB-1950][ING][API] REST API for ActiveEntity stats

- user model changes: yes

Added ActiveEntity stats REST API.

- storage format changes: no

- interface changes: yes

Changed ActiveEntityEventsListener & EventSubscrber interfaces.

Added getStats method to IActiveRuntime.

Details:

1. Added HttpAPI for active feed stats.

2. Replaced FeedEventSubscriber with WaitForStateSubscriber.

3. Added StatsSubscriber for monitoring stats request.

4. Moved the message related methods from FeedEventsListener to

ActiveEntityEventsListener for possible reuses in other cases.

Change-Id: I46b48b52a1c9906510c5bdce778d1672845f75ca

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1748

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -3
    • +193
    ./ActiveEntityEventsListener.java
  1. … 40 more files in changeset.
Remove static cc application context instance

Change-Id: Ia2e250405967ec880e7af6387aa981f39b3392c0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1606

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 181 more files in changeset.
Enable Feed Changes to work with BAD project

Extracts the ActiveListener

Enables listeners to survive after job executions

Change-Id: Ib62184b67aff564475ef9b58790ff96409195b77

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1524

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -0
    • +55
    ./ActiveEntityEventsListener.java
  1. … 1 more file in changeset.
Feed Connection Refactoring

1. The feed subscription network using FeedJoint is removed.

2. FeedConnection metadata dataset is added (pkeys: dataverseName,

feedName, datasetName).

3. Replaced the old intake job + collect job combination with one single

job using SplitOperator.

4. Now one feed can connect to multiple datasets.

5. The disconnect feed job is replaced by ActiveManagerMessage.

6. The new feed life cycle is:

- Create feed

- Connect feed to dataset0, dataset1, dataset2, etc.

- Start feed

- Stop feed

- Disconnect feed

7. New feedEventListner framework by Abdullah

Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1259

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -69
    • +0
    ./ActiveLifecycleEventSubscriber.java
  1. … 191 more files in changeset.
Introduce IStorageComponentProvider

Change-Id: If86750cdb2436c713f6598e54d4aaaf23d9f7bbf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1451

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 427 more files in changeset.
Runtime ErrorCode fix in external data

1. Refactored most of the exceptions in external-data package to use

ErrorCode.

2. This change only covers RuntimeDataException/HyracksDataExcetion.

3. Refactor errorcode names

Change-Id: Ida810a56bf4aef1394879f088a6a5e8f82c60b74

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1374

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 96 more files in changeset.
ASTERIXDB-1711: remove more mentions of Asterix

Change-Id: I21666fd54075ac56de8aab76bb7d5cdbec020121

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1387

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Steven Jacobs <sjaco002@ucr.edu>

  1. … 570 more files in changeset.
Enhanced Insert AQL

The optional "as Variable" provides a variable binding for the inserted records

The optional "returning Query" allows users to run simple

queries/functions on the records returned by the insert, and can refer

to the variable bound in "as Variable"

Allow commits to be non-sink operators (contnue job pipeline after commit)

Additionally, this change makes small modifications to

the extension code to prepare for the BAD extension

Also made the OptimizerTests able to work for Extensions

Change-Id: I65789d2a861d15232dd29156a6987d0635ec6c94

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1150

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +67
    ./ActiveLifecycleEventSubscriber.java
    • -66
    • +0
    ./FeedLifecycleEventSubscriber.java
  1. … 78 more files in changeset.
Refactor Messaging

In this refactoring, each message implementation includes

a handle method. This avoids bloating of message brokers

and enable better extensibility for messaging.

Change-Id: I7c918bf504058c98ecf89f5b019503278e9aa01f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1128

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 126 more files in changeset.
Refactor Active Listeners

Change-Id: I260c8608329523f56dc54780d87d796f838505cf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1118

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 25 more files in changeset.
Add Asterix Extension Manager

More extension support is added. A user can now provide implementations

for the IExtension interface which will give them more control over

the behavior of the system and give them the ability to add custom

features.

Initial customizations include:

1. Metadata Tuple Translators

2. Metadata Datasets

3. Query Translators

4. Statement Handlers

5. Lang Compilation Provider

Change-Id: I280268495cc3aad00f898cba21f7299f7120ce5c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1017

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 232 more files in changeset.
Refactor General Active Classes

This change is the first of a series of changes aiming to refactor

feed related classes into generic active and feed specific.

In this change, we introduce asterix-active which will be one level

below asterix-external-data. The ultimate goal is to have a very

flexible framewrok for long running activities within asterix.

Change-Id: I0a8f33ee5b45c5e090b08c24a102e369aae43c04

Reviewed-on: https://asterix-gerrit.ics.uci.edu/977

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -0
    • +644
    ./FeedEventsListener.java
  1. … 121 more files in changeset.
Add Unit Tests for Feed Runtime Input Handler

Change-Id: I7088f489a7d53dee8cf6cdbf5baa7cd8d3884f55

Reviewed-on: https://asterix-gerrit.ics.uci.edu/866

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

  1. … 16 more files in changeset.
Cleanup Feed CodeBase

In order to expedite cleaning feeds up and reaching a maintainable

state, we disabled the following:

1. Policies (At least once, throttling, discarding, elasticity).

2. Statistics Reporting.

3. Load management.

4. Feed re-activation upon System reboot.

Right now on master, none of these features work reliably.

We will re-introduce them one feature at a time.

The rules followed in this change:

1. Keep X if X is tested in a test case.

2. Remove X if X is not used in test cases.

After a few meetings with Mike and Till, the policies

1. Buffering

2. Spill

3. Discard

4. Throttle

have been fixed and unit tests have been added.

Change-Id: I545bc4f8560564e4c868a80d27c77a4edd97a8b8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/798

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -0
    • +204
    ./ConcurrentFramePool.java
  1. … 135 more files in changeset.