Clone Tools
  • last updated 27 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[NO ISSUE][NET] Log ChannelReadInterface Stats on Failure

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- When the read buffers of a FullFrameChannelReadInterface

are exceeded, log its stats to help in debugging the issue.

- Warn when a partial frame is being written over a full

frame write channel.

- Rename ThreadSafetyGuaranteedBy -> GuardedBy.

- Annotate networking calls with their synchronization

guards.

Change-Id: I89eed0c06dbf4b0e86747538bff286dc37853957

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/3663

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Al Hubail <mhubail@uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

    • -0
    • +5
    ./MessagingChannelInterfaceFactory.java
  1. … 12 more files in changeset.
[NO ISSUE][FAIL] Drop fractional seconds when reporting response timeout

Change-Id: Ia628b5607cdd69a65b4e08daf1c47ecf95b851bb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3527

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

[NO ISSUE] Apply / enforce java import order

The process-sources target will now sort imports as well as

format source code; the source-format job will likewise verify

import order in addition to source code format

Change-Id: I55d976c4df10d9919c6a25683be2a3e3304e65d9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3288

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

Reviewed-by: Till Westmann <tillw@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -1
    • +1
    ./MessagingChannelInterfaceFactory.java
  1. … 623 more files in changeset.
[NO ISSUE][OTH] Introduce Whispered Messages

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- Add whispered flag to IMessage API.

- Log whispered messages as trace rather than info level

to avoid filling the info log with high frequency messages.

Change-Id: I5c24205f5c25bf9f6aa245e8f1200b046390a915

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3220

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 3 more files in changeset.
[ASTERIXDB-2490][NET] Support Encrypted Multiplexed Connections

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- Use SocketChannelFactory in multiplex connections

to support both unencrypted and encrypted sockets.

- Adapt TCPEndpoint to socket channels that require

handshake.

- Adapt test cases to API changes.

Change-Id: I9cbed93c162018bad17923d50d4987011cbba16c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3071

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 17 more files in changeset.
[NO ISSUE][NET] Ensure CLOSE Is Not Sent After Channel ERROR

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Currently it is possible to send network channel

CLOSE command after a channel ERROR was sent. When this

happens and the channel was recycled to be reused

on the receiver side, the CLOSE command will result

in NPE. There is no need to send a CLOSE command

after an ERROR command because when an ERROR command

is received, it is treated as ERROR + CLOSE on the

receiver side.

- Avoid registering partition requests for failed jobs.

Change-Id: I17a769a46f4d13220adb22dd255e56dc4ccc458d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2954

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 3 more files in changeset.
[NO ISSUE][NET] Skip Channel Write on Connection Failure

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Do not attempt to adjust channel writability if

the failure was due to connection failure since

no more messages will be sent on that connection.

This is done to prevent a possible deadlock between

network IOThread that detected connection failure

and another thread that might be accessing the channel.

- Make sending error code conditions more explicit since

we currently have a single error code that is sent.

Change-Id: Ic25f05ac2c0d02699324f2d1b80c51f392654106

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2892

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 5 more files in changeset.
[ASTERIXDB-2204][STO] Fix implementations and usages of IIndexCursor

- user model changes: no

- storage format changes: no

- interface changes: yes

- IIndexCursor.close() is now idempotent and can be called on

a closed cursor.

- IIndexCursor.destroy() is now idempotent and can be called

on a destroyed cursor.

- Add IIndexAccessor.destroy() letting the accessor know it is

safe to destroy its reusable cursors and operation contexts.

- Add IIndexOperationContext.destroy() letting the context

know that the user is done with it and allow it to release

resources

details:

- Previously, implementations of the IIndexCursor interface

didn't enforce the interface contract. This change enforces

the contract for all the implementations.

- With the enforcement of the contract, all the users of the

cursors are expected to follow and enforce the expected lifecycle.

- Test cases were added.

Change-Id: I98a7a8b931eb24dbe11bf2bdc61b754ca28ebdf9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2324

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 132 more files in changeset.
[ASTERIXDB-2281][RT] Consider reserved txn ids when determining max

Change-Id: I88f14fb351976db239ed752693e59882da62d588

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2368

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

  1. … 21 more files in changeset.
[NO ISSUE] More multi-CC support, ConfigManager updates

- add ability for OptionTypes to natively parse JsonNodes

- allow all options to be overridden at the NC level, not just NC options (i.e. common, cc)

- accept controller id from the CC, avoid configuring this on NCs

- update all CCs with metadata bootstrap, not just the primary CC

- remove TxnIdFactory static singleton, management by metadata node

- remove unused build-properties style test configs

- cleanup test iodevices

Change-Id: Iff60887bf71ce3f3ed7201afd9499612bfc83485

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2344

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 67 more files in changeset.
[ASTERIXDB-2256] Reformat sources using code format template

Change-Id: I4faa141c1a8c9700d5e9ac50b839acc9d1eede73

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2310

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 984 more files in changeset.
[ASTERIXDB-2110] Introduce Cluster Controller Id

Change-Id: Iec1b01444bfbd923e38f5c162c5244e17c4d5f03

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2323

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 79 more files in changeset.
[ASTERIXDB-1564][CONF] Consistently Use Log4j2

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Replace java.util.logging by Log4j2.

- Excluded classes due to their tests:

- IoUtil

- Tracer

Change-Id: Ic137571292f45de1f1994c61d328b97185012197

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2226

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -7
    • +8
    ./MessagingChannelInterfaceFactory.java
  1. … 388 more files in changeset.
[ASTERIXDB-2019][CLUS] Update cluster state on partitions changes

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Set the cluster to UNUSABLE when no partitions are registered

- Update cluster state after partitions register/de-register

- Reject unregistered nodes queries on CC

- Avoid NPE when trying to send to a node that was de-registered

Change-Id: I7d11733a1dcd86136e157d80517bff4abcfc776b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1918

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 3 more files in changeset.
[ASTERIXDB-1950][ING][API] REST API for ActiveEntity stats

- user model changes: yes

Added ActiveEntity stats REST API.

- storage format changes: no

- interface changes: yes

Changed ActiveEntityEventsListener & EventSubscrber interfaces.

Added getStats method to IActiveRuntime.

Details:

1. Added HttpAPI for active feed stats.

2. Replaced FeedEventSubscriber with WaitForStateSubscriber.

3. Added StatsSubscriber for monitoring stats request.

4. Moved the message related methods from FeedEventsListener to

ActiveEntityEventsListener for possible reuses in other cases.

Change-Id: I46b48b52a1c9906510c5bdce778d1672845f75ca

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1748

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 41 more files in changeset.
Change DataflowHelperFactory not to require Task Context

Change-Id: I9dcd95dbefca131c4bbdb43306f00f6f8ea60800

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1758

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 19 more files in changeset.
Enable HTTP API processing on NCs

- Query/Status/Result are answered by NC nodes

- other HTTP requests are proxied to the CC node

- SessionConfig refactoring – split into config and output (SessionOutput)

- TestExecutor now can send http requests do multiple nodes (round robin)

Change-Id: I19414a23e163fc4deef9805c8f9089609f1ebe07

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1709

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 41 more files in changeset.
Remove static cc application context instance

Change-Id: Ia2e250405967ec880e7af6387aa981f39b3392c0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1606

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

    • -3
    • +3
    ./MessagingChannelInterfaceFactory.java
  1. … 179 more files in changeset.
Unify structure of Cluster Controller and Node Controller

Change-Id: Ife3a002371a07ee9cdd32e1ffd50cc775bf1d453

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1514

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 160 more files in changeset.
Cleanup and bug fixes in Feeds pipeline

The bug fix is for the MessagingFrameTupleAppender. It used

to consume one extra byte per frame.

Cleanups are for feeds. These include:

1. Remove the use of feed dataflow marker. Feeds which

need to send progress can and should do that without

a marker thread.

2. Lock the memory component for feed commit to be able

to add information to the memory component's metadata

page safely.

In addition, this change introduces a frame level callback

for index operations.

Change-Id: Ie97b2133ebecb7380cf0ba336e60ed714d06f8ee

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1523

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 72 more files in changeset.
Implements concurrent query management support.

The following changes are included:

-- factor out JobManager, NodeManager, and ResourceManager from ClusterControllerService;

-- let each application plugin its own IJobCapacityController implementation;

-- let each job specify its required cluster capacity;

-- add a required cluster capacity estimation visitor for optimized query plans;

-- add admission control and queuing for queries, but always executes DDLs and DMLs immediately;

-- add tests for JobManger, NodeManager, ClusterCapacity, ClusterCapacityVisitor, and IJobCapacityController;

-- enlarge the -Xmx setting for ManagixSqlppExecutionTest.

Change-Id: I8fb6fda57efa139114dd234e08cc7de7129468c8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1424

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 119 more files in changeset.
ASTERIXDB-1711: remove more mentions of Asterix

Change-Id: I21666fd54075ac56de8aab76bb7d5cdbec020121

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1387

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Steven Jacobs <sjaco002@ucr.edu>

  1. … 570 more files in changeset.
Small Cleanup Towards Fixing LifeCycle Issues

Before this change, dataset lifecycle manager was providing a set

of functionalities that are loosly related to management of datasets

and indexes. However, it was not clear what the possible states of

a dataset or an index and what is the responsibility of each object.

This change takes the first step towards fixing this area. Indexes

of a dataset are now grouped together under a single lifecycle class

A resource aka dataset must be created outside the lifecycle manager

and registered with it before it can be assigned resources (memory)

and before it can be used by any operation. This is still not the

case.

Change-Id: I84005a33837725f41ae63297a3711215dccce1d8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1148

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 69 more files in changeset.
Refactor Messaging

In this refactoring, each message implementation includes

a handle method. This avoids bloating of message brokers

and enable better extensibility for messaging.

Change-Id: I7c918bf504058c98ecf89f5b019503278e9aa01f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1128

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 125 more files in changeset.
Introduce MessagingNetworkManager for NC2NC AppMessaging

This change introduces MessagingNetworkManager to NodeControllerService.

The MessagingNetworkManager is used to open channels that are used for NC2NC

application messaging. The read and write interfaces of the messaging channels

are set by a MessagingChannelInterfaceFactory which is set by the application.

Change-Id: I5c0bd7c11c1e78954ebceff49cb274d8073a64bd

Reviewed-on: https://asterix-gerrit.ics.uci.edu/897

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

    • -0
    • +152
    ./MessagingChannelInterfaceFactory.java
    • -0
    • +64
    ./MessagingChannelReadInterface.java
    • -0
    • +72
    ./MessagingChannelWriteInterface.java
  1. … 71 more files in changeset.
Add Asterix Extension Manager

More extension support is added. A user can now provide implementations

for the IExtension interface which will give them more control over

the behavior of the system and give them the ability to add custom

features.

Initial customizations include:

1. Metadata Tuple Translators

2. Metadata Datasets

3. Query Translators

4. Statement Handlers

5. Lang Compilation Provider

Change-Id: I280268495cc3aad00f898cba21f7299f7120ce5c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1017

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 231 more files in changeset.
Refactor General Active Classes

This change is the first of a series of changes aiming to refactor

feed related classes into generic active and feed specific.

In this change, we introduce asterix-active which will be one level

below asterix-external-data. The ultimate goal is to have a very

flexible framewrok for long running activities within asterix.

Change-Id: I0a8f33ee5b45c5e090b08c24a102e369aae43c04

Reviewed-on: https://asterix-gerrit.ics.uci.edu/977

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 130 more files in changeset.
Asterix-1389 Fix Deadlocks in Feed Connections

This change ensures the completion of the Feed connect and feed

disconnect statement each as an atomic operation.

Previously, we assumed that with the intake ready on all nodes

and the connect started, the connect is complete. That is not

true. In order for the connect to be complete, we need to ensure

that the connect subscribe to the intake in all intake nodes.

Likewise, the disconnect shouldn't return until the connect

job terminates.

Change-Id: Ib2778b4d7f156c7e06ac9f561a26783c4933a22c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/792

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 11 more files in changeset.
Merge branch 'master' into hyracks-merge2

  1. … 484 more files in changeset.