Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[NO ISSUE][NET] Log ChannelReadInterface Stats on Failure

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- When the read buffers of a FullFrameChannelReadInterface

are exceeded, log its stats to help in debugging the issue.

- Warn when a partial frame is being written over a full

frame write channel.

- Rename ThreadSafetyGuaranteedBy -> GuardedBy.

- Annotate networking calls with their synchronization

guards.

Change-Id: I89eed0c06dbf4b0e86747538bff286dc37853957

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/3663

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Al Hubail <mhubail@uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

    • -0
    • +7
    ./comm/channels/ReadBufferFactory.java
  1. … 12 more files in changeset.
[ASTERIXDB-2479][API] Add Network Diagnostics API

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Add a new API that shows the state of node to node

connections and their logical channels to help diagnose

networking issues.

- Add channel details to the waiting thread name in

NetworkOutputChannel.

- Add test case.

Change-Id: Id6fd5a96c56e7078d1404bebcbab8afe93ba8f64

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3025

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Michael Blow <mblow@apache.org>

    • -2
    • +2
    ./comm/channels/NetworkOutputChannel.java
  1. … 14 more files in changeset.
Revert "[NO ISSUE][LOG] Include destination IP in thread name on push back"

This reverts commit 2498d28f0584d6ab002625417622be429f5c8131.

Change-Id: I221071a8a2851bde839e67d7f0380816c249366d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2941

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

    • -11
    • +1
    ./comm/channels/NetworkOutputChannel.java
  1. … 3 more files in changeset.
[NO ISSUE][LOG] Include destination IP in thread name on push back

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- In certain cases, we noticed that threads of certain jobs are

waiting on the network boundaries to send tuples.

- In that case and in a big cluster, it is difficult to tell which

node is pushing back.

- To make this easy, we include the IP of the receiving node

in the thread name.

Change-Id: I81b48cb7a79de0a37f2171a1bd3b82141f89b49d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2929

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

(cherry picked from commit 2498d28f0584d6ab002625417622be429f5c8131)

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2942

    • -1
    • +11
    ./comm/channels/NetworkOutputChannel.java
  1. … 3 more files in changeset.
[NO ISSUE][LOG] Include destination IP in thread name on push back

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- In certain cases, we noticed that threads of certain jobs are

waiting on the network boundaries to send tuples.

- In that case and in a big cluster, it is difficult to tell which

node is pushing back.

- To make this easy, we include the IP of the receiving node

in the thread name.

Change-Id: I81b48cb7a79de0a37f2171a1bd3b82141f89b49d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2929

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -1
    • +11
    ./comm/channels/NetworkOutputChannel.java
  1. … 3 more files in changeset.
[NO ISSUE][NET] Propagate Network Failure Error Codes

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Pass the network error code to the channel

to determine whether the error should be

sent to the receiver or not.

Change-Id: I29cffa916df9fecc9942f449ba6c5ca7c84b43b3

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2912

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

    • -2
    • +2
    ./comm/channels/NetworkOutputChannel.java
  1. … 4 more files in changeset.
[NO ISSUE][OTH] Log Level Fixes

- user model changes: no

- storage format changes: no

- interface changes: no

Change-Id: Ib5fb7e3140bf36cf4ba5857b9fda8e6cc4bdfa20

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2883

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 13 more files in changeset.
[NO ISSUE][OTH] Move Network Tracing Logs to Trace Level

- user model changes: no

- storage format changes: no

- interface changes: no

Change-Id: Iebc13ff6cb7b3c9086c19597ec982061672a891f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2848

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -2
    • +2
    ./comm/channels/NetworkInputChannel.java
  1. … 9 more files in changeset.
[ASTERIXDB-2314][HYR] Dataset in class names in Hyracks

- user model changes: no

- storage format changes: no

- interface changes: yes

Change-Id: I260add3a2dfb4a722440af5008a1a75f9446579e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2478

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

    • -154
    • +0
    ./comm/channels/DatasetNetworkInputChannel.java
    • -0
    • +154
    ./comm/channels/ResultNetworkInputChannel.java
  1. … 119 more files in changeset.
[NO ISSUE][RT] Abort tasks on local network failures

- user model changes: no

- storage format changes: no

- interface changes: yes

Add error code to IInputChannelMonitor.notifyFailure

Details:

- Previously, there was an assumption that all failures

reported to an IInputChannelMonitor come from a remote

task.

- This assumption is not always true and could lead

to jobs hanging.

- To fix this, we report an error code indicating whether

the failure is local or remote and if the failure is local

then we fail the local task and report the failure to cc.

Change-Id: I7ea5b9008383faaac7c563671242b03919090b35

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2806

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -1
    • +1
    ./comm/channels/NetworkInputChannel.java
    • -2
    • +2
    ./comm/channels/NetworkOutputChannel.java
  1. … 12 more files in changeset.
[NO ISSUE][NET] Networking Fixes

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Ensure received partitions requests after job failure

are aborted to prevent leaked network channels.

- Do not send channel close after channel write error

since the contract is to close the channel when remote

errors are received.

- Only remove closed outgoing connections to establish

new connections since incoming connections need to be

reestablished by the remote destination.

- Do not perform further operations on failed multiplexed

connections to avoid CanceledKeyException.

- Add test case for received partition requests after

job failure.

Change-Id: Idc45f47fdf0419bf75d461e16f028237a5143de7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2613

Reviewed-by: Michael Blow <mblow@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Michael Blow <mblow@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -4
    • +6
    ./comm/channels/NetworkOutputChannel.java
  1. … 11 more files in changeset.
[NO ISSUE] Incremental cleanup of deprecated exception ctors

Change-Id: I1e7c3655828fc6530cef83ea502a6cfbf41acddf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2533

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

    • -1
    • +1
    ./comm/channels/NetworkInputChannel.java
    • -1
    • +1
    ./comm/channels/NetworkOutputChannel.java
  1. … 238 more files in changeset.
[ASTERIXDB-2249][API] Add Max Result Reads to API

- user model changes: no

- storage format changes: no

- interface changes: yes

- IRequestParameters: add ResultProperties

- IDatasetPartitionManager: add maxReads

Details:

- Add option to specify max result reads and default

it to 1.

- Fix exception handling in DatasetPartitionReader.

- Add option to specify maxResultReads in tests.

- Use new option in async-repeated test.

- Add test case for exhausted result.

Change-Id: I86f75c791f034142c5b046445870bd91378c5b3a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2292

Reviewed-by: Michael Blow <mblow@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -1
    • +2
    ./comm/channels/NetworkOutputChannel.java
  1. … 37 more files in changeset.
[ASTERIXDB-1564][CONF] Consistently Use Log4j2

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Replace java.util.logging by Log4j2.

- Excluded classes due to their tests:

- IoUtil

- Tracer

Change-Id: Ic137571292f45de1f1994c61d328b97185012197

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2226

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -5
    • +5
    ./comm/channels/NetworkInputChannel.java
  1. … 389 more files in changeset.
Introduce MessagingNetworkManager for NC2NC AppMessaging

This change introduces MessagingNetworkManager to NodeControllerService.

The MessagingNetworkManager is used to open channels that are used for NC2NC

application messaging. The read and write interfaces of the messaging channels

are set by a MessagingChannelInterfaceFactory which is set by the application.

Change-Id: I5c0bd7c11c1e78954ebceff49cb274d8073a64bd

Reviewed-on: https://asterix-gerrit.ics.uci.edu/897

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

    • -5
    • +6
    ./comm/channels/NetworkInputChannel.java
    • -1
    • +1
    ./comm/channels/NetworkOutputChannel.java
    • -15
    • +7
    ./comm/channels/ReadBufferFactory.java
  1. … 70 more files in changeset.
Move Hyracks to subfolder

    • -0
    • +154
    ./comm/channels/DatasetNetworkInputChannel.java
    • -0
    • +28
    ./comm/channels/IChannelConnectionFactory.java
    • -0
    • +145
    ./comm/channels/NetworkInputChannel.java
    • -0
    • +127
    ./comm/channels/NetworkOutputChannel.java
    • -0
    • +55
    ./comm/channels/ReadBufferFactory.java
  1. … 4423 more files in changeset.