Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Support Sending Messages Alongside Frame Data

This change supports sending messages with records. The tuple Appender

reserves 100 bytes for a message. Before sending the frame, it appends

The message in the last tuple position. The message is read from the

task context as the shared object between different operators in the

pipeline. The first use of this feature will be within feeds to request

acks for at least once semantics.

Change-Id: Iaa23e9f8a909ddcafc1c3ee95181092eb04ee1ad

Reviewed-on: https://asterix-gerrit.ics.uci.edu/605

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 12 more files in changeset.
Add flush() to IFrameWriter

This method is expected to be used with feeds to push

frames all the way to storage when needed. As of now, it is

needed in two cases:

1. No activities in ingestion node and need to push content

so it can be stored.

2. When the ingestion node needs to move the checkpoint ahead

if the at least once semantics are used.

Two feeds make use of this function. The filesystem feed and

couchbase feed which was introduced as well in this change.

Change-Id: Id862ce9e9b1360864c6976f2aea2137092f51203

Reviewed-on: https://asterix-gerrit.ics.uci.edu/585

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

    • -0
    • +6
    ./dataflow/CollectTransformFeedFrameWriter.java
    • -0
    • +5
    ./dataflow/DistributeFeedFrameWriter.java
    • -1
    • +9
    ./dataflow/FeedRuntimeInputHandler.java
    • -3
    • +1
    ./watch/StorageSideMonitoredBuffer.java
  1. … 94 more files in changeset.
remove end-of-line whitespace

Change-Id: I5c0415f47d4c3a9827574fbdab949b45718d9ea4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/601

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -1
    • +1
    ./dataflow/FeedCollectRuntimeInputHandler.java
    • -2
    • +2
    ./dataflow/FeedFrameTupleAccessor.java
    • -1
    • +1
    ./management/FeedConnectionRequest.java
  1. … 126 more files in changeset.
Feed Fixes and Cleanup

1. Introduce filesystem feed data source.

2. Fix the order of closing feed stages on disconnection.

3. Added Twitter feed to the compatibility utility to allow

using it with its alias.

4. First part of the feed log space.

5. Fixed the handling of duplicate key exception.

Change-Id: I4e8db26a810efd1fbaa52ceeb3efd0c8328ab070

Reviewed-on: https://asterix-gerrit.ics.uci.edu/574

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

    • -0
    • +34
    ./api/ICentralFeedManager.java
    • -0
    • +43
    ./api/IExceptionHandler.java
    • -0
    • +75
    ./api/IFeedConnectionManager.java
    • -0
    • +39
    ./api/IFeedFrameHandler.java
    • -0
    • +121
    ./api/IFeedJoint.java
    • -0
    • +36
    ./api/IFeedLifecycleEventSubscriber.java
    • -0
    • +28
    ./api/IFeedLifecycleIntakeEventSubscriber.java
    • -0
    • +56
    ./api/IFeedLifecycleListener.java
    • -0
    • +60
    ./api/IFeedLoadManager.java
    • -0
    • +72
    ./api/IFeedManager.java
    • -0
    • +58
    ./api/IFeedMemoryComponent.java
    • -0
    • +58
    ./api/IFeedMemoryManager.java
    • -0
    • +52
    ./api/IFeedMessage.java
    • -0
    • +34
    ./api/IFeedMessageService.java
  1. … 403 more files in changeset.