Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
ASTERIXDB-1302 ASTERIXDB-1301 Fix Socket Feed Connection

A bug causes a read lock to never be released when a feed is

connected with "wait-for-completion" set to false. The bug

was fixed and a test case was added.

Another bug was causing the socket feed to not receive

connections correctly. The bug was fixed and a test case

was added.

Additionally, this change ensures that adapters have absolute

partitions to ensure consistency with regards to feed log

manager.

Change-Id: I8f6e982440d3577343f2479c3779653a9c3db614

Reviewed-on: https://asterix-gerrit.ics.uci.edu/660

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ildar Absalyamov <ildar.absalyamov@gmail.com>

    • -3
    • +3
    ./factory/LocalFSInputStreamProviderFactory.java
    • -2
    • +1
    ./factory/SocketInputStreamProviderFactory.java
    • -2
    • +1
    ./factory/TwitterFirehoseStreamProviderFactory.java
    • -1
    • +0
    ./provider/TwitterFirehoseInputStreamProvider.java
  1. … 44 more files in changeset.
Enabled Feed Tests and Added External Library tests

Feed tests had been switched off for a while due to having too many

sporadic failures. Now, we are switching them back on.

In addition, a new set of tests have been added to test that external

library works as expected.

Change-Id: Idd1fccd136fa2645b2707bbf7c04e60991ae8d4a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/625

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -2
    • +1
    ./factory/LocalFSInputStreamProviderFactory.java
    • -1
    • +2
    ./factory/TwitterFirehoseStreamProviderFactory.java
    • -0
    • +18
    ./provider/HDFSInputStreamProvider.java
    • -17
    • +17
    ./provider/LocalFSInputStreamProvider.java
    • -0
    • +10
    ./provider/SocketInputStreamProvider.java
    • -4
    • +28
    ./provider/TwitterFirehoseInputStreamProvider.java
  1. … 194 more files in changeset.
Remove Trailing Whitespace

This change removes all trailing whitespace in

java files in asterixdb

Change-Id: I32cda384f71dd5237a3c63e83c661f1cf64dc0eb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/646

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -1
    • +1
    ./factory/TwitterFirehoseStreamProviderFactory.java
    • -1
    • +1
    ./provider/HDFSInputStreamProvider.java
  1. … 325 more files in changeset.
Add flush() to IFrameWriter

This method is expected to be used with feeds to push

frames all the way to storage when needed. As of now, it is

needed in two cases:

1. No activities in ingestion node and need to push content

so it can be stored.

2. When the ingestion node needs to move the checkpoint ahead

if the at least once semantics are used.

Two feeds make use of this function. The filesystem feed and

couchbase feed which was introduced as well in this change.

Change-Id: Id862ce9e9b1360864c6976f2aea2137092f51203

Reviewed-on: https://asterix-gerrit.ics.uci.edu/585

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

    • -183
    • +0
    ./TwitterFirehoseInputStreamProvider.java
    • -1
    • +1
    ./factory/LocalFSInputStreamProviderFactory.java
    • -1
    • +1
    ./factory/SocketInputStreamProviderFactory.java
    • -0
    • +119
    ./provider/HDFSInputStreamProvider.java
    • -0
    • +64
    ./provider/LocalFSInputStreamProvider.java
    • -0
    • +38
    ./provider/SocketInputStreamProvider.java
    • -0
    • +184
    ./provider/TwitterFirehoseInputStreamProvider.java
  1. … 91 more files in changeset.
Feed Fixes and Cleanup

1. Introduce filesystem feed data source.

2. Fix the order of closing feed stages on disconnection.

3. Added Twitter feed to the compatibility utility to allow

using it with its alias.

4. First part of the feed log space.

5. Fixed the handling of duplicate key exception.

Change-Id: I4e8db26a810efd1fbaa52ceeb3efd0c8328ab070

Reviewed-on: https://asterix-gerrit.ics.uci.edu/574

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

    • -18
    • +24
    ./LocalFSInputStreamProvider.java
    • -0
    • +125
    ./LocalFileSystemInputStream.java
    • -1
    • +1
    ./TwitterFirehoseInputStreamProvider.java
    • -13
    • +28
    ./factory/LocalFSInputStreamProviderFactory.java
  1. … 412 more files in changeset.
First stage of external data cleanup

In this change, different parts of external data were refactored.

The goal was to make it more modular, easier to maintain and allow

higher flexibility for extension in addition to reducing code redundancy.

Change-Id: I04a8c4e494d8d1363992b6fe0bdbe6b2b3b7b767

Reviewed-on: https://asterix-gerrit.ics.uci.edu/566

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

    • -0
    • +34
    ./AInputStreamReader.java
    • -0
    • +86
    ./BasicInputStream.java
    • -0
    • +118
    ./HDFSInputStreamProvider.java
    • -0
    • +56
    ./LocalFSInputStreamProvider.java
    • -0
    • +101
    ./SocketInputStream.java
    • -0
    • +36
    ./SocketInputStreamProvider.java
    • -0
    • +183
    ./TwitterFirehoseInputStreamProvider.java
    • -0
    • +139
    ./factory/LocalFSInputStreamProviderFactory.java
    • -0
    • +130
    ./factory/SocketInputStreamProviderFactory.java
    • -0
    • +96
    ./factory/TwitterFirehoseStreamProviderFactory.java
  1. … 333 more files in changeset.