Clone Tools
  • last updated 12 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Control Number of Readers for LocalFS Data

Change-Id: Ib9d5ece656220d5f562cc385f882c5ddfd3283a6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/776

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 24 more files in changeset.
ASTERIXDB-1360 Fix Error Message for Unknown Format

This change includes some refactoring for external

data. This refactoring makes the code less error prone

and separate data source selection from data parser

selection. It also fixes issue ASTERIXDB-1366 and adds

a test case for it as well.

Change-Id: Ib4aac833e30bd7c5a7706f5c8116383c2362c964

Reviewed-on: https://asterix-gerrit.ics.uci.edu/767

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 33 more files in changeset.
Move AsterixDB to Subfolder

    • -110
    • +0
    ./external/CentralFeedManager.java
    • -762
    • +0
    ./external/ExternalIndexingOperations.java
    • -402
    • +0
    ./external/ExternalLibraryUtils.java
    • -780
    • +0
    ./external/FeedJobNotificationHandler.java
    • -510
    • +0
    ./external/FeedLifecycleListener.java
    • -96
    • +0
    ./external/FeedMessageReceiver.java
    • -187
    • +0
    ./external/FeedTrackingManager.java
    • -206
    • +0
    ./external/FeedWorkCollection.java
    • -269
    • +0
    ./external/FeedWorkRequestResponseHandler.java
  1. … 44146 more files in changeset.
Improve Error Handling in Local Directory Feeds

This change improves handling of two error types for filesystem

based feeds. The first one is the handling of IO Errors which

causes the input stream to be closed, and the second one is

reacting to missed filesystem events. In both cases, we scan the

directory and compare it with the history we have in order to

resume from where we last left off.

In addition, this change includes some refactoring in external

data. Particularly, we get rid of the stream provider layer and

instead, stream factories create input streams directly. This

is consistent with record reader factories which create readers

directly without reader providers.

Change-Id: I08d89229e33c91532b1038ba9f7a372f7ca1fdb5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/720

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 144 more files in changeset.
Fixed Feed Connect Statement

This change includes two fixes:

1. Feed connect doesn't return until the connection is complete.

2. When using wait for completion, it waits until all the jobs

complete.

Change-Id: I416bf4917b1f5cea687d1202c435f7183136cf1f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/726

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

    • -58
    • +96
    ./external/FeedJobNotificationHandler.java
    • -12
    • +23
    ./external/FeedLifecycleListener.java
  1. … 38 more files in changeset.
Support Change Feeds and Ingestion of Records with MetaData

This change allows feeds to perform upserts and deletes

in order to perform replication of an external data source.

The change does so by performing the following:

1. The adapter produces [PK][Record]. (Record == null --> delete)

2. The insert is replaced by an upsert operator.

Change-Id: If136a03d424970132dfb09f0dda56e160d4c0078

Reviewed-on: https://asterix-gerrit.ics.uci.edu/621

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -3
    • +3
    ./external/ExternalIndexingOperations.java
  1. … 270 more files in changeset.
Coverity Fixes

FeedMessageOperatorNodePushable

- CID 68721 (#1 of 1): Useless call (USELESS_CALL)

FeedLoadManager

- CID 68719 (#1 of 1): Useless call (USELESS_CALL)

RuntimeRecordTypeInfo

- CID 68717 (#1 of 1): Unused value (UNUSED_VALUE)

HelpCommand

- CID 68591 (#1 of 1): Missing break in switch (MISSING_BREAK)

- CID 68716 (#1 of 1): Unused value (UNUSED_VALUE)

SimilarityJaccardEvaluator

- CID 68714 (#1 of 1): Arguments in wrong order (SWAPPED_ARGUMENTS)

DatasetLifecycleManager

- CID 68713 (#1 of 1): Dereference before null check (REVERSE_INULL)

FeedMetaComputeNodePushable

- CID 68712 (#1 of 2): Dereference before null check (REVERSE_INULL)

- CID 68712 (#2 of 2): Dereference before null check (REVERSE_INULL)

FeedRuntimeInputHandler

- CID 68709 (#1 of 1): Dereference before null check (REVERSE_INULL)

DataflowControllerProvider

- CID 68708 (#1 of 1): Resource leak on an exceptional path

(RESOURCE_LEAK)

TokenLoad

- CID 68697 (#1 of 1): Resource leak (RESOURCE_LEAK)

RecordGroupLengthRange

- CID 68696 (#1 of 1): Resource leak (RESOURCE_LEAK)

RecordManagerGeneratorMojo

- CID 68691 (#1 of 1): Resource leak (RESOURCE_LEAK)

APIServlet

- CID 68687 (#1 of 1): Resource leak (RESOURCE_LEAK)

Change-Id: I8927126f505481e83dd5b9e6e5ccd75bb67d8188

Reviewed-on: https://asterix-gerrit.ics.uci.edu/715

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 12 more files in changeset.
ASTERIXDB-1302 ASTERIXDB-1301 Fix Socket Feed Connection

A bug causes a read lock to never be released when a feed is

connected with "wait-for-completion" set to false. The bug

was fixed and a test case was added.

Another bug was causing the socket feed to not receive

connections correctly. The bug was fixed and a test case

was added.

Additionally, this change ensures that adapters have absolute

partitions to ensure consistency with regards to feed log

manager.

Change-Id: I8f6e982440d3577343f2479c3779653a9c3db614

Reviewed-on: https://asterix-gerrit.ics.uci.edu/660

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ildar Absalyamov <ildar.absalyamov@gmail.com>

  1. … 49 more files in changeset.
Enabled Feed Tests and Added External Library tests

Feed tests had been switched off for a while due to having too many

sporadic failures. Now, we are switching them back on.

In addition, a new set of tests have been added to test that external

library works as expected.

Change-Id: Idd1fccd136fa2645b2707bbf7c04e60991ae8d4a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/625

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -0
    • +110
    ./external/CentralFeedManager.java
    • -0
    • +762
    ./external/ExternalIndexingOperations.java
    • -0
    • +402
    ./external/ExternalLibraryUtils.java
    • -0
    • +742
    ./external/FeedJobNotificationHandler.java
    • -0
    • +190
    ./external/FeedJoint.java
    • -0
    • +499
    ./external/FeedLifecycleListener.java
    • -0
    • +301
    ./external/FeedLoadManager.java
    • -0
    • +96
    ./external/FeedMessageReceiver.java
    • -0
    • +254
    ./external/FeedOperations.java
    • -0
    • +187
    ./external/FeedTrackingManager.java
    • -0
    • +206
    ./external/FeedWorkCollection.java
    • -0
    • +269
    ./external/FeedWorkRequestResponseHandler.java
    • -0
    • +118
    ./external/FeedsActivator.java
  1. … 192 more files in changeset.