Clone Tools
  • last updated 20 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Merge commit '8930f0d' from stabilization-f69489

Change-Id: I64cf862bc5ebe19a24acbccf821f68556699222e

    • -1
    • +1
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.11.adm
    • -1
    • +1
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.12.adm
    • -1
    • +1
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.13.adm
    • -1
    • +1
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.14.adm
    • -1
    • +1
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.15.adm
    • -1
    • +1
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.16.adm
    • -1
    • +1
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.17.adm
    • -1
    • +1
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.6.adm
    • -1
    • +1
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.7.adm
  1. … 15 more files in changeset.
[NO ISSUE][ING][EXT] Handle filters on meta() fields for feeds

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

This fix is to handle filters on meta() fields where the feed name

is not supplied as an arguement to the meta() when connecting the feed.

Both of these statement should work:

CONNECT FEED ds_feed TO DATASET ds WHERE meta(ds_feed).id LIKE "%WEB%"

CONNECT FEED ds_feed TO DATASET ds WHERE meta().id LIKE "%WEB%"

Change-Id: I1dbf40525601584efa29eb3adf4ba41062ebe53a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3557

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -0
    • +9
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.11.adm
    • -0
    • +5
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.12.adm
    • -0
    • +5
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.13.adm
    • -0
    • +5
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.14.adm
    • -0
    • +4
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.15.adm
    • -0
    • +5
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.16.adm
    • -0
    • +9
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.17.adm
    • -0
    • +9
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.3.adm
    • -0
    • +5
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.4.adm
    • -0
    • +5
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.5.adm
    • -0
    • +5
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.6.adm
    • -0
    • +4
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.7.adm
    • -0
    • +5
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.8.adm
    • -0
    • +9
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.9.adm
  1. … 21 more files in changeset.
[NO ISSUE][ING][EXT] Include count in test cases for meta() filter

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

This patch is to modify the test cases for feed with filter

on meta() to include the count to check the number of tuples

correctly ingested.

Change-Id: Ib7c258e00b869facaeeaf2dc47d227c304eefaec

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3069

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

    • -9
    • +9
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.11.adm
    • -5
    • +5
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.12.adm
    • -5
    • +5
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.13.adm
    • -5
    • +5
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.14.adm
    • -4
    • +4
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.15.adm
    • -5
    • +5
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.16.adm
    • -9
    • +9
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.17.adm
    • -9
    • +9
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.3.adm
    • -5
    • +5
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.4.adm
    • -5
    • +5
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.5.adm
    • -5
    • +5
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.6.adm
    • -4
    • +4
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.7.adm
    • -5
    • +5
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.8.adm
    • -9
    • +9
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.9.adm
  1. … 14 more files in changeset.
[NO ISSUE][ING][EXT] Handle filters on meta() fields for feeds

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

This fix is to handle filters on meta() fields where the feed name

is not supplied as an arguement to the meta() when connecting the feed.

Both of these statement should work:

CONNECT FEED ds_feed TO DATASET ds WHERE meta(ds_feed).id LIKE "%WEB%"

CONNECT FEED ds_feed TO DATASET ds WHERE meta().id LIKE "%WEB%"

Change-Id: I42bf6c156809416f4386970dd115ce602656eb48

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3062

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -0
    • +9
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.11.adm
    • -0
    • +5
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.12.adm
    • -0
    • +5
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.13.adm
    • -0
    • +5
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.14.adm
    • -0
    • +4
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.15.adm
    • -0
    • +5
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.16.adm
    • -0
    • +9
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.17.adm
    • -0
    • +9
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.3.adm
    • -0
    • +5
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.4.adm
    • -0
    • +5
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.5.adm
    • -0
    • +5
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.6.adm
    • -0
    • +4
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.7.adm
    • -0
    • +5
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.8.adm
    • -0
    • +9
    ./change-feed-with-filter-on-meta/change-feed-with-filter-on-meta.9.adm
  1. … 21 more files in changeset.
[ASTERIXDB-2480][ING] Fix http-feed compatibility with JSON parser

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

The generic record used in current http-feed returns length of "-1"

which will cause the JSON parser to fail. For compatibility

consideration, we should use CharArrayRecord instead.

Change-Id: I719c98376583a3316dbc291696df65a97dd4b26e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3026

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Wail Alkowaileet <wael.y.k@gmail.com>

    • -0
    • +1
    ./http_feed_json/http_feed.1.adm
    • -0
    • +1
    ./http_feed_json/http_feed.2.adm
  1. … 8 more files in changeset.
[ASTERIXDB-2264][ING] Introduce Http Feed Adapter

- user-model changes: add http_adapter for feed.

- storage format changes: no

- interface changes: no

Details:

1. Added http feed. User may use following syntax to create a http feed:

create feed TweetFeed with {

"adapter-name" : "http_adapter",

"addresses" : "asterix_nc2:10002,asterix_nc1:10001",

"address-type" : "NC",

"type-name" : "TweetMessageType",

"format" : "adm"

};

2. Refactored insert-feed test case to avoid waiting for 10s.

3. Refactored some constants in feeds.

Change-Id: I3c197a3df557ecc01f07f0907688c4ea81379e40

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2994

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +1
    ./http_feed/http_feed.1.adm
    • -0
    • +1
    ./http_feed/http_feed.2.adm
    • -0
    • +1
    ./http_feed/http_feed.3.adm
  1. … 20 more files in changeset.
[NO ISSUE] User-defined Function Documentation update

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

1. Updated the UDF documentation to be consisten with current master.

2. Cleaned default UDF package to remove useless UDFs.

3. Added the example in documentation as a test case for IT.

4. Reorganized the documentation to keep up with the new structure.

5. Minor changes to other documentation pages to keep style consistent.

Change-Id: I17b1b4d639ca38689298ce88145257e794eb90e1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2804

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Taewoo Kim <wangsaeu@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -12
    • +21
    ./feed-with-external-function/feed-with-external-function.1.adm
  1. … 27 more files in changeset.
[ASTERIXDB-2429] Fix the upsert of primary key index

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Previously the primary key index is not properly maintained during

upsert. Since there is no secondary key in the primary key index, the

old value would always point to the primary key, which is always equal

to the new value. As a result, the primary key index is nevered

maintainined during upsert.

- This patch fixes this bug with two changes:

First, if there is a primary key index, we would perform upsert anyway

no matter whether old value == new value

Second, use a boolean variable to indicate whether the operation

is upsert or delete since for the primary key index, old value cannot

provide such information.

Change-Id: I925bd42ba67f70e94f5f5bc2d24151c8e2e20baf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2825

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -0
    • +1
    ./change-feed-with-meta-pk-index/change-feed-with-meta-pk-index.1.adm
  1. … 21 more files in changeset.
[NO ISSUE][TEST] Fix Sporadic failures in CancellationTests

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Multiple queries in a single request can cause an abrupt drop

of connection from the server if the first query was processed

successfully and then an error occurred.

Change-Id: I4df583e2afb0213361f420dc2490e1b5f3d8d835

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2606

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

    • -2
    • +0
    ./push-socket-with-auuid/push-socket-with-auuid.1.adm
    • -0
    • +1
    ./push-socket-with-auuid/push-socket-with-auuid.2.adm
    • -0
    • +1
    ./push-socket-with-auuid/push-socket-with-auuid.3.adm
  1. … 11 more files in changeset.
[ASTERIXDB-2319][TEST] Split Queries in start-feed Test

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Split queries to multiple files to avoid

job cancellation of the second query after

the result of the first query has been read.

Change-Id: I06978c83bca612a41337944c61027a1cf3849ac0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2491

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +1
    ./start-feed/start-feed.6.adm
    • -0
    • +1
    ./start-feed/start-feed.7.adm
  1. … 6 more files in changeset.
[ASTERIXDB-2227][ING] Enabling filitering incoming data in feed

- user model changes: yes

Add syntax support for specifying predicate in connect feed

- storage format changes: no

- interface changes: no

Details:

In some use cases, a user may want to filter the incombing data with

certain attributes. One example can be only store the incoming tweets

with geo locations. This patch enables the <WHERE> clause in connect

feed statement. User can subset the incoming data using following

syntax:

connect feed feeds.TweetFeed to dataset Tweets3 using policy `Basic`

WHERE id NOT LIKE 'nc1:10%' OR username = 'BronsonMike';

Change-Id: I0b3cc6fe9d7fb5f5645dd9c759da448bfe1e88f1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2255

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -0
    • +15
    ./feeds_13/feeds_13.1.adm
  1. … 13 more files in changeset.
[ASTERIXDB-2182][ING] Use JSON-like syntax for feed configuration

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

To be consistent with the other JSON-like syntax in defining

configurations, CREATE FEED will change to use JSON object instead of

key-value pairs. Also, the <USING> adapter_name syntax is removed. The

adapter name becomes as part of the configuration. After this change,

the feed configuration are specified as:

<CREATE> <FEED> testFeed <WITH> {

"adapter-name" : "localfs",

"path" : "localhost://../../../../../tweets.json",

"format" : "adm",

"type-name" : "typeTweet"

};

This patch also fixes two minor mistakes when drop feed and adapter in

Metadata.

Change-Id: I2dfb0fdf74d28ae0c45dafcf6babb624bd8b2149

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2209

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 107 more files in changeset.
[ASTERIXDB-1983] Feed pipeline refactoring for SQL++

- user model changes: no

- storage format changes: no

- interface changes: no

Current implementation of feed uses handcraft AQL queries for creating

feed pipeline. This causes a lot of issues and does not support SQL++

very well. Also, there is an overhead for parsing the query everytime.

In this patch, it's replaced with compiled statement in SQL++ which

provides support for attaching UDF to feed as well.

Details:

1. Remove SubscribeFeedStatement.

2. Remove SubscribeFeed related query compilation code, and reuse the

upsert dataflow.

3. Added SQL++ User Defined Function support for feed, including adding

multiple functions to one feed.

4. Related test cases added.

5. Change the default behavior of feed to be upsert instead of upsert.

'insert-feed' option is provided for experiment uses.

6. This patch also fixes several feed related bugs: [ASTERIXDB-2085]

[ASTERIXDB-2124].

Change-Id: I0ae5a837613780a4d2c90c98139fdc6d5e040cc9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2059

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -4
    • +0
    ./connect-feed-with-aql-function/connect-feed-with-aql-function.1.adm
    • -0
    • +4
    ./connect-feed-with-function/connect-feed-with-function.1.adm
    • -0
    • +6
    ./insert-feed/insert-feed.1.adm
  1. … 89 more files in changeset.
[ASTERIXDB-2114][MTD] Fixed incorrectly persisted FeedPolicy

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- New FeedPolicy was incorrectly persisting PolicyName & Description fields

Change-Id: I04beb24c861b116525a9da467fafc1742f4a73d4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2035

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Steven Jacobs <sjaco002@ucr.edu>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -1
    • +1
    ./create-policy-from-file/create-policy-from-file.1.adm
  1. … 1 more file in changeset.
[ASTERIXDB-1949][FUN][ING] Function usage check refactoring in feed

- user model changes: no

- storage format changes: no

- interface changes: yes

Removed several interefaces for maintaining reference count.

Added getFunctions and getFeeds method for MetadataNode.

Details:

1. Change the usage check from reference count to scan feed connections

in all dataverses.

2. Add test case for drop dataverse which has function that is being

used in other dataverse. This operation should be blocked.

3. Fix small bug in apply function metadata which the function dataverse

was not recorded properly.

Change-Id: I568f7af18e69216ff2a5fd3b623a1667809b2f95

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1787

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 25 more files in changeset.
Fix function usage check in feed

1. Add reference count field to Function entity.

2. Add reference count check when drop function.

3. Increate reference count when apply function to feed. Decrease the

reference count when disconnect feed.

4. Add test cases for drop function that is being used and drop function

that is no longer in use.

Change-Id: I04bed3836455997269a4cae7088dcc25c745a3df

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1763

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +0
    ./drop-function-used-by-feed/drop-function-used-by-feed.1.adm
  1. … 23 more files in changeset.
Add function signature check to Connect Feed

1. Revise the exception info when apply an unknown function to feed.

2. Fix the possible NPE in connect feed statement.

3. Add test case for applying undefined function.

Change-Id: I1462b394d84ea7e1eae5a03f98fe8cd39213eb8e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1674

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -0
    • +0
    ./feed-with-undefined-function/feed-with-external-function.1.adm
  1. … 8 more files in changeset.
Fix continue ingestion bug when exception happens

1. Fix the bug when exception happens, localfs adapter couldn't pick up

a new file to continue the ingestion.

2. Change the exception handling from string to error code.

3. Added a test case.

4. Removed some tailing . in error message.

Change-Id: Ie8656a4d1afabbc1b481eb97509a861b22478676

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1713

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -0
    • +1
    ./record-reader-with-malformed-input-stream/record-reader-with-malformed-input-stream.1.adm
  1. … 13 more files in changeset.
Feed Policy Refactoring

1. Fix the framesize inconsistency in feed dataflow.

2. Add a runtime test case for create feed with policy.

3. Fix the FeedRuntimeInputHandler flush() logic. Only flush after the

writer is opened.

4. Refactor FeedPolicyAccessor and BuiltinPolices. Now we only have

spill and discard polices.

5. Remove PolicyEnforcer. Merge the functionality into

FeedPolicyAccessor.

6. Revise SocketServerInputStream. Make the expected exception more

friendly.

7. Fixed one test case fail in change feed.

8. Refactor FeedRuntimeInputHandler consumer thread logic. Change the

poison consumer thread mechanism.

Change-Id: Ibc10139925cfedee66d1263990ba80b94675f182

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1591

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +10
    ./create-feed-with-policy/create-feed-with-policy.1.adm
  1. … 26 more files in changeset.
Fix record loss for certain certain feed type

1. Fix blindly replace connector between FeedCollector and

AssignOperator.

2. Wrap AssignOperator into the FeedMetaOperator to make sure the

operators inside (udf, accessor, etc.) can handle messages in the

feed workflow.

3. Revise feed connection job merge function.

4. Test case fix.

Change-Id: I4e1f7dd3621482a11feb675a93b826ae2cb965a7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1652

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -1
    • +1
    ./change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.4.adm
    • -1
    • +1
    ./change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.5.adm
  1. … 5 more files in changeset.
ASTERIXDB-1854 Fix Feed with user defined function

1. Fix the bug that AQL Function applied to feed can only $x as

parameter.

2. Fix Connect feed statement can only take one user defined function.

3. Fix one metadata bug when store multiple UDF in feed connection.

Change-Id: Ic60582b4198614a1f6a6026fe4c8675c9fec8a97

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1635

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Steven Jacobs <sjaco002@ucr.edu>

    • -0
    • +4
    ./connect-feed-with-aql-function/connect-feed-with-aql-function.1.adm
  1. … 14 more files in changeset.
ASTERIXDB-1310 Autogenerated uuid

Added test case for how to use autogenerated uuid with feed properly.

Change-Id: Ief4fba8a7227a40c5d22293758cbdf4575b14401

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1601

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Steven Jacobs <sjaco002@ucr.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +3
    ./push-socket-with-auuid/push-socket-with-auuid.1.adm
  1. … 10 more files in changeset.
Feed Connection Refactoring

1. The feed subscription network using FeedJoint is removed.

2. FeedConnection metadata dataset is added (pkeys: dataverseName,

feedName, datasetName).

3. Replaced the old intake job + collect job combination with one single

job using SplitOperator.

4. Now one feed can connect to multiple datasets.

5. The disconnect feed job is replaced by ActiveManagerMessage.

6. The new feed life cycle is:

- Create feed

- Connect feed to dataset0, dataset1, dataset2, etc.

- Start feed

- Stop feed

- Disconnect feed

7. New feedEventListner framework by Abdullah

Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1259

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -1
    • +1
    ./change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.4.adm
    • -1
    • +1
    ./change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.5.adm
    • -0
    • +3
    ./connect-feed/connect-feed.1.adm
    • -0
    • +2
    ./start-feed/start-feed.1.adm
  1. … 192 more files in changeset.
Fix for ASTERIXDB-1738: Change feed fails to delete record with meta PK

Change-Id: I5ce8d9c69f96593ee305cfad4b44e486c9f1d6cc

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1351

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +2
    ./change-feed-with-meta-csv/change-feed-with-meta-csv.3.adm
  1. … 8 more files in changeset.
Add upsert option for feed

For ASTERIXDB-1567. Provide "upsert-feed"="true" for

feed configuration, which changes the default record

insert to upsert. Added one test case for upsert feed.

Change-Id: Ic5133e7c6941fea4110cc9983f99502f364dc810

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1068

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

    • -0
    • +6
    ./upsert-feed/upsert-feed.1.adm
  1. … 15 more files in changeset.
ASTERIXDB-1135 Fix Feed Policy Creation From File

Change-Id: I452ac11714e20e2ecc31ab8b96a9964ca9d9d6cd

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1018

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -0
    • +1
    ./create-policy-from-file/create-policy-from-file.1.adm
    • -11
    • +0
    ./issue_711_feeds/issue_711_feeds.1.adm
  1. … 71 more files in changeset.
Cast Record Before Insert for Open Indexes

This change modifies the insert plan to perform cast for open

indexes before inserting to primary index. This avoids modification

of indexes and then doing complex work to try and undo the insert.

In addition, feeds can simply log and remove tuples which have

open fields of incompatible type.

Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914

Reviewed-on: https://asterix-gerrit.ics.uci.edu/997

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

    • -0
    • +1
    ./feed-with-external-parser-with-open-index/feed-with-external-parser-with-open-index.1.adm
    • -0
    • +1
    ./feed-with-external-parser-with-two-open-indexes/feed-with-external-parser-with-two-open-indexes.1.adm
  1. … 29 more files in changeset.
Update adm.grammar and printers for double/float

adm.grammar was not accepting suffixed doubles. This was breaking

a lot of round-trip ADM cases. I also changed the default print for

float and double to not have a suffix, because this makes the output

very odd (NaNd, NaNf, -Infinityf). The print mode for integers

has also been changed to omit the suffix, but, similarly the old

format is still accepted.

Change-Id: I6164d74f6c286fa160a30cff9ad47e97a30e0770

Reviewed-on: https://asterix-gerrit.ics.uci.edu/932

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

    • -1
    • +1
    ./change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.4.adm
    • -1
    • +1
    ./change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.5.adm
    • -5
    • +5
    ./change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.6.adm
    • -1
    • +1
    ./change-feed-with-meta-with-mixed-index/change-feed-with-meta-with-mixed-index.3.adm
    • -10
    • +10
    ./feed-push-socket/feed-push-socket.1.adm
    • -100
    • +100
    ./feed-with-external-parser/feed-with-external-parser.1.adm
    • -1
    • +1
    ./feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.4.adm
    • -1
    • +1
    ./feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.5.adm
    • -100
    • +100
    ./feed-with-multiple-indexes/feed-with-multiple-indexes.1.adm
  1. … 490 more files in changeset.
Fix Indexing on Open fields and Meta fields

This change fix the following cases:

1. Build a secondary index on a meta field: success

2. Build an open index on a meta field: failure

3. Build a secondary index on an open field in record part: success

Testing ingestion and querying are working correctly for these cases.

Change-Id: I6195149940f150250a65f2515e9ac9d6de2a33f9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/930

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -0
    • +1
    ./change-feed-with-meta-pk-in-meta-index-after-ingest/change-feed-with-meta-pk-in-meta-index-after-ingest.4.adm
    • -0
    • +1
    ./change-feed-with-meta-pk-in-meta-index-in-meta/change-feed-with-meta-pk-in-meta-index-in-meta.4.adm
    • -0
    • +1
    ./change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest/change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest.4.adm
    • -0
    • +1
    ./change-feed-with-meta-pk-in-meta-open-index-in-value/change-feed-with-meta-pk-in-meta-open-index-in-value.4.adm
    • -0
    • +1
    ./change-feed-with-meta-pk-in-meta-open-index-with-missing/change-feed-with-meta-pk-in-meta-open-index-with-missing.4.adm
    • -0
    • +1
    ./change-feed-with-meta-with-mixed-index/change-feed-with-meta-with-mixed-index.3.adm
  1. … 73 more files in changeset.
Improve External Data

This change enable an adapter to specify its data parser and not have

the user specify it and pass it as part of the adapter configurations.

In addition, it introduces a new parameter "parser-factory" that can be

used to specifies a parser factory instead of using parser to specify

a parser factory.

Change-Id: Iae2560c73fa63e9454f731b8e893ae779a2ac7d9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/906

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 34 more files in changeset.