Clone
 

steven glenn jacobs <sjaco002@ucr.edu> in asterixdb

[ASTERIXDB-2402][COMP] Allow AND for IsMissing in Groupby

AccessMethodUtils.findLOJIsMissingFuncInGroupBy()

looks for the not(is-missing($VAR)) pattern

in a group by, but failed to detect it if it is part of

an and, e.g. and(not(is-missing($VAR1)),not(is-missing($VAR2)))

This changes fixes this and adds a test case

Change-Id: I9547fba5e4ba02226b5b2c2504080b091d3b8d5e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2735

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

[ASTERIXDB-2405][COMP] Allow IntroduceJoin to start at delegates

- user model changes: no

- storage format changes: no

- interface changes: no

details:

Previously, IntroduceJoinAccessMethodRule would only start at

the top of a plan (commit or distribute result)

Commit is the only delegate operator in master.

We have one more delegate operator in BAD, which is also the

top of the plan.

This change allows all delegates to be considered by

IntroduceJoinAccessMethodRule as a starting point.

Change-Id: I00172958df8c9ce6dc4cc6b405bc1d6665e2d514

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2740

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Xikui Wang <xkkwww@gmail.com>

[COMP][ASTERIXDB-2400] Fix UNION handling in InlineAllNtsInSubplanVisitor

Makes the visitor correctly allow substitution of the input ops

for the output op in future operators

Added test

Change-Id: I8b7e3e24749ed88e27d6cd38be5584ea748f39fc

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2703

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

[ASTERIXDB-2391][SQL] Insure that var for return clause gets optimized

Places the variable created for the return clause below the insert

so future optimizations can work on this variable

Added test

Change-Id: Id285435c4dc8a603c60b177dacd9f09722faac21

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2689

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

[ASTERIXDB-2373][HYR,CLUS] Allow upsert of JobSpecs for Deployed Jobs

- user model changes: none

- storage format changes: none

- interface changes: new methods added

This change adds the upsertDeployedJobSpec method, enabling

a Deployed Job to update its Job Specification.

Added call in test.

Removed DeployedJobService (moved methods to BAD codebase).

Change-Id: I01fd5d43896d520fe75e1007d7bd39324f6f6e4b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2619

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Xikui Wang <xkkwww@gmail.com>

[ASTERIXDB-2333][COMP] Allow filters to work alongside autogenerated keys

- user model changes: no

- storage format changes: no

- interface changes: no

Details: Fixes a bug preventing autogenerated keys from working

when there is a filtering field present.

Added test.

Change-Id: I450b4e20175992414955d59116e672450d442f81

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2504

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Xikui Wang <xkkwww@gmail.com>

[ASTERIXDB-2332][RT] Fix concurrency issue with RecordMerge and RecordRemoveFields

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Removed shared instance of PointableHelper.STRING_BINARY_COMPARATOR

which was not thread safe.

Change-Id: I4c5db58184235474f7aed7e5a4b91b6b5685fc06

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2492

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

[ASTERIXDB-2199][COMP] Fix PushFieldAccessRule for nested partitioning keys

Fixes an issue where nested partitioning keys

were ignored by PushFieldAccessRule

Added Test and fixed changed plans

Change-Id: I874c1fd15719b6bdeb7b0913fbafc04a58d32ed4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2246

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ildar Absalyamov <ildar.absalyamov@gmail.com>

  1. … 17 more files in changeset.
[ASTERIXDB-2180][FUN] Prevent dropping of entities used by functions

Add dependencies to Functional Metadata

Check dependencies before dropping datasets or functions

Add tests

Change-Id: I2f08ff150dfd57432b88381c507814ddb57bd67b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2253

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 15 more files in changeset.
[ASTERIXDB-2181][FUN] Check whether a function is usable at creation

Verify rewrite step on a function before allowing creation

Add negative tests for bad function declarations

Change-Id: I262b8cfd29117c18f452973fed147696b83f1249

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2242

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 33 more files in changeset.
[ASTERIXDB-2167][TX][RT] Remove TxnId from JobSpecification

- user model changes: no

- storage format changes: no

- interface changes: IJobEventListenerFactory

details:

- Remove the TxnId from the compiled job specification

- This enables one job spec to be used by multiple jobs/transactions

- Runtime operators who need the TxnId will pull it from the EventListener

Change-Id: I9526d50b31aebc3bf971d95ba3edf29c0c1066a7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2154

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 13 more files in changeset.
[ASTERIXDB-1911][HYR,RT,CLUS] Fixes and Improvements for Deployed Jobs

Rename Predistributed Jobs to Deployed Jobs

Enable job executions to have a map of job parameters

Add an Asterix function to retrieve these parameters

which are can be assigned when the job is run, e.g. for Deployed jobs

Allow Deployed jobs to have new TxnIds and JobIds for each execution

Allow simultaneous execution of one Deployed Job

Change-Id: I8f493c1fa977d07dfe8a875f9ebe9515d01d1473

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2045

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Xikui Wang <xkkwww@gmail.com>

  1. … 59 more files in changeset.
Put back function used by BAD project

Change-Id: I6ac0c71c7f0e1f121a7597d7f09dce8943ed943d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1749

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ildar Absalyamov <ildar.absalyamov@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Allow update statements in the tests to return adm

Allows subscribe statements to be tested in BAD

Change-Id: Ia39123a1e5c1ca5dde368f77f40be00b9c0d130d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1693

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Fix Pre-Distributed Jobs

This fix prevents ResultStateSweeper from deinitializing

pre-distributed jobs before they are dropped.

Also fixes issues with new lock manager code

Change-Id: Id50c52fbc7c891761dcabd654fb9b853b5f7a91d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1656

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

ASTERIXDB-1327, ASTERIXDB-1362 Fixed circle-point intersect function

Change-Id: I2512c73c9dcd593dc7e6690435da67f8086ff0db

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1558

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Jianfeng Jia <jianfeng.jia@gmail.com>

Enable Feed Changes to work with BAD project

Extracts the ActiveListener

Enables listeners to survive after job executions

Change-Id: Ib62184b67aff564475ef9b58790ff96409195b77

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1524

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

ASTERIXDB-1747 Implemented full lifecycle capabilities for pre-distributed jobs

Added distribute and destroy functionality

Removed serialization and bytes when running pred-distributed jobs

Cleaned up methods

Enabled Mockito testing for CCS and NCS

Added Unit Test for Distributed Jobs

Change-Id: I59c3422d5c1ab7756a6a4685ac527dfe50434954

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1377

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 27 more files in changeset.
Prepare AsterixDB for Pre-Distributed Jobs

Change-Id: Id809f4b563bbed808c7764d1af664a15919db35b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1366

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

ASTERIXDB-1608, ASTERIXDB-1617 Match user query for nonpure function calls

This fix makes it so that nonpure functions are called in

the same place and with the same number of executions

as specified by the user in the query. This also means

that indexes cannot be used for queries that compare

with a nonpure call that is made on a per-record basis.

Added optimizer tests

Change-Id: I2dec322b30835625430c06acd7626d902bada137

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1057

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 32 more files in changeset.
ASTERIXDB-1573 Allow Extension of Rewrite Rules

Add IRuleSetFactory which builds rewrite Rule Set

Can be extended by Asterix Extensions

Removed some obsolete extension code

Change-Id: I7f59dea86b0ef4ee9d31b56766a97bd2b34ec02c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1304

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 7 more files in changeset.
Enhanced Insert AQL

The optional "as Variable" provides a variable binding for the inserted records

The optional "returning Query" allows users to run simple

queries/functions on the records returned by the insert, and can refer

to the variable bound in "as Variable"

Allow commits to be non-sink operators (contnue job pipeline after commit)

Additionally, this change makes small modifications to

the extension code to prepare for the BAD extension

Also made the OptimizerTests able to work for Extensions

Change-Id: I65789d2a861d15232dd29156a6987d0635ec6c94

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1150

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 66 more files in changeset.
ASTERIXDB-1587 Test upsert when using foreign datatype

Change-Id: I4c374a44389af5916c8398afb3be8628b7872389

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1091

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

ASTERIXDB-1587 Made upsert look in the correct dataverse for datatypes

Change-Id: Iccf04c2210a1535cd7de6eeea88dfefec9ba206a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1085

Reviewed-by: Preston Carman <prestonc@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>