asterix-om

Change-Id: Ib9d5ece656220d5f562cc385f882c5ddfd3283a6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/776

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

Change-Id: I3d02e697624865b955e5f1991e7ea509d1dc0d21

Reviewed-on: https://asterix-gerrit.ics.uci.edu/762

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

This change improves handling of two error types for filesystem

based feeds. The first one is the handling of IO Errors which

causes the input stream to be closed, and the second one is

reacting to missed filesystem events. In both cases, we scan the

directory and compare it with the history we have in order to

resume from where we last left off.

In addition, this change includes some refactoring in external

data. Particularly, we get rid of the stream provider layer and

instead, stream factories create input streams directly. This

is consistent with record reader factories which create readers

directly without reader providers.

Change-Id: I08d89229e33c91532b1038ba9f7a372f7ca1fdb5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/720

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

- Allow replication port assignment per NC.

- Allow replication to be enabled on virtual cluster.

- Wait for JOB_ABORT ACK from remote replicas.

- Fix LSM component mask file name.

- Fix index directory deletion on index drop.

- Eliminate multiple partition takeover requests.

- Free LogFlusher thread from sending replication ACKs.

- Fix possible deadlock between LogFlusher and Logs Replication Thread.

- Remove wait for FLUSH_LOG for replicated LSM components:

This wait is not needed since on node failure, complete remote recovery is done.

Change-Id: I34a38f59c4915a19242adb6a4eaa6ee1c82d2372

Reviewed-on: https://asterix-gerrit.ics.uci.edu/743

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Change-Id: I8fcf41c009f9faf51bc8bccc0c3c7217b7769149

Change-Id: I80e7995e814180fe567818263c79493901af284c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/703

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

This change allows feeds to perform upserts and deletes

in order to perform replication of an external data source.

The change does so by performing the following:

1. The adapter produces [PK][Record]. (Record == null --> delete)

2. The insert is replaced by an upsert operator.

Change-Id: If136a03d424970132dfb09f0dda56e160d4c0078

Reviewed-on: https://asterix-gerrit.ics.uci.edu/621

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

FeedMessageOperatorNodePushable

- CID 68721 (#1 of 1): Useless call (USELESS_CALL)

FeedLoadManager

- CID 68719 (#1 of 1): Useless call (USELESS_CALL)

RuntimeRecordTypeInfo

- CID 68717 (#1 of 1): Unused value (UNUSED_VALUE)

HelpCommand

- CID 68591 (#1 of 1): Missing break in switch (MISSING_BREAK)

- CID 68716 (#1 of 1): Unused value (UNUSED_VALUE)

SimilarityJaccardEvaluator

- CID 68714 (#1 of 1): Arguments in wrong order (SWAPPED_ARGUMENTS)

DatasetLifecycleManager

- CID 68713 (#1 of 1): Dereference before null check (REVERSE_INULL)

FeedMetaComputeNodePushable

- CID 68712 (#1 of 2): Dereference before null check (REVERSE_INULL)

- CID 68712 (#2 of 2): Dereference before null check (REVERSE_INULL)

FeedRuntimeInputHandler

- CID 68709 (#1 of 1): Dereference before null check (REVERSE_INULL)

DataflowControllerProvider

- CID 68708 (#1 of 1): Resource leak on an exceptional path

(RESOURCE_LEAK)

TokenLoad

- CID 68697 (#1 of 1): Resource leak (RESOURCE_LEAK)

RecordGroupLengthRange

- CID 68696 (#1 of 1): Resource leak (RESOURCE_LEAK)

RecordManagerGeneratorMojo

- CID 68691 (#1 of 1): Resource leak (RESOURCE_LEAK)

APIServlet

- CID 68687 (#1 of 1): Resource leak (RESOURCE_LEAK)

Change-Id: I8927126f505481e83dd5b9e6e5ccd75bb67d8188

Reviewed-on: https://asterix-gerrit.ics.uci.edu/715

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Change-Id: Ia21a29049364bd2f99bfd766efdd1518aeffe79f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/689

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Change-Id: Ie65417b6baf209ca0ab413cfa4a5f7fc5156ca63

Reviewed-on: https://asterix-gerrit.ics.uci.edu/685

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Change-Id: I3aadec904af64cc2b7b03e6dff399a2288c21a7b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/451

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Fixed the AqlExpressionToPlanTranslator for union expression;

Eliminate shared operator references in translated logical plans;

Removed IntroduceUnionRule;

Fixed tuple source for SQL++ join clause;

Added regression tests.

Change-Id: Ib79a899ec4b35ece6f8f9f59b80126fed0c48851

Reviewed-on: https://asterix-gerrit.ics.uci.edu/629

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Change-Id: I302f1547c5668250cf4801dbe26ae2aa6f455aa6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/647

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Change-Id: Ib1239b5e6c83b6b93da4f02d83492118bd0e97c2

Reviewed-on: https://asterix-gerrit.ics.uci.edu/652

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

This change removes all trailing whitespace in

java files in asterixdb

Change-Id: I32cda384f71dd5237a3c63e83c661f1cf64dc0eb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/646

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

- Allow Failed NCs to failback and takeover their partitions.

- New cluster API servlet for cluster state description.

- Remove nodeId from txn logs except remote FLUSH_LOG.

- Add partition id in UPDATE and Entity_COMMIT logs.

- Adapt remote recovery to new logs format.

- Refactor RecoveryManager and split Analysis and Redo phase.

- Spill remote recover logs to temporary file.

- Replicate files to remote replicas for partitions of interest only.

- Introduce NC active/inactive partitions concept.

- Test case for failback.

Change-Id: Id17819542d6b9c4e32647e64737c4a467b630f24

Reviewed-on: https://asterix-gerrit.ics.uci.edu/613

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Change-Id: Iee8e9b87145d9b3511874cd11d61ae87acdc94c7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/636

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

ExternalLookupOperator has been removed in this change

and UnnestMapOperator was used instead. different visitors

and methods in the UnnestMapOperator were changed to

support both external and internal data.

Change-Id: Ic9a9400586b75fd8cb3c0b8f4aeb45aee76cdb92

Reviewed-on: https://asterix-gerrit.ics.uci.edu/592

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Change-Id: If132857cf8d707ce96951f12ba6dd86f54aa4069

Reviewed-on: https://asterix-gerrit.ics.uci.edu/615

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

instead of AlgebricksException (which should be used at compile-time)

Change-Id: I39b3c8bbe3935255afffc49ce8847db66b4abeb1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/612

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

- Updated UUID implementation to eliminate marshalling to/from a pair

of longs, replacing with byte [].

- Eliminate all but one call to SecureRandom at init time, avoid risk

of entropy exhaustion

- Fix exceptions thrown

- Incorporate constant to represent number of bytes, String chars in

an instance of AUUID

- Split generated from parsed AUUID

- Eliminate intermediate StringBuilder construction on priting paths,

optimize imports

Change-Id: I9e90d42f6b62f80ad180dbd0c8c852db85b14173

Reviewed-on: https://asterix-gerrit.ics.uci.edu/610

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Reviewed-by: Till Westmann <tillw@apache.org>

Change-Id: I94eed0d97d9c3137b128a79b7e6e5e8ca00913a3

Reviewed-on: https://asterix-gerrit.ics.uci.edu/618

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>