main

Ensure that replication identifiers comparisons are not influenced by

address resolution

Change-Id: I6d7aa87777d17a3416d16cfad23b2773b1614a02

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3536

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Issue warnings for all hints that were not processed

  by the SQL++ parser. A hint is not processed because

  it is either unknown or unexpected at given location

Change-Id: Iec46335a8db4360e5431b145cd9e98c7075c7fda

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3517

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Change-Id: I139c265ba998c32ec049c8a8bbd7a5a213895d2a

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Added compile/runtime warnings for bitwise functions.

- Added a new error code to represent values out of acceptable

range.

Change-Id: I9ada529d91d8fdc881447e8b4cf17fc8e71aa87f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3504

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Hussain Towaileb <hussainht@gmail.com>

Reviewed-by: Michael Blow <mblow@apache.org>

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Do not delete corrupted checkpoint files until a valid

checkpoint is persisted. This ensure a forged checkpoint

that will force recovery to start from the begining is

always used until a valid checkpoint is found.

- Attempt to read the latest checkpoint file right after

writing it and before attempting cleaning up invalid and

old checkpoint files.

- Use on disk files to determine next checkpoint id to

account for existing corrupted checkpoints.

- Maintain two older checkpoint files in addition to the

latest one.

- Catch all exceptions on checkpointing failures since it is

a try operation.

- Add test scenairo for the new checkpoints clean up behavior.

Change-Id: Iea689f5a644351491d9748273bb2158e8179f54d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3496

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Till Westmann <tillw@apache.org>

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Task, added getter for NodeControllerService.

- RecordWithMetadataAndPK, added getter for metaTypes.

- Added extra error codes.

Change-Id: Ic3fcc6b771b4393542988cd0aa6fd992ccb81028

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3423

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Hussain Towaileb <hussainht@gmail.com>

Reviewed-by: Michael Blow <mblow@apache.org>

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Add IWarningCollector accessor to IEvaluatorContext

- Provide compiler's warning collector to function evaluators

during constant folding, so they can emit warnings

Change-Id: I61dd53adfbcad2f8479333579185258876f074e9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3484

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Add support for compiler warnings during parsing,

AST rewriting and query optimization phases

- SQL++ parser emits warnings for unknown hints in

relational expression, between expression,

groupby clause and function call.

- Optimizer emits warning if hash group by hint cannot

be applied because there's no serializable aggregate

- Added testcases for these new warnings

Change-Id: Ic3c1e98c183cd214eea3e4fee24b2b7c46366b52

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3476

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Change-Id: I18115329ce7ab3501e7fcc9c6dc06d0e87c97688

- POST existing UDF format to /admin/udf/$DATAVERSE/$LIBNAME

- DELETE against that URL to remove UDFs

Change-Id: I6be9fef54c010bdb32f5c78af9b973f9843f442f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3386

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- Before synchronizing replicas, stop datasets checkpointing to

prevent new files from being generated due to async IO operations

triggered by checkpointing.

- Instead of sync'ing current files to replicas then scheduling a flush

and sync'ing any newly generated files, just flush datasets before

the initial sync then sync all the files in one go.

Change-Id: I058fd48bc0fb89a1e16448ce516c3410bb4d681d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3469

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Till Westmann <tillw@apache.org>

Change-Id: I01e1810f1c44d9ecaf569dd22577c33e8702c682

- user model changes: yes

- storage format changes: no

- interface changes: yes

Details:

- Add the ability to add runtime warnings per task

and return the generated warnings as part of the

task profile on task completion.

- On successful job completion, aggregate warnings

from all task profiles of a job.

- Return the generated warnings in the query service

response as an array of "warnings" each with a code,

which is currently hard-coded to 1, and a message.

- Fix propagating source location to scalar aggregate

functions.

- Add a flag in test cases definition to indicate whether

or not to check for expected warnings and default it to

false.

- Generate warnings when min/max functions encounter

incomparable type or unsupported input.

- Add support in test framework to extract warnings

along with extracting the result field.

- Add support in test framework to validate generated

and expected warnings.

- Add test cases for min/max generated warnings.

Change-Id: I52fa5b807799487d62e67a8861068e1547aa629a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3451

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- Add API in DatasetLifecycleManager to wait for IO on datasets

matching a replication strategy.

- Before synchronizing the files on a replica, wait for any

on-going IO operations on replicated datasets to make sure

the set of files won't change while the replica is being

synchronized (e.g. a merge operation won't delete a file

that we wanted to send to a replica).

Change-Id: I01ed5c9379cf7ae249faeef624d5226ea699cf22

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3446

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Add an API that returns the on disk total size of a dataset

and its indexes on a node.

Change-Id: Iaff87bbe1f2417689f7827deaf03fcddd64ca7e4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3425

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Michael Blow <mblow@apache.org>

- user model changes: no

- storage format changes: no

- interface changes: no

Change-Id: I87d75f7264c1eecf178300a1b0a67b351cce0216

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3431

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Support IGNORE NULLS / RESPECT NULLS modifiers in window

  functions LEAD(), LAG(), FIRST_VALUE(), LAST_VALUE(), NTH_VALUE()

  (RESPECT NULLS is the default)

- Add testcases for each function

- Fix CollectionMemberResultType and PushAggregateIntoNestedSubplanRule

  to handle remaining non-list arguments

Change-Id: I400c9c95bac9159efa17fca4e97047fd089eb931

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3407

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- Introduce ResponsePrinter that is responsible for printing

the returned response from query service.

- Make ResponsePrinter responsible for adding separators between

fields in the returned response.

- Introduce IResponseFieldPrinter and encapsulate each response

field in its own printer.

- Introduce ICodedMessage and use it for any object that

has code<->message (e.g errors and warnings).

Change-Id: I797e6615a72c886391fed26281fc648b38fa748f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3410

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Remove the codegen related files.

- Remove the codegen plugin.

- Remove the codegen plugin dependency from pom files.

Change-Id: I2c17f9f3f0c73f2ec3048a39da3cbbdd5f24e816

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3396

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Hussain Towaileb <hussainht@gmail.com>

Allow extensions to mandate that a rebalance is required in order for

the cluster to go active

Change-Id: I863f8bf3fe1ce8d59522c9a28a1283006ffa414c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3394

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Refactor how memory requirements are maintained in a query plan

- Introduce LocalMemoryRequirements class. Its instances are held

by each physical operator and could be altered by the optimizer.

- Introduce optimizer rule SetMemoryRequirementsRule that

initializes and configures memory requirements for each operator

Change-Id: I3481ddfe163c6ce786290c540cbd05db16a7f64f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3374

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Create NCExtensionManager in NCApplication instead of

NCAppRuntimeContext (aligned with how CCApplication creates

CCExtensionManager)

- Pass extension manager to application context via constructor

instead of setter method (for both CC and NC contexts)

- Move getExtensionManager() from ICcApplicationContext

to IApplicationContext, so it's available in NC application context

- Add compilation provider to NC application context

(NCExtensionManager.getCompilationProvider())

- Add compilation provider to NCQueryServiceServlet constructor

Change-Id: Ibb4dc6d8dbc00c8a701fddb1a3205d1070078cb4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3378

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Use proper exception type for cast exception.

- Move cast exception message to error properties file.

- Improve cast exception message.

- Update test cases to new error message.

Change-Id: I07c6c3c344b930ecbfaf6111a89c62b5d13868d9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3382

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

- user model changes: different error message

- storage format changes: no

- interface changes: no

Change-Id: I5bbcb63f266685befc80de957687249191345063

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3381

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Remove usage of DEBUG flag in GlobalConfig.

- Remove unused methods/constants in GlobalConfig.

- Move storage constants from GlobalConfig to StorageConstants.

Change-Id: Ie59a646d7d2eedaf1360c97c355f827d80917adf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3376

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Introduced data generator function to generate data with accordance

to the TPC Benchmark DS specifications.

- Data generator function accepts 2 arguments, table name and scaling

factor, this makes the same function reusable for all the desired

tables and scaling factors to be generated.

- Added test cases for the data generator functions.

Change-Id: Idb6bd6f697628395c70008e6f730bc5ca403da5e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3360

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- This annotation is for functions that respect the missing/null in ->

missing/null out behavior.

- Removed all the hardcoded functions inside the NullMissingTest and

ExceptionIT test classes.

- Functions annotated with the MissingNullInOutFunction are

automatically included in the NullMissingTest and ExceptionIT tests.

- Renamed ExceptionIT class to NullMissingExceptionTest.

- Moved NullMissingTest and NullMissingExceptionTest to their

appropriate packages to read the Logger config properly.

Change-Id: I253373ae4c8e23d9f5ed38f5fbbdefe9680e04b4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3361

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Introduce ConfigValidator that can be used to validate

runtime changes to configurations.

- On storage compression scheme configuration changes, ensure

new value is a registered scheme.

Change-Id: I7e9dcde9c93feb2c9e2bdfe790f121b751d4c8ad

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3359

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Change-Id: Ie6bb64e331666ef44b07034d430ba2ee563bba82

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3340

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

- user model changes: no

- storage format changes: yes (backwards-compatible)

- interface changes: no

Details:

- add marker on inintial IndexCheckpoint, propagate on subsequent checkpoints

- ignore extra JSON fields when deserializing IndexCheckpoint

Change-Id: Id93ab9d16887b37cf6c0d011950e7c57f1a1d646

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3324

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>