java

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

UNION should not be allowed when getting a comparator by tag.

The comparator provider returns a generic comparator with types

ANY when UNION is passed as a tag. This could cause problems if

the actual type is a complex type. UNION should not be allowed.

Change-Id: Id8816a0dc5584f0a27410c512f3a44ccfc6c3151

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3404

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Implement support for ARRAY_AGG() aggregate function.

- Add new testcases and update documentation

- Add AbstractScalarDistinctAggregateDescriptor.createDescriptorFactory()

to uniformly set required type inferer for all distinct scalar aggregates

- Propagate correct item type to GenericScalarDistinctAggregateFunction

Change-Id: I704e031a1252493e83ad8d45c38b75e0b15c1896

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3390

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Variables introduced by LET clauses should not be

considered by "Single Variable Resolution" rule

- In INSERT/UPSERT statements with RETURNING expression

the automatically introduced variable should be named

as the target dataset

- Refactor VariableCheckAndRewriteVisitor and

other code related to name resolution

- Added testcases and updated documentation

Change-Id: I9b9f70a1671378f5216ec4e30416d4d67c3df089

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3322

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

The process-sources target will now sort imports as well as

format source code; the source-format job will likewise verify

import order in addition to source code format

Change-Id: I55d976c4df10d9919c6a25683be2a3e3304e65d9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3288

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

Reviewed-by: Till Westmann <tillw@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Support aliases for SQL aggregate functions

- Add function aliases: stddev() -> stddev_samp(),

variance() -> var_samp(), variance_samp() -> var_samp(),

variance_pop() -> var_pop()

- stddev_pop() and var_pop() must return 0 if input is a single item

Change-Id: I0fe04164164726cfb4e17f951f0fd6c2a823c162

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3278

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

- user model changes: parse-only request parameter has been added

- storage format changes: no

- interface changes: no

Details:

- Support for new request parameter named parse-only;

Returns as result an object with single key-value pair.

The key is statement-parameters and the is a sorted array

containing positional and named free parameters.

- Added test cases

Change-Id: Idd2f461c22b05a5fcaa50a6e4f9b7dcd91acc184

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3085

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Till Westmann <tillw@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Implement aggregate window functions:

agg_func() OVER (frame_var AS)? (PARTITION BY ... ORDER BY ... frame_spec)

- Where agg_func is a SQL/SQL++ aggregate function

- Fix percent_rank() to always return 0 for the first tuple

- Fix ntile() to handle NULL argument

- Log query after each rewrite rule in SqlppQueryRewriter

- Implement toString() for ADayTimeDuration, fix it for AYearMonthDuration

- Add seek() method to RunFileReader

Change-Id: If0f71118a04c2dbd3462070673d52e67f076b7e1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3049

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Added slice parser syntax. [list][start:end], the syntax accepts

end expression as an optional argument, the expression can be

written as [list][start:].

- Added slice parser test cases.

Change-Id: Ie83283bfd0a04257b59b573de3dab6b3e47de1bf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3046

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

- user model changes: yes

- Add new configuration in the with-caluse to enable compression

- Add new nc configuration in the config file

- storage format changes: yes

- Pages of the primary index can be compressed

- Add a companion file (Look Aside File) with the compressed index

- Allow optional values in the LocalResource

- Add compression information in Metadata.Dataset

- interface changes: yes

- ICCApplicationContext:

- Add getCompressionManager()

- IBufferCache:

- Add getCompressedFileWriter(int fileId)

- ICachedPageInternal:

- Add setCompressedPageOffset(long offset)

- Add getCompressedPageOffset()

- Add setCompressedPageSize(int size)

- Add getCompressedPageSize()

Details:

- Add new integration test for this patch

- Fix ASTERIXDB-2464

- Add ddl-with-clause type validator

Additional details in the design document:

https://cwiki.apache.org/confluence/display/ASTERIXDB/Compression+in+AsterixDB

Change-Id: Idde6f37c810c30c7f1a5ee8bcbc1e3e5f4410031

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2857

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Implement window functions with SQL syntax:

function() OVER ((PARTITION BY expr1, expr2, ...)? ORDER BY exprA, exprB, ...)

- Where supported functions are:

ROW_NUMBER(), RANK(), DENSE_RANK(), PERCENT_RANK(), NTILE()

Change-Id: Ia28af8773cb11049c38d440c51b9c3cd1ed2bab4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3002

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

- user model changes: yes

- storage format changes: no

- interface changes: yes

details:

- new plan for sort operation which includes sampling and

replicating the stream of data to be sorted. Sort-merge connector

is removed from the plan. The sorted result now is in multiple partitions.

- new optimization rule to check whether full parallel sort is applicable.

- new Forward operator to read the replicated sort input stream and

to receive the ouput of the sampling.

- new sequential merge connector to merge a globally ordered result residing

in multiple partitions (in addition to the connector's partition computer).

- "asterix-lang-aql/pom.xml" is changed as a result of refactoring

code related to the range map handling.

- new private sampling function to generate the range map object

(local & global functions) & their type computers.

user model changes:

- new compiler property is added to enable and disable parallel sort.

interface changes:

- "ILogicalOperatorVisitor.java" includes Forward Operator.

- "ITuplePartitionComputer.java" includes initialize() to enable partitioner

to do some initialization. FieldRangePartitionComputerFactory uses it to

pick a range map.

- "ITuplePartitionComputerFactory.java". createPartitioner() is changed to

createPartitioner(IHyracksTaskContext hyracksTaskContext). Context is needed

for transferring the range map throught the context.

Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2393

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Add coalesce() function which is an alias to if_missing_or_null()

Change-Id: Ic462bc08173f2f2349cc208c687d432a219d62b9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2923

Reviewed-by: Till Westmann <tillw@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Introduce LET clauses for column alias expressions

instead of inlining them into ORDER BY/LIMIT clauses

- Fix incorrect visitor used in SubstituteGroupbyExpressionWithVariableVisitor

for rewriting LET clauses after GROUP BY

Change-Id: If925cbb803f2ad5fe955ba343ddc62585589b894

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2905

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Contrib: Till Westmann <tillw@apache.org>

- user model changes: no

- storage format changes: no

- interface changes: no

Change-Id: Iafff39073d0fedaff74a26ef7e3260008a79ff0c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2893

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- In many cases, we would like to associate a value with a key

in MetadataProvider to be accessed during the compilation

of jobs. However, currently, there is no place to store

such values, so we ended up storing them in the config

map.

- The config map is a <String, String> map and so, we would

write our values as a string and then parse them when needed.

- To avoid this, and to avoid introducing a new map, we simply

change the config stored in MetadataProvider from <String,String>

to <String, Object>.

Change-Id: I55b392ad199d74b0f3cffdc38b54593b12ec1a06

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2795

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Integration-Tests: Murtadha Hubail <mhubail@apache.org>

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Introduce utils to convert pointables into adm nodes.

- Such functions can be useful for tuple translation of

Metadata entities, reading constant records in with

clauses in DDLs, and for debugging runtime.

Change-Id: I705e75d573a7f2c888dd7d83b4c24a2c7050eb88

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2733

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

The hints are not copied over when Copy or Clone OperatorExpr.

Change-Id: I5eef37e525b7d1bec842e3488d8b63f88ddc05ea

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2722

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

- user model changes: yes

- storage format changes: no

- interface changes: yes

Details:

- Support statement parameters: named ($name) and positional ($1 or ?)

- Enhance query service API to accept these parameters in the request

- Remove [?] index accessor from SQL++ grammar because it conflicts

with positional parameters ([0] can be used instead)

- Add testcases for parameterized queries

Change-Id: Ia612f731cd2370fccd54c4796bd9787fbea16766

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2707

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

- user model changes: no

- storage format changes: no

- interface changes: yes

- IRetryPolicy.retry now takes a Throwable parameter

Details:

- This change improves the retry policy for active

entities by providing the failure causing the last

failure.

- The change also removes the lock over the active

notification handler on the recover call.

Change-Id: I4246e2a276e1f80569a07630e182aab8f49dd115

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2665

Reviewed-by: Michael Blow <mblow@apache.org>

Contrib: Michael Blow <mblow@apache.org>

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Currently, before we do certain language rewritings, we take a snapshot

of the var counter in the context and pass it to the rewriter. This

works fine in most of the cases, but it can fail when the rewriter

introduces new variables. This will cause different variables map to the

same variable id. Although this error may be fixed by another

ScopingVisitor, the compiler could fail if there is no more visitor down

in the rewriting workflow.

Change-Id: I8ea1ee7de140d6556af03f07a5ec7bc278b011ba

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2673

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Minor cleanup in source location handling

- Created error codes for unknown node group / function / type / index

Change-Id: Ia5e0b67c49f66dd27c67832707f5003892453fc3

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2676

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Change-Id: Id7a5a6cd252867e8ee69fd2bd6157ff64668256f

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Add aliases for type functions:

is_atom(), is_bool(), is_num(), is_str(), is_obj(),

to_atom(), to_bool(), to_num(), to_str(), to_obj()

Change-Id: I83508d3a439cd7b707c0ce9f43d7b7482aa162f0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2670

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Add source locations to compiler and runtime error messages

- Add source location enforcement to the test framework

and enable it for SqlppExecutionTest

Change-Id: Ie279ce345d1edcb5dea8e55cdb0233151c7bfd66

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2659

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Make "/" operator return double if both operands are integers

- Add "DIV" operator which is the same as "/" except that

it returns integer if both operands are integers

(as "/" did before this change)

- Add "MOD" operator which is an alias for "%" operator

- Remove "IDIV" operator from the grammar

Change-Id: I7c6b0704ce60a03dd3c10e1c75cb9761acc56536

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2630

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Implement the following conditional functions:

null_if(), missing_if(), nan_if(), posinf_if(), neginf_if()

Change-Id: I5399cae1a225b1881d67b87541e46a9d4fe158ec

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2616

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Change-Id: Iff6678efff502b566020896320ecdf170ade720a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2576

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Implement to_array(), to_atomic(), to_object() functions

- Add function aliases: to_bool, to_str

Change-Id: I6391a060ecd86cd397dd0a59f4930c4c55216d3b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2539

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- add job start timestamp to the joblet context

- make CommonFunctionMapUtil extensible by products

- enhance DateTimeFormatUtils.parseDateTime() to return

timezone information and not throw exception on invalid data

- add GregorianCalendarSystem.getDayOfYear(), getWeekOfYear()

- change sleep() to sleep first then evaluate the argument

- skip fractional trailing 0s when printing nano-duration

- fix documentation about using column aliases in GroupBy

Change-Id: I190663ec5e709584ef449f8279b1d2a5a0b099dd

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2511

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>