Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[NO ISSUE] Fix bug to block illegal inverted index

Fix bug to block inverted index creation with a variable-length primary

key (e.g. string as the primary key). Currently, we don't support such

an inverted index in this case.

Change-Id: I715d55d48246fd08817b72f7104bc4d436578600

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/7523

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

  1. … 2 more files in changeset.
[NO ISSUE][COMP] UDF improvements

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- SQL++ grammar changes

1. CREATE FUNCTON

Support AS .. AT .. for external identifier

Remove LANGUAGE, INLINE

Move WITH record contents into its 'resources' field

Move DETERMINISTIC, NULLCALL into WITH fields

2. CREATE ADAPTER

Support AS .. AT .. for external identifier

3. Add DROP ADAPTER

- CreateFunctionStatement, CreateAdapterStatement

1. Add library dataverse name

2. Support IF NOT EXISTS

3. Remove language

- Support IF EXISTS in AdapterDropStatement and

LibraryDropStatement

- Handle AdapterDropStatement in QueryTranslator

- FunctionTupleTranslator writes external identifier as

an ordered list instead of encoding it into function body

- Changed external identifier format for Python UDFs to

"package.module", "class.function"

- Add validation for external identifier depending

on the library language

- Update documentation and testcases

Change-Id: I3f2125b5508f2321cfa9b561b5250d6255135f91

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/7483

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

    • -87
    • +203
    ./translator/QueryTranslator.java
  1. … 59 more files in changeset.
Merge branch 'gerrit/mad-hatter'

Change-Id: I78f3e2ee495feec5bf311261188c48dd770b66ad

  1. … 1 more file in changeset.
[ASTERIXDB-2176][RT] Improved Python IPC

- use msgpack serialization over tcp loopback

- convert directly from ADM binary format to msg

Change-Id: I5cbbc367944b489aee651ea050e74990dcf65521

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6883

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 59 more files in changeset.
[NO ISSUE] DROP DATASET IF EXISTS must not fail if dataverse not found

Details:

- Fixed regression in DROP DATASET IF EXISTS.

The statement must not fail if dataset's dataverse

is not found.

Change-Id: I613c0b84cd57ce690b4e7d28e3e376ddfd633791

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/7325

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 1 more file in changeset.
[NO ISSUE][COMP] Improve error message when function not found

Details:

- Remove arity from the error message raised when

the function is not found

- Also remove "null." prefix from that message if

the function call did not specify a dataverse

Change-Id: I384ca2af87078e72aab23dccd3651a8edf02a3b4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/7307

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 5 more files in changeset.
[NO ISSUE][COMP] Support dropping dataverse only if it's empty

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Add ifEmpty flag to DataverseDropStatement to indicate that

the dataverse must be dropped only if it does not contain any

objects. This flag is currently not supported in the grammar,

but may be used by product extensions.

- Add validateDataverseStateBeforeDrop() method that is called

before a dataverse is about to be dropped. It may be used

by product extensions to perform additional checks before

dropping the dataverse.

- Refactor handleCreateIndexStatement() to align with other

handle*() methods (acquires locks, then calls doCreateIndex()).

- Ensure that all doDrop*() methods return a boolean value

indicating whether the entity was actually dropped.

Change-Id: I86aca8ab02cf1bf0e0600237d3e2f71293166b08

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/7204

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

    • -39
    • +86
    ./translator/QueryTranslator.java
  1. … 5 more files in changeset.
[ASTERIXDB-2760][COMP] Improve function call resolution

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Refactor function call resolution and perform it earlier,

before GROUP BY expressions are substituted

- Consolidate gby-expr testcases into a single one

- Introduce QueryTranslator.handleDeclareFunctionStatement()

- Rename existing SqlppCaseExpressionVisitor to

SqlppCaseAggregateExtractionVisitor

- Move CASE expression rewriting code from

SqlppBuiltinFunctionRewriteVisitor into a separate visitor:

SqlppCaseExpressionVisitor

- Remove function name normalization code from SqlppAstPrintVisitor

and updated existing reference results

Change-Id: I527ffbff487b8534f66ca27d48788e96c65aec9b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/7264

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 1162 more files in changeset.
[NO ISSUE] Redact sensitive data from statement logging

Change-Id: Ibd63ca9167c769eea4d03982dbf7fa543913dc67

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/7165

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

    • -1
    • +1
    ./message/ExecuteStatementRequestMessage.java
  1. … 6 more files in changeset.
[NO ISSUE][COMP] Refactor drop dataverse and drop function

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Refactor feed handling when processing drop dataverse

and drop function

- Move feed dependency checking from QueryTranslator

to MetadataNode to align with other entities

Change-Id: I493dfffd77c596bb3485eccb00b417d40f47b647

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/7066

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Xikui Wang <xkkwww@gmail.com>

    • -66
    • +31
    ./translator/QueryTranslator.java
  1. … 4 more files in changeset.
[NO ISSUE][COMP][RT] New library deployment model

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- Deploy external libraries using Hyracks jobs

- Library manager is no longer available on CC

- Add IServlet.init() which is invoked during web server startup

- External libraries can no longer provide implementations of

IExternalDataSourceFactory because external libraries are

not available on CC

- Added testcase for an external adapter

Change-Id: If64f99f6a15b81b1e426239bde63360f5ef57059

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6863

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

    • -0
    • +140
    ./external/ExternalLibraryUtil.java
    • -55
    • +0
    ./external/ExternalLibraryUtils.java
    • -50
    • +0
    ./nc/task/ExternalLibrarySetupTask.java
    • -2
    • +0
    ./replication/NcLifecycleCoordinator.java
    • -14
    • +217
    ./translator/QueryTranslator.java
  1. … 85 more files in changeset.
Merge commit 'c25c36d1b0' from 'gerrit/mad-hatter'

Change-Id: I8a78950cb81f0a29ceae94fa439eec700c17d06d

  1. … 11 more files in changeset.
[NO ISSUE] Pass item type when creating external dataset properties

Change-Id: I8b6693d229a36358eda584ddfb95d1819f76ce5c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6963

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Hussain Towaileb <hussainht@gmail.com>

[ASTERIXDB-2751][EXT] Issue a warning if external dataset configuration returns no files

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Issue a warning when creating an external dataset if the dataset

configuration returns no files.

- Issue a warning when querying an external dataset if no files

are returned.

Change-Id: I0b2be49bf3c69c3e055d942a5c3f0106b3069d52

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6843

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Hussain Towaileb <hussainht@gmail.com>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 53 more files in changeset.
Merge branch 'gerrit/mad-hatter'

Change-Id: I5cd06cc3cd69068faecced6a2b1cc558eecd0784

  1. … 4 more files in changeset.
[NO ISSUE][MISC] IRetryPolicy requires non-null throwable

Change-Id: I4835266435fc31f5973f4312e6ddcbdb85d1e839

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6923

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 1 more file in changeset.
[NO ISSUE][COMP] Improve error reporting in TypeTranslator

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Add error code and source location to exceptions

raised by TypeTranslator

- Pass dataset item type when calling

QueryTranslator.validateExternalDatasetProperties()

Change-Id: I2b62c162f37e72245cdbf70fe9f8307bbf9b2bf1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6885

Reviewed-by: Till Westmann <tillw@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 4 more files in changeset.
[ASTERIXDB-2730][STO] Optimize flush in GVBC

- user model changes: no

- storage format changes: no

- interface changes: yes. Introduce a new storage property:

storage.memorycomponent.max.concurrent.flushes (default 0)

Details:

- Introduce a new storage property to allow concurrent flushes

by GVBC. The default value is 0, which means that the flush concurency

will be the same as the number of NC partitions.

- Move cleaning up of a memory component out of the synchronization block

on op tracker because this may take a relatively long time (a full scan over

all GVBC pages).

- Introduce a minor fix to make sure the memory component is unwritable

before requesting flushing it by GVBC

Change-Id: Id8867fa3ac65da319723b804cc1e39dc8eb6bde5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6624

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 11 more files in changeset.
Merge commit '0163af68d0' from 'gerrit/mad-hatter'

Change-Id: I5b53384961e42fe070d2e1b885e762428f6f8c45

  1. … 4 more files in changeset.
Merge branch 'gerrit/mad-hatter'

Change-Id: I85867c451c84aaa94f008c6b718ab7805983c503

  1. … 5 more files in changeset.
[ASTERIXDB-2741][EXT] Support include/exclude filters for S3 external datasets

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Added support for include/exclude filters.

- Added test cases for include/exclude.

Change-Id: I21dbc5c08456f95a2cad1a8fd9147472fad15376

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6604

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Hussain Towaileb <hussainht@gmail.com>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 89 more files in changeset.
[NO ISSUE][API][SQL] retrieve request and statement information

- user model changes: no

- storage format changes: no

- interface changes: no

details:

- Get user agent and remote address from request reference.

- Extract statement properties that are determined during parsing.

Change-Id: I26dab3ccee21293963496f38abe44b9a653a35e4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5783

Tested-by: Michael Blow <mblow@apache.org>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -2
    • +5
    ./message/ExecuteStatementRequestMessage.java
    • -0
    • +10
    ./message/ExecuteStatementResponseMessage.java
    • -9
    • +18
    ./translator/RequestParameters.java
  1. … 12 more files in changeset.
[NO ISSUE] Improve handling of udf parameter/return types

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Prohibit parameter/return type specification for inline udfs

- Prohibit parameter/return type quantifier (?) for external udfs

- Allow external udfs to refer to types from other dataverses

- Do not store type quantifiers in udf metadata

- Align names of anonymous types automatically created by

QueryTranslator for datasets and udfs

- Reserve user-defined type names starting from '$' for system use

- Add testcases

- Prohibit UdfServlet operations if cluster is not active

- Return correct path from IOManager.getWorkspacePath()

- Test framework: support multiple commands in .lib.sqlpp files

- Remove CC files when deleting old instance data in

AsterixHyracksIntegrationUtil

Change-Id: I09bdfd8c7b26750d12339034d2143a16102d1212

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6545

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -66
    • +90
    ./translator/QueryTranslator.java
  1. … 35 more files in changeset.
Merge commit 'a548307b9f' from 'gerrit/mad-hatter'

Change-Id: I57eb5d96af8f227e5a9550381418f9468eb400d8

  1. … 8 more files in changeset.
[ASTERIXDB-2722][EXT] Require minimum permissions + improve error reporting

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Creating an external dataset requires S3:ListBucket permission.

- Querying an external dataset requires S3:ListBucket and

S3:GetObject permissions.

- Improved error reporting.

- Added test cases for the above mentioned items (where applicable).

Change-Id: Idc266cf63b8f92a07af7341118d2636673913160

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6463

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 18 more files in changeset.
[ASTERIXDB-2738][STO] Create Mask File Before Merge Operations

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Before starting a merge operation, create a mask file

(.mask_C_startSeq_endSeq) for the merged component to

indicate that this component isn't valid yet.

- On the merge operation successful completion, delete the

merged component mask file.

- In the case of any unexpected failure during the merge

operation, all files of the failed merged component will

be deleted on node startup/shutdown, including the mask

file.

- Halt on any IO opeartion failure.

- Add a test case that ensures only masked merged components

are deleted but not the original components that were

supposed to be merged.

Change-Id: I476dd3be5e75468e83044b3aaf0f6c2d8beadf1c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6425

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Luo Chen <cluo8@uci.edu>

  1. … 6 more files in changeset.
Merge commit '0f407d9224' from 'gerrit/mad-hatter'

Change-Id: I89eaf1bace87130d8d35ec0ce8e0c263ac87b59a

  1. … 1 more file in changeset.
Merge commit '594e7eca21' from 'gerrit/mad-hatter'

Change-Id: I975228925366adc94f136bbbb014cb6817bf180e

  1. … 5 more files in changeset.
Merge commit '6904d36' from 'gerrit/mad-hatter'

Change-Id: Ib118bb2d1b18a93412015b1b652684c3fbb11e52

    • -18
    • +24
    ./translator/QueryTranslator.java
  1. … 2 more files in changeset.
[ASTERIXDB-2725][COMP] Check if dataverse exists before continuing the DDL execution

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Added dataverse check at multiple places before proceeding

with the execution of the DDL.

- Added a test cases for each newly added check.

Change-Id: I8d54b0b10741d65df91cbb4b7596d363f7ddbd97

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6203

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 14 more files in changeset.