Clone Tools
  • last updated 10 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[NO ISSUE][API][SQL] retrieve request and statement information

- user model changes: no

- storage format changes: no

- interface changes: no

details:

- Get user agent and remote address from request reference.

- Extract statement properties that are determined during parsing.

Change-Id: I26dab3ccee21293963496f38abe44b9a653a35e4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5783

Tested-by: Michael Blow <mblow@apache.org>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -7
    • +20
    ./api/http/server/NCQueryServiceServlet.java
    • -41
    • +65
    ./api/http/server/QueryServiceServlet.java
    • -0
    • +10
    ./app/message/ExecuteStatementResponseMessage.java
    • -9
    • +18
    ./app/translator/RequestParameters.java
  1. … 7 more files in changeset.
[NO ISSUE][*DB] Clean up rebalance servlet API to remove quotes around node list

Change-Id: I61048478f2ca93eb49ffed603e24bba509559d4a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6603

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

    • -2
    • +1
    ./api/http/server/RebalanceApiServlet.java
  1. … 22 more files in changeset.
[NO ISSUE] Improve handling of udf parameter/return types

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Prohibit parameter/return type specification for inline udfs

- Prohibit parameter/return type quantifier (?) for external udfs

- Allow external udfs to refer to types from other dataverses

- Do not store type quantifiers in udf metadata

- Align names of anonymous types automatically created by

QueryTranslator for datasets and udfs

- Reserve user-defined type names starting from '$' for system use

- Add testcases

- Prohibit UdfServlet operations if cluster is not active

- Return correct path from IOManager.getWorkspacePath()

- Test framework: support multiple commands in .lib.sqlpp files

- Remove CC files when deleting old instance data in

AsterixHyracksIntegrationUtil

Change-Id: I09bdfd8c7b26750d12339034d2143a16102d1212

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6545

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -2
    • +15
    ./api/http/server/UdfApiServlet.java
    • -66
    • +90
    ./app/translator/QueryTranslator.java
  1. … 34 more files in changeset.
Merge commit 'a548307b9f' from 'gerrit/mad-hatter'

Change-Id: I57eb5d96af8f227e5a9550381418f9468eb400d8

    • -5
    • +17
    ./app/translator/QueryTranslator.java
  1. … 8 more files in changeset.
[ASTERIXDB-2722][EXT] Require minimum permissions + improve error reporting

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Creating an external dataset requires S3:ListBucket permission.

- Querying an external dataset requires S3:ListBucket and

S3:GetObject permissions.

- Improved error reporting.

- Added test cases for the above mentioned items (where applicable).

Change-Id: Idc266cf63b8f92a07af7341118d2636673913160

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6463

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -5
    • +17
    ./app/translator/QueryTranslator.java
  1. … 18 more files in changeset.
[ASTERIXDB-2738][STO] Create Mask File Before Merge Operations

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Before starting a merge operation, create a mask file

(.mask_C_startSeq_endSeq) for the merged component to

indicate that this component isn't valid yet.

- On the merge operation successful completion, delete the

merged component mask file.

- In the case of any unexpected failure during the merge

operation, all files of the failed merged component will

be deleted on node startup/shutdown, including the mask

file.

- Halt on any IO opeartion failure.

- Add a test case that ensures only masked merged components

are deleted but not the original components that were

supposed to be merged.

Change-Id: I476dd3be5e75468e83044b3aaf0f6c2d8beadf1c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6425

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Luo Chen <cluo8@uci.edu>

  1. … 6 more files in changeset.
Merge commit '0f407d9224' from 'gerrit/mad-hatter'

Change-Id: I89eaf1bace87130d8d35ec0ce8e0c263ac87b59a

    • -3
    • +30
    ./app/translator/QueryTranslator.java
  1. … 1 more file in changeset.
Merge commit '594e7eca21' from 'gerrit/mad-hatter'

Change-Id: I975228925366adc94f136bbbb014cb6817bf180e

  1. … 5 more files in changeset.
Merge commit '6904d36' from 'gerrit/mad-hatter'

Change-Id: Ib118bb2d1b18a93412015b1b652684c3fbb11e52

    • -18
    • +24
    ./app/translator/QueryTranslator.java
  1. … 2 more files in changeset.
[ASTERIXDB-2725][COMP] Check if dataverse exists before continuing the DDL execution

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Added dataverse check at multiple places before proceeding

with the execution of the DDL.

- Added a test cases for each newly added check.

Change-Id: I8d54b0b10741d65df91cbb4b7596d363f7ddbd97

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6203

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -0
    • +25
    ./app/translator/QueryTranslator.java
  1. … 14 more files in changeset.
[NO ISSUE][EXT] Minor refactoring of external datasets details validation

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Minor refactoring of external datasets details validation.

- Added a missing error message in Hyracks.

Change-Id: I975228925366adc94f136bbbb014cb6817bf180e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6325

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 3 more files in changeset.
Merge commit 'aa7ff7b' from 'gerrit/mad-hatter'

Change-Id: Ibfc874ca45692420fb86038537d10e172894730a

    • -1
    • +2
    ./api/http/server/RebalanceApiServlet.java
    • -15
    • +23
    ./app/translator/QueryTranslator.java
  1. … 18 more files in changeset.
[NO ISSUE][COMP] Refactor locking in CREATE DATASET

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Refactor locking in CREATE DATASET to make it extensible

by products (introduce QueryTranslator.createDatasetBegin())

- Refactor MetadataLockUtil.createDatasetBegin() into two methods

Change-Id: I1b430a5e8f4a3cdf1a360397e3d160f7b8c95d26

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6323

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

    • -22
    • +38
    ./app/translator/QueryTranslator.java
  1. … 1 more file in changeset.
Merge commit 'f1cd417' from 'gerrit/mad-hatter'

Change-Id: I0f30399a6485708f4b154af09ea36377fe4d3dbf

    • -16
    • +38
    ./app/translator/QueryTranslator.java
  1. … 10 more files in changeset.
Merge commit '4bf7048' from 'gerrit/mad-hatter'

Change-Id: I06432e8daa1669862d1f23a04efdfec5b1e6d7b5

    • -2
    • +15
    ./app/translator/QueryTranslator.java
  1. … 3 more files in changeset.
[ASTERIXDB-2731][API] GET is not supported for the query service servlet

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Remove the restriction of only supporting POST requests in the query service servlet

while maintaining that GET requests cannot change the state of the system

(i.e. no modifying statements can be evaluated when using the GET method).

Change-Id: I989ff489d863697808cdc26898a52b867a9621d6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6303

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Till Westmann <tillw@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -2
    • +11
    ./api/http/server/QueryServiceServlet.java
  1. … 16 more files in changeset.
Merge commit '0c91bd7c564e1bd5cb5d54b137f41a4d9b9db076' into HEAD

Change-Id: Iebc71d2b9797e5f645580ea8ae2b7e149e749a03

    • -27
    • +10
    ./app/translator/QueryTranslator.java
Merge commit '3e2623cf64827e891ecd8c6dea13d03f5599c6a5' into HEAD

Change-Id: I89091d8bf8575c73d26f497d5fbffe780adbedf9

    • -23
    • +120
    ./app/translator/QueryTranslator.java
  1. … 8 more files in changeset.
[NO ISSUE][COMP] Introduce QueryTranslator.validateIndexType()

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Introduce QueryTranslator.validateIndexType() that

can be overriden by product extensions to customize

supported index types.

Change-Id: I000ba078579b8e0a96d0219cefad137f41abf791

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6204

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -17
    • +25
    ./app/translator/QueryTranslator.java
Merge commit 'bc0f7e0' from branch 'gerrit/mad-hatter'

Change-Id: I629850e0f6ac7e1afae3fb5c6e1f1f859b2ead72

    • -1
    • +23
    ./app/translator/QueryTranslator.java
    • -3
    • +6
    ./hyracks/bootstrap/CCApplication.java
  1. … 17 more files in changeset.
[ASTERIXDB-2176] Python UDFs

- user model changes: yes

- storage format changes: no

- interface changes: yes

Details:

- Allow UDFs to be shiv-packaged python modules with their

dependencies

- Use pyro for python RPC

- Maven build for pyro package into server

- Remove JObject spatial types

- Simpler conversion from java primitive/standard types and

collections

to JObject equivalents

Change-Id: Ibea23a2e9308132f343d80eff04ede9a235aa021

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5526

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

    • -52
    • +88
    ./api/http/server/UdfApiServlet.java
    • -134
    • +6
    ./app/external/ExternalLibraryUtils.java
    • -0
    • +4
    ./app/io/PersistedResourceRegistry.java
    • -3
    • +1
    ./app/nc/task/ExternalLibrarySetupTask.java
    • -7
    • +4
    ./hyracks/bootstrap/CCApplication.java
  1. … 99 more files in changeset.
[ASTERIXDB-2715][STO] Dynamic Memory Component Architecture

- user model changes: yes

- storage format changes: no

- interface changes: yes

Details:

- Introduce a dynamic memory component architecture that uses a global

virtual buffer cache to manage the write memory for all LSM-trees.

- When the overall write memory is nearly full, we flush a dataset

partition at a time using a round-robin way. Additionally, we allow

users to configure the maximum size of filtered memory components

to provide better pruning capability.

- Clean up legacy code for statically allocating write memory to each

dataset.

- Remove the following parameters:

storage.metadata.memorycomponent.numpages

storage.max.active.writable.datasets

- Add the following parameters:

storage.memorycomponent.flush.threshold (default: 0.9)

storage.filtered.memorycomponent.max.size (default 0)

Change-Id: Ia6a0f4de020acd7af89ef630322526c4be5076e0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5824

Reviewed-by: Luo Chen <cluo8@uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -10
    • +14
    ./app/nc/NCAppRuntimeContext.java
  1. … 44 more files in changeset.
[NO ISSUE][HTTP] Http Encoding refactoring / cleanup

Change-Id: I1e6eef71f4f17a7563f182400dd1e815e4a6b8ff

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5943

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Contrib: Michael Blow <mblow@apache.org>

    • -1
    • +2
    ./api/http/server/RebalanceApiServlet.java
  1. … 2 more files in changeset.
[NO ISSUE] Fix UDF Metadata Transactions

- Move UDF Metadata Transactions to be initiated from the CC

directly

- Remove old UDF initialization code

- Make DROP DATAVERSE remove libraries

- Make NCs properly read libraries on init with multipart dataverse

names

Change-Id: Ibda23a2e8308937f343d80eff04ede9a235da0d2

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4923

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

    • -30
    • +136
    ./api/http/server/UdfApiServlet.java
    • -198
    • +27
    ./app/external/ExternalLibraryUtils.java
    • -1
    • +1
    ./app/nc/task/ExternalLibrarySetupTask.java
    • -0
    • +14
    ./app/translator/QueryTranslator.java
    • -1
    • +1
    ./hyracks/bootstrap/CCApplication.java
  1. … 38 more files in changeset.
[ASTERIXDB-2713][EXT] CSV & TSV support for external dataset p2

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- validate adapter configuration

- ignore empty lines in CSV/TSV files

- require "header" parameter for CSV/TSV formats

- make some parameters case-insensitive

- few fixes and clean-ups

Change-Id: I2f523de0d482a358ada0c27236ff24616ad0d7da

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5848

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

    • -3
    • +21
    ./app/translator/QueryTranslator.java
  1. … 37 more files in changeset.
[NO ISSUE][COMP] Extensible item type check in CREATE DATASET

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Introduce QueryTranslator.validateDatasetItemType().

By overriding this method product extensions can further

restrict types that are allowed for dataset creation.

Change-Id: I426430707766b7b6049abd2687bee69c011335fb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5785

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Hussain Towaileb <hussainht@gmail.com>

    • -12
    • +15
    ./app/translator/QueryTranslator.java
[ASTERIXDB-2713][EXT] Add CSV & TSV support for external dataset

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Add CSV support for external dataset.

- support S3

- add boolean parser to Hyracks

Change-Id: Id1790fa73461e9f4a5fb443c51c1905ac588cee6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5743

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 48 more files in changeset.
[NO ISSUE][COMP] Prohibit anonymous type use by CREATE DATASET

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- CREATE DATASET should not be able to use anonymous types

Change-Id: I0dcdd2684180b054ff48f0f0a8ccd6041e8bd00c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5664

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Hussain Towaileb <hussainht@gmail.com>

    • -2
    • +15
    ./app/translator/QueryTranslator.java
  1. … 8 more files in changeset.
[NO ISSUE][COMP] CREATE INDEX with Geometry datatype

Details:

- Support geometry datatype for RTree Index:

CREATE INDEX geomIndex ON geoDataset(geometry) TYPE rtree;

- Enable ST functions for rtree with geometry datatype

- Add testcases

Change-Id: I8f4a82c43b950fc3573cae5aa7c0782b475f962d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5584

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Mehnaz Tabassum Mahin <mmahi004@ucr.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -0
    • +2
    ./app/io/PersistedResourceRegistry.java
  1. … 48 more files in changeset.
[NO ISSUE][COMP] Introduce Index Only Compiler Property

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Introduce "compiler.indexonly" compiler property to

enable/disable index-only plans.

- Replace usage of 'noindexonly' query property by

the new compiler property.

- Update docs and test cases.

Change-Id: I82d05bbfacbaca17eb8d8a2b6d825aa180e9ab02

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5567

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 33 more files in changeset.