asterixdb

Clone Tools
  • last updated 18 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Merge branch 'gerrit/mad-hatter' into 'master'

Change-Id: I892dfdc0f2f61cb20e17995d797598b420e8c554

[NO ISSUE][STO] Persist Bloom Filter Existence in Index Metadata

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- To clearly identify whether an index has a bloom filter or

not for BTree indexes, persist this information in the index's

metadata stored on each NC.

- For backward compatibility, when reading an index's metadata that

was created before adding the hasBloomFilter field, default its

value based on whether or not the index is a primary key index.

- Remove unused special readObject from LSMBTreeLocalResource.

Change-Id: Icec570d490987de401c036790ee9567238a60301

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/3804

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 6 more files in changeset.
[NO ISSUE][RT] Report Disk IO Per Operator

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Do not collect per thread stats when runtime

profiling is disabled.

- Include per operator disk IO in task profile.

- Remove aggregated disk IO count from the query

response metrics.

- Adapt test cases.

Change-Id: Iced174ae5d4fa9ca13f597325cbb66ff80819e71

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/3783

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Murtadha Hubail <mhubail@apache.org>

  1. … 4 more files in changeset.
[NO ISSUE][OTH] Eliminate Object Creation on Thread Stats Lookup

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Use the thread object for thread stats lookup

rather than the thread id to avoid boxed objects

creation on stats lookup.

Change-Id: I5b8b26be9d3439889e21cf097ea86c413e973e1d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/3763

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Michael Blow <mblow@apache.org>

Merge branch 'gerrit/mad-hatter' into 'master'

Change-Id: I7e921b559f30adfab704143818dd7ee59e194eb4

    • -16
    • +7
    /asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
[ASTERIXDB-2648][COMP] Fix dataset order in broadcast hint

- user model changes: yes

- storage format changes: no

- interface changes: no

details:

- Provides hash-bcast hint to replace bcast hint which makes

broadcast join consistent with other types of joins for choosing

the broadcasting dataset based on dataset order in FROM clause.

- Chooses the build dataset to get broadcast based on the order

in the FROM clause not the WHERE clause (which was the case for

bcast hint).

Change-Id: I505fc8e0bc32c760ae0471c6b302bbfd8499be4a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/3566

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 37 more files in changeset.
[ASTERIXDB-2655][STO] Default storage compression to enabled

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Use 'snappy' as default 'storage.compression.block' value

- Update tests to reflect the new default

Change-Id: I62428c01ec24bc72bdf3cdf903655b3d0d7acbdd

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/3683

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Michael Blow <mblow@apache.org>

Reviewed-by: Michael Blow <mblow@apache.org>

Reviewed-by: Hussain Towaileb <hussainht@gmail.com>

[ASTERIXDB-2651][RT][FUN] Handle matching a literal backslash properly in LIKE

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- fixed an issue in the build() of RegExpMatcher where the "flags" argument gets ignored if the flags

argument does not change in the subsequent evaluation of the regexp_* functions over the tuples of dataset.

- added the missing Java regex meta characters.

- split test case "regexp_replace" into two, "regexp_replace" & "regexp_replace_with_flag".

Change-Id: I4f218f87f0c1736e2d8b5de07221d7efa0f9aef1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/3614

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 86 more files in changeset.
[NO ISSUE] Fix type inference for ANY in UDFs

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

When data type is available, we should use defined data type. If not, we

should try to infer the data type from the data.

Change-Id: I6a03749fe041ec6f43de230d3e784122f7664545

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/3405

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

[NO ISSUE][NET] Log ChannelReadInterface Stats on Failure

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- When the read buffers of a FullFrameChannelReadInterface

are exceeded, log its stats to help in debugging the issue.

- Warn when a partial frame is being written over a full

frame write channel.

- Rename ThreadSafetyGuaranteedBy -> GuardedBy.

- Annotate networking calls with their synchronization

guards.

Change-Id: I89eed0c06dbf4b0e86747538bff286dc37853957

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/3663

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Al Hubail <mhubail@uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Merge branch 'mad-hatter' into 'master'

Change-Id: I24f42dc32931a214a27c2c88fde163f665cc83db

[NO ISSUE][API] Add ability to identify secure servlet requests

Change-Id: Ia693e29a47b513a63fdce80383da90ba165c28d6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/3603

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

[NO ISSUE][COMP] Cleanup Identifier constructors

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Remove default constructors from

Identifier, VarIdentifier, VariableExpr

- Make Identifier immutable

Change-Id: I3a0b4da0e2b621d309b8d9aa3c47540eb18566eb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/3607

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

    • -18
    • +15
    /asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
[ASTERIXDB-2634][COMP] String functions return null on data/type errors

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Functions changed:

substring(string, start_idx),

repeat(string, num_times),

string_join([string], string_separator),

replace(string, search_string, string_replacement, num_times),

regexp_replace(string, string_pattern, string_replacement, num_times)

Those functions (except string_join) can return NULL even if the

arguments types are valid since the argument values could be not

valid at runtime. Their type computer is always nullable.

- int argument can be double/float on the condition that its value

is integer

- clean-ups:

UnaryStringInt64TypeComputer & AbstractStringTypeComputer.

ExceptionUtil:

- changed signature of toExpectedTypeString(). Overloading

with varargs with Object made it confusing.

- changed some args to Supplier to allow code sharing.

- removed StringIntToStringTypeComputer since now the instances

are not being used by the above functions.

- reorganized/renamed test cases

Change-Id: Ia85a0d08888021ae439a1d9f2f5858bcd52c79f3

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/3605

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 49 more files in changeset.
[NO ISSUE][DOC] Improve window functions documentation

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Minor improvements in the documentation

for window functions and OVER clause

Change-Id: I349d9b93edca3f3a7ffb340b016402e9ae8320fc

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/3606

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[ASTERIXDB-2649][FUN] TPC-DS datasource function, generate proper data types

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Changed the TPC-DS datasource function to generate

the TPC-DS data with proper data types according to

the TPC-DS schema.

- Updated the TPC-DS test cases.

Change-Id: I20f6b8d043906ad62652d098e09ab70eb1d78b1b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/3604

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Hussain Towaileb <hussainht@gmail.com>

Merge commit 'eaa38d8c2c' into 'master'

Change-Id: Ic03c45b9a79784b357ea66502956778bf105b9b3

[NO ISSUE][FUN] Clean up of warn utility method

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

clean up of warnTypeMismatch methods to avoid confusion

betweent the two methods.

Change-Id: I7ae7c85fa3cc549cfd622f179f36486e016d9643

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/3581

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

[NO ISSUE][TXN] Remove Synchronization on TransactionContext

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Currently the flag indicating if a transaction timed out

synchronizes on the transaction context which makes it

a hot spot when multiple threads on the same transactions

attempting to access that flag. This change removes that

synchronization and replaces it with a volatile variable.

Change-Id: Ib14c07a051d27a5b47c62ff35769670e6ab01fd2

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3579

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Merge commit '819c57b440'

Change-Id: I82594526b6f921af058982f450efbdfdad78b106

Merge commit 'e8e59d0e6e' from 'stabilization-f69489'

Change-Id: I73ad57791bb7264a670bb36f4eb6f50640b4a86f

[NO ISSUE] Update Guava, Jackson dependencies

Change-Id: I1a415367b0e403237deab782dd99214b927138db

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3576

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

[ASTERIXDB-2541][STO] Introduce GreedyScheduler

- user model changes: yes.

Add new option: storage.io.scheduler (async/greedy)

- storage format changes: no.

- interface changes: yes.

Introduce IIndexCursorStats

Details:

- Introduce GreedyScheduler that always executes the merge

operation with the smallest number of remaining pages to minimize

the number of disk components

- Introduce IIndexCursorStats to collect the statistics of index scans.

This allows GreedyScheduler to know the remaning pages of merge

operations.

- Extend AbstractIoOperation so that GreedyScheduler can pause/resume

merge operations if needed.

Change-Id: I38fe394d1180d4e3f6796064c0e6c6630b6ad303

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3284

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Michael Blow <mblow@apache.org>

  1. … 59 more files in changeset.
[ASTERIXDB-2647][COMP] Random partitioner shouldn't preserve input's properties

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Random partitioner operator currently always propagates

the local properties of its input. If the data of the

input operator is partitioned, the random partitioner

should not propagate the local properties of the input

since the random partitioner destroys them.

Change-Id: I0378597451b3e6b25e8b45295159efec262abae2

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3575

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

[ASTERIXDB-2635][*DB] Fix JSON plan pretty printing

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Follow up change to fix printing "expressions" field

Change-Id: Ia994c0e0001cee2eca31f0bdf51ddb52320ae752

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3574

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

[NO ISSUE] Fix comment of concurrent merge policy test

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Fix the comment of the test case using-concurrent-merge-policy.

Change-Id: I363f5739ac4bb2a9dd14df7be97372eb1e838dd6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3572

Reviewed-by: Luo Chen <cluo8@uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

[NO ISSUE][TEST] Increase Test Buffer Cache Memory Budget

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Increase test buffer cache memory budget to 128MB

to avoid test failures due to small buffer cache.

Change-Id: I76d29bf7d81de189e2157d902382a60e6c9772e0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3571

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Michael Blow <mblow@apache.org>

[NO ISSUE][COMP] Fix type computers of some array functions

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Fix the type computer of some array functions to return a nullable

type if the function might return NULL.

Change-Id: I21b747f3116e78f4dcbec079a38972c1f1abc2ce

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3570

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Contrib: Till Westmann <tillw@apache.org>

[ASTERIXDB-2635][*DB] Fix JSON plan pretty printing

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

Use Jackson to pretty print the logical plan as JSON.

- fixed JsonLogicalPlanTest to validate JSON plans

produced by optimizer tests.

Change-Id: Ic5ada2f31afebc3f219b0584b5ae527ddf9e326e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3567

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 7 more files in changeset.
[NO ISSUE][*DB] pass max-warnings via session config

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

Pass max-warnings from servlet to APIFramework

and QueryTranslator through the session config instead

of creating an internal parameter (REQUEST_MAX_WARNINGS)

and storing it in the metadata provider.

Change-Id: I4145bcccc2702fae4b0c33872f1b98c47ab299f7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3569

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>