asterixdb

Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[NO ISSUE][DOC] Documentation improvements

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Add variable.* to SELECT clause

- Add section and example for variable scope in JOIN subclauses

Change-Id: I654fb35f3632cb46fc0ecdce3f12af72c2356091

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2993

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

[NO ISSUE][TEST] Allow Requests To NC Endpoints By Port

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Currently, the test framework allows sending requests to

NC endpoints that are referenced by name if that name was

added to the list of available NC endpoints. This change

allows a request to be sent to any endpoint by specifying

the endpoint port in the test file.

- Modify a test case to utilize port-based NC endpoints.

Change-Id: If0ad80bed625c45da01aa018e4267409c329be9e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2991

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

[NO ISSUE][COMP] allow optimization rule to inspect other functions + fixes

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

1. allow the optimization rule "InjectTypeCastForSwitchCaseRule"

(renamed to "InjectTypeCastForFunctionArgumentsRule" now) to consider other

functions specified by the user in addition to the ones already considered

by the rule.

2. fix an issue related to ConstantFoldingRule that affects some existing

functions where the rule would have the function compute the value but fail

to deserialize it (read it) back due to differences in compile-time type

and runtime type.

3. fix static casting of a list constructor function which fails to cast

open its items when the item is a function that returns a closed derived

type.

Change-Id: I65679e934a41e00b04bc3fd479ed404af293eb21

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2980

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Merge release-0.9.4 into master

Change-Id:Ied2b685f015e0f35563ebd0510231a5d663494f4

    • -1
    • +1
    /asterixdb/asterix-external-data/pom.xml
    • -1
    • +1
    /asterixdb/asterix-lang-common/pom.xml
    • -1
    • +1
    /asterixdb/asterix-test-framework/pom.xml
  1. … 24 more files in changeset.
[NO ISSUE] Enable Build/Run w/ Java 11

- upgrade hadoop libraries (2.2.0 -> 2.8.5)

Change-Id: I04377b91ea7535bd358c396fbd9fcb019c734814

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2986

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3030

Reviewed-by: Till Westmann <tillw@apache.org>

    • -5
    • +9
    /asterixdb/asterix-external-data/pom.xml
    • -0
    • +8
    /asterixdb/asterix-test-framework/pom.xml
  1. … 3 more files in changeset.
[NO ISSUE] Enable Build/Run w/ Java 11

- upgrade hadoop libraries (2.2.0 -> 2.8.5)

Change-Id: I04377b91ea7535bd358c396fbd9fcb019c734814

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2986

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -5
    • +9
    /asterixdb/asterix-external-data/pom.xml
    • -0
    • +8
    /asterixdb/asterix-test-framework/pom.xml
  1. … 3 more files in changeset.
[ASTERIXDB-2456][DOC] Re-add insert to primer

Change-Id: I404246ea4a8568e28190cd2371356175db1471cc

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2983

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Carey <dtabass@gmail.com>

[NO ISSUE][FUN] Add more data types for external library

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

1. Added more types for external library.

2. Added test case for data types.

3. Fixed minor bug where the parameter type is not trimmed in UDF.

Change-Id: I6ce73e791533b0617074536e0d841242d9e0ee31

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2871

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

[NO ISSUE][FUN][COMP] fix ifnan(), ifnanorinf(), ifinf()

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

This patch is to fix the mentioned functions. The fix is

to make ifnan() & ifnanorinf() skip the argument being

inspected if it's a missing value and check the next one.

For ifinf(), it should return missing if the argument

being inspected is a missing value. Also, number of args

is checked now.

Change-Id: Ib7cc3d18ca0443857ae271b6e14f2c9e920f1c1f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2981

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

[NO ISSUE][LOG] Ensure NC logDir ends with separator

- user model changes: no

- storage format changes: no

- interface changes: no

Change-Id: I492428fdbf84ed6f1c4afd136dce33ac84aea3c0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2982

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

[NO ISSUE] User-defined Function Documentation update

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

1. Updated the UDF documentation to be consisten with current master.

2. Cleaned default UDF package to remove useless UDFs.

3. Added the example in documentation as a test case for IT.

4. Reorganized the documentation to keep up with the new structure.

5. Minor changes to other documentation pages to keep style consistent.

Change-Id: I17b1b4d639ca38689298ce88145257e794eb90e1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2804

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Taewoo Kim <wangsaeu@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +358
    /asterixdb/asterix-doc/src/main/data_ingestion/feeds.md
  1. … 13 more files in changeset.
[ASTERIXDB-2453] Add Improved Constant Merge Policy

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- The current constant merge policy is unsuable because of its high

merge cost, i.e., O(N*N) where N is the number of flushes. This patch

replaces the previous constant merge policy with a more efficient policy

that still enforces a maximum number of components but greatly lowers

the merge cost.

- Extend AbstractLSMIndex with a method to return the total number of

flushes, based on the file name sequencer. This is required by the new

policy.

Change-Id: Ie5f83a4d5fdd3f036b823c906df1760f5110ae0a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2971

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[NO ISSUE][NET] Avoid Sleeping The Network Thread on Failures

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- We currently sleep the networking thread incrementally with

every failure. This sleep was added to avoid CPU spinning

back when failures on pending networking operations were not

handled properly which led to processing the same networking

message that caused the failure over and over. This sleep is

not needed anymore since every failed read/write/send/connect

network operations will not be attempted again.

Change-Id: I9f7ddc088868f8cf4d0a15ec5349021af8ccae36

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2978

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

[ASTERIXDB-2455][DOC] Deprecate AQL documentations

- user model changes: no

- storage format changes: no

- interface changes: no

details:

- Create [Deprecated] section and move AQL docs to there.

- Move some docs from /aql directory to /sqlpp directory.

Change-Id: I677dd7a8d114197eaa2ae93e0405184526b31a03

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2977

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +147
    /asterixdb/asterix-doc/src/site/markdown/sqlpp/filters.md
    • -0
    • +114
    /asterixdb/asterix-doc/src/site/markdown/sqlpp/fulltext.md
    • -0
    • +227
    /asterixdb/asterix-doc/src/site/markdown/sqlpp/similarity.md
    • -11
    • +11
    /asterixdb/asterix-doc/src/site/site.xml
[NO ISSUE][TEST] Add Test Case For Chunked Http Requests

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Add test case for chunked HTTP requests to ensure

the HTTP server can handle requests from clients

sending chunked requests.

Change-Id: I64ad01ca5ca322fb3efd4e86f053b70eb74e1079

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2975

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

[ASTERIXDB-2437][COMP] Ensure the index-only plan on a composite index

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Ensure the proper build of an index-only plan on a composite index

where both fields are used in the SELECT condition and only one field

is returned.

Change-Id: Idcc4cbe08323e0c6edb4a01637b2017128da1ab5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2854

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Merge commit 'bd728afe' from stabilization-f69489

Change-Id: Ic382adcb9e6f0a084a029206a0785d979a45faad

[NO ISSUE][REPL] Ensure Valid Component ID is Initialized On Replica Sync

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- Currently, the first time a replica is synchronized from master,

the valid component id on each replicated index's initial checkpoint

will be the initial value of a component id (-1). This value is

fixed when the the replica receives a flushed component from

the index. However, if the master fails before any component is

flushed to a replica and that replica is promoted to master, it

will start from an invalid component id. This change ensures that

the initial checkpoint of replicated indexes is initialized to

the maximum component id that appears on master. This will ensure

that if the replica is promoted, it will at least start from

a component that wasn't previously used on master.

- Replace assertion of component ids validation by illegal state.

Change-Id: I85395ad823a630725c4cab4bead1c61546dc61ae

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2973

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[NO ISSUE] Don't close Keep-Alive connections on non-200/401

Change-Id: Ia39f8f9030be5a738a71de4c336a98e326ee896f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2972

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[NO ISSUE][STO] Ensure Files From Failed Bulkload Are Deleted

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- If a bulkload is interrupted at the end-stage (i.e. after all

tuples have been added but before the component is added to

the index's disk components), then the bulkload operation will

fail but the files from the failed operation are not deleted.

This change ensures that if a bulkload fails in such stage,

the files of the generated component are deleted to avoid

file is already mapped exception if the bulkload is attempted

again.

- Ensure disk component is only destroyed once in case of

bulkload cleanup.

- Stop attempting to write a checkpoint early if the failure was

due to the thread being interrupted.

Change-Id: I7f63054ac72a9482e779e49eb4da658a08fe7e9d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2970

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

[NO ISSUE][OTH] Ensure Stats Refresh is Thread Safe

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Make variables used in determining the need to refresh

stats volatile to ensure refresh won't be called by

multiple thread if not needed.

Change-Id: I07d76e53d193f93e0aaf08715e3f4c82b3b1524e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2969

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

[ASTERIXDB-2174] Use ListSet when clone partition property

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Use ListSet instead of HashSet when cloning partitioning properties

to ensure it has the consistent behavior as before.

Change-Id: Ib2b8c1715d75b387a73504f4709524d9ab9ce123

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2968

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

[ASTERIXDB-2452] Recompute type env after ListifyUnnestingFunctionRule

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

-Recompute type env of the parent operator after firing

ListifyUnnestingFunctionRule. Otherwise the type env of the parent operator

would still point to the old operator, which is not correct.

Change-Id: I97bf721cb7fc9f36ec14214677d5545ae4aa1d2a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2967

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

[ASTERIXDB-2174] Fix partitioning improvement of group by

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Fix the improvement of partitioning propagation of group by

introduced by https://asterix-gerrit.ics.uci.edu/#/c/2176/.

- The problem is that we cannot modify the partitioning property of the

input operator directly, since that property is owned by the input.

Otherwise, the property of the child operator would be modified even

before the group by operator, and this caused some tests regarding

common sub operator extration to fail (introducing REPLICATE).

Change-Id: I81a74ab1242772c790175414281e3f200537b386

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2966

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Merge commit '142aad08' from stabilization-f69489

Change-Id: I7ebe1f339fa27fc269f56f4e394622b452310638

[NO ISSUE] Avoid NPE logging shutdown hooks if shutdown already in progress

Change-Id: Id7102a975bd6e2d653198067bb5b5e7573ded7e7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2962

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[ASTERIXDB-2444][STO] Avoid Using System Clock in Checkpoints

- user model changes: no

- storage format changes: yes

- interface changes: yes

Details:

- Replace the usage of system clock timestamps in checkpoints

by a sequencer.

- Update Asterix/Hyracks storage version to reflect the recent

changes in storage.

- This change is expected to break storage backward compatibility.

Change-Id: Idc061e6eaccfb308b29a5a263b77a0a849694d4f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2961

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

[NO ISSUE][TX] Create New Log File Before Deleting Old Files

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Ensure next log file is created after a sharp check

point before deleting old files. This is to prevent

the case if a crash happens right after deleting the

old files but before creating the new one, then the

next time the system starts up, it will start with

log file id 0 which is wrong.

- Log the details of latest index checkpoint when the

low watermark of the new checkpoint is less than the

low watermakr of the latest checkpoint.

Change-Id: I4817f697b43daff55726909ab074ec30a1c224ce

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2959

Reviewed-by: Michael Blow <mblow@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[NO ISSUE][STO] Ensure Index Component Sequence Uniqueness

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Currently, if two or more flushes are scheduled before any

of them create its files on disk, we will end up reusing

the same component sequence since the last used component

sequence is based on the files on disk. This change ensures

that the last use component sequence is only initialized

from the disk files and after that the last used component

sequence is incremented in memory.

Change-Id: I478ca3e2c9a98bab14a9145cc8b23eadbd0eab08

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2958

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Till Westmann <tillw@apache.org>

[NO ISSUE][STO] Report batch operation failure before exiting

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- In certain cases, caller of a batch operation call are

interested in failure events.

- In those cases, we used to report failure after exiting

the components but with this change, failure reporting

happens before the exit.

Change-Id: I0c22b6bddfe8f12ef8e3c59dae0b0c585137a126

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2956

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>