asterixdb

Clone Tools
  • last updated 25 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[ASTERIXDB-2444][STO] Avoid Using System Clock in Checkpoints

- user model changes: no

- storage format changes: yes

- interface changes: yes

Details:

- Replace the usage of system clock timestamps in checkpoints

by a sequencer.

- Update Asterix/Hyracks storage version to reflect the recent

changes in storage.

- This change is expected to break storage backward compatibility.

Change-Id: Idc061e6eaccfb308b29a5a263b77a0a849694d4f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2961

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

[NO ISSUE][TX] Create New Log File Before Deleting Old Files

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Ensure next log file is created after a sharp check

point before deleting old files. This is to prevent

the case if a crash happens right after deleting the

old files but before creating the new one, then the

next time the system starts up, it will start with

log file id 0 which is wrong.

- Log the details of latest index checkpoint when the

low watermark of the new checkpoint is less than the

low watermakr of the latest checkpoint.

Change-Id: I4817f697b43daff55726909ab074ec30a1c224ce

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2959

Reviewed-by: Michael Blow <mblow@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[NO ISSUE][STO] Ensure Index Component Sequence Uniqueness

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Currently, if two or more flushes are scheduled before any

of them create its files on disk, we will end up reusing

the same component sequence since the last used component

sequence is based on the files on disk. This change ensures

that the last use component sequence is only initialized

from the disk files and after that the last used component

sequence is incremented in memory.

Change-Id: I478ca3e2c9a98bab14a9145cc8b23eadbd0eab08

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2958

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Till Westmann <tillw@apache.org>

[NO ISSUE][STO] Report batch operation failure before exiting

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- In certain cases, caller of a batch operation call are

interested in failure events.

- In those cases, we used to report failure after exiting

the components but with this change, failure reporting

happens before the exit.

Change-Id: I0c22b6bddfe8f12ef8e3c59dae0b0c585137a126

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2956

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

[NO-ISSUE][COMP] Avoid adding redundant var in AbstractIntroduceGroupByCombinerRule

- user model changes: no

- storage format changes: no

- interface changes: no

For live variables added in new Group-by op, they should not be added

again.

Change-Id: Ic1ab9aee31db95d5782385bc3d53777da54f6d83

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2933

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

[NO ISSUE][ING] Fix race between active recovery and rebalance

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- In certain cases, a rebalance active suspend starts before

recovery of an active job starts.

- When that happens, sometimes, the recovery task exists and

the active job is not resumed after rebalance.

Change-Id: I66edb73950bb82baa1a1dfd892cb4b23bb7046be

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2950

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

[NO ISSUE][NET] Ensure CLOSE Is Not Sent After Channel ERROR

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Currently it is possible to send network channel

CLOSE command after a channel ERROR was sent. When this

happens and the channel was recycled to be reused

on the receiver side, the CLOSE command will result

in NPE. There is no need to send a CLOSE command

after an ERROR command because when an ERROR command

is received, it is treated as ERROR + CLOSE on the

receiver side.

- Avoid registering partition requests for failed jobs.

Change-Id: I17a769a46f4d13220adb22dd255e56dc4ccc458d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2954

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Michael Blow <mblow@apache.org>

Advance version to AsterixDB 0.9.5-SNAPSHOT and Hyracks 0.3.5-SNAPSHOT

Change-Id:Ied2b685f015e0f35563ebd0510231a5d663494f7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2953

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

    • -1
    • +1
    /asterixdb/asterix-client-helper/pom.xml
    • -1
    • +1
    /asterixdb/asterix-external-data/pom.xml
  1. … 85 more files in changeset.
Release Apache AsterixDB 0.9.4 and 0.3.4

Change-Id:Ib015ae6e13f38eb726512a9c5ddeeda428ad1dff

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2952

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

    • -1
    • +1
    /asterixdb/asterix-client-helper/pom.xml
    • -1
    • +1
    /asterixdb/asterix-external-data/pom.xml
  1. … 86 more files in changeset.
[ASTERIXDB-2446][COMP] Detect duplicate field names in select clause

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Throw compile-time error if select clause has duplicate projection names

Change-Id: Idd97c2edd6b4a627ad1a0225d95c26915d4ad5f7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2951

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Contrib: Till Westmann <tillw@apache.org>

Merge commit '95f508bbb1d07650392c21da2958ba1c53f2a03d' from stabilization-f69489

Change-Id: Ib6a428564681938dceb6ad4a2ba91f15256dc8a2

[NO ISSUE][OTH] Remove Unused Variable

- user model changes: no

- storage format changes: no

- interface changes: no

Change-Id: Ie782fd0bc081f3daa5eb77d2db556a1effb5600e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2949

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

[NO ISSUE][STO] Ensure First Component ID is Initialized

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- Initialize the component id generator from the primary

index checkpoint, if exits, as soon as it is created.

- Ensure the first component id is passed to all indexes.

Change-Id: I246f9373f950e2f9a2c63f86746462e42a3f1c62

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2948

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[ASTERIXDB-2449][FUN] Incorrect NULL/MISSING handling by concat functions

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Infer function return type as unknownable if

its input list item can be NULL/MISSING

- Always return MISSING if there is a MISSING argument

Change-Id: Idc364b061f3e74bdc9d7715bbadedc957e9e8223

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2946

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

[NO ISSUE] asterix-docker profilization

Change-Id: I71eb349de45b4d424e171d82e1ae598235b04b48

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2945

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

    • -20
    • +25
    /asterixdb/asterix-docker/pom.xml
[NO ISSUE] Remove nonexistent source dependency

Change-Id: I0cf543092867a475d031860c7d6192cc005658d8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3245

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

[NO ISSUE] Remove nonexistent source dependency

Change-Id: I0cf543092867a475d031860c7d6192cc005658d8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3199

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

[NO ISSUE] Remove nonexistent source dependency

Change-Id: I0cf543092867a475d031860c7d6192cc005658d8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2944

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

[NO ISSUE] Put asterix-server components in folder

Change-Id: I6281454bcd6b3f4a6c0f14d0c5b1d082efdd0955

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2880

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3198

Tested-by: Michael Blow <mblow@apache.org>

  1. … 14 more files in changeset.
[NO ISSUE] Put asterix-server components in folder

Change-Id: I6281454bcd6b3f4a6c0f14d0c5b1d082efdd0955

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2880

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 14 more files in changeset.
[NO ISSUE][STO] Add API to Get Total Disk Usage

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- Add new API to get total disk usage of all

NC io devices.

Change-Id: Iba1538205dbadefdddd8b50be6373dbe7525d0ef

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2943

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Revert "[NO ISSUE][LOG] Include destination IP in thread name on push back"

This reverts commit 2498d28f0584d6ab002625417622be429f5c8131.

Change-Id: I221071a8a2851bde839e67d7f0380816c249366d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2941

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

[NO ISSUE][RT] Allow getting the value of a span duration

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Enable getting the duration for spans. This is useful for

delay calculations.

Change-Id: Icce0ac0a0d47a26655bf640f3f1a92506d87c30e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2940

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

[NO ISSUE][CLUS] Ensure Active Jobs Capacity is Released Only Once

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Ensure active jobs capacity is released only once.

- Warn if the cluster maximum capacity is exceeded.

Change-Id: Ia53c6918a68f7050bd8af482dbe8e161d1315844

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2938

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Michael Blow <mblow@apache.org>

[ASTERIXDB-2448][COMP] Fix NullPointerException in IntroduceSelectAccessMethodRule

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Fixed NPE when intersecting two indexes and predicate containing

between operator:

a) If index intersection plan is not possible then exit the rule

b) Rewrite between operator into ge/le pair instead of le/le pair

Change-Id: I34d4cde6adf3785feb6a8280fcdb964a5fb72803

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2937

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

[ASTERIXDB-2447][COMP] Parameter rewriting for if_missing, if_null functions

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Introduce cast operation for parameters of if_missing, if_null functions

(as performed for parameters of the switch_case function)

- Fix union type handling in TypeResolverUtil.generalizeTypes()

Change-Id: I768d8236f5b0ccb9a850304ffedd3686d911702d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2935

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

[NO ISSUE][FUN] Add expected result for coalesce function test case

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Add expected result for coalesce function test case

so it does not share one with if_missing_or_null

Change-Id: I7d8135f702d10690823d88d8232d011505ee82f9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2934

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

[ASTERIXDB-2444][STO] Avoid Using System Clock in Storage

- user model changes: no

- storage format changes: yes

- interface changes: yes

Details:

- Replace the usage of system clock timestamps in LSM

index components file names by a sequencer. The next

sequence id to use is determined by checking the list

of existing components on disk. Note that due to a

rollback, an index checkpoint file may have last valid

component sequence which is greater than what is on disk.

This should not cause any issues since only components

that have a sequence greater than that appears in the

checkpoint will be deleted.

- Replace the usage of system clock timestamps in LSM

index components ids by a monotonically increasing

sequencer. The sequencer is initialized after restarts

by the last valid component id that appears in the

index checkpoint.

- Refactor the logic to generate flush/merge file names.

- Refactor the logic to check invalid components.

- Adapt test cases to new naming format.

Change-Id: I9dff8ffb38ce8064a199d03b070ed1f5b924b8a4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2927

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 9 more files in changeset.
[ASTERIXDB-2445][COMP] Nullable category handling in TypeComputeUtils.getResultType()

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- TypeComputeUtils.getResultType() currently makes the output type unknownable when

nullable category is passed. It should make it nullable instead

Change-Id: I0bb1e96c5a4f83a3d281811246da183d1a31e0f1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2932

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

[ASTERIXDB-2446][COMP] Improve error raised for a duplicate field name

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Improve error raised in case when there is a duplicate field name

in the SELECT clause. Throw compilation exception instead of

IllegalStateException at runtime

Change-Id: I2f442acd73b3018058909f65626694dfbb49acc6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2930

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>