Clone Tools
  • last updated 27 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[ASTERIXDB-2520][RT] Make Dataset Memory Reservation Idempotent

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Ensure thread-safety of metadata node takeover.

- To account for multiple failures of metadata node, make

dataset memory reservation idempotent.

Change-Id: I360226187e176ce3a0ccdcd7a1b611a01d906394

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3207

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 2 more files in changeset.
[NO ISSUE][OTH] Add API To Ensure Request Requirements

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- Refactor IRequestParameters as ICommonRequestParameters

and IRequestParameters to break cyclic dependencies.

- Add new API to ensure request can be scheduled for execution.

Change-Id: Ifb0513e0baf2b473006d4aa23040c86751fbb4fc

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3181

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Integration-Tests: Murtadha Hubail <mhubail@apache.org>

Tested-by: Murtadha Hubail <mhubail@apache.org>

  1. … 6 more files in changeset.
[NO ISSUE][OTH] Allow Cancellation of Requests Before Job Start

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Currently clients' requests are marked cancellable only after

their jobs start. This change allows cancellation of a request

after its job is compiled but before the job starts.

Change-Id: I8b7e811a16f37326d00d1b9d145819bdf6763fe0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3182

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +1
    ./resources/asx_errormsg/en.properties
  1. … 2 more files in changeset.
[ASTERIXDB-2518][RT] Introduce Request Tracker

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- Introduce IReceptionist to generate request references.

- Track all requests by uuid.

- Add more information to active_requests response.

- Replace StatementExecutorContext by RequestTracker.

- Deprecate StatementExecutorContext (to be removed)

- Allow extensions to set optional parameters in query service.

- Return forbidden when a cancellation is attempt on a request

that is not cancellable.

Change-Id: If08ecd91c55881743b2ecf40a628fa3d4166c554

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3163

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +48
    ./java/org/apache/asterix/common/api/IReceptionist.java
    • -0
    • +30
    ./java/org/apache/asterix/common/api/IReceptionistFactory.java
    • -0
    • +45
    ./java/org/apache/asterix/common/api/IRequestReference.java
    • -0
    • +71
    ./java/org/apache/asterix/common/api/IRequestTracker.java
    • -0
    • +84
    ./java/org/apache/asterix/common/api/RequestReference.java
  1. … 31 more files in changeset.
[NO ISSUE] Ensure serialVersionUIDs are present

Serializable classes should define a serialVersionUID; define those

where missing. For classes present in 0.9.4.x, initialize the value

to the auto-configured serialVersionUID on those versions of the class

Change-Id: Ia92d0c81f3681dce7ec0f643e2edf77e2ac4dbc5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3141

Tested-by: Michael Blow <mblow@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 49 more files in changeset.
[ASTERIXDB-2513][FUN] Add Active_Requests Function

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- Add a datasource function (active_requests) which

returns the active jobs that the user specified

client_context_id for.

- This function runs on a single NC and uses messaging

to get the currently running jobs from CC.

- Currently, the function returns the following fields:

-- clientContextId: the user specified clientContextId.

-- requestTime: a timestamp at which the request reference

was created.

-- jobId: optionally, the job id that belongs to this request.

- The function may be improved later to return all jobs and it may

return additional fields such as (request uuid, statement,

executionTime, elapsedTime, nodeAddress, userAgent, etc..)

- Add test case.

- Do not allow cancellation test to cancel queries with

clientContextId to avoid intermittent failures.

Change-Id: I95962742161ed18c4cf2e09c8541c8ad3b35356c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3136

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 24 more files in changeset.
Merge commit '1ff69c0' from 'stabilization-f69489' into 'master'

Change-Id: Ic8aa90be3ba3645ad22d21cc54abc83ab7008598

  1. … 24 more files in changeset.
[ASTERIXDB-2509][FUN] Implement CUME_DIST() and RATIO_TO_REPORT()

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Implement CUME_DIST() and RATIO_TO_REPORT() window functions

- Add internal WIN_PARTITION_LENGTH() window function

- Fixed typos in javadocs

Change-Id: I921d3e0488b3ca277cffed3b99a9a8e770ac9a24

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3113

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

    • -0
    • +2
    ./resources/asx_errormsg/en.properties
  1. … 29 more files in changeset.
[ASTERIXDB-2506][STO] Checkpoint Datasets

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- Add new config to specify dataset checkpoint interval

and default its value to 10 minutes.

- Flush datasets which were not flushed for the dataset

checkpoint interval.

- Run dataset checkpoint logic as part of the CheckpointThread.

- Improve dataset async flush API to accept any LSM index predicate.

- Cleanup exception handling in CheckpoinThread.

- Add test case for dataset checkpoint.

Change-Id: I38a73a43a4b1b7d3a8ac79dd579ed4ef8c9c6a9b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3106

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 14 more files in changeset.
[NO ISSUE] Fix typos

- user model changes: no

- storage format changes: no

- interface changes: no

Change-Id: I10bb0ec96b8a12819f5b2fe5fd8bd698d326a551

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3102

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

    • -1
    • +1
    ./resources/asx_errormsg/en.properties
  1. … 2 more files in changeset.
[ASTERIXDB-2488][COMP] Support aggregate window functions

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Implement aggregate window functions:

agg_func() OVER (frame_var AS)? (PARTITION BY ... ORDER BY ... frame_spec)

- Where agg_func is a SQL/SQL++ aggregate function

- Fix percent_rank() to always return 0 for the first tuple

- Fix ntile() to handle NULL argument

- Log query after each rewrite rule in SqlppQueryRewriter

- Implement toString() for ADayTimeDuration, fix it for AYearMonthDuration

- Add seek() method to RunFileReader

Change-Id: If0f71118a04c2dbd3462070673d52e67f076b7e1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3049

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

    • -0
    • +3
    ./resources/asx_errormsg/en.properties
  1. … 853 more files in changeset.
[ASTERIXDB-2494][RT] Ensure Dataset Ref Counters Are Thread-safe

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Dataset/Index reference counters are modified concurrently by

multiple threads and should be thread-safe. Thread-safety issues

in these counters could result in leaving a dataset allocated

in memory even though it can be evicted.

Change-Id: I9328df660f463bd45bfc003b1e44c9df2702cc90

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3089

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -6
    • +7
    ./java/org/apache/asterix/common/context/Info.java
[ASTERIXDB-2497][TX] Ensure Log Record Flush LSN is Set

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Set the flush LSN of log records before giving it

to the log flusher to avoid reading invalid value

if the flush is completed before setting the LSN.

- Ensure log record LSN is thread-safe.

- Warn in case of a flush with LSN = 0.

Change-Id: Ifc605c2d794339a3dc5004b462eca50ec103c717

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3087

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 1 more file in changeset.
[ASTERIXDB-2497][TX] Ensure Log Record Flush LSN is Set

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Set the flush LSN of log records before giving it

to the log flusher to avoid reading invalid value

if the flush is completed before setting the LSN.

- Ensure log record LSN is thread-safe.

- Warn in case of a flush with LSN = 0.

Change-Id: Ifc605c2d794339a3dc5004b462eca50ec103c717

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3087

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Till Westmann <tillw@apache.org>

(cherry picked from commit 440e3a56b6f7d888c07575a898b79fec8848e06a)

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3573

  1. … 1 more file in changeset.
[ASTERIXDB-2494][RT] Log Allocated Datasets on Allocation Failure

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Log the currently allocated datasets on dataset memory

allocation failure.

Change-Id: Iefabb9c080c8937a2d2fd928dadde4993d4dd226

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3070

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

[ASTERIXDB-2479][API] Add Network Diagnostics API

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Add a new API that shows the state of node to node

connections and their logical channels to help diagnose

networking issues.

- Add channel details to the waiting thread name in

NetworkOutputChannel.

- Add test case.

Change-Id: Id6fd5a96c56e7078d1404bebcbab8afe93ba8f64

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3025

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 14 more files in changeset.
[ASTERIXDB-2422][STO] Introduce compressed storage

- user model changes: yes

- Add new configuration in the with-caluse to enable compression

- Add new nc configuration in the config file

- storage format changes: yes

- Pages of the primary index can be compressed

- Add a companion file (Look Aside File) with the compressed index

- Allow optional values in the LocalResource

- Add compression information in Metadata.Dataset

- interface changes: yes

- ICCApplicationContext:

- Add getCompressionManager()

- IBufferCache:

- Add getCompressedFileWriter(int fileId)

- ICachedPageInternal:

- Add setCompressedPageOffset(long offset)

- Add getCompressedPageOffset()

- Add setCompressedPageSize(int size)

- Add getCompressedPageSize()

Details:

- Add new integration test for this patch

- Fix ASTERIXDB-2464

- Add ddl-with-clause type validator

Additional details in the design document:

https://cwiki.apache.org/confluence/display/ASTERIXDB/Compression+in+AsterixDB

Change-Id: Idde6f37c810c30c7f1a5ee8bcbc1e3e5f4410031

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2857

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

    • -4
    • +6
    ./resources/asx_errormsg/en.properties
  1. … 187 more files in changeset.
[ASTERIXDB-2474][COMP] Remove physical optimization static variable

- user model changes: no

- storage format changes: no

- interface changes: no

details:

This patch is to remove the physical optimization config static

variable used in IndexUtil to prevent side effects.

Change-Id: I83f58715e5d643fa5b52f6e0a0d1ad1cb839f895

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3020

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 12 more files in changeset.
[ASTERIXDB-2472][CONF] More Restrictive Configuration Options

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- To avoid accepting invalid configuration types, more restrictive

configurations types (UNSIGNED_INTEGER) and (POSITIVE_INTERGER)

were introduced.

- Update configuration options that accept INTEGER type to use

the more restrictive INTEGER types.

Change-Id: I7236089d93f90ebdb57a87a8efa0a64cf98df845

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3017

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 4 more files in changeset.
[ASTERIXDB-2286][COMP] Parallel sort changes p.2

- user model changes: no

- storage format changes: no

- interface changes: no

details:

This patch is change the way the SequentialMergeExchangePOperator

connector computes its local property instead of blindly

propagating the child's local property.

The patch also includes minor code clean-ups (moved some methods down)

Change-Id: Ie37e03b6fc6e55fc21f8324c0f09a7fa05b51769

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3005

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 8 more files in changeset.
[ASTERIXDB-2466][FUN] Implement window functions

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Implement window functions with SQL syntax:

function() OVER ((PARTITION BY expr1, expr2, ...)? ORDER BY exprA, exprB, ...)

- Where supported functions are:

ROW_NUMBER(), RANK(), DENSE_RANK(), PERCENT_RANK(), NTILE()

Change-Id: Ia28af8773cb11049c38d440c51b9c3cd1ed2bab4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3002

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

    • -0
    • +1
    ./resources/asx_errormsg/en.properties
  1. … 166 more files in changeset.
[ASTERIXDB-2467][STO] Fix locking protocol with bad tuples

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- The current locking protocol is not correct when there are bad tuples,

e.g., duplicates. When locking fails, the write must push partial frames

so that the previous acquired locks are released. However, if there are

bad tuples in the frame, after bad tuples are removed from the frame, the

state of pushed partial frames will be completely lost.

Change-Id: I86baa1f21ef7390777c8e2b0ce226095528e7f18

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3003

Reviewed-by: Xikui Wang <xkkwww@gmail.com>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

[ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

- user model changes: yes

- storage format changes: no

- interface changes: yes

details:

- new plan for sort operation which includes sampling and

replicating the stream of data to be sorted. Sort-merge connector

is removed from the plan. The sorted result now is in multiple partitions.

- new optimization rule to check whether full parallel sort is applicable.

- new Forward operator to read the replicated sort input stream and

to receive the ouput of the sampling.

- new sequential merge connector to merge a globally ordered result residing

in multiple partitions (in addition to the connector's partition computer).

- "asterix-lang-aql/pom.xml" is changed as a result of refactoring

code related to the range map handling.

- new private sampling function to generate the range map object

(local & global functions) & their type computers.

user model changes:

- new compiler property is added to enable and disable parallel sort.

interface changes:

- "ILogicalOperatorVisitor.java" includes Forward Operator.

- "ITuplePartitionComputer.java" includes initialize() to enable partitioner

to do some initialization. FieldRangePartitionComputerFactory uses it to

pick a range map.

- "ITuplePartitionComputerFactory.java". createPartitioner() is changed to

createPartitioner(IHyracksTaskContext hyracksTaskContext). Context is needed

for transferring the range map throught the context.

Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2393

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -0
    • +1
    ./resources/asx_errormsg/en.properties
  1. … 355 more files in changeset.
[NO ISSUE][COMP][RT] Enable multiway similarity joins

- Enable the FuzzyJoinRule that transforms

a nested-loop-similarity-join plan to a three-stage-similarity join.

- Modify FuzzyJoinRuleCollections.

- Add the ExtractCommonExpressionRule to extract common expressions

in the star-like multiple similarity join substitutions.

- Add the InlineSubplanInputForNestedTupleSourceRule to translate

the generated subplan from the similarity function-derived

substitution into join in case of nested schemas.

- Use similarity-jaccard-prefix to enable the pp+ join strategy.

- Use the right side to build the heavy hash join on

the prefix tokens from both sides.

- Add RemoveAssign/Variables/AggRules to iteratively remove unused

assign/vars once FuzzyJoinRule is applied in each round.

- Add three new optimization cases for multiway similarity joins.

- link-like multiway similarity joins

- star-like multiway similarity joins

- hybrid multiway similarity joins with the both styles of similarity joins.

- Add a check whether a similarity function is on

a select over an existing similarity join.

- Change the inverted-index-based similarity join to the three-stage-similarity join

due to efficiency considerations.

Change-Id: I8736f104905eeda763d39709e002c2b9629278cc

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1076

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Taewoo Kim <wangsaeu@gmail.com>

    • -0
    • +1
    ./resources/asx_errormsg/en.properties
  1. … 259 more files in changeset.
[NO ISSUE][REPL] Ensure Valid Component ID is Initialized On Replica Sync

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- Currently, the first time a replica is synchronized from master,

the valid component id on each replicated index's initial checkpoint

will be the initial value of a component id (-1). This value is

fixed when the the replica receives a flushed component from

the index. However, if the master fails before any component is

flushed to a replica and that replica is promoted to master, it

will start from an invalid component id. This change ensures that

the initial checkpoint of replicated indexes is initialized to

the maximum component id that appears on master. This will ensure

that if the replica is promoted, it will at least start from

a component that wasn't previously used on master.

- Replace assertion of component ids validation by illegal state.

Change-Id: I85395ad823a630725c4cab4bead1c61546dc61ae

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2973

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 6 more files in changeset.
[ASTERIXDB-2444][STO] Avoid Using System Clock in Checkpoints

- user model changes: no

- storage format changes: yes

- interface changes: yes

Details:

- Replace the usage of system clock timestamps in checkpoints

by a sequencer.

- Update Asterix/Hyracks storage version to reflect the recent

changes in storage.

- This change is expected to break storage backward compatibility.

Change-Id: Idc061e6eaccfb308b29a5a263b77a0a849694d4f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2961

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

  1. … 4 more files in changeset.
[NO ISSUE][TX] Create New Log File Before Deleting Old Files

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Ensure next log file is created after a sharp check

point before deleting old files. This is to prevent

the case if a crash happens right after deleting the

old files but before creating the new one, then the

next time the system starts up, it will start with

log file id 0 which is wrong.

- Log the details of latest index checkpoint when the

low watermark of the new checkpoint is less than the

low watermakr of the latest checkpoint.

Change-Id: I4817f697b43daff55726909ab074ec30a1c224ce

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2959

Reviewed-by: Michael Blow <mblow@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 3 more files in changeset.
[NO ISSUE][OTH] Remove Unused Variable

- user model changes: no

- storage format changes: no

- interface changes: no

Change-Id: Ie782fd0bc081f3daa5eb77d2db556a1effb5600e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2949

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

[NO ISSUE][STO] Ensure First Component ID is Initialized

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- Initialize the component id generator from the primary

index checkpoint, if exits, as soon as it is created.

- Ensure the first component id is passed to all indexes.

Change-Id: I246f9373f950e2f9a2c63f86746462e42a3f1c62

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2948

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 8 more files in changeset.
[ASTERIXDB-2444][STO] Avoid Using System Clock in Storage

- user model changes: no

- storage format changes: yes

- interface changes: yes

Details:

- Replace the usage of system clock timestamps in LSM

index components file names by a sequencer. The next

sequence id to use is determined by checking the list

of existing components on disk. Note that due to a

rollback, an index checkpoint file may have last valid

component sequence which is greater than what is on disk.

This should not cause any issues since only components

that have a sequence greater than that appears in the

checkpoint will be deleted.

- Replace the usage of system clock timestamps in LSM

index components ids by a monotonically increasing

sequencer. The sequencer is initialized after restarts

by the last valid component id that appears in the

index checkpoint.

- Refactor the logic to generate flush/merge file names.

- Refactor the logic to check invalid components.

- Adapt test cases to new naming format.

Change-Id: I9dff8ffb38ce8064a199d03b070ed1f5b924b8a4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2927

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 19 more files in changeset.