Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[ASTERIXDB-2738][STO] Create Mask File Before Merge Operations

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Before starting a merge operation, create a mask file

(.mask_C_startSeq_endSeq) for the merged component to

indicate that this component isn't valid yet.

- On the merge operation successful completion, delete the

merged component mask file.

- In the case of any unexpected failure during the merge

operation, all files of the failed merged component will

be deleted on node startup/shutdown, including the mask

file.

- Halt on any IO opeartion failure.

- Add a test case that ensures only masked merged components

are deleted but not the original components that were

supposed to be merged.

Change-Id: I476dd3be5e75468e83044b3aaf0f6c2d8beadf1c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6425

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Luo Chen <cluo8@uci.edu>

    • -0
    • +53
    ./PersistentLocalResourceRepositoryTest.java
  1. … 6 more files in changeset.
[NO ISSUE] Make MetadataProvider extensible

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Allow product extensions to provide custom

implementation of MetadataProvider

- Modify 'async-deferred' testcases to

request 'Clean-JSON' output

Change-Id: I485073e6a9ec7e36083da6f50a8df63a37b2668b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4224

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

    • -1
    • +1
    ./IndexDropOperatorNodePushableTest.java
  1. … 35 more files in changeset.
[NO ISSUE] Support multipart dataverse names

- user model changes: yes

- storage format changes: no

- interface changes: yes

Details:

- Support dataverse names that consist of multiple parts

- Introduce DataverseName class and adopt it throughout the system

- Add testcases for multipart dataverse names

Change-Id: I1313b2dc0e8df6a9b0ded48dea122afc3ba5dbe7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4004

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

    • -2
    • +3
    ./IndexDropOperatorNodePushableTest.java
  1. … 284 more files in changeset.
[NO ISSUE][STO] Ensure Invalid Merged Components Are Deleted

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- When cleaning up an index's invalid components, check the

components end sequence in addition to the start sequence

to ensure invalid merged components with are cleaned too.

- Add test case.

Change-Id: I80d72b1b614718e2e283bc72a874c140d178d1e0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3331

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -13
    • +17
    ./PersistentLocalResourceRepositoryTest.java
  1. … 1 more file in changeset.
[ASTERIXDB-2444][STO] Avoid Using System Clock in Storage

- user model changes: no

- storage format changes: yes

- interface changes: yes

Details:

- Replace the usage of system clock timestamps in LSM

index components file names by a sequencer. The next

sequence id to use is determined by checking the list

of existing components on disk. Note that due to a

rollback, an index checkpoint file may have last valid

component sequence which is greater than what is on disk.

This should not cause any issues since only components

that have a sequence greater than that appears in the

checkpoint will be deleted.

- Replace the usage of system clock timestamps in LSM

index components ids by a monotonically increasing

sequencer. The sequencer is initialized after restarts

by the last valid component id that appears in the

index checkpoint.

- Refactor the logic to generate flush/merge file names.

- Refactor the logic to check invalid components.

- Adapt test cases to new naming format.

Change-Id: I9dff8ffb38ce8064a199d03b070ed1f5b924b8a4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2927

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -17
    • +8
    ./PersistentLocalResourceRepositoryTest.java
  1. … 23 more files in changeset.
[NO ISSUE][STO] Delete Corrupted Resources Metadata Files

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Use a mask file to determiner whether a resource metadata

file was completely written to disk or not.

- Delete corrupted resources metadata files that were

not completely written to disk on NC startup/shutdown.

- Add test case.

Change-Id: Ib205453a6620734930b9dde7652277d2c5a1d6ed

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2763

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -0
    • +22
    ./PersistentLocalResourceRepositoryTest.java
  1. … 3 more files in changeset.
[NO ISSUE][STO] Add consistency to flush lifecycle

- user model changes: no

- storage format changes: yes

- renamed AbstractLSMIOOperationCallbackFactory

to LSMIOOperationCallbackFactory

- useless classes have been removed.

- LSMBTreeIOOperationCallbackFactory

- LSMBTreeWithBuddyIOOperationCallbackFactory

- LSMInvertedIndexIOOperationCallbackFactory

- LSMRTreeIOOperationCallbackFactory

- interface changes: yes

Details:

- Previously, flushes have different lifecycle depending

on the memory component state

- not allocated

- allocated

- modified

- In certain cases, flush operations are skipped alltogether

- IO Operation callbacks became complicated and difficult

to maintain since calls are done differently in different

cases.

- In certain cases, afterFinalize is called on the IO

Operation callbacks even if beforeOperation was never

called.

- In this change, flushes go through the same lifecycle

events regardless of the state of the memory component.

- In addition, primary and secondary memory components

would reside in different virtual buffer caches due

to skipped flushes, or due to having the secondary

index created when the primary index's memory component

is residing on the virtual buffer cache with index !=0.

- Moreover, when flushes are lagging and all memory

components are being flushed, search operations assumes

the oldest of the memory component is the newest and

produces incorrect results.

- In addition, in case of a failed flush of a component,

the IO scheduler would skip it and flush the next

component. This would produce a bad state on disk.

- In this change, a failed flush can be retried. otherwise,

all future flushes of the component fail due to the failure

of the previously failed flush.

- Previously, when a component fails to modify an index due

to flush failures, it assumes disk is full.

- With this change, the modification failure reports the

original cause of the failed flush.

Change-Id: I29f7992ec6c0f71c5b63d45800b2fb590d651e4b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2584

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Murtadha Hubail <mhubail@apache.org>

  1. … 161 more files in changeset.
[ASTERIXDB-2316][STO] Fix Merging Components For Full Merge

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Pass correct merging components in case of

full merge.

- Add full merge test case.

Change-Id: I17890a1f85f569e0d3dc5eba046f61c91571c9c1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2471

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Luo Chen <cluo8@uci.edu>

    • -0
    • +73
    ./LsmIndexLifecycleTest.java
  1. … 1 more file in changeset.
[ASTERIXDB-2204][STO] Fix implementations and usages of IIndexCursor

- user model changes: no

- storage format changes: no

- interface changes: yes

- IIndexCursor.close() is now idempotent and can be called on

a closed cursor.

- IIndexCursor.destroy() is now idempotent and can be called

on a destroyed cursor.

- Add IIndexAccessor.destroy() letting the accessor know it is

safe to destroy its reusable cursors and operation contexts.

- Add IIndexOperationContext.destroy() letting the context

know that the user is done with it and allow it to release

resources

details:

- Previously, implementations of the IIndexCursor interface

didn't enforce the interface contract. This change enforces

the contract for all the implementations.

- With the enforcement of the contract, all the users of the

cursors are expected to follow and enforce the expected lifecycle.

- Test cases were added.

Change-Id: I98a7a8b931eb24dbe11bf2bdc61b754ca28ebdf9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2324

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 132 more files in changeset.
[ASTERIXDB-2256] Reformat sources using code format template

Change-Id: I4faa141c1a8c9700d5e9ac50b839acc9d1eede73

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2310

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

    • -15
    • +15
    ./IndexDropOperatorNodePushableTest.java
  1. … 983 more files in changeset.
[NO ISSUE][STO] Remove Storage Auto Migration Code

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- It was decided to not support auto storage migration

from old to newer version at this point in time. This

change removes the storage migration code and its test.

Change-Id: I6faa816ca71913f18ff6017e86c8a29253728572

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2305

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -103
    • +0
    ./MigrateStorageResourcesTaskTest.java
  1. … 7 more files in changeset.
[ASTERIXDB-2234][STO] Delete Invalid Components

- user model changes: no

- storage format changes: no

- interface changes: yes

- IIndexCheckpointManager:

(+) getValidComponentTimestamp

(+) getCheckpointCount

(-) advanceLowWatermark

Details:

- Delete any component with start timestamp

after the index checkpoint valid component

timestamp on NC startup/shutdown.

- Add test case for deleting invalid components.

Change-Id: Ib11782edd79c7ef0c8949cb08e863b0ec1687a87

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2270

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

    • -0
    • +70
    ./PersistentLocalResourceRepositoryTest.java
  1. … 5 more files in changeset.
[ASTERIXDB-2195][REPL] Clean Masked Files

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Clean invalid masked files before promoting

a partition or sending partition files list

to master.

- Let replica calculate component id instead

of sending it from master.

- Add tests for:

- Deleting masked component.

- Deleting masked file.

Change-Id: Ib0f0159159faf87b9f5fd2eca3956dd90633bcfa

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2268

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

    • -0
    • +99
    ./PersistentLocalResourceRepositoryTest.java
  1. … 8 more files in changeset.
[ASTERIXDB-1723] Delete managix

- user model changes: yes

- storage format changes: no

- interface changes: yes

Details:

- Delete managix installer method and all dependencies

- Remove xml-based configuration

Change-Id: I5c710b6ec870bedd49dd46b74ed96a3665670e9b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2155

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -13
    • +13
    ./IndexDropOperatorNodePushableTest.java
    • -12
    • +11
    ./MigrateStorageResourcesTaskTest.java
  1. … 1376 more files in changeset.
[ASTERIXDB-2177][STO] Use Fixed Storage Root Dir Name

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Eliminate the need to read the storage root dir name

from cluster properties and use a fixed name (storage).

- Eliminate the need to maintain root_metadata file.

Change-Id: I4e9772e9da10cff33f11353610788ba541a35571

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2182

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -2
    • +2
    ./MigrateStorageResourcesTaskTest.java
  1. … 14 more files in changeset.
[ASTERIXDB-2161][TEST] Add indexes to MultiPartitionTest

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- This change adds secondary indexes to multi partition

LSM indexes tests.

- This enables testing of specific concurrency scenarios

and ensuring properties stored in primary and secondary

indexes are consistent.

- In addition, the call for flushDataset in

DatasetLifecycleManager now throws an

IllegalStateException if the number of active

operations is not 0. Some tests used to call this

function when there are ongoing operations and that

is expected to never be the case in the actual system.

Change-Id: I5aea71a87f149b01f6c7310867fc15b5a340b93c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2173

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Luo Chen <cluo8@uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 19 more files in changeset.
[NO ISSUE][STO] Adapt Storage Structure To Rebalance

- user model changes: no

- storage format changes: no

- interface changes: yes

-- Added IResource#setPath to use for the resource

storage migration.

Details:

- Unify storage structure to support dataset rebalance:

Old format:

./storage/partition_#/dataverse/datasetName_idx_indexName

New format:

./storage/partition_#/dataverse/datasetName/rebalanaceNum/indexName

- Adapt recovery and replication to new storage structure.

- Add old structure -> new structure NC migration task.

- Add CompatibilityUtil to ensure NC can be upgraded during

NC startup.

- Centralize the logic for parsing file path to its components in

ResourceReference/DatasetResourceReference.

- Add storage structure migration test case.

- Add test case for recovery after rebalance.

Change-Id: I0f968b9f493bf5aa2d49f503afe21f0d438bb7f0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2181

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -0
    • +104
    ./MigrateStorageResourcesTaskTest.java
  1. … 40 more files in changeset.
[ASTERIXDB-2171][STO] Remove Temporary Datasets

- user model changes: no

- storage format changes: no

- interface changes: yes

- Removed all reference to temporary datasets.

Details:

- Remove temporary datasets grammar but reserve

<TEMPORARY> token.

- Remove special callbacks for temp datasets.

- Remove metadata garbage collector thread.

- Remove isTemp flag from datasets.

- Remove special storage path for temp datasets.

- Remove temp datasets test cases.

Change-Id: Ib50ea27e2ec9c114508cd4a109cf646e2f20dcb0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2172

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

    • -1
    • +1
    ./IndexDropOperatorNodePushableTest.java
  1. … 368 more files in changeset.
[ASTERIXDB-2169][STO][TX] Unblock modifications during full scan

- user model changes: no

- storage format changes: no

- interface changes: yes

- added ILSMHarness.replaceMemoryComponentsWithDiskComponents

details:

- During a long running query aka full scan, two things block

incoming modifications:

1) Memory component gets full, is flushed but can't be recycled

because of the search operation inside the component.

2) Read latches on the memory component not being released and

the memory component search cursor is not advancing.

The two cases are addressed in this change for the LSMBTree but

not yet addressed for other indexes.

The proposed solution for case (1) is to poll memory components

states every n records during the search operation. If a memory

component was found to have been flushed, its cursor is moved

to the corresponding disk component allowing the memory

component to be recycled.

The proposed solution for case (2) is to check memory component

cursor every n records. If the cursor has not advanced and the

component has writers, then the latches over the leaf page are

released, and the cursor re-do the operation entering from the

tree root.

- Added a test case.

- Added performance traces for enter and exit components.

Change-Id: I37ba52f6324ed1c5a78465c3a8cbcd351f1ed5bc

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2166

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Luo Chen <cluo8@uci.edu>

    • -14
    • +14
    ./IndexDropOperatorNodePushableTest.java
  1. … 49 more files in changeset.
[NO ISSUE][TX] Introduce Atomic Transactions

- user model changes: no

- storage format changes: no

- interface changes: yes

- Redesign and simplify ITransactionManager API

- Redesign and simplify ITransactionContext API

Details:

- Introduce atomic transactions. Unlike entity level transaction,

atomic transaction do not generate any entity commit logs and

may modify multiple primary indexes. Therefore, either all the

operations of an atomic transaction will be committed or nothing.

Atomic transactions are used by metadata transactions, while other

transactions still use entity level transactions.

- Add index resource id to AbstractOperationCallback.

- Refactor metadata index modification code.

- Remove unused class MutableResourceId

- Remove unused class FieldsHashValueGenerator

- Add test case for concurrent metadata transactions.

Change-Id: I13db1c15f8afbdaae608ff0a7468fe62bf1daccd

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2156

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 38 more files in changeset.
[ASTERIXDB-1871][ASTERIXDB-2095] Stop Consumer Thread on Deallocate

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Currently there is a chance that a MaterializingPipelinedPartition

is deallocated before the consuming thread starts (e.g. due to job

abort) and therefore the consuming thread will not be interrupted

leading to leaked threads and files. This change checks if partition

was deallocated before the consumer starts, then the consumer thread

cleans up any files then exits.

- Make TaskAttemptId non-final class to mock it.

- Add test case.

Change-Id: I18c9fb085c149f41a202fff83aa6ec3aaeba6a77

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2143

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -0
    • +90
    ./DeallocatableTest.java
  1. … 5 more files in changeset.
[ASTERIXDB-2130][TX] Prevent Dropping Index Pending Txn Completion

- user model changes: no

- storage format changes: no

- interface changes: yes

Added resource id to ITransactionOperationTracker

before/after txn operations.

Details:

- Currently, an index could be evicted/dropped while a transaction

waiting for its completion (commit/rollback). This change prevents

that by incrementing the reference counter of the indexes registered

in the transaction.

- Add test case.

Change-Id: If6a938cbb5a9c3b7f5cc59505c07ae45b3425223

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2132

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

    • -0
    • +59
    ./IndexDropOperatorNodePushableTest.java
  1. … 4 more files in changeset.
[ASTERIXDB-1954][STO][RT] Add Index Drop Options

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Add options to allow drop index to ignore

index does not exist and retry on index in-use.

- Add test case for new index drop options.

Change-Id: Id6f8fa52489bbe64d2f48c5c3d0a07be60f30b1b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2121

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

    • -0
    • +182
    ./IndexDropOperatorNodePushableTest.java
  1. … 12 more files in changeset.
[NO ISSUE][STO] Fix memory leaks in storage

- user model changes: no

- storage format changes: no

- interface changes: yes

- Added javadocs to:

-- IBufferCache

-- IExtraPageBlockHelper

- Moved IBufferCache.setPageDiskId -> ICachedPage.setDiskPageId

- Renamed:

-- IBufferCache.flushDirtyPage -> IBufferCache.flush

-- IBufferCache.getNumPages -> IBufferCache.getPageBudget

- Removed:

-- IBufferCache.adviseWontNeed [not used]

-- IBufferCache.tryPin [not used]

details:

- Previously, when adding a kv pair to the metadata of a memory

component, we add a new Pair item to the ArrayList. After

this change, we only update it if it exists.

- VirtualBufferCache used to leak pages when reclaiming pages

of a file after deletion. This has also been fixed.

- New tests for VirtualBufferCache added:

- Checks for memory budget after end of testDisjointPins

- Concurrent Users pinning pages concurrently

- Test for large pages and ensuring allocated large

pages are accounted for through removal of cached

free pages.

Change-Id: I4ae9736c9b5fdba5795245bdf835c023e3f73b15

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2115

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 39 more files in changeset.
[ASTERIXDB-1995][STO] Abort write txn when index cannot be flushed

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Fix LSM memory component state transition on flush/merge failure

- When index cannot be flushed, abort waiting threads

- Prevent NPE in MateralizerTaskState when file creation fails

- Check parent dirs creation for index metadata file

Change-Id: I28592c30c788f4a6f44db8b47a84bc77f6b3f8f3

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1896

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -0
    • +173
    ./DiskIsFullTest.java
  1. … 10 more files in changeset.