Clone Tools
  • last updated 24 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[ASTERIXDB-2541][STO] Introduce GreedyScheduler

- user model changes: yes.

Add new option: storage.io.scheduler (async/greedy)

- storage format changes: no.

- interface changes: yes.

Introduce IIndexCursorStats

Details:

- Introduce GreedyScheduler that always executes the merge

operation with the smallest number of remaining pages to minimize

the number of disk components

- Introduce IIndexCursorStats to collect the statistics of index scans.

This allows GreedyScheduler to know the remaning pages of merge

operations.

- Extend AbstractIoOperation so that GreedyScheduler can pause/resume

merge operations if needed.

Change-Id: I38fe394d1180d4e3f6796064c0e6c6630b6ad303

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3284

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Michael Blow <mblow@apache.org>

  1. … 72 more files in changeset.
[ASTERIXDB-2540][STO] Optimize Storage Disk I/O

- user model changes: yes. Add a new storage option:

storage.disk.force.bytes (default 16MB),

- storage format changes: no.

- interface changes: yes.

Introduced IPageWriteCallback to LSM indexes

Details:

- Bypass all queuing (from BufferCache and IOManager) for disk writes.

This queuing is unnecessary but destroys fairness among multiple

writers.

- Introduce IPageWriteCallback to control the behavior of disk page

writes. Currently, this interface is used to perform disk forces

regularly for each writer thread.

Change-Id: I1f618dc7c186623e860239b4d97640fe3528e75b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3285

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

    • -1
    • +1
    ./hyracks/storage/am/common/api/IPageManager.java
  1. … 139 more files in changeset.
[ASTERIXDB-2532][RT] per-operator profiling

Enables profiling in queries at the operator-level when the analyze

variable is set in a query.

Change-Id: Ie16f3901ae5b32920d8552d5fd1ec8bb6e2ec8ae

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3226

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 60 more files in changeset.
[NO ISSUE][RT] Add Thread-Based Stats Collector

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- Add infra to allow collecting thread-based stats during

runtime for any thread that belongs to a task.

- Collect number of pinned pages per thread and report it

in the TaskProfile.

- Aggregate pinned pages counters from all job tasks and

report it as diskIoCount in the metrics field in the json

response. The plan is to move this stats to the profile

field when it is introduced.

- Collecting pinned pages stats is currently enabled by

default for any job with IndexSearchOperatorNodePushable.

The plan is to allow enabling/disabling as part of the

job profiling change.

- Add test case for diskIoCount metric.

- Remove unused IndexSearchOperatorNodePushable constructor.

Change-Id: I44dfcedcadb3d0f48815b521e7d495e473b02e3d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3555

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 27 more files in changeset.
[NO ISSUE][RT] Improve PreclusteredGroupWriter

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Modified PreclusteredGroupWriter to only save group fields

from a last tuple in a frame instead of the whole frame

- move PermutingFrameTupleReference and PermutingTupleReference

from 'hyracks-storage-am-common' to 'hyracks-dataflow-common'

Change-Id: Ic75de2e6b64d0aacaf48096ecc9d47fc8e95c9cf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3351

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 32 more files in changeset.
Merge commit '153b2ce970' from 'stabilization-f69489'

Change-Id: I5b0d2756d839b0bc2e7c59723306921e698a7630

  1. … 6 more files in changeset.
[NO ISSUE][RT] Fix Secondary Indexes Bulkload/Upsert

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Prevent null values from being upserted into

secondary indexes.

- Use secondary index tuple filter when bulkloading

to prevent null/missing values from being inserted

into indexes of optional fields.

Change-Id: I7b847aece06a4387cc41389b0566d0c0dc98487d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3304

Reviewed-by: Luo Chen <cluo8@uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 10 more files in changeset.
[NO ISSUE] Apply / enforce java import order

The process-sources target will now sort imports as well as

format source code; the source-format job will likewise verify

import order in addition to source code format

Change-Id: I55d976c4df10d9919c6a25683be2a3e3304e65d9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3288

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

Reviewed-by: Till Westmann <tillw@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 623 more files in changeset.
[ASTERIXDB-2491][TXN] Recovery fixes

- user model changes: no

- storage format changes: yes[1]

- interface changes: no

Details:

- Change field offset type from integer16 to integer32

- Add recovery test

- Add version to log entries

[1] LogRecord format change with this patch; old LogRecord format can still

be processed, however old instances cannot read new log format

Change-Id: Iaf14b9a73a0239763bfeb0ce2d81cf952e6d72d3

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3065

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

    • -0
    • +59
    ./hyracks/storage/am/common/tuples/SimpleTupleReferenceV0.java
  1. … 23 more files in changeset.
[ASTERIXDB-2422][STO] Introduce compressed storage

- user model changes: yes

- Add new configuration in the with-caluse to enable compression

- Add new nc configuration in the config file

- storage format changes: yes

- Pages of the primary index can be compressed

- Add a companion file (Look Aside File) with the compressed index

- Allow optional values in the LocalResource

- Add compression information in Metadata.Dataset

- interface changes: yes

- ICCApplicationContext:

- Add getCompressionManager()

- IBufferCache:

- Add getCompressedFileWriter(int fileId)

- ICachedPageInternal:

- Add setCompressedPageOffset(long offset)

- Add getCompressedPageOffset()

- Add setCompressedPageSize(int size)

- Add getCompressedPageSize()

Details:

- Add new integration test for this patch

- Fix ASTERIXDB-2464

- Add ddl-with-clause type validator

Additional details in the design document:

https://cwiki.apache.org/confluence/display/ASTERIXDB/Compression+in+AsterixDB

Change-Id: Idde6f37c810c30c7f1a5ee8bcbc1e3e5f4410031

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2857

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 190 more files in changeset.
[ASTERIXDB-2444][STO] Avoid Using System Clock in Checkpoints

- user model changes: no

- storage format changes: yes

- interface changes: yes

Details:

- Replace the usage of system clock timestamps in checkpoints

by a sequencer.

- Update Asterix/Hyracks storage version to reflect the recent

changes in storage.

- This change is expected to break storage backward compatibility.

Change-Id: Idc061e6eaccfb308b29a5a263b77a0a849694d4f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2961

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

  1. … 6 more files in changeset.
[NO ISSUE][OTH] Remove Unused Imports

- user model changes: no

- storage format changes: no

- interface changes: no

Change-Id: Iafff39073d0fedaff74a26ef7e3260008a79ff0c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2893

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

  1. … 66 more files in changeset.
[NO ISSUE] Report all BufferCache write failures.

- user model changes: no

- storage format changes: no

- interface changes: yes

+ IPageWriteFailureCallback: used to notify async

IO caller when something goes wrong.

Details:

- Before this change, it is possible for failures to

be lost and for bulkload operations to not be

aware of failure to write some pages. This can be

dangerous.

- To avoid this, when sending a page to be written

a PageWriteFailureCallback is associated with the

page to notify the caller that a failure took place.

Change-Id: I97fd3dccff85dab84d644359be6f66b15ee708ef

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2787

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Luo Chen <cluo8@uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

    • -1
    • +10
    ./hyracks/storage/am/common/api/IPageManager.java
  1. … 44 more files in changeset.
[ASTERIXDB-1130][STO]: JSON serialization for persisted objects

- user model changes: no

- storage format changes:

This change replaces the use of Java serialization for persisted objects

such as dataset/index metadata, checkpoints, ect...

This will break backward compatibly with any existing AsterixDB instance.

However, the change is needed to enable future backward compatibility support

for persisted objects.

- interface changes:

IJsonSerializable: contains API to serialize a class as a JsonNode.

IPersistedResourceRegistry: contains a mapping between an IJsonSerializable

class and a unique type id. An IPersistedResourceRegistry is responsible

for generating the class identifier in the JSON output.

The class identifier will always contain the following attributes:

@type: a unique type id that identifies the object type.

@version: the version of the serialized class.

@class: the serialized class full name.

Any registered class with PersistedResourceRegistry must provide

a static fromJson(IPersistedResourceRegistry, JsonNode) method for

deserialization. This is ensured during the class registration process.

Change-Id: I5b103e06eab6627dbfe9d531caae1a3ac4b296da

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2752

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Integration-Tests: Murtadha Hubail <mhubail@apache.org>

Tested-by: Murtadha Hubail <mhubail@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 125 more files in changeset.
[NO ISSUE][STO] Misc Storage Fixes and Improvements

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- This change introduces some improvements to storage

operations.

- Local RecoveryManager is now extensible.

- Bulk loaders now call the IO callback similar to

Flushes, making them less special and creating a

unified lifecycle for adding an index component.

- As a result, The IndexCheckpointManager doesn't need

to have a special treatment for components loaded

through the bulk load operation.

- Component Id have been added to the index checkpoint

files.

- Cleanup for the code of local recovery for failed flush

operations.

- Ensure that after local recovery of flushes, primary

and secondary indexes have the same index for mutable

memory component.

- The use of WAIT logs to ensure in-flight flushes

are scheduled didn't work as expected. A new log type

WAIT_FOR_FLUSHES was introduced to acheive the expected

behavior.

- The local test framework was made Extensible to support

more use cases.

- Test cases were added for component ids in checkpoint files.

The following scenarios were covered:

- Primary and secondary both have values when a flush is

shceduled.

- Primary have values but not secondary when a flush is

scheduled.

- Primary is empty and an index is created through bulk

load.

- Primary has a single component and secondary is created

through bulk load.

- Primary has multiple components and secondary is created

through bulk load.

- Each primary opTracker now keeps a list of ongoing flushes.

- FlushDataset now waits only for flushes only and

not all io operations.

- Previously, we had many flushes scheduled on open datasets.

This was not detected but after this change, a failure

is thrown in such cases.

- Flush operations dont need to extend the comparable

interface anymore since they are FIFO per index.

Change-Id: If24c9baaac2b79e7d1acf47fa2601767388ce988

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2632

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 89 more files in changeset.
[ASTERIXDB-2344] LIMIT pushdown for primary index

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

-Implement LIMIT pushdown for primary index scan/lookup. In case of

a select operator, we also push the select condition to the priamry

search operator to ensure correctness.

Change-Id: I824fcad79995325e12a1a81d629160025294b915

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2541

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -1
    • +2
    ./hyracks/storage/am/common/api/ITupleFilter.java
  1. … 52 more files in changeset.
[NO ISSUE][STO] Add consistency to flush lifecycle

- user model changes: no

- storage format changes: yes

- renamed AbstractLSMIOOperationCallbackFactory

to LSMIOOperationCallbackFactory

- useless classes have been removed.

- LSMBTreeIOOperationCallbackFactory

- LSMBTreeWithBuddyIOOperationCallbackFactory

- LSMInvertedIndexIOOperationCallbackFactory

- LSMRTreeIOOperationCallbackFactory

- interface changes: yes

Details:

- Previously, flushes have different lifecycle depending

on the memory component state

- not allocated

- allocated

- modified

- In certain cases, flush operations are skipped alltogether

- IO Operation callbacks became complicated and difficult

to maintain since calls are done differently in different

cases.

- In certain cases, afterFinalize is called on the IO

Operation callbacks even if beforeOperation was never

called.

- In this change, flushes go through the same lifecycle

events regardless of the state of the memory component.

- In addition, primary and secondary memory components

would reside in different virtual buffer caches due

to skipped flushes, or due to having the secondary

index created when the primary index's memory component

is residing on the virtual buffer cache with index !=0.

- Moreover, when flushes are lagging and all memory

components are being flushed, search operations assumes

the oldest of the memory component is the newest and

produces incorrect results.

- In addition, in case of a failed flush of a component,

the IO scheduler would skip it and flush the next

component. This would produce a bad state on disk.

- In this change, a failed flush can be retried. otherwise,

all future flushes of the component fail due to the failure

of the previously failed flush.

- Previously, when a component fails to modify an index due

to flush failures, it assumes disk is full.

- With this change, the modification failure reports the

original cause of the failed flush.

Change-Id: I29f7992ec6c0f71c5b63d45800b2fb590d651e4b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2584

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Murtadha Hubail <mhubail@apache.org>

  1. … 161 more files in changeset.
[NO ISSUE][RT] Inherit Closeable in IFrameOperationCallback

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- Primary upsert operator uses an IFrameOperationCallback

to perform per frame operations. In this change,

IFrameOperationCallback extends Closeable and allow the

callback to release resources on the close of the pipeline.

Change-Id: Iaacd6538c27d5498868256616a793e0ebcec3729

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2536

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 4 more files in changeset.
[NO ISSUE] Incremental cleanup of deprecated exception ctors

Change-Id: I1e7c3655828fc6530cef83ea502a6cfbf41acddf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2533

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 238 more files in changeset.
[NO ISSUE][STO] Misc fixes of inverted index

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

This patch mainly improves the inverted index merge perf

in the following ways:

- Fix the bloom filter check on deleted btree

- Remove unnecessary btree search during inverted index range search

- Remove unnecessary page pins when merging inverted lists by

making the cursor remembering the last pinned page

- Fix close/destroy operations of range search cursor

Change-Id: I7bbf6fe2d332db0e0368a4ce0f1ab218ba6da81d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2514

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 12 more files in changeset.
[ASTERIXDB-1952][TX][IDX] Filter logs pt.2

- user model changes: no

- storage format changes: yes

- interface changes: yes

Details:

- Add a log type specifically for filters

- Only log change when filter actually widens

- Stop logging of index + filter tuple during modification

- Redo index and filter tuples separately via their logs

Change-Id: Ie9e7795d9c8c212e8610dcb9bb5d26ec9fbbee8a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1857

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

    • -0
    • +37
    ./hyracks/storage/am/common/api/IExtendedModificationOperationCallback.java
  1. … 42 more files in changeset.
[NO ISSUE][STO] Move the IO threads from BufferCache to IOManager

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Move the IO threads from BufferCache to IOManager to cover all

IO uses that go through the IOManager.

Change-Id: Ic02b456826ae7abc2619a7eec3f90b48717b0adb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2417

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 17 more files in changeset.
[ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained Inverted index search

- user-model changes: add text.searchmemory parameter

- storage format changes: no

- interface changes: IInvertedIndexSearcher, IInPlaceInvertedIndex,

IInvertedIndexAccessor, IInvertedListCursor

IObjectFactory, IPartitionedInvertedIndex,

IIndexAccessor

Details:

- Introduce text.searchmemory parameter in the configuration

to conduct budget-constrained inverted index search to prevent

a possible OOM exception

- Remove non-standard hyracks task context from the inverted-index-search

Change-Id: Ib2b2ef7c0b8c55ef66a5322be5d97ebbbf287bf5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2251

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 99 more files in changeset.
[NO ISSUE][RT] follow IFrameWriter protocol in SplitOperatorDescriptor

- user model changes: no

- storage format changes: no

- interface changes: no

details:

- Previously, the SplitOperatorDescriptor didn't follow the

IFrameWriter protocol in case of failure which lead to having

some open resources after the job.

- This caused so many failures in Cancellation tests.

- This change also increases the rate of cancellation during the

cancellation tests to ensure that similar problems are found.

Change-Id: I3166895589e1ab7355d689397f676f7da5c9809f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2399

Reviewed-by: Taewoo Kim <wangsaeu@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 11 more files in changeset.
[ASTERIXDB-1972][COMP][RT][TX] index-only plan

- user model changes: no

- storage format changes: no

- interface changes: IAccessMethod, ILSMIndexOperationContext,

IIndexAccessor

Details:

- Implement an index-only plan

- Add a SET option that disables the index-only plan

Change-Id: Ifd5c9ab1cf2e4bedb7d8db582441919875e74d51

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1866

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Taewoo Kim <wangsaeu@gmail.com>

  1. … 423 more files in changeset.
[ASTERIXDB-2204][STO] Fix implementations and usages of IIndexCursor

- user model changes: no

- storage format changes: no

- interface changes: yes

- IIndexCursor.close() is now idempotent and can be called on

a closed cursor.

- IIndexCursor.destroy() is now idempotent and can be called

on a destroyed cursor.

- Add IIndexAccessor.destroy() letting the accessor know it is

safe to destroy its reusable cursors and operation contexts.

- Add IIndexOperationContext.destroy() letting the context

know that the user is done with it and allow it to release

resources

details:

- Previously, implementations of the IIndexCursor interface

didn't enforce the interface contract. This change enforces

the contract for all the implementations.

- With the enforcement of the contract, all the users of the

cursors are expected to follow and enforce the expected lifecycle.

- Test cases were added.

Change-Id: I98a7a8b931eb24dbe11bf2bdc61b754ca28ebdf9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2324

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +158
    ./hyracks/storage/am/common/util/ResourceReleaseUtils.java
  1. … 126 more files in changeset.
[NO ISSUE][STO] Recover from failure in memory allocation callback

- user model changes: no

- storage format changes: no

- interface changes: no

details:

- Previously, if an exception is thrown in the

ILSMIOOperationCallback.allocated call, then the memory component

is allocated but the flag memoryComponentsAllocated is false.

- Any subsequent attempt to modify the index will try to allocate

the component but since it has already been allocated, it will fail

with the exception: File is already mapped.

- In this change, if an exception is thrown from the callback, then

the component is de-allocated before throwing the exception.

- Test is case is added.

Change-Id: I80e605461df18c7f6d7785cd7504ca3acb4f45b1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2336

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 12 more files in changeset.
[ASTERIXDB-2256] Reformat sources using code format template

Change-Id: I4faa141c1a8c9700d5e9ac50b839acc9d1eede73

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2310

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 970 more files in changeset.
[NO ISSUE][STO] Introduce ILSMIndexCursor

- user model changes: no

- storage format changes: no

- interface changes: yes

- introduce ILSMIndexCursor as a sub interface for IIndexCursor

- move getFilterMinTuple and getFilterMaxTuple to the

ILSMIndexCursor interface.

details:

- IIndexCursor.getFilterMinTuple and IIndexCursor.getFilterMaxTuple

don't make sense for any non LSM cursor and all the existing

implementations return null for those calls.

- In this change, we move those methods to a new interface for

LSM Index cursors.

Change-Id: I2fabe46e4c3a6370c48c2ccd9689fce507b37009

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2296

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -0
    • +36
    ./hyracks/storage/am/common/api/ILSMIndexCursor.java
  1. … 18 more files in changeset.
[NO ISSUE][OTR] Logging Cleanup

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- No pretty printing for replica status.

- Log ignored exception in debug level.

Change-Id: I7e4a37e9621ef5bdb27105b17e883800212e594b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2276

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 1 more file in changeset.