hyracks-storage-am-lsm-common

Clone Tools
  • last updated 21 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Move merged files

  1. … 1 more file in changeset.
Exclude Temporary Indexes From Replication

- Exclude non-durable LSM indexes from replication.

- Stop heartbeat task after NC has stopped to avoid false failures detection.

Change-Id: Icce91a203e04cb068a7a5aa541720bbd0289eacb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/777

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 1 more file in changeset.
ASTERIXDB-1377: Prevent Operations from Exiting Components Multiple Times

Change-Id: I5907b1b4c76ce48366f5447f2940f3561c474cfe

Reviewed-on: https://asterix-gerrit.ics.uci.edu/769

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 6 more files in changeset.
Move Hyracks to subfolder

  1. … 4413 more files in changeset.
Merge branch 'release-0.2.17' into master

Change-Id: I8fcf41c009f9faf51bc8bccc0c3c7217b7769147

  1. … 60 more files in changeset.
Support Change Feeds and Ingestion of Records with MetaData

This change allows feeds to perform upserts and deletes

in order to perform replication of an external data source.

The change does so by performing the following:

1. The adapter produces [PK][Record]. (Record == null --> delete)

2. The insert is replaced by an upsert operator.

Change-Id: I3749349e2b9f1b03c8b310eb99d3f44d08be77df

Reviewed-on: https://asterix-gerrit.ics.uci.edu/620

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 26 more files in changeset.
ASTERIXDB-1337: Dataset Memory Management on Multi-Partition NC

As sugggested in ASTERIXDB-1337, this change is to maintain a per-

partition MultitenantVirtualBufferCache budget, as opposed to sharing

the budget across the dataset.

Change-Id: Ie554ab6d9b0d01ed149ab0729d20cdac3c969ec1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/711

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 6 more files in changeset.
More Coverity Fixes

- CID 68210 (#1 of 1): Check of thread-shared field evades lock

acquisition (LOCK_EVASION)

6. thread1_overwrites_value_in_field: Thread1 sets index to a new

value. Now the two threads have an inconsistent view of index and

updates to fields of index or fields correlated with index may be

lost.

- CID 68209 (#1 of 1): Check of thread-shared field evades lock

acquisition (LOCK_EVASION)

6. thread1_overwrites_value_in_field: Thread1 sets index to a new

value. Now the two threads have an inconsistent view of index and

updates to fields of index or fields correlated with index may be

lost.

- CID 68208 (#1 of 1): Check of thread-shared field evades lock

acquisition (LOCK_EVASION)

6. thread1_overwrites_value_in_field: Thread1 sets index to a new

value. Now the two threads have an inconsistent view of index and

updates to fields of index or fields correlated with index may be

lost.

- CID 68435 (#1 of 1): UR: Uninitialized read of field in constructor

(FB.UR_UNINIT_READ)

1. defect: Uninitialized read of btreeCmp.

- CID 68416 (#1 of 1): UL: Lock not released on all paths

(FB.UL_UNRELEASED_LOCK_EXCEPTION_PATH)

1. defect:

org.apache.hyracks.storage.am.lsm.invertedindex.inmemory.PartitionedIn

MemoryInvertedIndex.updatePartitionIndexes(short) does not release

lock on all exception paths.

- CID 68415 (#1 of 1): UL: Lock not released on all paths

(FB.UL_UNRELEASED_LOCK)

1. defect:

org.apache.hyracks.storage.common.buffercache.BufferCache.sweepAndFlus

h(int, boolean) does not release lock on all paths.

- CID 68414 (#1 of 1): UL: Lock not released on all paths

(FB.UL_UNRELEASED_LOCK)

1. defect:

org.apache.hyracks.storage.common.buffercache.BufferCache.dumpState()

does not release lock on all paths.

- CID 68412 (#1 of 1): UL: Lock not released on all paths

(FB.UL_UNRELEASED_LOCK)

1. defect:

org.apache.hyracks.storage.am.lsm.common.impls.VirtualBufferCache.dele

teFile(int, boolean) does not release lock on all paths.

- CID 68337 (#1 of 1): RV: Bad use of return value

(FB.RV_ABSOLUTE_VALUE_OF_RANDOM_INT)

1. defect: Bad attempt to compute absolute value of signed random

integer.

- CID 68336 (#1 of 1): RV: Bad use of return value

(FB.RV_ABSOLUTE_VALUE_OF_RANDOM_INT)

1. defect: Bad attempt to compute absolute value of signed random

integer.

- CID 68335 (#1 of 1): RV: Bad use of return value

(FB.RV_ABSOLUTE_VALUE_OF_RANDOM_INT)

1. defect: Bad attempt to compute absolute value of signed random

integer.

- CID 68334 (#1 of 1): RV: Bad use of return value

(FB.RV_ABSOLUTE_VALUE_OF_RANDOM_INT)

1. defect: Bad attempt to compute absolute value of signed random

integer.

- CID 68259 (#1 of 1): Result is not floating-point

(UNINTENDED_INTEGER_DIVISION)

integer_division: Dividing integer expressions 2147483647 and 2, and

then converting the integer quotient to type double. Any remainder,

or fractional part of the quotient, is ignored.

Change-Id: I0f63f7fb75b1ac8ffc1fc1d70e66bb19b788fe8e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/661

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 16 more files in changeset.
Remove Trailing White Spaces

This change removes all trailing white spaces in

java files in hyracks

Change-Id: I1c2ea4ff1190a965a857c0c3760cbebdf21860b3

Reviewed-on: https://asterix-gerrit.ics.uci.edu/645

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 651 more files in changeset.
[maven-release-plugin] prepare for next development iteration

  1. … 67 more files in changeset.
[maven-release-plugin] prepare release apache-asterixdb-hyracks-0.2.17-incubating-rc1

  1. … 67 more files in changeset.
remove end-of-line whitespace

Change-Id: If3542874d7f3a536a6fe524aba2fffaf6447a614

Reviewed-on: https://asterix-gerrit.ics.uci.edu/600

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 241 more files in changeset.
Revert "[maven-release-plugin] prepare for next development iteration"

This reverts commit ba7ed18443fb0789526f9d03af7aa34e953aaa6d.

  1. … 67 more files in changeset.
Revert "[maven-release-plugin] prepare release apache-asterixdb-hyracks-0.2.17-incubating-rc0"

This reverts commit 4112bf370fac4479b404ca59ef83b3bb9485a4c7.

  1. … 67 more files in changeset.
Add Support for Upsert Operation

This change adds support for upsert operations. it includes

creating a primary and secondary upsert operators in addition

to adding a new function "before" to the index operation call

back to correctly perform locking for the upsert operation.

Change-Id: I2705f43b6e6d187ee29b9ba5a7946d422990022a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/476

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 53 more files in changeset.
Add flush() to IFrameWriter

This change introduces flush() method in frame writers. It is

intended to be used for long running jobs (Feeds) to flush contents

of frames all the way to storage.

Change-Id: I85424bab7965b71aac709280af066e1655457aa3

Reviewed-on: https://asterix-gerrit.ics.uci.edu/584

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 86 more files in changeset.
[maven-release-plugin] prepare for next development iteration

  1. … 67 more files in changeset.
[maven-release-plugin] prepare release apache-asterixdb-hyracks-0.2.17-incubating-rc0

  1. … 67 more files in changeset.
Reduce Object Creation in Index Operations

Removed proportioanl object creation in btrees and rtrees during insert and

search operations. Instead, we re-use objects in OpContexts.

Change-Id: I7fab280372951522db02f8c2ff1d7d5b15529cd7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/245

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 25 more files in changeset.
Add Method to Get LSN Byte Offset in LSM Disk Components

The LSN byte offset is needed to support updating LSNs in remote replicas.

Change-Id: I2eb407e607157fe9c71a9cf22b04accc832be31b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/560

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 2 more files in changeset.
ASTERIXDB-1105 - LICENSE per submodule

This is a good part of what we need to have a specific set of

dependencies per binary assembly. The main key here is using the

maven remote resources plugin, and supplemental resources to

fill in info from the POM of dependencies where it is lacking.

For things that aren't described in Maven dependencies,

like Javascript files, they are added as LICENSE fragments

that get included in binary assemblies.

Change-Id: I2a6c4fb809b6fcad578007b8f48bf8bce179cd83

Reviewed-on: https://asterix-gerrit.ics.uci.edu/520

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 74 more files in changeset.
Make LSM bulkload append-only and write-once.

Allows for usage of LSM indexes with underlying storage that is append-only.

This also results in a small improvement for LSM component bulk load speed.

- Tree metadata (filters, etc) now lie at the back of the tree file in

append-only mode.

-- Note that you should *not* ever give the append-only flag on bulk-load,

if the tree is ever to be modified in place.

- Append-only operations bypass the buffer cache for writes, but utilize

the buffer cache for memory allocation and reads.

- Addresses ASTERIXDB-1059

Change-Id: I80fb891b5310252143854a336b591bf3f8cd4ba7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/255

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 86 more files in changeset.
ASTERIXDB-1058: ensure memory availablity before allocation

This change includes the following:

- IResourceMemeoryManager responsible for resources memory budget calculation.

- ResourceHeapBufferAllocator which checks with IResourceManager before allocation.

Change-Id: I846ff45402410835f5ed0afd2f701509abc95222

Reviewed-on: https://asterix-gerrit.ics.uci.edu/480

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 7 more files in changeset.
ASTERIXDB-1058: Lazy LSM memory components allocation

Change-Id: I476e756f8d71260ea614c8c072fc9503053866c9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/405

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

  1. … 8 more files in changeset.
ASTERIXDB-1136: Make Java compiler target configurable via profile

Change-Id: I10378f9d3fdec9b4951f53c9b370b7741668e537

Reviewed-on: https://asterix-gerrit.ics.uci.edu/273

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 57 more files in changeset.
Moved LSMOperationType to LSM API package

Change-Id: Ib6f0b7373388fc88605188e5a8089bd183d23af1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/447

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 9 more files in changeset.
ASTERIXDB-139: Add temp workspace files deletion to IOManager

This change includes the following:

- Add a method to delete temp workspace files (WAF)

- Expose LSMComponents files suffixes to Asterix

Change-Id: I760074764755e7aee100ff33c14b13bf4b29ec2e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/337

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 11 more files in changeset.
[maven-release-plugin] prepare for next development iteration

  1. … 67 more files in changeset.
[maven-release-plugin] prepare release fullstack-0.2.16-incubating

  1. … 67 more files in changeset.
Change license headers

Change-Id: I98b18f24a20dcd8dc75e828e47fb0ab88179a5be

Reviewed-on: https://asterix-gerrit.ics.uci.edu/386

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 2055 more files in changeset.