Clone Tools
  • last updated 27 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
ASTERIXDB-1377: Prevent Operations from Exiting Components Multiple Times

Change-Id: I5907b1b4c76ce48366f5447f2940f3561c474cfe

Reviewed-on: https://asterix-gerrit.ics.uci.edu/769

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 9 more files in changeset.
Move Hyracks to subfolder

  1. … 4413 more files in changeset.
Fixed Bugs Caused by Disabling Installer Tests

Change-Id: Ic77f4ee76a77f4b49cac6835bbb976eb9dc92d3a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/736

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 2 more files in changeset.
ASTERIXDB-1337: Dataset Memory Management on Multi-Partition NC

As sugggested in ASTERIXDB-1337, this change is to maintain a per-

partition MultitenantVirtualBufferCache budget, as opposed to sharing

the budget across the dataset.

Change-Id: Ie554ab6d9b0d01ed149ab0729d20cdac3c969ec1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/711

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 5 more files in changeset.
Fix Double-Checked Locking (Coverity)

Fix CIDs 68208,68209,68210 - eliminate problematic double-checked locking

Change-Id: I3383035b5e8e0aa3cfd54f803f12d56090f006cf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/667

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 2 more files in changeset.
More Coverity Fixes

- CID 68210 (#1 of 1): Check of thread-shared field evades lock

acquisition (LOCK_EVASION)

6. thread1_overwrites_value_in_field: Thread1 sets index to a new

value. Now the two threads have an inconsistent view of index and

updates to fields of index or fields correlated with index may be

lost.

- CID 68209 (#1 of 1): Check of thread-shared field evades lock

acquisition (LOCK_EVASION)

6. thread1_overwrites_value_in_field: Thread1 sets index to a new

value. Now the two threads have an inconsistent view of index and

updates to fields of index or fields correlated with index may be

lost.

- CID 68208 (#1 of 1): Check of thread-shared field evades lock

acquisition (LOCK_EVASION)

6. thread1_overwrites_value_in_field: Thread1 sets index to a new

value. Now the two threads have an inconsistent view of index and

updates to fields of index or fields correlated with index may be

lost.

- CID 68435 (#1 of 1): UR: Uninitialized read of field in constructor

(FB.UR_UNINIT_READ)

1. defect: Uninitialized read of btreeCmp.

- CID 68416 (#1 of 1): UL: Lock not released on all paths

(FB.UL_UNRELEASED_LOCK_EXCEPTION_PATH)

1. defect:

org.apache.hyracks.storage.am.lsm.invertedindex.inmemory.PartitionedIn

MemoryInvertedIndex.updatePartitionIndexes(short) does not release

lock on all exception paths.

- CID 68415 (#1 of 1): UL: Lock not released on all paths

(FB.UL_UNRELEASED_LOCK)

1. defect:

org.apache.hyracks.storage.common.buffercache.BufferCache.sweepAndFlus

h(int, boolean) does not release lock on all paths.

- CID 68414 (#1 of 1): UL: Lock not released on all paths

(FB.UL_UNRELEASED_LOCK)

1. defect:

org.apache.hyracks.storage.common.buffercache.BufferCache.dumpState()

does not release lock on all paths.

- CID 68412 (#1 of 1): UL: Lock not released on all paths

(FB.UL_UNRELEASED_LOCK)

1. defect:

org.apache.hyracks.storage.am.lsm.common.impls.VirtualBufferCache.dele

teFile(int, boolean) does not release lock on all paths.

- CID 68337 (#1 of 1): RV: Bad use of return value

(FB.RV_ABSOLUTE_VALUE_OF_RANDOM_INT)

1. defect: Bad attempt to compute absolute value of signed random

integer.

- CID 68336 (#1 of 1): RV: Bad use of return value

(FB.RV_ABSOLUTE_VALUE_OF_RANDOM_INT)

1. defect: Bad attempt to compute absolute value of signed random

integer.

- CID 68335 (#1 of 1): RV: Bad use of return value

(FB.RV_ABSOLUTE_VALUE_OF_RANDOM_INT)

1. defect: Bad attempt to compute absolute value of signed random

integer.

- CID 68334 (#1 of 1): RV: Bad use of return value

(FB.RV_ABSOLUTE_VALUE_OF_RANDOM_INT)

1. defect: Bad attempt to compute absolute value of signed random

integer.

- CID 68259 (#1 of 1): Result is not floating-point

(UNINTENDED_INTEGER_DIVISION)

integer_division: Dividing integer expressions 2147483647 and 2, and

then converting the integer quotient to type double. Any remainder,

or fractional part of the quotient, is ignored.

Change-Id: I0f63f7fb75b1ac8ffc1fc1d70e66bb19b788fe8e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/661

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 15 more files in changeset.
Remove Trailing White Spaces

This change removes all trailing white spaces in

java files in hyracks

Change-Id: I1c2ea4ff1190a965a857c0c3760cbebdf21860b3

Reviewed-on: https://asterix-gerrit.ics.uci.edu/645

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 653 more files in changeset.
remove end-of-line whitespace

Change-Id: If3542874d7f3a536a6fe524aba2fffaf6447a614

Reviewed-on: https://asterix-gerrit.ics.uci.edu/600

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 252 more files in changeset.
Add Support for Upsert Operation

This change adds support for upsert operations. it includes

creating a primary and secondary upsert operators in addition

to adding a new function "before" to the index operation call

back to correctly perform locking for the upsert operation.

Change-Id: I2705f43b6e6d187ee29b9ba5a7946d422990022a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/476

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 54 more files in changeset.
ASTERIXDB-1273: Get file references from components to be merged

This change includes the following:

- Get file references from components to be merged instead of file map manager.

- Fix an issue with passing inverted index files to replication job.

Change-Id: I7d0f06c002db86a2a34e6393d06c49fb528c22b7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/596

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 4 more files in changeset.
Reduce Object Creation in Index Operations

Removed proportioanl object creation in btrees and rtrees during insert and

search operations. Instead, we re-use objects in OpContexts.

Change-Id: I7fab280372951522db02f8c2ff1d7d5b15529cd7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/245

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 20 more files in changeset.
Make Index Path Relative to NC IO Devices

This change includes the following:

- Store relative index path in persisted local resource as resource name.

- Fetch index absolute path based on FileSplit IO device and index relative path.

This change will enable a node to perform operations on an index that was created

on another node by finding the absolve path based on the node IO deviec and

the index relative path. This is needed in case a node leaves the cluster and

a replica node is assigned to take over and recover the index partitions

that belonged to the failed node.

Change-Id: I09fabd097b8a995ca959fe60286acdb43edfcfa1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/563

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 19 more files in changeset.
Make LSM bulkload append-only and write-once.

Allows for usage of LSM indexes with underlying storage that is append-only.

This also results in a small improvement for LSM component bulk load speed.

- Tree metadata (filters, etc) now lie at the back of the tree file in

append-only mode.

-- Note that you should *not* ever give the append-only flag on bulk-load,

if the tree is ever to be modified in place.

- Append-only operations bypass the buffer cache for writes, but utilize

the buffer cache for memory allocation and reads.

- Addresses ASTERIXDB-1059

Change-Id: I80fb891b5310252143854a336b591bf3f8cd4ba7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/255

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 92 more files in changeset.
ASTERIXDB-1058: Lazy LSM memory components allocation

Change-Id: I476e756f8d71260ea614c8c072fc9503053866c9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/405

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

  1. … 10 more files in changeset.
Moved LSMOperationType to LSM API package

Change-Id: Ib6f0b7373388fc88605188e5a8089bd183d23af1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/447

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 23 more files in changeset.
ASTERIXDB-1053: change IndexLifecycleManager API to use resource name

Change-Id: I2e8e974fc2f746959639ce94351f8e419a7f9093

Reviewed-on: https://asterix-gerrit.ics.uci.edu/343

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 15 more files in changeset.
ASTERIXDB-139: Add temp workspace files deletion to IOManager

This change includes the following:

- Add a method to delete temp workspace files (WAF)

- Expose LSMComponents files suffixes to Asterix

Change-Id: I760074764755e7aee100ff33c14b13bf4b29ec2e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/337

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 22 more files in changeset.
Change license headers

Change-Id: I98b18f24a20dcd8dc75e828e47fb0ab88179a5be

Reviewed-on: https://asterix-gerrit.ics.uci.edu/386

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 2055 more files in changeset.
Change folder structure for Java repackage

Change only the folders, not the files, for our package name change.

This will break the build, and needs to be followed by a change to

the package name in all of the source files. However performing

the folder move and file change in two steps lets Git understand

that the files are the same, and lets us track revisions across

those files.

Change-Id: I08aff75e25ac7c6298c32cf3402febbc4a318c2a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/307

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Chris Hillery <ceej@lambda.nu>

  1. … 3879 more files in changeset.
Introducing data replication API to LSM indexes

Change-Id: I80565fc9d74e30440d2df5917911904ba8f33c25

Reviewed-on: https://asterix-gerrit.ics.uci.edu/322

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 34 more files in changeset.
Change Java package from edu.uci.ics to org.apache

Change-Id: I99172d856e88954b00cf7cfb24d33bb400f53994

Reviewed-on: https://asterix-gerrit.ics.uci.edu/308

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 2019 more files in changeset.
1. Fix the "writerCount!=0 during component flushing" issue 2. Fix the duplicate LSM disk component file name issue by avoiding duplicate timestamps for different components.

Note that this change includes https://asterix-gerrit.ics.uci.edu/#/c/268/.

Change-Id: I805eab33603f52e19a1b76f1c315f9b75b6e3c03

Reviewed-on: https://asterix-gerrit.ics.uci.edu/278

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

  1. … 17 more files in changeset.
Add a flag for LSM-based indices to indicate whether force pages to disk devices during flush and merge.

Change-Id: I988716c03cffe30b008e144d3a478ee25e367212

Reviewed-on: https://asterix-gerrit.ics.uci.edu/240

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

  1. … 69 more files in changeset.
avoid duplication of Pointable code in SerializerDeserializer

Change-Id: Ia98985fc994e48d7d6a37dfaade0178b6644d836

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/221

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 44 more files in changeset.
- Fixed Type Casting issue - Reorganized duplicated internal class in the DelimitedDataParser and DelimitedDataParserFactory - Prevented a user from creating an inverted index on a dataset with a variable-length PK

Change-Id: Ic5606501223b8d860b49a258ff49afacd7d76b9a

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/191

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <westmann@gmail.com>

  1. … 65 more files in changeset.
Changes to allow having the no-merge policy as an option in asterix.

Change-Id: I573b6a09185d51df1ec115edc38a89bd029574d5

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/107

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 18 more files in changeset.
Added LSM component-level filters for all indexes.

Change-Id: I898cf885c9f88feae85c99799a00fd8ec036efea

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/81

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 116 more files in changeset.
Adding external indexes

In Hyracks side, this change include the following:

1. The addition of three indexes:

a) external b-tree index

b) external r-tree index

c) external b-tree with buddy b-tree index

2. creating an additional logical operator in algebricks for performing lookup operations over external data and modify the different visitors to work with this operator

3. Added copyright header to all new files

Change-Id: Iecfbd86f06aff3caaf3a9652b63420666745ebb9

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/69

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Zachary Heilbron <zheilbron@gmail.com>

Reviewed-by: Sattam Alsubaiee <salsubaiee@gmail.com>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

  1. … 55 more files in changeset.
Various bug and performance fixes for the lsm indexes.

  1. … 2 more files in changeset.
Fix a bug related to merges in LSMRTree