Clone
 

luochen01 <cluo8@uci.edu> in asterixdb

[ASTERIXDB-2453] Add Improved Constant Merge Policy

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- The current constant merge policy is unsuable because of its high

merge cost, i.e., O(N*N) where N is the number of flushes. This patch

replaces the previous constant merge policy with a more efficient policy

that still enforces a maximum number of components but greatly lowers

the merge cost.

- Extend AbstractLSMIndex with a method to return the total number of

flushes, based on the file name sequencer. This is required by the new

policy.

Change-Id: Ie5f83a4d5fdd3f036b823c906df1760f5110ae0a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2971

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[ASTERIXDB-2174] Use ListSet when clone partition property

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Use ListSet instead of HashSet when cloning partitioning properties

to ensure it has the consistent behavior as before.

Change-Id: Ib2b8c1715d75b387a73504f4709524d9ab9ce123

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2968

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

[ASTERIXDB-2452] Recompute type env after ListifyUnnestingFunctionRule

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

-Recompute type env of the parent operator after firing

ListifyUnnestingFunctionRule. Otherwise the type env of the parent operator

would still point to the old operator, which is not correct.

Change-Id: I97bf721cb7fc9f36ec14214677d5545ae4aa1d2a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2967

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

[ASTERIXDB-2174] Fix partitioning improvement of group by

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Fix the improvement of partitioning propagation of group by

introduced by https://asterix-gerrit.ics.uci.edu/#/c/2176/.

- The problem is that we cannot modify the partitioning property of the

input operator directly, since that property is owned by the input.

Otherwise, the property of the child operator would be modified even

before the group by operator, and this caused some tests regarding

common sub operator extration to fail (introducing REPLICATE).

Change-Id: I81a74ab1242772c790175414281e3f200537b386

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2966

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

[NO ISSUE] Misc Fixes for Point MBR RTree

- user-model changes: no

- storage format changes: no

- interface changes: no

Details:

1. Fix the MBR calculation of Point MBR RTrees when finalizing the

bulkload process.

2. Fix the nullFlagBytes of Point MBR Rtrees.

3. Add unit test cases to cover Point MBR RTrees.

Change-Id: Ice24112152a2d93c7d2316b7506e6d6e81f2df44

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2911

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

[ASTERIXDB-2125] Fix RTree Bulkload Exception

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- When bulkloading a rtree, it's possible that we saw the

NotImplementedException. It turns out this bug is caused by that we

forgot to reset the previous MBR when calculating the MBR for missing

fields.

Change-Id: I4267a7ce0b8621471e7a8504c448ba1f500e92db

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2864

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[ASTERIXDB-2429] Fix the upsert of primary key index

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Previously the primary key index is not properly maintained during

upsert. Since there is no secondary key in the primary key index, the

old value would always point to the primary key, which is always equal

to the new value. As a result, the primary key index is nevered

maintainined during upsert.

- This patch fixes this bug with two changes:

First, if there is a primary key index, we would perform upsert anyway

no matter whether old value == new value

Second, use a boolean variable to indicate whether the operation

is upsert or delete since for the primary key index, old value cannot

provide such information.

Change-Id: I925bd42ba67f70e94f5f5bc2d24151c8e2e20baf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2825

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 7 more files in changeset.
[NO ISSUE][RT] Eliminate HashMap lookup during comparision

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Eliminate the use of hash map to store type tags. Replace it with a

simple TypeTag[] to improve performance.

Change-Id: Ie5b8efdcb226a8818deaf57a6305a18107544feb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2655

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Taewoo Kim <wangsaeu@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[NO ISSUE][STO] Improve error handling of writing deleted file

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- When the bulkloader is aborted, the file would be deleted from the

buffer cache but there could still be unflushed pages in the write

queue. Instead of throwing exceptions, simply ignore the case when

writting to non-existing files (as before).

Change-Id: I3f693ef178e0ab68c2a28b0edc6ffeaaeca1c520

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2651

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

[ASTERIXDB-2367][STO] Various Fixes for BufferCache

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Ensure fileInfo is deleted from FileInfoMap when delete a file from

buffer cache

- When delete old components after merge, ensure components are first

purged to avoid sweep old pages

- When open/create/delete files from buffer cache, use file-level

synchronization to reduce blocking. Certain operations such as

createFile/deleteFile can take long time if the disk is busy.

Change-Id: I8a199d3e83592425ab5055bd12bde519e80deb13

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2604

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

[NO ISSUE][STO] Eliminate S Lock for Disk Components

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Eliminate S lock for tuples returned from disk components, since LSM

disk components only contain committed data and S lock is not needed to

prevent from reading uncommitted data.

Change-Id: Id6ec999b131cd6609d588966d7ae7788f429ab9d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2637

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

[ASTERIXDB-2344] LIMIT pushdown for primary index

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

-Implement LIMIT pushdown for primary index scan/lookup. In case of

a select operator, we also push the select condition to the priamry

search operator to ensure correctness.

Change-Id: I824fcad79995325e12a1a81d629160025294b915

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2541

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 39 more files in changeset.
[ASTERIXDB-2357] ADMParser Improvements

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- The current ADMParser heavily relies on string operations,

which results in a lot of objects being created. This patch optimize

this by directly operating on char[].

- Improved exception handling in ADMParser.

Change-Id: I106b58e79746b0a6f3d8b79473202653341a7009

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2573

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[NO ISSUE] Use Async Write Mode in ResultState

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Use Async write mode in ResultState to improve it's throughput.

- For concurrent reader, use the same file handle (since it's a

RandomAccessFile) for both read/write. Reference counting is used to

ensure the file is properly opened/closed.

Change-Id: Ia053f8e258759881583deb509425b585bc3c4bf2

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2560

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

[NO ISSUE] Increase Default Log Page Size

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- The current log page default size (128KB) is too small for high

throughput disk writes. Increase it to 4MB.

Change-Id: I83595e12abb531293a64eb12a1be3e15bafca463

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2553

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

[ASTERIXDB-2280][IDX] Fix Index on Optional Nested Field

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Fix the creation of indexes on optional nested field

by handling AUnionType properly.

- Fix the nullability check of primary keys.

Change-Id: If098ff4a45db3c4e8b65d098cbb0940c8b7ab845

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2477

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[ASTERIXDB-2334] Fix Range Predicate for Composite Key Search

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Currently during a primary btree search, we always use the range limit

of the first search key. However, this is incorrect for prefix range

search. This patch fixes this bug by using the correct range limit

during index search.

Change-Id: Ie2c432c78043913a9dcf9b8a2eb31db8c548d8d2

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2532

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Michael Blow <mblow@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

[ASTERIXDB-2339] Add a new inverted index merge cursor

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Implement a new inverted index merge cursor which uses two priority queues,

one for tokens and one for keys. For each token, we merge their inverted

lists using the key queue. After that, we fetch the next token and merge

their lists again. This reduces unnecessary token comparision a lot.

- Along this change, created a fast path for inverted index bulkloader.

Based on how the token+key pair is created, there is no need to copy

bulkloaded tuple and check whether it's a new token during merge.

Change-Id: I57d039cd7e08033884529a204bff9acffd96d9bb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2519

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 7 more files in changeset.
[NO ISSUE][STO] Misc fixes of inverted index

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

This patch mainly improves the inverted index merge perf

in the following ways:

- Fix the bloom filter check on deleted btree

- Remove unnecessary btree search during inverted index range search

- Remove unnecessary page pins when merging inverted lists by

making the cursor remembering the last pinned page

- Fix close/destroy operations of range search cursor

Change-Id: I7bbf6fe2d332db0e0368a4ce0f1ab218ba6da81d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2514

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

[ASTERIXDB-2304] Ensure Flush is Finished in FlushRecoveryTest

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Previous in LSMFlushRecoveryTest, it's possible during recovery

we check component ids before the flush is finished (since flush is

asynchronous), and thus causes intermittent failures. This patch fixes

this problem by waiting for active IOs before checking component ids.

- Fix the problem the override config options in TestNodeController not

working.

- Also fix [ASTERIXDB-2309] to ensure only indexes of a given partition

are flushed upon seeing a FLUSH record.

Change-Id: I1704c6606c7c7bef226ae31961c347c6ebb76c2a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2437

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[ASTERIXDB-2301][TX] Fix Abort of DELETE operation

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Fix the undo logic of the DELETE operation. Previously undo

of DELETE is implemented as re-inserting the old value. However,

if the deleted record is from the disk component, the old value could be

null. In this case, we simply need to physically deleted the anti-matter

key from the memory component, exactly the same as undo upsert.

- Add test cases for undo.

Change-Id: I5002d639399f024be8837da1c539101e6d62a159

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2432

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[ASTERIXDB-2299] Set log type properly during modifications

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Previously we have a deadlock-free protocol during normal ingestion

pipeline. When the try lock fails, we flush the frame partially so that

they can release locks, and log a WAIT record to wake up after that.

However, after logging the WAIT record, we didn't set the log type

back to UPDATE. This seriously degrades the ingestion performance

afterwords since all updates log records would become WAIT log records,

which require heavier logic to process upon the log is flushed to

disk.

Change-Id: Ibcf93072ca0833cb24ba6719796f58df56384c3b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2424

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

[ASTERIXDB-2188] Ensure recovery of component ids

- user model changes: no

- storage format changes: yes.

Flush log record format changes.

- interface changes: no

Details:

- Add flush component ids to the flush log record. Upon

seeing a flush log record during recovery, schedule

a flush to all indexes in this partition s.t. LSN>maxDiskLSN

to ensure component ids are properly maintained upon

failed flushes.

- Add a test case to ensure the correctness of the recovery logic

of component ids

Change-Id: I8c1fc2b209cfb9d3dafa216771d2b7032eb99e75

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2408

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 3 more files in changeset.
[ASTERIXDB-2231][STO] Separate primary op tracker for each partition

- user model changes: no

- storage format changes: no.

- interface changes: yes.

Details:

- Separate primary index operation tracker for each partition, instead

of having a global one on each NC to achieve better scalability.

- As a coordinated change, separate component id generator for each

partition as well.

- Add partition to transaction context so that transaction operations

can operate on proper op tracker.

- Fixes [ASTERIXDB-2232] to calculate dataset partitions correctly.

Change-Id: I9eb3854d2343e45beeccb87b0d434e5f4efd69c9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2263

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 48 more files in changeset.
[ASTERIXDB-2250] Clean up files after failed flush/merge

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Currently we didn't clean up component files if the flush/merge

operation fails. As a result, when a failure happens, the subsequent

retries must fail as well because the files already exist.

- This patch cleans up component files when there is exception thrown

during flush/merge operation

- Added a test case on failed merge

Change-Id: I94630613cfe68de9d5784e022ca3834de959aa02

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2300

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

[ASTERIXDB-2243][STO] Fix BloomFilter size estimation

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Fix the bloom filter size estimation by using the

actual number of elements after bulk loading. This prevents

the bloom filter size grows larger and large under an update

heavy workloads, where most of ingested records are deleted

through merge.

Change-Id: Ib4054797d969efcfceb86f91b5321d34480e25c3

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2285

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[ASTERIXDB-2184] Add Immutable DiskBTree

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Add a immutable DiskBTree to for LSM disk components. This DiskBTree

only supports two operations, i.e., search and bulkload. No concurrency

control is performed at all, since it's immutable.

- Change LSMBTree/InvertedIndex to use this DiskBTree

- Add a DiskBTree point search cursor to optimize point lookups

Change-Id: I8f2a9281478c4b8665589dc695769d0497af9961

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2193

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 7 more files in changeset.
[ASTERIXDB-2149] Refactor key normalizer with longer keys

- user model changes: no

- storage format changes: no

- interface changes: yes. The interface of key normalized is changed.

Details:

- Refactored key normalizer to work with longer normalized keys composed

of multiple integers.

- Add tests for key normalizers

- Add key normalizer for UUID type to improve sort performance.

Change-Id: Idba747285af74195ef9953ed9bf5f6f217511380

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2225

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 10 more files in changeset.
[ASTERIXDB-920] Fix BTree Composite Key Search

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

When we search a primary BTree with composite keys, we should always add

another select operator to filter out results, unless only the last key

is a range search. The reason is that when we search a BTree, we iterate

from the start index (based on the composite low range) to the end index

(based on the composite high range). However, during the iteration,

there could be a lot of false positives.

This patch fixes this bug by preserving the select operator in case of

composite keys.

Change-Id: I8654e52aec59601c533255cd57c2824612066fc0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2227

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 8 more files in changeset.
[ASTERIXDB-2186][STO] Cache-friendly Bloom Filter

- user model changes: no

- storage format changes: yes. Change the format of bloom filter

- interface changes: no

Details:

- Introduce blocked bloom filter to guarantee only 1 random memory

access is required during each membership check. It improves bloom

filter performance by 2x - 4x, depending on the ratio of positive

queries.

- For legacy bloom filters, we fall back to previous implementation

based on the stored version in the metadata.

- Add pinAllPages/unpinAllPages method to reduce pin/unpin overhead.

Change-Id: I0e8e0db9b60d5addfaf61ebb372a1bcb2d2d5957

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2201

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>