Clone
Abdullah Alamoudi <bamousaa@gmail.com>
committed
on 28 Apr 18
[NO ISSUE][STO] Add consistency to flush lifecycle
- user model changes: no
- storage format changes: yes
- renamed AbstractLSMIOOperation… Show more
[NO ISSUE][STO] Add consistency to flush lifecycle

- user model changes: no

- storage format changes: yes

 - renamed AbstractLSMIOOperationCallbackFactory

   to LSMIOOperationCallbackFactory

 - useless classes have been removed.

   - LSMBTreeIOOperationCallbackFactory

   - LSMBTreeWithBuddyIOOperationCallbackFactory

   - LSMInvertedIndexIOOperationCallbackFactory

   - LSMRTreeIOOperationCallbackFactory

- interface changes: yes

Details:

- Previously, flushes have different lifecycle depending

 on the memory component state

 - not allocated

 - allocated

 - modified

- In certain cases, flush operations are skipped alltogether

- IO Operation callbacks became complicated and difficult

 to maintain since calls are done differently in different

 cases.

- In certain cases, afterFinalize is called on the IO

 Operation callbacks even if beforeOperation was never

 called.

- In this change, flushes go through the same lifecycle

 events regardless of the state of the memory component.

- In addition, primary and secondary memory components

 would reside in different virtual buffer caches due

 to skipped flushes, or due to having the secondary

 index created when the primary index's memory component

 is residing on the virtual buffer cache with index !=0.

- Moreover, when flushes are lagging and all memory

 components are being flushed, search operations assumes

 the oldest of the memory component is the newest and

 produces incorrect results.

- In addition, in case of a failed flush of a component,

 the IO scheduler would skip it and flush the next

 component. This would produce a bad state on disk.

- In this change, a failed flush can be retried. otherwise,

 all future flushes of the component fail due to the failure

 of the previously failed flush.

- Previously, when a component fails to modify an index due

 to flush failures, it assumes disk is full.

- With this change, the modification failure reports the

 original cause of the failed flush.

Change-Id: I29f7992ec6c0f71c5b63d45800b2fb590d651e4b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2584

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Murtadha Hubail <mhubail@apache.org>

Show less