Clone
Murtadha Makki Al Hubail
committed
on 07 Dec 17
[NO ISSUE][STO] Introduce Index Checkpoints
- user model changes: no
- storage format changes: yes
- Add index checkpoints.
- Use index … Show more
[NO ISSUE][STO] Introduce Index Checkpoints

- user model changes: no

- storage format changes: yes

 - Add index checkpoints.

 - Use index checkpoint to determine low watermark

   during recovery.

- interface changes: yes

 - Introduce IIndexCheckpointManager for managing

   indexes checkpoints.

 - Introduce IIndexCheckpointProvider for tracking

   IIndexCheckpointManager references.

Details:

- Unify LSM flush/merge operations completion order.

- Introduce index checkpoints which contains:

  - Index low watermark.

  - Latest valid LSM component

  - Mapping between master replica and local replica.

- Use index checkpoints instead of LSM component metadata

 for identifying low watermark in recovery.

- Use index checkpoints in replication instead of overwriting

 LSN byte offset in replica component metadata.

- Replace LSN_MAP used in replication by index checkpoints.

- Replace NIO Files.find by Commons FileUtils.listFiles to

 avoid no NoSuchFileException on any file deletion.

Change-Id: Ib22800002bf8ea3660242e599b3f5f20678301a8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2200

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Show less

master + 3 more