Clone
luochen <cluo8@uci.edu>
committed
on 25 Apr
[ASTERIXDB-2715][STO] Dynamic Memory Component Architecture
- user model changes: yes
- storage format changes: no
- interface changes: yes… Show more
[ASTERIXDB-2715][STO] Dynamic Memory Component Architecture

- user model changes: yes

- storage format changes: no

- interface changes: yes

Details:

- Introduce a dynamic memory component architecture that uses a global

virtual buffer cache to manage the write memory for all LSM-trees.

- When the overall write memory is nearly full, we flush a dataset

partition at a time using a round-robin way. Additionally, we allow

users to configure the maximum size of filtered memory components

to provide better pruning capability.

- Clean up legacy code for statically allocating write memory to each

dataset.

- Remove the following parameters:

 storage.metadata.memorycomponent.numpages

 storage.max.active.writable.datasets

- Add the following parameters:

 storage.memorycomponent.flush.threshold (default: 0.9)

 storage.filtered.memorycomponent.max.size (default 0)

Change-Id: Ia6a0f4de020acd7af89ef630322526c4be5076e0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5824

Reviewed-by: Luo Chen <cluo8@uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Show less