Clone Tools
  • last updated 19 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
ASTERIXDB-1892: Sets a proper hash table cardinality during hash-group by

- Set a proper hash table cardinality during the merge phase

of the external hash group-by operator.

- Currently, the number of tuples in a spilled partition is

used as the hash table cardinality. And this can cause an issue

since compiler.groupmemory size is not considered.

- So, like the initial group-by build phase, the hash table

cardinality will be set properly based on the memory budget for

the group-by operator.

Change-Id: I651139b2b559ad4d2f6137a5c844814606516a90

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1702

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 6 more files in changeset.
Fix ASF license headers.

Change-Id: I5a91d8b7a4eb219374114c460caa879dac683af3

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1428

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 22 more files in changeset.
ASTERIXDB-1556, ASTERIXDB-1733: Hash Group By and Hash Join conform to the memory budget

- External Hash Group By and Hash Join now conform to the memory budget (compiler.groupmemory and compiler.joinmemory)

- For Optimzed Hybrid Hash Join, we calculate the expected hash table size when the build phase is done and

try to spill one or more partitions if the freespace can't afford the hash table size.

- For External Hash Group By, the number of hash entries (hash table size) is calculated based on

an estimation of the aggregated tuple size and possible hash values for the given field size in that tuple.

- Garbage Collection feature has been added to SerializableHashTable. For external hash group-by,

whenever we spill a data partition to the disk, we also check the ratio of garbage in the hash table.

If it's greater than the given threshold, we conduct a GC on Hash Table.

Change-Id: I2b323e9a2141b4c1dd1652a360d2d9354d3bc3f5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1056

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -0
    • +128
    ./algebra/operators/physical/ExternalGroupByPOperatorTest.java
  1. … 41 more files in changeset.