Clone Tools
  • last updated 26 mins ago
Constraints: committers
Constraints: files
Constraints: dates
[ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

- user model changes: yes

- storage format changes: no

- interface changes: yes


- new plan for sort operation which includes sampling and

replicating the stream of data to be sorted. Sort-merge connector

is removed from the plan. The sorted result now is in multiple partitions.

- new optimization rule to check whether full parallel sort is applicable.

- new Forward operator to read the replicated sort input stream and

to receive the ouput of the sampling.

- new sequential merge connector to merge a globally ordered result residing

in multiple partitions (in addition to the connector's partition computer).

- "asterix-lang-aql/pom.xml" is changed as a result of refactoring

code related to the range map handling.

- new private sampling function to generate the range map object

(local & global functions) & their type computers.

user model changes:

- new compiler property is added to enable and disable parallel sort.

interface changes:

- "" includes Forward Operator.

- "" includes initialize() to enable partitioner

to do some initialization. FieldRangePartitionComputerFactory uses it to

pick a range map.

- "". createPartitioner() is changed to

createPartitioner(IHyracksTaskContext hyracksTaskContext). Context is needed

for transferring the range map throught the context.

Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f


Tested-by: Jenkins <>

Contrib: Jenkins <>

Integration-Tests: Jenkins <>

Reviewed-by: Dmitry Lychagin <>

    • -0
    • +2
  1. … 357 more files in changeset.
Support rebalancing datasets with indexes.

- Remove type arguments from the methods in IndexUtils

that generate index operation (e.g., create, load, compact)

jobs. Do type extraction inside SecondaryIndexOperationsHelper.

Change-Id: I9c0720382440ae44441a8f8847e75649a3822fa2


Sonar-Qube: Jenkins <>

Tested-by: Jenkins <>

BAD: Jenkins <>

Reviewed-by: Dmitry Lychagin <>

Integration-Tests: Jenkins <>

Reviewed-by: Till Westmann <>

    • -0
    • +25
  1. … 35 more files in changeset.