Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

- user model changes: yes

- storage format changes: no

- interface changes: yes

details:

- new plan for sort operation which includes sampling and

replicating the stream of data to be sorted. Sort-merge connector

is removed from the plan. The sorted result now is in multiple partitions.

- new optimization rule to check whether full parallel sort is applicable.

- new Forward operator to read the replicated sort input stream and

to receive the ouput of the sampling.

- new sequential merge connector to merge a globally ordered result residing

in multiple partitions (in addition to the connector's partition computer).

- "asterix-lang-aql/pom.xml" is changed as a result of refactoring

code related to the range map handling.

- new private sampling function to generate the range map object

(local & global functions) & their type computers.

user model changes:

- new compiler property is added to enable and disable parallel sort.

interface changes:

- "ILogicalOperatorVisitor.java" includes Forward Operator.

- "ITuplePartitionComputer.java" includes initialize() to enable partitioner

to do some initialization. FieldRangePartitionComputerFactory uses it to

pick a range map.

- "ITuplePartitionComputerFactory.java". createPartitioner() is changed to

createPartitioner(IHyracksTaskContext hyracksTaskContext). Context is needed

for transferring the range map throught the context.

Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2393

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -0
    • +51
    ./ngram-contains-panic_ps.sqlpp
    • -0
    • +51
    ./ngram-contains_ps.sqlpp
    • -0
    • +63
    ./olist-edit-distance-check-panic_ps.sqlpp
    • -0
    • +63
    ./olist-edit-distance-check_ps.sqlpp
    • -0
    • +63
    ./olist-edit-distance-panic_ps.sqlpp
    • -0
    • +63
    ./olist-edit-distance_ps.sqlpp
    • -0
    • +65
    ./olist-fuzzyeq-edit-distance_ps.sqlpp
    • -0
    • +51
    ./word-contains_ps.sqlpp
  1. … 350 more files in changeset.
[ASTERIXDB-2366][TEST] Optimizer tests cleanup for SQL++

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

The current optimizerTest actually doesn't use the SQL++ test cases.

The existed test cases for SQLPP also have various issues.

This patch cleans part of the test cases which failed at variable names

changes in the result query plan.

Change-Id: I8dbe67d6376d517a4919e8478a6e88326b3e1cc0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2591

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -0
    • +49
    ./ngram-contains-panic.sqlpp
    • -0
    • +49
    ./ngram-contains.sqlpp
    • -0
    • +48
    ./ngram-edit-distance-check.sqlpp
    • -0
    • +48
    ./ngram-edit-distance.sqlpp
    • -0
    • +52
    ./ngram-fuzzyeq-edit-distance.sqlpp
    • -0
    • +54
    ./ngram-fuzzyeq-jaccard.sqlpp
    • -0
    • +50
    ./ngram-jaccard-check.sqlpp
  1. … 445 more files in changeset.
Full-text implementation step 4

- Follows the new error handling proposal.

- Introduces Full-TextTypeComputer.

Change-Id: Ib909882c43a3fc1401eceec1e1ddd981f40b6dee

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1422

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -2
    • +0
    ./fulltext-contains-without-option.aql
  1. … 44 more files in changeset.
Full-text implementation step 3

- Implemented index-based full-text contains() function.

- Fixed a bug in the full-text code that can't process join cases well.

- Introduced full-text type index syntax. The functionality was already

in the codebase but it was not being used. So, using this syntax, we

can utilize the given index type - Not the length partitioned keyword index.

- Added the support for SQL++

- Default option will be set if a user doesn't provide any option.

Change-Id: I1087854ac7cf5b6ef5094e27a1646f12f6a8653f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1388

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

    • -0
    • +55
    ./fulltext-contains-without-option.aql
    • -0
    • +55
    ./fulltext-contains.aql
  1. … 162 more files in changeset.