Clone Tools
  • last updated 10 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[ASTERIXDB-2611][*DB] Fix GroupAll with empty input

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

This patch is to fix the issue with group by all where

some partitions are empty. The patch in general handles

propagating outer scope variables through group by all by using

subplans instead of using decor variables when translating

the AST.

- changed IsomorphismVariableMappingVisitor to stop re-visiting

the same nested tuple operator pair again

- moved getFdColumns() up in hierarchy to be shared by others

- added a check to prevent group-all with decoration fields

Change-Id: I56916fc60c696f6c702717210f108f2247b2a397

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3521

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -14
    • +14
    ./query-ASTERIXDB-1581-correlated.plan
    • -141
    • +167
    ./query-ASTERIXDB-1581.plan
  1. … 36 more files in changeset.
[ASTERIXDB-2511][COMP] Refactor LET/WHERE clauses into a single list

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Consolidate LET and WHERE clauses into a single list in the SelectBlock

Change-Id: I9265dfc3adb9ef4d5eb963b0aa4d39c2ad0cc068

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3118

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 29 more files in changeset.
[ASTERIXDB-2488][COMP] Support aggregate window functions

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Implement aggregate window functions:

agg_func() OVER (frame_var AS)? (PARTITION BY ... ORDER BY ... frame_spec)

- Where agg_func is a SQL/SQL++ aggregate function

- Fix percent_rank() to always return 0 for the first tuple

- Fix ntile() to handle NULL argument

- Log query after each rewrite rule in SqlppQueryRewriter

- Implement toString() for ADayTimeDuration, fix it for AYearMonthDuration

- Add seek() method to RunFileReader

Change-Id: If0f71118a04c2dbd3462070673d52e67f076b7e1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3049

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

    • -32
    • +32
    ./query-ASTERIXDB-1581-correlated.plan
  1. … 849 more files in changeset.
[ASTERIXDB-2468] Extend CountVarToCountOneRule to support no group-by

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Extend CountVarToCountOneRule to support the no group-by case so

that count queries without group-by can be handled as well.

- Also extend this rule to support SQL_COUNT. In this case, count(v)

is replaced with count(1) only v is not nullable.

Change-Id: Ie0808912cf45d3a914ebf7f69ec42f73b33c071f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3024

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -68
    • +66
    ./query-ASTERIXDB-1581-correlated.plan
  1. … 30 more files in changeset.
[ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

- user model changes: yes

- storage format changes: no

- interface changes: yes

details:

- new plan for sort operation which includes sampling and

replicating the stream of data to be sorted. Sort-merge connector

is removed from the plan. The sorted result now is in multiple partitions.

- new optimization rule to check whether full parallel sort is applicable.

- new Forward operator to read the replicated sort input stream and

to receive the ouput of the sampling.

- new sequential merge connector to merge a globally ordered result residing

in multiple partitions (in addition to the connector's partition computer).

- "asterix-lang-aql/pom.xml" is changed as a result of refactoring

code related to the range map handling.

- new private sampling function to generate the range map object

(local & global functions) & their type computers.

user model changes:

- new compiler property is added to enable and disable parallel sort.

interface changes:

- "ILogicalOperatorVisitor.java" includes Forward Operator.

- "ITuplePartitionComputer.java" includes initialize() to enable partitioner

to do some initialization. FieldRangePartitionComputerFactory uses it to

pick a range map.

- "ITuplePartitionComputerFactory.java". createPartitioner() is changed to

createPartitioner(IHyracksTaskContext hyracksTaskContext). Context is needed

for transferring the range map throught the context.

Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2393

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -0
    • +58
    ./query-ASTERIXDB-1596_ps.plan
  1. … 357 more files in changeset.
[ASTERIXDB-2174] Fix partitioning improvement of group by

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Fix the improvement of partitioning propagation of group by

introduced by https://asterix-gerrit.ics.uci.edu/#/c/2176/.

- The problem is that we cannot modify the partitioning property of the

input operator directly, since that property is owned by the input.

Otherwise, the property of the child operator would be modified even

before the group by operator, and this caused some tests regarding

common sub operator extration to fail (introducing REPLICATE).

Change-Id: I81a74ab1242772c790175414281e3f200537b386

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2966

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -72
    • +76
    ./query-ASTERIXDB-1581-correlated.plan
  1. … 7 more files in changeset.
[ASTERIXDB-2441][COMP] Improve column alias handling in ORDERBY/LIMIT

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Introduce LET clauses for column alias expressions

instead of inlining them into ORDER BY/LIMIT clauses

- Fix incorrect visitor used in SubstituteGroupbyExpressionWithVariableVisitor

for rewriting LET clauses after GROUP BY

Change-Id: If925cbb803f2ad5fe955ba343ddc62585589b894

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2905

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Contrib: Till Westmann <tillw@apache.org>

    • -32
    • +32
    ./query-ASTERIXDB-1581-correlated.plan
  1. … 41 more files in changeset.
[ASTERIXDB-2394][COMP] Align VarCounter in query rewritting

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Currently, before we do certain language rewritings, we take a snapshot

of the var counter in the context and pass it to the rewriter. This

works fine in most of the cases, but it can fail when the rewriter

introduces new variables. This will cause different variables map to the

same variable id. Although this error may be fixed by another

ScopingVisitor, the compiler could fail if there is no more visitor down

in the rewriting workflow.

Change-Id: I8ea1ee7de140d6556af03f07a5ec7bc278b011ba

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2673

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -32
    • +32
    ./query-ASTERIXDB-1581-correlated.plan
  1. … 28 more files in changeset.
[ASTERIXDB-2170][SQL] Fix resolution order of implicit field access

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Improved name resolution rules

- Resolve field access to the nearest variable in scope

instead of raising compile-time error

- Do not rely on type information when resolving names

- Cleanup group variable handling in GroupBy clause,

no longer use ‘with’ map for it

- Fix ByNameToByIndexFieldAccessRule to use type environment

of its input operator when analyzing its expression

- Fix ExternalGroupByPOperator to use input schema of its

aggregate function when generating runtime for that function

- Fix invalid free variable computation for GroupBy clause

Change-Id: I50bc823ff53da06507a5454b30f4f500b862d4bf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2207

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -32
    • +32
    ./query-ASTERIXDB-1581-correlated.plan
  1. … 310 more files in changeset.
[ASTERIXDB-2174][COMP] Improve partitioning propagation in GroupBy

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Improve partitioning property computation in

AbstractPreclusteredGroupByPOperator

If "gby v1 as v2" and input partitioned on (v1)

then output is partitioned on (v2)

Change-Id: Ib679278d6ee6c53b1d9f581438e8bd04c56c2b08

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2176

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -194
    • +188
    ./query-ASTERIXDB-1581-correlated.plan
    • -130
    • +121
    ./query-ASTERIXDB-1581.plan
  1. … 8 more files in changeset.
[ASTERIXDB-1753][SQL] Disable auto-plural for group-by.

- user model changes: do not perform auto-scalar-to-plural

transition for variables that are defined prior to a group-by

group clause after the group-by clause.

- storage format changes: no

- interface changes: no

details:

- migrated existing test queries that use auto-plural;

- added a negative test query.

Change-Id: I5d6383fc7453fe7537fc291c3483708b59df3871

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1860

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -36
    • +36
    ./query-ASTERIXDB-1581-correlated.plan
  1. … 259 more files in changeset.
Fix ASTERIXDB-1875 UDF in SQL++

1. Fix the bug that if a function created with SQLPP, the callExpr in

the function body is not normalzied to list input functions.

2. Fix the bug that in SQL++ Query Rewritter, the variables are not

expanded before the inline the functions.

3. Add test case for this issue.

4. Make SQL++ visitor visits decorVariableExpression.

5. Optimizer test cases update.

Change-Id: I7cf72c690b452dd141efe56f5d7098a6f00cab0f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1666

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -36
    • +36
    ./query-ASTERIXDB-1581-correlated.plan
  1. … 19 more files in changeset.
Disable hash merge exchange.

- Hash merge exchange does not work very well on large clusters.

- Hash merge exchange is used only when a user explicitly enable it.

Change-Id: If82c4e4db1d4340cdd53d17e48a193ab2b7a05e0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1345

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 41 more files in changeset.
Changed the physical tag of ReplicatePOperator (SPLIT -> REPLICATE)

Change-Id: Ic298f90c5bc9875cea1017aff17a524214596b1e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1219

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -19
    • +19
    ./query-ASTERIXDB-1581-correlated.plan
  1. … 91 more files in changeset.
ASTERIXDB-1572 and ASTERIXDB-1591: fix and regression tests.

- push aggregates into subplans;

- fix recursive variable mapping in subquery decorrelation.

Change-Id: I7092dd2fa7c9193ff919b27464854936f48261b0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1161

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -35
    • +35
    ./query-ASTERIXDB-1581-correlated.plan
    • -0
    • +151
    ./query-ASTERIXDB-1591.plan
  1. … 28 more files in changeset.
ASTERIXDB-1581: fix subquery decorrelation.

- fix concurrent modification exception;

- fix variable substitution ordering;

- fix required partitioning property for DataSourceScan;

- fix partitioning property enforcer for nested operators;

- fix recursive subplan decorrelation;

- fix CardinalityInferenceVisitor;

- add a rule to switch inner join branches;

- fix SimpleUnnestToProductRule;

- add test cases which are variants of ASTERIXDB-1581.

Change-Id: Ia2fa4b5b836eafee1975bd1164ae7c22199a4af0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1125

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -0
    • +281
    ./query-ASTERIXDB-1581-correlated.plan
    • -0
    • +180
    ./query-ASTERIXDB-1581.plan
  1. … 72 more files in changeset.
ASTERIXDB-1592: fix and regression tests.

Change-Id: Id59ae2fd79302211973cfb3b82829eb3836da4bc

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1110

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +27
    ./query-ASTERIXDB-1596.plan
  1. … 9 more files in changeset.
ASTERIXDB-1580: add regression tests.

Change-Id: Ib046f56ca296b79cd44d2af934c25ab927e8b09e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1081

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -0
    • +74
    ./query-ASTERIXDB-1580.plan
  1. … 17 more files in changeset.