Clone Tools
  • last updated 20 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[NO ISSUE][ING] Allow external UDF to use runtime parallelism

- user model changes: no

- storage format changes: no

- interface changes:

Details:

1. Enable UDF in feed to use the runtime parallelism.

2. Fix the DefaultNodeDomain where the nodes should be mutliSet but not

list, for comparison purpose.

Change-Id: Ic3b54617be115f51b6a48b9a61581c26b5be8d9d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2398

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 7 more files in changeset.
[ASTERIXDB-2256] Reformat sources using code format template

Change-Id: I4faa141c1a8c9700d5e9ac50b839acc9d1eede73

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2310

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

    • -41
    • +43
    ./IPartitioningRequirementsCoordinator.java
  1. … 981 more files in changeset.
[NO ISSUE][HYR][*DB] Minor refactoring / address SonarQube comments

Change-Id: Icf10b6df0fdc006675d8f0da6fd06d50200c6b6a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2098

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 60 more files in changeset.
Supports flexible degree of parallelism.

Detailed list of changes include:

- Add the IClusterInfoCollector interface and feed that to the compiler;

- Add the number of cores into the heartbeat;

- Add partition calculation in APIFramework;

- Fix partitioning property requirements in binary operators such as Join and UnionAll;

- Fix PushSubplanIntoGroupByRule for type propagation;

- Fix InlineLeftNtsInSubplanJoinFlatteningVisitor for the returned top join operator;

- Add unit test for partition calculation in APIFrameworkTest

- Add integration test SqlppExecutionFullParallelismIT, SqlppExecutionLessParallelismIT,

AqlExecutionFullParallelismIT, and AqlExecutionLessParallelismIT for different

parallelism settings;

- Refactor AQLExecutionTest and SqlppExecutionTest;

- Separate one cluster state test out from AqlExecutionTest.

Change-Id: If70faf52bed995c7098cc343f2fabd1b6c8d96e7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1398

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 115 more files in changeset.
Cleanup FileSplit and FileReference

This change gives FileSplit and FileReference specific meaning to

avoid confusion of an absolute vs relative, local vs global, inside

an IO device vs outside IO devices.

In addition, it enables better abstraction of global partitions and

delegate the responsibility of choosing which partition goes to which

IO device to the IO Manager through the introduction of FileDeviceComputer

In details:

Previously, the LocalResource in Hyracks had partition (storage partition)

and there is no such thing in Hyracks. This scope leak is bad. In addition

The local resource had a name and a path. they were always the same and so

the name was removed.

The storage partition was instead moved to asterixdb implementation of the

serialized object in the local resource.

With all of these changes, the cluster controller (compiler) only needs to

know about partitions and relative paths. It doesn't need to worry about

heterogenous Node setups and different io device configurations. For File

assignment to IO devices, a new interface (IFileDeviceComputer) was

introduced which can be overriden by applications to have their own

strategy for distributing files among IO devices.

Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1352

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 287 more files in changeset.
merge master

  1. … 21 more files in changeset.
ASTERIXDB-1592: fix and regression tests.

Change-Id: Id59ae2fd79302211973cfb3b82829eb3836da4bc

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1110

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 9 more files in changeset.
ASTERIXDB-1571: fix and add regression tests.

- also fix the left outer type propagation policy.

Change-Id: I45d6dbabdef709c9ba3cb883412301ab7fd69a85

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1075

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 19 more files in changeset.
Finalize the range in context.

  1. … 30 more files in changeset.
Clean up GROUP BY and WITH clause.

- fix ASTERIXDB-971, i.e., the expression to plan translator for FLWOGR without a "for";

- fix the scoping for nested WITH clause;

- fix the scoping for nested GROUP BY clause;

- fix default fields in group variables;

- fix/unify the expression substition AST visitor;

- enhance RemoveUnusedAssignAndAggregateRule to be able to remove unused GROUP BY

decoration varaibles;

- clearly sperate SQL++ aggregate functions and SQL-92 aggregate functions, i.e.,

SQL-92 aggregate sugars cannot apply to SQL++ aggregate functions;

- fix PushAggregateIntoGroupbyRule for nested plan deletions;

- enhance RemoveCartesianProductWithEmptyBranchRule;

- add ExtractGroupByDecorVariablesRule;

- add an aggregate function first-element;

- fix ASTERIXDB-1560, an error message issue.

Change-Id: I62fca7f937aa007d97ed87c75cef19f6aa3e5ade

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1050

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 235 more files in changeset.
snapshot for range state transition

    • -14
    • +12
    ./OrderedPartitionedProperty.java
  1. … 69 more files in changeset.
Support UNION ALL.

- support heterogenous input types for UNION ALL and the output is resolved to a mimimally generalized type;

- fix the partitioning property inference for UnionAllPOperator;

- fix for ASTERIXDB-1354, a bug in IsomorphismVariableMappingVisitor.

Change-Id: Ifadf1707bb2b6bed22f8fc5792c635e87291a468

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1035

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -1
    • +1
    ./IPartitioningRequirementsCoordinator.java
  1. … 72 more files in changeset.
ASTERIXDB-1531: fix ORDER BY with primary keys as non-prefix sort keys.

- clean up the local property inference implemenation;

- avoid side-effects for data properties during property matching.

Change-Id: Iee7fcdd6eb1279e8ee14ba75ee31ac118b00c806

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1022

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 13 more files in changeset.
Sonar fixes and other clean up items.

Change-Id: Ieb72a0fef3c181282cc4c71b881a7cb52da86818

Reviewed-on: https://asterix-gerrit.ics.uci.edu/999

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 67 more files in changeset.
merge with master

  1. … 12 more files in changeset.
Support implicit variable name and column name.

1. Support WITH clause for general cases;

2. Fixed open/closed type optimization;

3. Fixed scoping for nested aggregations;

4. Fixed side effects during partition property analysis;

5. Fixed type propagation policy for Subplan operator.

Change-Id: I409b9ba139c9f000a6b9b84d519d172d0069e4bb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/950

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 225 more files in changeset.
sonar update

  1. … 45 more files in changeset.
ASTERIXDB-1228: Add MISSING into the data model.

1. MISSING repsents the value of a non-existing field in a record

or an out-of-bound index access of a collection;

2. NULL represents that the value of an optional field in a record

is unknown or the value of existing collection entry is unknown.

3. Unit tests for all missing/null-in-missing/null-out scalar functions.

Change-Id: Ia49ed8474bfc5d6604231819065117468c5b0897

Reviewed-on: https://asterix-gerrit.ics.uci.edu/846

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 551 more files in changeset.
great merge - hyracks

  1. … 63 more files in changeset.
Move Hyracks to subfolder

    • -0
    • +90
    ./AbstractGroupingProperty.java
    • -0
    • +64
    ./BroadcastPartitioningProperty.java
    • -0
    • +70
    ./DefaultNodeGroupDomain.java
    • -0
    • +76
    ./FileSplitDomain.java
    • -0
    • +41
    ./FilteredVariablePropagationPolicy.java
    • -0
    • +57
    ./FunctionalDependency.java
    • -0
    • +54
    ./ILocalStructuralProperty.java
    • -0
    • +25
    ./ILogicalPropertiesVector.java
    • -0
    • +91
    ./IPartitioningProperty.java
    • -0
    • +102
    ./IPartitioningRequirementsCoordinator.java
    • -0
    • +47
    ./IPhysicalPropertiesVector.java
    • -0
    • +27
    ./IPropertiesComputer.java
    • -0
    • +27
    ./IStructuralProperty.java
    • -0
    • +107
    ./LocalGroupingProperty.java
  1. … 4413 more files in changeset.