Clone Tools
  • last updated 15 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[ASTERIXDB-2648][COMP] Fix dataset order in broadcast hint

- user model changes: yes

- storage format changes: no

- interface changes: no

details:

- Provides hash-bcast hint to replace bcast hint which makes

broadcast join consistent with other types of joins for choosing

the broadcasting dataset based on dataset order in FROM clause.

- Chooses the build dataset to get broadcast based on the order

in the FROM clause not the WHERE clause (which was the case for

bcast hint).

Change-Id: I505fc8e0bc32c760ae0471c6b302bbfd8499be4a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/3566

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -0
    • +108
    ./logical/visitors/BroadcastSideSwitchingVisitor.java
  1. … 51 more files in changeset.
[ASTERIXDB-2647][COMP] Random partitioner shouldn't preserve input's properties

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Random partitioner operator currently always propagates

the local properties of its input. If the data of the

input operator is partitioned, the random partitioner

should not propagate the local properties of the input

since the random partitioner destroys them.

Change-Id: I0378597451b3e6b25e8b45295159efec262abae2

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3575

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -4
    • +18
    ./physical/RandomPartitionExchangePOperator.java
[ASTERIXDB-2635][*DB] Fix JSON plan pretty printing

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

Use Jackson to pretty print the logical plan as JSON.

- fixed JsonLogicalPlanTest to validate JSON plans

produced by optimizer tests.

Change-Id: Ic5ada2f31afebc3f219b0584b5ae527ddf9e326e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3567

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 21 more files in changeset.
[NO ISSUE][COMP] Introduce connector for partial broadcasts

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Introduce MToNPartialBroadcastConnectorDescriptor

- Refactor FieldRangePartitionComputerFactory to

use RangeMapSupplier

Change-Id: I4a6f8f17d1709862300db7ab386161b4dfbfee5a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3524

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

    • -10
    • +7
    ./physical/RangePartitionExchangePOperator.java
    • -2
    • +4
    ./physical/RangePartitionMergeExchangePOperator.java
  1. … 12 more files in changeset.
[ASTERIXDB-2611][*DB] Fix GroupAll with empty input

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

This patch is to fix the issue with group by all where

some partitions are empty. The patch in general handles

propagating outer scope variables through group by all by using

subplans instead of using decor variables when translating

the AST.

- changed IsomorphismVariableMappingVisitor to stop re-visiting

the same nested tuple operator pair again

- moved getFdColumns() up in hierarchy to be shared by others

- added a check to prevent group-all with decoration fields

Change-Id: I56916fc60c696f6c702717210f108f2247b2a397

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3521

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -18
    • +28
    ./logical/visitors/IsomorphismVariableMappingVisitor.java
    • -2
    • +33
    ./physical/AbstractGroupByPOperator.java
    • -17
    • +0
    ./physical/AbstractPreclusteredGroupByPOperator.java
    • -15
    • +4
    ./physical/ExternalGroupByPOperator.java
    • -4
    • +4
    ./physical/MicroPreclusteredGroupByPOperator.java
    • -1
    • +2
    ./physical/PreclusteredGroupByPOperator.java
    • -15
    • +3
    ./physical/SortGroupByPOperator.java
  1. … 31 more files in changeset.
[NO ISSUE][COMP] Static range map in full parallel sort

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Support static range map in full parallel sort

- Add static range map option to ORDERED_PARTITIONED

structural property

- Add parseExpression() methods to SQL++ and AQL parsers

- Simplify RangeMapBuilder.parseHint()

Change-Id: I7eab6e6ede8c2dbb714a27801a76ad64bd9be1b8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3513

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -22
    • +39
    ./physical/AbstractStableSortPOperator.java
    • -3
    • +3
    ./physical/RangePartitionExchangePOperator.java
    • -2
    • +3
    ./physical/RangePartitionMergeExchangePOperator.java
  1. … 21 more files in changeset.
[NO ISSUE][COMP] Improve IPartitioningProperty.substituteColumnVars()

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- IPartitioningProperty.substituteColumnVars() should return

a new instance of IPartitioningProperty if variable

substitution was performed for given variables

- Compute schema for a new Select operator introduced by

PullSelectOutOfEqJoin rule

- Refactor delivered properties computation in

AbstractPreclusteredGroupByPOperator

Change-Id: Iee7ef7de26f9d960b205d6d43e2820aaa396cb15

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3499

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

    • -24
    • +26
    ./physical/AbstractPreclusteredGroupByPOperator.java
  1. … 6 more files in changeset.
[NO ISSUE][COMP] Fix variable handling in Intersect operator

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Create separate lists for output compare and

extra variables in Intersect logical operator

- Align accessor method for input extra variables

with the one for input compare variables

- Remove getInputVariables() from Intersect operator

because these variables are already provided by

getInputCompareVariables() and getInputExtraVariables()

Change-Id: I4f436566e743d7e908a72bfa502c138ae7b99c6c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3460

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

    • -52
    • +70
    ./logical/IntersectOperator.java
    • -7
    • +22
    ./logical/visitors/IsomorphismOperatorVisitor.java
    • -5
    • +24
    ./logical/visitors/IsomorphismVariableMappingVisitor.java
    • -19
    • +14
    ./logical/visitors/LogicalOperatorDeepCopyWithNewVariablesVisitor.java
    • -10
    • +11
    ./logical/visitors/OperatorDeepCopyVisitor.java
    • -19
    • +23
    ./physical/IntersectPOperator.java
  1. … 5 more files in changeset.
[NO ISSUE][COMP] Stop reusing variables of Intersect operator

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Intersect operator's input variables should not be reused as

its output variables (fixed IntroduceSelectAccessMethodRule

and IntroduceLSMComponentFilterRule)

- Remove HeuristicOptimizer.isHyracksOp() and updated

ExtractCommonOperatorsRule to use isMicroOperator() instead

- When copying Intersect operator OperatorDeepCopyVisitor now

properly handles its two kinds input variables

- Clear additional state in IntroduceSelectAccessMethodRule.clear()

and OptimizableOperatorSubTree.reset()

- Added testcases for index intersection

Change-Id: I551819b828450760797f6b29af02aad4e124646c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3450

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 26 more files in changeset.
[NO ISSUE][FUN] Support IGNORE NULLS in window functions

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Support IGNORE NULLS / RESPECT NULLS modifiers in window

  functions LEAD(), LAG(), FIRST_VALUE(), LAST_VALUE(), NTH_VALUE()

  (RESPECT NULLS is the default)

- Add testcases for each function

- Fix CollectionMemberResultType and PushAggregateIntoNestedSubplanRule

  to handle remaining non-list arguments

Change-Id: I400c9c95bac9159efa17fca4e97047fd089eb931

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3407

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

    • -5
    • +6
    ./logical/visitors/LogicalOperatorDeepCopyWithNewVariablesVisitor.java
    • -4
    • +13
    ./physical/AbstractWindowPOperator.java
  1. … 42 more files in changeset.
[NO ISSUE][API] Return Async Query Results As JSON

- user model changes: yes

- storage format changes: no

- interface changes: yes

Details:

- Add ResultMetadata that can hold metadata about a job's

result.

- Add required CC functions to get ResultMetadata.

- Add IJobResultCallback that is called when a job's result

is written successfully.

- Calculate a job's processed objects on the job completion.

- Calculate a job's duration on the job completion.

- Use the async query request format when serving the result

from QueryResultApiServlet.

- Return a proper JSON envelop in QueryResultApiServlet

which includes the results as well as metrics fields.

- Add test case for returning JSON response for async

query.

Change-Id: Ic4812a14925099a677f9e77a0040f881d2600724

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3401

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -1
    • +8
    ./logical/DistributeResultOperator.java
    • -2
    • +2
    ./physical/DistributeResultPOperator.java
  1. … 57 more files in changeset.
[ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

This patch changes the hash join operator to use the join condition

to evaluate if tuples are equal when joining. Binary physical comparators

have been removed. The join condition evaluator is in TuplePairEvaluator.

- extraced TuplePairEvaluatorFactory out of nested loop join class

into a separate class so that it is shared among nested loop and

hash join.

- switched from FrameTuplePairComparator to ITuplePairComparator in

in OptimizedHybridHashJoin and InMemoryHashJoin.

- moved debugging code from OptimizedHybridHashJoin into a separate

class, JoinUtil.

- temporarily made the logical comparison of multisets use raw binary

comparison instead of returning null until the logic is implemented.

- made IBinaryBooleanInspector a functional interface and updated

the implementations.

- updated record and array test cases to reflect the new

behaviour of hash join where logical comparison could produce null.

Also, updated sorting, group by and distinct test cases since

the input data has been modified.

- added two new input files arrays1nulls.adm & arrays2nulls.adm

to be used by the open dataset. previous arrays1.adm & arrays2.adm

are used by the closed dataset since it cannot accept arrays with

null values.

Change-Id: If1834967fdd913fdc76003f09636b2450d07cd5e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3387

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

    • -100
    • +23
    ./physical/HybridHashJoinPOperator.java
    • -13
    • +15
    ./physical/InMemoryHashJoinPOperator.java
    • -127
    • +2
    ./physical/NestedLoopJoinPOperator.java
  1. … 46 more files in changeset.
[NO ISSUE][COMP] Make memory requirements an operator property

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Refactor how memory requirements are maintained in a query plan

- Introduce LocalMemoryRequirements class. Its instances are held

by each physical operator and could be altered by the optimizer.

- Introduce optimizer rule SetMemoryRequirementsRule that

initializes and configures memory requirements for each operator

Change-Id: I3481ddfe163c6ce786290c540cbd05db16a7f64f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3374

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

    • -4
    • +11
    ./physical/AbstractGroupByPOperator.java
    • -3
    • +14
    ./physical/AbstractPhysicalOperator.java
    • -2
    • +2
    ./physical/AbstractPreclusteredGroupByPOperator.java
    • -3
    • +10
    ./physical/AbstractStableSortPOperator.java
    • -7
    • +7
    ./physical/ExternalGroupByPOperator.java
    • -17
    • +10
    ./physical/HybridHashJoinPOperator.java
    • -7
    • +5
    ./physical/InMemoryHashJoinPOperator.java
    • -2
    • +3
    ./physical/MicroPreclusteredGroupByPOperator.java
    • -2
    • +2
    ./physical/MicroStableSortPOperator.java
    • -6
    • +2
    ./physical/NestedLoopJoinPOperator.java
    • -2
    • +3
    ./physical/PreclusteredGroupByPOperator.java
  1. … 17 more files in changeset.
[ASTERIXDB-2552][RT] Reuse objects on reseting buffer manager

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

The "VariableFrameMemoryManager" releases the physical frames

objects, the logical frames objects and the free slots objects

on reset(). This patch is to retain those objects and only

release them on close() (as also stated in the interface).

- added close() to IFrameFreeSlotPolicy to make the interface similar

to the other interfaces (IFrameBufferManager, IFramePool, ... etc)

where close() is supposed to release while reset() is supposed to only

reset but retain resources.

- renamed "InMemorySortRuntimeFactory" to "MicroSortRuntimeFactory".

Change-Id: I88bea27e5024c621412ef609475e2a7ba1913afa

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3372

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -2
    • +2
    ./physical/MicroStableSortPOperator.java
  1. … 10 more files in changeset.
[NO ISSUE][COMP] Refactor physical window operator

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Create a new physical operator (WindowStreamPOperator)

for window operators that do not require partition materialization

- Create AbstractWindowPOperator which is now a base

class for both physical window operators

- Rename WindowSimpleRuntime* to WindowStreamRuntime*

Change-Id: I3863fa3d298aef53d4098be9fc17b0451eb2c23e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3369

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

    • -0
    • +289
    ./physical/AbstractWindowPOperator.java
    • -279
    • +49
    ./physical/WindowPOperator.java
    • -0
    • +62
    ./physical/WindowStreamPOperator.java
  1. … 12 more files in changeset.
[ASTERIXDB-2552][RT] Implement micro external sort for subplans

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

This patch is to change the in memory sort used in subplans to

be a micro external sort to avoid out of memory error.

- added reset() to the runs merger to allow reusing the runs merger.

- renamed "InMemoryStableSortPOperator" to "MicroStableSortPOperator".

- changed the tag from "IN_MEMORY_STABLE_SORT" to "MICRO_STABLE_SORT".

- added test cases.

Change-Id: I930849d644c60d461d2869c9773b85e49b46fbdb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3353

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -3
    • +5
    ./physical/AbstractStableSortPOperator.java
    • -87
    • +0
    ./physical/InMemoryStableSortPOperator.java
    • -0
    • +88
    ./physical/MicroStableSortPOperator.java
  1. … 30 more files in changeset.
[NO ISSUE][COMP] Refactor physical operator assignment rules

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Refactor SetAlgebricksPhysicalOperatorsRule and make it extensible

- Make SetAsterixPhysicalOperatorsRule a subclass of

SetAlgebricksPhysicalOperatorsRule

- Remove SetAlgebricksPhysicalOperatorsRule from Asterix rule set,

replace its invocations with SetAsterixPhysicalOperatorsRule

Change-Id: I502f367464a6fabc595cff804722f793e052570f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3367

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 5 more files in changeset.
[NO ISSUE][COMP] Refactor physical operators for GroupBy and DistinctBy

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Introduce AbstractGroupByPOperator -

base class for physical GroupBy operators

- Introduce AbstractDistinctByPOperator -

base class for physical DistinctBy operators

- Set memory limit for MicroPreclusteredGroupByPOperator

- Remove unused PhysicalOperatorTag.HASH_GROUP_BY

- Rename GroupByOperator.getGbyVarList() to getGroupByVarList()

- Remove constructors that do not specify memory limit from

PreclusteredGroupOperatorDescriptor and PreclusteredGroupWriter

Change-Id: I93f17a6a3e0df6587e518e18eea01ca54a289b4c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3364

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

    • -0
    • +69
    ./physical/AbstractDistinctByPOperator.java
    • -0
    • +54
    ./physical/AbstractGroupByPOperator.java
    • -38
    • +3
    ./physical/AbstractPreSortedDistinctByPOperator.java
    • -26
    • +6
    ./physical/AbstractPreclusteredGroupByPOperator.java
    • -38
    • +8
    ./physical/ExternalGroupByPOperator.java
    • -1
    • +1
    ./physical/MicroPreSortedDistinctByPOperator.java
    • -4
    • +3
    ./physical/MicroPreclusteredGroupByPOperator.java
    • -1
    • +1
    ./physical/NestedTupleSourcePOperator.java
    • -1
    • +1
    ./physical/PreSortedDistinctByPOperator.java
    • -4
    • +1
    ./physical/PreclusteredGroupByPOperator.java
    • -34
    • +5
    ./physical/SortGroupByPOperator.java
  1. … 15 more files in changeset.
[ASTERIXDB-2553][SQL] Make DIV and MOD Operators Case Insensitive

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Make DIV and MOD operators case insensitive.

- Fix imports order.

- Add test case.

Change-Id: I3f28a8324aeedcb05bef628751f1ff8e33ee08a0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3354

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

    • -1
    • +1
    ./physical/RangePartitionMergeExchangePOperator.java
  1. … 4 more files in changeset.
[NO ISSUE] Create an abstraction for the ForwardOperatorDescriptor

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Abstract the activities out of the ForwardOperatorDescriptor so we can reuse

the same basic framework for multiple forward Operators.

- Abstract the ForwardOperatorDescriptor out of the ForwardPOperator so we can

reuse the same basic framework for multiple forward Operators.

Change-Id: Icc3db4b386e69a98c2a1c40dadc96eb3e1a5d4fa

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3338

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -2
    • +2
    ./logical/visitors/LogicalOperatorDeepCopyWithNewVariablesVisitor.java
    • -0
    • +146
    ./physical/AbstractForwardPOperator.java
    • -136
    • +0
    ./physical/ForwardPOperator.java
    • -0
    • +34
    ./physical/SortForwardPOperator.java
  1. … 9 more files in changeset.
[NO ISSUE][COMP] Improve window operator consolidation rule

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Consolidate window operators computing FIRST_VALUE()

and LAST_VALUE() over the same window specification

Change-Id: I4714081d711c98c6e843901aa8a02bc4a0bd214e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3347

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 11 more files in changeset.
[NO ISSUE][COMP][RT] Handle type mismatch in window frame boundaries

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- If window frame boundary is defined as N preceding/following

and the incoming value is not of a numeric or temporal type

then assume that the frame is empty for this value because

its boundaries cannot be computed

- Add tests for NULL/MISSING/complex types for window operator

- Fix typo in the name resolution documentation

Change-Id: I4dc1b010674eb9a8b679039dc68c81163d156956

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3342

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

    • -2
    • +7
    ./logical/visitors/LogicalOperatorDeepCopyWithNewVariablesVisitor.java
    • -15
    • +8
    ./physical/StableSortPOperator.java
  1. … 24 more files in changeset.
[NO ISSUE][FUN] Modify array functions to allow comparing complex

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

This change is to modify the array functions to compare complex

types like arrays and records. The change also includes a small

fix to IsomorphismVariableMappingVisitor where the

NestedTupleSourceOperator visit method would downcast the args

without checking first they are of the same operator types.

- changed test cases

- changed NullMissingTest to pass the args types for functions

that need them.

Change-Id: I416989cf902eda38224a3b00340e478fc4e3a60c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3313

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -0
    • +3
    ./logical/visitors/IsomorphismVariableMappingVisitor.java
  1. … 108 more files in changeset.
[NO ISSUE] Apply / enforce java import order

The process-sources target will now sort imports as well as

format source code; the source-format job will likewise verify

import order in addition to source code format

Change-Id: I55d976c4df10d9919c6a25683be2a3e3304e65d9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3288

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

Reviewed-by: Till Westmann <tillw@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -1
    • +0
    ./logical/AbstractBinaryJoinOperator.java
    • -3
    • +2
    ./logical/AbstractDataSourceOperator.java
    • -1
    • +0
    ./logical/AbstractOperatorWithNestedPlans.java
    • -1
    • +0
    ./logical/DistributeResultOperator.java
    • -1
    • +1
    ./logical/LeftOuterUnnestMapOperator.java
    • -1
    • +0
    ./logical/RunningAggregateOperator.java
  1. … 611 more files in changeset.
[ASTERIXDB-2551][*DB] Use Ordered Property in range merge POperator

- user model changes: no

- storage format changes: no

- interface changes: no

details:

- Create the IPartitioningProperty using the orderedPartitionedProperty

instead of creating the list to be used in an unorderedPartitionProperty.

Change-Id: I044e2219fff7564864f68fa0df849d1102b4bb20

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3261

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

    • -7
    • +2
    ./physical/RangePartitionMergeExchangePOperator.java
[ASTERIXDB-2523][RT][COMP] add support for hashing array fields

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Add support for hashing array fields.

- Modified AMurmurHash3BinaryHashFunctionFamily and extracted the hashing function

into a private named hashing function "GenericHashFunction". Added hashing arrays.

- Modified hash join to include generating hash functions for the right branch

since now hash functions are type-dependent and cannot use the same hash functions

generated for the left branch.

- Added test cases.

Change-Id: Ibd0dc7f270730140226f54445705822049f5c863

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3241

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -11
    • +13
    ./physical/HybridHashJoinPOperator.java
    • -6
    • +9
    ./physical/InMemoryHashJoinPOperator.java
  1. … 53 more files in changeset.
[ASTERIXDB-2521][COMP] Add rule to eliminate isomorphic subplans

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Add optimizer rule that finds two isomorphic subplans

and eliminates one of them

Change-Id: I1142ea4805e7508a5d0a778081093504cf4f526a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3238

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 7 more files in changeset.
[ASTERIXDB-2516][RT] prepare physical comparators for deep comparison

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

This change is to make physical comparators type-aware in order to do

deep comparison of complex types like arrays and records. The IAType

is propagated to the comparators.

- added new methods in IBinaryComparatorFactoryProvider to accept the

type of left and right inputs for operations like hash join where

the join key types come from different dataset sources.

- defaulted some arrays functions to use the old comparator behaviour temporarily

until complex comparison is implemented

- modified AObjectAscBinaryComparatorFactory & AObjectDescBinaryComparatorFactory to

create a comparator with IAType information. Changed the serialization/deserialization

of their instances to take care of the newly added fields since they are not

present in old instances.

Change-Id: I02011e7151398d5f5f9ba9c1e1db6518484b9fe5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3229

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -1
    • +7
    ./physical/AbstractHashJoinPOperator.java
    • -18
    • +23
    ./physical/HybridHashJoinPOperator.java
    • -6
    • +9
    ./physical/InMemoryHashJoinPOperator.java
  1. … 22 more files in changeset.
[NO ISSUE][RT] Window operator runtime optimization

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Improve memory management for window operators

- Add "compiler.windowmemory" property that specifies memory

budget for each window operator (default is 4MB, min is 160KB)

- Consolidated negative window operator testcases into a single one

Change-Id: I6756e92046883f79db339ef490cca8bc8b7b1fb8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3227

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 58 more files in changeset.
[NO ISSUE][RT] Window operator runtime optimization

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Runtime optimization for window operators with accumulating

frames (unbounded preceding to current row or n following)

- Refactor window function properties into generic

builtin function properties

Change-Id: I8d1574defc73076ad960c4067432da29ead160a5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3151

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 29 more files in changeset.