Clone Tools
  • last updated 13 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Added replicate operator with materialization

be more aggressive to find shared plans in ExtractCommonOperatorRule

- find all the isomorphic subgraphs instead of just the ones on join build branches

- while expanding candidates handle the operators with multiple inputs

- analyze the DAG to find all the operators that can be co-scheduled, and infer the dependencies between clusters

- based on the dependencies, decide which outputs of a replicate operator needs materialization

- if the shared branch needs materialization, and it consists of only trivial operators (such as assign, unnest, datasource scan), that branch is discarded from the candidates

- modified the replicate operator descriptor to materialize the input if needed, and read from the materialized file for the outputs that requires materialization

- removed redundant decor variables in group-by

- fixed a bug on computing live variables for unnest-map operator: if the operator does not propagate inputs, those input variables should not be live anymore

- fixed a bug in ComplexUnnestToProductRule

Change-Id: If221d1507844f9409bf1163f93b0c04ef5848578

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/86

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 42 more files in changeset.
Added LSM component-level filters for all indexes.

Change-Id: I898cf885c9f88feae85c99799a00fd8ec036efea

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/81

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 130 more files in changeset.
1. fix asterixdb issue 782 --- push nested pipeline before a nested group-by operator into the combiner group-by operator in the AbstractIntroduceGroupByCombinerRule --- add a processNullTest abstract method in the AbstractIntroduceGroupByCombinerRule -- fix the join order in a subplan 2. allow user-configurable buffer cache page size (B-tree page size) in Pregelix

commit 4d9a11d0c05281a41bbabe03066478fe851b3a2b

Author: buyingyi <buyingyi@gmail.com>

Change-Id: Ib7761370df8606c55ac34c126554319586e824f0

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/64

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <westmann@gmail.com>

  1. … 5 more files in changeset.
Several major changes in hyracks: -- reduced CC/NC communications for reporting partition request and availability; partition request/availability are only reported for the case of send-side materialized (without pipelining) policies in case of task re-attempt. -- changed buffer cache to dynamically allocate memory based on needs instead of pre-allocating -- changed each network channel to lazily allocate memory based on needs, and changed materialized connectors to lazily allocate files based on needs -- changed several major CCNCCFunctions to use non-java serde -- added a sort-based group-by operator which pushes group-by aggregations into an external sort -- make external sort a stable sort

1,3,and 4 is to reduce the job overhead.

2 is to reduce the unecessary NC resource consumptions such as memory and files.

5 and 6 are improvements to runtime operators.

One change in algebricks:

-- implemented a rule to push group-by aggregation into sort, i.e., using the sort-based gby operator

Several important changes in pregelix:

-- remove static states in vertex

-- direct check halt bit without deserialization

-- optimize the sort algorithm by packing yet-another 2-byte normalized key into the tPointers array

Change-Id: Id696f9a9f1647b4a025b8b33d20b3a89127c60d6

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/35

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <westmann@gmail.com>

  1. … 275 more files in changeset.
fixed issue 731, 740, and more

commit 8911cc529e72e2bb544d9b472d6e10f173d173af

Author: Young-Seok <kisskys@gmail.com>

Date: Sun May 18 11:28:28 2014 -0700

another fix for picking available index for leftouterjoin plan

commit 9bce43087615fee53613467a027833dd53e190f9

Merge: c8e85ac efab69f

Author: Young-Seok <kisskys@gmail.com>

Date: Sun May 11 22:22:10 2014 -0700

merged master to kisskys/left-outer-join-issue branch

commit c8e85aca31545c13b2a02ff6dc259943e2cf66ad

Author: Young-Seok <kisskys@gmail.com>

Date: Sun May 11 20:17:17 2014 -0700

changes for left-outer-join to pick available indexes

Change-Id: Ib0fc186bc9388802f95445edee92c428b3bb69cc

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/34

Reviewed-by: Inci Cetindil <icetindil@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 51 more files in changeset.
fixing issue #352

Merge branch 'master' into zheilbron/hyracks_msr_demo

  1. … 2 more files in changeset.
Revert changes to InlineVariablesRule.

reverted the change of removing adjacent exchange operators

Updated the policy to have a boolean function for entering nested plans.

Code style fix.

The check for do not inline functions has been moved to a policy that is in Asterix.

do not apply PullSelectOutOfEqJoin for LOJ

Update the interface to include better names and arguments.

ensure limits are copied down as far as possible and not through select operators

    • -0
    • +105
    ./uci/ics/hyracks/algebricks/rewriter/rules/CopyLimitDownRule.java
  1. … 3 more files in changeset.
don't inline non-functional functions

disable common subexpression elimination for non-functional functions

Fixed the incorrect exchange merging introduced by the previous commit; updated the IntroHashPartitionMergeExchange rule to handle the hash-merge-exchange operator.

Fixed a bug on omitted order by columns when added an exchange operator to enforce the group-by property.

  1. … 1 more file in changeset.
Fixed a bug on unclosed running aggregation runtime; fixed an issue on two adjacent exchange operators (connectors) when duplicate sort operator is removed.

  1. … 4 more files in changeset.
Fixed a bug on unclosed running aggregation runtime; fixed an issue on two adjacent exchange operators (connectors) when duplicate sort operator is removed.

checkpoint: fixed a issue when introducing the HashMergeExcnahge but have not compute its delivered property, which could cause a NPE when its delivered property by other operator.

make more rules aware of non-functional functions

add a rule to eliminate empty-key gby

add a rule to converet left outer join to inner join

add a rule to converet left outer join to inner join

merge from zheilbron/hyracks_msr

  1. … 291 more files in changeset.
fix the execution mode of operators and the location constraint of algebricks meta operator

  1. … 29 more files in changeset.
Creating a branch for changes required for VXQuery. This commit includes the patch for issue 99.

add/update license headers

  1. … 2273 more files in changeset.