Clone Tools
  • last updated 12 mins ago
Constraints: committers
Constraints: files
Constraints: dates
This change list includes several fixes: 1. Adds a rule to push subplan into group-by 2. Adds a rule to eliminate subplan with input cardinality one 3. Fix the nested running aggregate runtime 4. Adds a wrapper of FrameTupleAppender to internally flush full frames. A TODO item is to cleanup existing usage of FrameTupleAppender to use the wrapper, which makes code simpler.

Change-Id: I647f9bce2f40700b18bdcad1fa64fb8f0a26838b


Tested-by: Jenkins <>

Reviewed-by: Preston Carman <>

Reviewed-by: Till Westmann <>

  1. … 10 more files in changeset.
Fix the data property inference: 1. Fix order property,a LocalOrderProperty stores an array of OrderColumns instead of one OrderColumn. A delivered order property D satisfies a required order property R if R's sorting columns are a prefix of D's sorting columns. 2. Fix partition proerty inference, a delivered partition property D satisfies a required partition property R if D's partitioning columns are a prefix of R's partitioning columns. 3. Fix the data property progatation, e.g., what data properties are left after passing through a project operator. 4. Fix the data property within a group. For example, order property ($1 ASC, $2 ASC) is delivered to a group-by operator with $1 as the group key, within a particular group, ($2 ASC) is a valid data property.

Change-Id: If812fe7dca9c1714780734af425a1bb363db125f


Reviewed-by: abdullah alamoudi <>

Tested-by: Jenkins <>

  1. … 13 more files in changeset.
- Added Tokenize Operator in addition to the bulkload operator changes that were made by Zachary Heilbron. The tokenize operator is only added to the logical plan when bulk-loading the data. - Each secondary index is now updated in the separate branch by using the replicate operator. - Sink Operator now accepts multiple inputs. - Fixed the bulk-load so that it correctly produces auto-generated PK.

Change-Id: Ifb591754dba5eb4a9207edaa4e658f4cc745893a


Reviewed-by: Young-Seok Kim <>

Tested-by: Jenkins <>

  1. … 54 more files in changeset.
Added replicate operator with materialization

be more aggressive to find shared plans in ExtractCommonOperatorRule

- find all the isomorphic subgraphs instead of just the ones on join build branches

- while expanding candidates handle the operators with multiple inputs

- analyze the DAG to find all the operators that can be co-scheduled, and infer the dependencies between clusters

- based on the dependencies, decide which outputs of a replicate operator needs materialization

- if the shared branch needs materialization, and it consists of only trivial operators (such as assign, unnest, datasource scan), that branch is discarded from the candidates

- modified the replicate operator descriptor to materialize the input if needed, and read from the materialized file for the outputs that requires materialization

- removed redundant decor variables in group-by

- fixed a bug on computing live variables for unnest-map operator: if the operator does not propagate inputs, those input variables should not be live anymore

- fixed a bug in ComplexUnnestToProductRule

Change-Id: If221d1507844f9409bf1163f93b0c04ef5848578


Tested-by: Jenkins <>

Reviewed-by: Yingyi Bu <>

  1. … 42 more files in changeset.
Added LSM component-level filters for all indexes.

Change-Id: I898cf885c9f88feae85c99799a00fd8ec036efea


Tested-by: Jenkins <>

Reviewed-by: Yingyi Bu <>

  1. … 130 more files in changeset.
1. fix asterixdb issue 782 --- push nested pipeline before a nested group-by operator into the combiner group-by operator in the AbstractIntroduceGroupByCombinerRule --- add a processNullTest abstract method in the AbstractIntroduceGroupByCombinerRule -- fix the join order in a subplan 2. allow user-configurable buffer cache page size (B-tree page size) in Pregelix

commit 4d9a11d0c05281a41bbabe03066478fe851b3a2b

Author: buyingyi <>

Change-Id: Ib7761370df8606c55ac34c126554319586e824f0


Tested-by: Jenkins <>

Reviewed-by: Till Westmann <>

  1. … 5 more files in changeset.
Several major changes in hyracks: -- reduced CC/NC communications for reporting partition request and availability; partition request/availability are only reported for the case of send-side materialized (without pipelining) policies in case of task re-attempt. -- changed buffer cache to dynamically allocate memory based on needs instead of pre-allocating -- changed each network channel to lazily allocate memory based on needs, and changed materialized connectors to lazily allocate files based on needs -- changed several major CCNCCFunctions to use non-java serde -- added a sort-based group-by operator which pushes group-by aggregations into an external sort -- make external sort a stable sort

1,3,and 4 is to reduce the job overhead.

2 is to reduce the unecessary NC resource consumptions such as memory and files.

5 and 6 are improvements to runtime operators.

One change in algebricks:

-- implemented a rule to push group-by aggregation into sort, i.e., using the sort-based gby operator

Several important changes in pregelix:

-- remove static states in vertex

-- direct check halt bit without deserialization

-- optimize the sort algorithm by packing yet-another 2-byte normalized key into the tPointers array

Change-Id: Id696f9a9f1647b4a025b8b33d20b3a89127c60d6


Tested-by: Jenkins <>

Reviewed-by: Till Westmann <>

  1. … 275 more files in changeset.
fixed issue 731, 740, and more

commit 8911cc529e72e2bb544d9b472d6e10f173d173af

Author: Young-Seok <>

Date: Sun May 18 11:28:28 2014 -0700

another fix for picking available index for leftouterjoin plan

commit 9bce43087615fee53613467a027833dd53e190f9

Merge: c8e85ac efab69f

Author: Young-Seok <>

Date: Sun May 11 22:22:10 2014 -0700

merged master to kisskys/left-outer-join-issue branch

commit c8e85aca31545c13b2a02ff6dc259943e2cf66ad

Author: Young-Seok <>

Date: Sun May 11 20:17:17 2014 -0700

changes for left-outer-join to pick available indexes

Change-Id: Ib0fc186bc9388802f95445edee92c428b3bb69cc


Reviewed-by: Inci Cetindil <>

Tested-by: Jenkins <>

  1. … 51 more files in changeset.
fixing issue #352

Merge branch 'master' into zheilbron/hyracks_msr_demo

  1. … 2 more files in changeset.
Revert changes to InlineVariablesRule.

reverted the change of removing adjacent exchange operators

Updated the policy to have a boolean function for entering nested plans.

Code style fix.

The check for do not inline functions has been moved to a policy that is in Asterix.

do not apply PullSelectOutOfEqJoin for LOJ

Update the interface to include better names and arguments.

ensure limits are copied down as far as possible and not through select operators

    • -0
    • +105
  1. … 3 more files in changeset.
don't inline non-functional functions

disable common subexpression elimination for non-functional functions

Fixed the incorrect exchange merging introduced by the previous commit; updated the IntroHashPartitionMergeExchange rule to handle the hash-merge-exchange operator.

Fixed a bug on omitted order by columns when added an exchange operator to enforce the group-by property.

  1. … 1 more file in changeset.
Fixed a bug on unclosed running aggregation runtime; fixed an issue on two adjacent exchange operators (connectors) when duplicate sort operator is removed.

  1. … 4 more files in changeset.
Fixed a bug on unclosed running aggregation runtime; fixed an issue on two adjacent exchange operators (connectors) when duplicate sort operator is removed.

checkpoint: fixed a issue when introducing the HashMergeExcnahge but have not compute its delivered property, which could cause a NPE when its delivered property by other operator.

make more rules aware of non-functional functions

add a rule to eliminate empty-key gby

add a rule to converet left outer join to inner join

add a rule to converet left outer join to inner join

merge from zheilbron/hyracks_msr

  1. … 291 more files in changeset.