Clone Tools
  • last updated 19 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Move Pregelix and Hivesterix codebase to new repositories: 1. Move Pregelix codebase to https://github.com/pregelix/pregelix; 2. Move Hivesterix codebase to https://code.google.com/p/hivesterix .

Change-Id: Iede698fcb92a0ad0a7a4918ea69b54886fd64fc7

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/155

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

    • -114
    • +0
    ./FunctionCallOperatorDescriptor.java
    • -177
    • +0
    ./IndexNestedLoopJoinFunctionUpdateOperatorDescriptor.java
    • -265
    • +0
    ./IndexNestedLoopJoinFunctionUpdateOperatorNodePushable.java
    • -123
    • +0
    ./IndexNestedLoopJoinOperatorDescriptor.java
    • -208
    • +0
    ./IndexNestedLoopJoinOperatorNodePushable.java
    • -332
    • +0
    ./IndexNestedLoopRightOuterJoinFunctionUpdateOperatorNodePushable.java
    • -283
    • +0
    ./IndexNestedLoopRightOuterJoinOperatorNodePushable.java
    • -279
    • +0
    ./IndexNestedLoopSetUnionFunctionUpdateOperatorNodePushable.java
    • -270
    • +0
    ./IndexNestedLoopSetUnionOperatorNodePushable.java
    • -105
    • +0
    ./ProjectOperatorDescriptor.java
    • -70
    • +0
    ./RuntimeHookOperatorDescriptor.java
    • -63
    • +0
    ./TreeIndexBulkReLoadOperatorDescriptor.java
    • -110
    • +0
    ./TreeIndexBulkReLoadOperatorNodePushable.java
    • -92
    • +0
    ./TreeSearchFunctionUpdateOperatorDescriptor.java
    • -261
    • +0
    ./TreeSearchFunctionUpdateOperatorNodePushable.java
  1. … 943 more files in changeset.
Support big vertex in Pregelix. --For those vertice beyond page size, we store them on HDFS as immutable files. --Updates on those big vertice will trigger creations of new immutable files.

Change-Id: I6b6f0528b6b5360c96dcdace1fa360d42c517f22

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/72

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Pouria Pirzadeh <pouria.pirzadeh@gmail.com>

  1. … 17 more files in changeset.
Several major changes in hyracks: -- reduced CC/NC communications for reporting partition request and availability; partition request/availability are only reported for the case of send-side materialized (without pipelining) policies in case of task re-attempt. -- changed buffer cache to dynamically allocate memory based on needs instead of pre-allocating -- changed each network channel to lazily allocate memory based on needs, and changed materialized connectors to lazily allocate files based on needs -- changed several major CCNCCFunctions to use non-java serde -- added a sort-based group-by operator which pushes group-by aggregations into an external sort -- make external sort a stable sort

1,3,and 4 is to reduce the job overhead.

2 is to reduce the unecessary NC resource consumptions such as memory and files.

5 and 6 are improvements to runtime operators.

One change in algebricks:

-- implemented a rule to push group-by aggregation into sort, i.e., using the sort-based gby operator

Several important changes in pregelix:

-- remove static states in vertex

-- direct check halt bit without deserialization

-- optimize the sort algorithm by packing yet-another 2-byte normalized key into the tPointers array

Change-Id: Id696f9a9f1647b4a025b8b33d20b3a89127c60d6

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/35

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <westmann@gmail.com>

    • -15
    • +13
    ./IndexNestedLoopJoinFunctionUpdateOperatorNodePushable.java
    • -6
    • +6
    ./IndexNestedLoopRightOuterJoinFunctionUpdateOperatorNodePushable.java
    • -5
    • +5
    ./IndexNestedLoopSetUnionFunctionUpdateOperatorNodePushable.java
    • -3
    • +13
    ./IndexNestedLoopSetUnionOperatorNodePushable.java
    • -1
    • +1
    ./TreeIndexBulkReLoadOperatorNodePushable.java
    • -5
    • +5
    ./TreeSearchFunctionUpdateOperatorNodePushable.java
    • -0
    • +78
    ./collectors/SortMergeFrameReader.java
    • -0
    • +72
    ./connectors/MToNPartitioningMergingConnectorDescriptor.java
    • -0
    • +50
    ./group/ClusteredGroupOperatorDescriptor.java
    • -0
    • +79
    ./group/ClusteredGroupOperatorNodePushable.java
    • -0
    • +144
    ./group/ClusteredGroupWriter.java
    • -0
    • +105
    ./group/IAggregatorDescriptor.java
    • -0
    • +35
    ./group/IClusteredAggregatorDescriptorFactory.java
    • -0
    • +114
    ./sort/ExternalSortRunGenerator.java
    • -0
    • +159
    ./sort/ExternalSortRunMerger.java
  1. … 262 more files in changeset.
fixed issue 731, 740, and more

commit 8911cc529e72e2bb544d9b472d6e10f173d173af

Author: Young-Seok <kisskys@gmail.com>

Date: Sun May 18 11:28:28 2014 -0700

another fix for picking available index for leftouterjoin plan

commit 9bce43087615fee53613467a027833dd53e190f9

Merge: c8e85ac efab69f

Author: Young-Seok <kisskys@gmail.com>

Date: Sun May 11 22:22:10 2014 -0700

merged master to kisskys/left-outer-join-issue branch

commit c8e85aca31545c13b2a02ff6dc259943e2cf66ad

Author: Young-Seok <kisskys@gmail.com>

Date: Sun May 11 20:17:17 2014 -0700

changes for left-outer-join to pick available indexes

Change-Id: Ib0fc186bc9388802f95445edee92c428b3bb69cc

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/34

Reviewed-by: Inci Cetindil <icetindil@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -6
    • +6
    ./IndexNestedLoopJoinFunctionUpdateOperatorDescriptor.java
    • -6
    • +6
    ./IndexNestedLoopJoinOperatorDescriptor.java
    • -2
    • +2
    ./TreeIndexBulkReLoadOperatorDescriptor.java
    • -2
    • +2
    ./TreeSearchFunctionUpdateOperatorDescriptor.java
  1. … 53 more files in changeset.
fix application lifecyle mgmt in hyracks nc

    • -2
    • +10
    ./IndexNestedLoopJoinFunctionUpdateOperatorNodePushable.java
    • -2
    • +10
    ./IndexNestedLoopRightOuterJoinFunctionUpdateOperatorNodePushable.java
    • -4
    • +13
    ./IndexNestedLoopSetUnionFunctionUpdateOperatorNodePushable.java
    • -5
    • +13
    ./TreeSearchFunctionUpdateOperatorNodePushable.java
  1. … 3 more files in changeset.
fix the pinned page issue during a node failure

    • -0
    • +1
    ./IndexNestedLoopJoinFunctionUpdateOperatorNodePushable.java
    • -0
    • +1
    ./IndexNestedLoopRightOuterJoinFunctionUpdateOperatorNodePushable.java
    • -1
    • +2
    ./TreeSearchFunctionUpdateOperatorNodePushable.java
  1. … 1 more file in changeset.
fix IIndexAccessor interface, add a boolean exclusiveMode parameter for the createSearchCursor method

    • -1
    • +1
    ./IndexNestedLoopJoinFunctionUpdateOperatorNodePushable.java
    • -1
    • +1
    ./IndexNestedLoopJoinOperatorNodePushable.java
    • -1
    • +1
    ./IndexNestedLoopRightOuterJoinFunctionUpdateOperatorNodePushable.java
    • -1
    • +1
    ./IndexNestedLoopRightOuterJoinOperatorNodePushable.java
    • -1
    • +1
    ./IndexNestedLoopSetUnionFunctionUpdateOperatorNodePushable.java
    • -1
    • +1
    ./IndexNestedLoopSetUnionOperatorNodePushable.java
    • -1
    • +1
    ./TreeSearchFunctionUpdateOperatorNodePushable.java
  1. … 37 more files in changeset.
fix in-place update

    • -2
    • +10
    ./IndexNestedLoopJoinFunctionUpdateOperatorNodePushable.java
    • -4
    • +12
    ./IndexNestedLoopRightOuterJoinFunctionUpdateOperatorNodePushable.java
    • -4
    • +12
    ./IndexNestedLoopSetUnionFunctionUpdateOperatorNodePushable.java
    • -2
    • +10
    ./TreeSearchFunctionUpdateOperatorNodePushable.java
  1. … 13 more files in changeset.
fix the issue found by Genomix P4 algorithm

    • -1
    • +1
    ./IndexNestedLoopJoinFunctionUpdateOperatorNodePushable.java
    • -2
    • +2
    ./IndexNestedLoopRightOuterJoinFunctionUpdateOperatorNodePushable.java
    • -2
    • +2
    ./IndexNestedLoopSetUnionFunctionUpdateOperatorNodePushable.java
    • -1
    • +1
    ./TreeSearchFunctionUpdateOperatorNodePushable.java
  1. … 10 more files in changeset.
fix fault-tolerance and error reporting to handle disk failures

    • -3
    • +2
    ./TreeIndexBulkReLoadOperatorNodePushable.java
  1. … 31 more files in changeset.
1. fix the node failure scenario in job scheduler; 2. add fault-tolerance support and tests in pregelix

    • -1
    • +9
    ./IndexNestedLoopJoinFunctionUpdateOperatorNodePushable.java
    • -1
    • +9
    ./IndexNestedLoopRightOuterJoinFunctionUpdateOperatorNodePushable.java
    • -0
    • +7
    ./IndexNestedLoopRightOuterJoinOperatorNodePushable.java
    • -1
    • +9
    ./IndexNestedLoopSetUnionFunctionUpdateOperatorNodePushable.java
    • -0
    • +7
    ./IndexNestedLoopSetUnionOperatorNodePushable.java
    • -1
    • +7
    ./TreeIndexBulkReLoadOperatorNodePushable.java
    • -1
    • +9
    ./TreeSearchFunctionUpdateOperatorNodePushable.java
  1. … 14 more files in changeset.
merge from zheilbron/hyracks_msr

    • -4
    • +22
    ./IndexNestedLoopJoinFunctionUpdateOperatorNodePushable.java
    • -7
    • +21
    ./IndexNestedLoopRightOuterJoinFunctionUpdateOperatorNodePushable.java
  1. … 290 more files in changeset.
add message overflow support

    • -4
    • +22
    ./IndexNestedLoopJoinFunctionUpdateOperatorNodePushable.java
    • -7
    • +21
    ./IndexNestedLoopRightOuterJoinFunctionUpdateOperatorNodePushable.java
  1. … 88 more files in changeset.
Merged master

    • -6
    • +3
    ./IndexNestedLoopJoinFunctionUpdateOperatorDescriptor.java
    • -28
    • +22
    ./IndexNestedLoopJoinFunctionUpdateOperatorNodePushable.java
    • -26
    • +21
    ./IndexNestedLoopRightOuterJoinFunctionUpdateOperatorNodePushable.java
    • -25
    • +20
    ./IndexNestedLoopSetUnionFunctionUpdateOperatorNodePushable.java
    • -0
    • +92
    ./TreeSearchFunctionUpdateOperatorDescriptor.java
    • -0
    • +236
    ./TreeSearchFunctionUpdateOperatorNodePushable.java
  1. … 83 more files in changeset.
fix inl right outer join

    • -1
    • +1
    ./IndexNestedLoopRightOuterJoinOperatorNodePushable.java
  1. … 1 more file in changeset.
checkin the fixed bulk reload for Sattam to debug

    • -0
    • +2
    ./TreeIndexBulkReLoadOperatorNodePushable.java
  1. … 1 more file in changeset.
fix parameters

    • -2
    • +2
    ./TreeIndexBulkReLoadOperatorDescriptor.java
  1. … 2 more files in changeset.
for sattam to debug

    • -3
    • +3
    ./IndexNestedLoopJoinFunctionUpdateOperatorNodePushable.java
    • -3
    • +3
    ./IndexNestedLoopJoinOperatorNodePushable.java
    • -3
    • +3
    ./IndexNestedLoopRightOuterJoinFunctionUpdateOperatorNodePushable.java
    • -3
    • +3
    ./IndexNestedLoopRightOuterJoinOperatorNodePushable.java
    • -3
    • +3
    ./IndexNestedLoopSetUnionFunctionUpdateOperatorNodePushable.java
    • -3
    • +3
    ./IndexNestedLoopSetUnionOperatorNodePushable.java
    • -3
    • +3
    ./TreeIndexBulkReLoadOperatorNodePushable.java
    • -4
    • +4
    ./TreeSearchFunctionUpdateOperatorNodePushable.java
  1. … 2 more files in changeset.
add LSM support in pregelix

    • -5
    • +2
    ./IndexNestedLoopJoinFunctionUpdateOperatorDescriptor.java
    • -2
    • +1
    ./IndexNestedLoopJoinOperatorDescriptor.java
  1. … 14 more files in changeset.
refactoring dataflow operators to be more general--suport both b-tree and lsm b-tree

    • -91
    • +0
    ./BTreeSearchFunctionUpdateOperatorDescriptor.java
    • -212
    • +0
    ./BTreeSearchFunctionUpdateOperatorNodePushable.java
    • -23
    • +17
    ./IndexNestedLoopJoinFunctionUpdateOperatorNodePushable.java
    • -11
    • +11
    ./IndexNestedLoopJoinOperatorNodePushable.java
    • -22
    • +17
    ./IndexNestedLoopRightOuterJoinFunctionUpdateOperatorNodePushable.java
    • -19
    • +14
    ./IndexNestedLoopRightOuterJoinOperatorNodePushable.java
    • -20
    • +15
    ./IndexNestedLoopSetUnionFunctionUpdateOperatorNodePushable.java
    • -18
    • +13
    ./IndexNestedLoopSetUnionOperatorNodePushable.java
    • -0
    • +91
    ./TreeSearchFunctionUpdateOperatorDescriptor.java
    • -0
    • +235
    ./TreeSearchFunctionUpdateOperatorNodePushable.java
  1. … 7 more files in changeset.
Pass a boolean argument to the bulkload to decides if checking for an empty index is needed.

    • -1
    • +1
    ./TreeIndexBulkReLoadOperatorNodePushable.java
  1. … 25 more files in changeset.
add/update license headers

    • -1
    • +1
    ./BTreeSearchFunctionUpdateOperatorDescriptor.java
    • -1
    • +1
    ./BTreeSearchFunctionUpdateOperatorNodePushable.java
    • -1
    • +1
    ./IndexNestedLoopJoinFunctionUpdateOperatorDescriptor.java
    • -1
    • +1
    ./IndexNestedLoopJoinFunctionUpdateOperatorNodePushable.java
    • -1
    • +1
    ./IndexNestedLoopJoinOperatorDescriptor.java
    • -1
    • +1
    ./IndexNestedLoopJoinOperatorNodePushable.java
    • -1
    • +1
    ./IndexNestedLoopRightOuterJoinFunctionUpdateOperatorNodePushable.java
    • -1
    • +1
    ./IndexNestedLoopRightOuterJoinOperatorNodePushable.java
    • -1
    • +1
    ./IndexNestedLoopSetUnionFunctionUpdateOperatorNodePushable.java
    • -1
    • +1
    ./IndexNestedLoopSetUnionOperatorNodePushable.java
    • -1
    • +1
    ./TreeIndexBulkReLoadOperatorDescriptor.java
    • -1
    • +1
    ./TreeIndexBulkReLoadOperatorNodePushable.java
  1. … 2273 more files in changeset.
Added memory manager

    • -1
    • +2
    ./BTreeSearchFunctionUpdateOperatorDescriptor.java
    • -1
    • +2
    ./BTreeSearchFunctionUpdateOperatorNodePushable.java
    • -1
    • +2
    ./IndexNestedLoopJoinFunctionUpdateOperatorDescriptor.java
    • -1
    • +2
    ./IndexNestedLoopJoinFunctionUpdateOperatorNodePushable.java
    • -1
    • +1
    ./IndexNestedLoopRightOuterJoinFunctionUpdateOperatorNodePushable.java
    • -1
    • +1
    ./IndexNestedLoopSetUnionFunctionUpdateOperatorNodePushable.java
  1. … 84 more files in changeset.
fix the multiple deletion issue in large amounts of deletions

    • -15
    • +5
    ./BTreeSearchFunctionUpdateOperatorNodePushable.java
    • -15
    • +6
    ./IndexNestedLoopJoinFunctionUpdateOperatorNodePushable.java
    • -42
    • +19
    ./IndexNestedLoopRightOuterJoinFunctionUpdateOperatorNodePushable.java
    • -30
    • +7
    ./IndexNestedLoopSetUnionFunctionUpdateOperatorNodePushable.java
  1. … 3 more files in changeset.
reintegrate fullstack_dynamic_deployment

  1. … 203 more files in changeset.
Merged fullstack_lsm_staging upto r3336

git-svn-id: https://hyracks.googlecode.com/svn/trunk/fullstack@3339 123451ca-8445-de46-9d55-352943316053

    • -6
    • +8
    ./BTreeSearchFunctionUpdateOperatorDescriptor.java
    • -9
    • +10
    ./BTreeSearchFunctionUpdateOperatorNodePushable.java
    • -12
    • +18
    ./IndexNestedLoopJoinFunctionUpdateOperatorDescriptor.java
    • -10
    • +10
    ./IndexNestedLoopJoinFunctionUpdateOperatorNodePushable.java
    • -12
    • +15
    ./IndexNestedLoopJoinOperatorDescriptor.java
    • -9
    • +10
    ./IndexNestedLoopJoinOperatorNodePushable.java
    • -9
    • +10
    ./IndexNestedLoopRightOuterJoinFunctionUpdateOperatorNodePushable.java
    • -9
    • +10
    ./IndexNestedLoopRightOuterJoinOperatorNodePushable.java
    • -9
    • +10
    ./IndexNestedLoopSetUnionFunctionUpdateOperatorNodePushable.java
    • -9
    • +10
    ./IndexNestedLoopSetUnionOperatorNodePushable.java
    • -9
    • +10
    ./TreeIndexBulkReLoadOperatorDescriptor.java
    • -80
    • +29
    ./TreeIndexBulkReLoadOperatorNodePushable.java
  1. … 887 more files in changeset.
cross merge fullstack_release_candidate into trunk

git-svn-id: https://hyracks.googlecode.com/svn/trunk/fullstack@3208 123451ca-8445-de46-9d55-352943316053

    • -1
    • +27
    ./BTreeSearchFunctionUpdateOperatorNodePushable.java
    • -109
    • +0
    ./FileWriteOperatorDescriptor.java
    • -21
    • +24
    ./FunctionCallOperatorDescriptor.java
    • -24
    • +25
    ./IndexNestedLoopJoinFunctionUpdateOperatorNodePushable.java
    • -29
    • +55
    ./IndexNestedLoopRightOuterJoinFunctionUpdateOperatorNodePushable.java
    • -25
    • +43
    ./IndexNestedLoopSetUnionFunctionUpdateOperatorNodePushable.java
  1. … 897 more files in changeset.
merged hyracks_lsm_tree and fullstack_asterix_stabilization

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_lsm_staging@3026 123451ca-8445-de46-9d55-352943316053

    • -6
    • +8
    ./BTreeSearchFunctionUpdateOperatorDescriptor.java
    • -9
    • +10
    ./BTreeSearchFunctionUpdateOperatorNodePushable.java
    • -12
    • +18
    ./IndexNestedLoopJoinFunctionUpdateOperatorDescriptor.java
    • -10
    • +10
    ./IndexNestedLoopJoinFunctionUpdateOperatorNodePushable.java
    • -12
    • +15
    ./IndexNestedLoopJoinOperatorDescriptor.java
    • -9
    • +10
    ./IndexNestedLoopJoinOperatorNodePushable.java
    • -9
    • +10
    ./IndexNestedLoopRightOuterJoinFunctionUpdateOperatorNodePushable.java
    • -9
    • +10
    ./IndexNestedLoopRightOuterJoinOperatorNodePushable.java
    • -9
    • +10
    ./IndexNestedLoopSetUnionFunctionUpdateOperatorNodePushable.java
    • -9
    • +10
    ./IndexNestedLoopSetUnionOperatorNodePushable.java
    • -9
    • +10
    ./TreeIndexBulkReLoadOperatorDescriptor.java
    • -80
    • +29
    ./TreeIndexBulkReLoadOperatorNodePushable.java
  1. … 751 more files in changeset.
Merged fullstack_asterix_stabilization -r 2813:2933

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_hyracks_ioc@2995 123451ca-8445-de46-9d55-352943316053

    • -21
    • +24
    ./FunctionCallOperatorDescriptor.java
  1. … 90 more files in changeset.
Merge fullstack_asterix_stabilization into fullstack_hyracks_result_distribution.

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_hyracks_result_distribution@2965 123451ca-8445-de46-9d55-352943316053

    • -21
    • +24
    ./FunctionCallOperatorDescriptor.java
  1. … 91 more files in changeset.