Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Move Pregelix and Hivesterix codebase to new repositories: 1. Move Pregelix codebase to https://github.com/pregelix/pregelix; 2. Move Hivesterix codebase to https://code.google.com/p/hivesterix .

Change-Id: Iede698fcb92a0ad0a7a4918ea69b54886fd64fc7

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/155

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

    • -59
    • +0
    ./resources/conf/cluster.properties
    • -26
    • +0
    ./resources/conf/debugnc.properties
    • -21
    • +0
    ./resources/conf/topology-template.xml
    • -124
    • +0
    ./resources/scripts/pregelix.bat
  1. … 943 more files in changeset.
1. fix asterixdb issue 782 --- push nested pipeline before a nested group-by operator into the combiner group-by operator in the AbstractIntroduceGroupByCombinerRule --- add a processNullTest abstract method in the AbstractIntroduceGroupByCombinerRule -- fix the join order in a subplan 2. allow user-configurable buffer cache page size (B-tree page size) in Pregelix

commit 4d9a11d0c05281a41bbabe03066478fe851b3a2b

Author: buyingyi <buyingyi@gmail.com>

Change-Id: Ib7761370df8606c55ac34c126554319586e824f0

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/64

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <westmann@gmail.com>

  1. … 7 more files in changeset.
Several major changes in hyracks: -- reduced CC/NC communications for reporting partition request and availability; partition request/availability are only reported for the case of send-side materialized (without pipelining) policies in case of task re-attempt. -- changed buffer cache to dynamically allocate memory based on needs instead of pre-allocating -- changed each network channel to lazily allocate memory based on needs, and changed materialized connectors to lazily allocate files based on needs -- changed several major CCNCCFunctions to use non-java serde -- added a sort-based group-by operator which pushes group-by aggregations into an external sort -- make external sort a stable sort

1,3,and 4 is to reduce the job overhead.

2 is to reduce the unecessary NC resource consumptions such as memory and files.

5 and 6 are improvements to runtime operators.

One change in algebricks:

-- implemented a rule to push group-by aggregation into sort, i.e., using the sort-based gby operator

Several important changes in pregelix:

-- remove static states in vertex

-- direct check halt bit without deserialization

-- optimize the sort algorithm by packing yet-another 2-byte normalized key into the tPointers array

Change-Id: Id696f9a9f1647b4a025b8b33d20b3a89127c60d6

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/35

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <westmann@gmail.com>

  1. … 276 more files in changeset.
compatibility for for bash versions <4.0

add (optional) CC_HTTPPORT and JOB_HISTORY_SIZE to conf

fix for linux setting

1. make startcc/nc scripts flexible for different physical memory size; 2. add dynamic optimization option in the Client

  1. … 4 more files in changeset.
make failure-recovery work in multi-JVM test

  1. … 6 more files in changeset.
merge from zheilbron/hyracks_msr

  1. … 288 more files in changeset.
add debugging utility scripts

    • -0
    • +7
    ./resources/scripts/copylog.sh
    • -0
    • +12
    ./resources/scripts/dumpAll.sh
    • -0
    • +15
    ./resources/scripts/dumptrace.sh
  1. … 3 more files in changeset.
cleanup hivesterix poms and assembly

  1. … 13 more files in changeset.
add/update license headers

    • -0
    • +14
    ./resources/conf/cluster.properties
    • -0
    • +14
    ./resources/conf/debugnc.properties
    • -0
    • +14
    ./resources/conf/topology-template.xml
    • -0
    • +14
    ./resources/scripts/startCluster.sh
    • -0
    • +14
    ./resources/scripts/startDebugNc.sh
  1. … 2273 more files in changeset.
update pregelix pom to let it have a distribution zip after build

    • -0
    • +41
    ./assembly/binary-assembly.xml
    • -0
    • +36
    ./resources/conf/cluster.properties
    • -0
    • +12
    ./resources/conf/debugnc.properties
    • -0
    • +1
    ./resources/conf/master
    • -0
    • +1
    ./resources/conf/slaves
    • -0
    • +2
    ./resources/conf/stores.properties
    • -0
    • +7
    ./resources/conf/topology-template.xml
    • -0
    • +43
    ./resources/scripts/getip.sh
    • -0
    • +111
    ./resources/scripts/pregelix
    • -0
    • +110
    ./resources/scripts/pregelix.bat
    • -0
    • +114
    ./resources/scripts/pregelixcc
    • -0
    • +115
    ./resources/scripts/pregelixnc
    • -0
    • +24
    ./resources/scripts/startAllNCs.sh
    • -0
    • +21
    ./resources/scripts/startCluster.sh
    • -0
    • +68
    ./resources/scripts/startDebugNc.sh
  1. … 8 more files in changeset.