Clone Tools
  • last updated 18 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Move Pregelix and Hivesterix codebase to new repositories: 1. Move Pregelix codebase to https://github.com/pregelix/pregelix; 2. Move Hivesterix codebase to https://code.google.com/p/hivesterix .

Change-Id: Iede698fcb92a0ad0a7a4918ea69b54886fd64fc7

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/155

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

    • -279
    • +0
    ./clusterconfig/ClusterConfig.java
  1. … 952 more files in changeset.
- Added Tokenize Operator in addition to the bulkload operator changes that were made by Zachary Heilbron. The tokenize operator is only added to the logical plan when bulk-loading the data. - Each secondary index is now updated in the separate branch by using the replicate operator. - Sink Operator now accepts multiple inputs. - Fixed the bulk-load so that it correctly produces auto-generated PK.

Change-Id: Ifb591754dba5eb4a9207edaa4e658f4cc745893a

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/78

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 54 more files in changeset.
Added LSM component-level filters for all indexes.

Change-Id: I898cf885c9f88feae85c99799a00fd8ec036efea

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/81

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 129 more files in changeset.
Support big vertex in Pregelix. --For those vertice beyond page size, we store them on HDFS as immutable files. --Updates on those big vertice will trigger creations of new immutable files.

Change-Id: I6b6f0528b6b5360c96dcdace1fa360d42c517f22

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/72

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Pouria Pirzadeh <pouria.pirzadeh@gmail.com>

  1. … 17 more files in changeset.
Several major changes in hyracks: -- reduced CC/NC communications for reporting partition request and availability; partition request/availability are only reported for the case of send-side materialized (without pipelining) policies in case of task re-attempt. -- changed buffer cache to dynamically allocate memory based on needs instead of pre-allocating -- changed each network channel to lazily allocate memory based on needs, and changed materialized connectors to lazily allocate files based on needs -- changed several major CCNCCFunctions to use non-java serde -- added a sort-based group-by operator which pushes group-by aggregations into an external sort -- make external sort a stable sort

1,3,and 4 is to reduce the job overhead.

2 is to reduce the unecessary NC resource consumptions such as memory and files.

5 and 6 are improvements to runtime operators.

One change in algebricks:

-- implemented a rule to push group-by aggregation into sort, i.e., using the sort-based gby operator

Several important changes in pregelix:

-- remove static states in vertex

-- direct check halt bit without deserialization

-- optimize the sort algorithm by packing yet-another 2-byte normalized key into the tPointers array

Change-Id: Id696f9a9f1647b4a025b8b33d20b3a89127c60d6

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/35

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <westmann@gmail.com>

    • -443
    • +0
    ./JobGenOuterJoinSingleSort.java
  1. … 271 more files in changeset.
fixed issue 731, 740, and more

commit 8911cc529e72e2bb544d9b472d6e10f173d173af

Author: Young-Seok <kisskys@gmail.com>

Date: Sun May 18 11:28:28 2014 -0700

another fix for picking available index for leftouterjoin plan

commit 9bce43087615fee53613467a027833dd53e190f9

Merge: c8e85ac efab69f

Author: Young-Seok <kisskys@gmail.com>

Date: Sun May 11 22:22:10 2014 -0700

merged master to kisskys/left-outer-join-issue branch

commit c8e85aca31545c13b2a02ff6dc259943e2cf66ad

Author: Young-Seok <kisskys@gmail.com>

Date: Sun May 11 20:17:17 2014 -0700

changes for left-outer-join to pick available indexes

Change-Id: Ib0fc186bc9388802f95445edee92c428b3bb69cc

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/34

Reviewed-by: Inci Cetindil <icetindil@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 55 more files in changeset.
fix for issue #732

  1. … 11 more files in changeset.
changes to fix issue 727

  1. … 11 more files in changeset.
fix to support non-default pregelix cc http ports

  1. … 1 more file in changeset.
support heterogenous cluster

    • -33
    • +33
    ./JobGenOuterJoinSingleSort.java
    • -21
    • +25
    ./clusterconfig/ClusterConfig.java
  1. … 31 more files in changeset.
let pregelix driver print perf. counter

  1. … 2 more files in changeset.
fix jobgen

1. add deployment retry 2. support plan switch

    • -0
    • +44
    ./JobGenFactory.java
  1. … 3 more files in changeset.
support multiple user-defined global aggregators

  1. … 31 more files in changeset.
fix file write race condition

    • -21
    • +23
    ./JobGenOuterJoinSingleSort.java
  1. … 19 more files in changeset.
fix the duplicate jobid issue

  1. … 1 more file in changeset.
support multiple concurrent jobs

  1. … 10 more files in changeset.
fix in-place update

  1. … 16 more files in changeset.
fix the issue found by Genomix P4 algorithm

  1. … 8 more files in changeset.
Merge branch 'master' into yingyi/fullstack_fix

  1. … 1 more file in changeset.
Allowed user to specify the merging policy

  1. … 22 more files in changeset.
fix raw binary comparator

  1. … 2 more files in changeset.
use raw binary comparator internally for pregelix

  1. … 2 more files in changeset.
support both quick sort and merge sort as in-memory sort algorithms, but merge sort is the default one

  1. … 11 more files in changeset.
fix the cache miss problem in the sort merge reader

  1. … 21 more files in changeset.
fix fault-tolerance and error reporting to handle disk failures

  1. … 29 more files in changeset.
1. fix the node failure scenario in job scheduler; 2. add fault-tolerance support and tests in pregelix

  1. … 16 more files in changeset.
fix the checkpoint writing for vertex checkpointing and message checkpointing

  1. … 4 more files in changeset.
support large-size global aggreate values

  1. … 22 more files in changeset.
clean up the checkpointing control

  1. … 1 more file in changeset.