Clone Tools
  • last updated 12 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Move Pregelix and Hivesterix codebase to new repositories: 1. Move Pregelix codebase to https://github.com/pregelix/pregelix; 2. Move Hivesterix codebase to https://code.google.com/p/hivesterix .

Change-Id: Iede698fcb92a0ad0a7a4918ea69b54886fd64fc7

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/155

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

    • -603
    • +0
    ./java/edu/uci/ics/pregelix/core/join/JoinTest.java
    • -51
    • +0
    ./resources/cluster/cluster.properties
    • -15
    • +0
    ./resources/cluster/data.properties
    • -32
    • +0
    ./resources/hadoop/conf/core-site.xml
    • -32
    • +0
    ./resources/hadoop/conf/hdfs-site.xml
    • -108
    • +0
    ./resources/hadoop/conf/log4j.properties
    • -39
    • +0
    ./resources/hadoop/conf/mapred-site.xml
    • -16
    • +0
    ./resources/hyracks-deployment.properties
  1. … 945 more files in changeset.
- Added Tokenize Operator in addition to the bulkload operator changes that were made by Zachary Heilbron. The tokenize operator is only added to the logical plan when bulk-loading the data. - Each secondary index is now updated in the separate branch by using the replicate operator. - Sink Operator now accepts multiple inputs. - Fixed the bulk-load so that it correctly produces auto-generated PK.

Change-Id: Ifb591754dba5eb4a9207edaa4e658f4cc745893a

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/78

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -7
    • +10
    ./java/edu/uci/ics/pregelix/core/join/JoinTest.java
  1. … 54 more files in changeset.
Several major changes in hyracks: -- reduced CC/NC communications for reporting partition request and availability; partition request/availability are only reported for the case of send-side materialized (without pipelining) policies in case of task re-attempt. -- changed buffer cache to dynamically allocate memory based on needs instead of pre-allocating -- changed each network channel to lazily allocate memory based on needs, and changed materialized connectors to lazily allocate files based on needs -- changed several major CCNCCFunctions to use non-java serde -- added a sort-based group-by operator which pushes group-by aggregations into an external sort -- make external sort a stable sort

1,3,and 4 is to reduce the job overhead.

2 is to reduce the unecessary NC resource consumptions such as memory and files.

5 and 6 are improvements to runtime operators.

One change in algebricks:

-- implemented a rule to push group-by aggregation into sort, i.e., using the sort-based gby operator

Several important changes in pregelix:

-- remove static states in vertex

-- direct check halt bit without deserialization

-- optimize the sort algorithm by packing yet-another 2-byte normalized key into the tPointers array

Change-Id: Id696f9a9f1647b4a025b8b33d20b3a89127c60d6

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/35

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <westmann@gmail.com>

  1. … 276 more files in changeset.
fix the cache miss problem in the sort merge reader

    • -10
    • +17
    ./java/edu/uci/ics/pregelix/core/join/JoinTest.java
  1. … 23 more files in changeset.
Pass a boolean argument to the bulkload to decides if checking for an empty index is needed.

  1. … 25 more files in changeset.
add/update license headers

    • -0
    • +14
    ./resources/cluster/cluster.properties
    • -0
    • +14
    ./resources/cluster/data.properties
    • -0
    • +14
    ./resources/hadoop/conf/core-site.xml
    • -0
    • +14
    ./resources/hadoop/conf/hdfs-site.xml
    • -0
    • +14
    ./resources/hadoop/conf/log4j.properties
    • -0
    • +14
    ./resources/hadoop/conf/mapred-site.xml
    • -0
    • +14
    ./resources/hyracks-deployment.properties
  1. … 2275 more files in changeset.
Merged fullstack_lsm_staging upto r3336

git-svn-id: https://hyracks.googlecode.com/svn/trunk/fullstack@3339 123451ca-8445-de46-9d55-352943316053

    • -24
    • +23
    ./java/edu/uci/ics/pregelix/core/join/JoinTest.java
  1. … 898 more files in changeset.
cross merge fullstack_release_candidate into trunk

git-svn-id: https://hyracks.googlecode.com/svn/trunk/fullstack@3208 123451ca-8445-de46-9d55-352943316053

    • -2
    • +2
    ./resources/hadoop/conf/mapred-site.xml
  1. … 900 more files in changeset.
fix code and scripts for rack-awareness

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_release_candidate@3181 123451ca-8445-de46-9d55-352943316053

    • -0
    • +7
    ./resources/topology.xml
  1. … 11 more files in changeset.
merged from fullstack_asterix_stabilization to fullstack_lsm_staging -r3100:3171

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_lsm_staging@3173 123451ca-8445-de46-9d55-352943316053

  1. … 22 more files in changeset.
Merge fullstack_asterix_stabilization with fullstack_hyracks_result_distribution.

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_hyracks_result_distribution@3170 123451ca-8445-de46-9d55-352943316053

  1. … 190 more files in changeset.
merged hyracks_lsm_tree and fullstack_asterix_stabilization

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_lsm_staging@3026 123451ca-8445-de46-9d55-352943316053

    • -21
    • +22
    ./java/edu/uci/ics/pregelix/core/join/JoinTest.java
  1. … 762 more files in changeset.
Merged fullstack_asterix_stabilization -r 2813:2933

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_hyracks_ioc@2995 123451ca-8445-de46-9d55-352943316053

    • -2
    • +2
    ./resources/hadoop/conf/mapred-site.xml
  1. … 90 more files in changeset.
Merge fullstack_asterix_stabilization into fullstack_hyracks_result_distribution branch.

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_hyracks_result_distribution@2862 123451ca-8445-de46-9d55-352943316053

    • -2
    • +2
    ./resources/hadoop/conf/mapred-site.xml
  1. … 389 more files in changeset.
1. let hyracks-hdfs support hadoop new API\n 2. let Pregelix use hyracks-hdfs functionality;\n 3. increase degree-of-parallelism in tests

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_asterix_stabilization@2816 123451ca-8445-de46-9d55-352943316053

    • -2
    • +2
    ./resources/hadoop/conf/mapred-site.xml
  1. … 11 more files in changeset.
Merged fullstack_staging -r 2426:2786

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_hyracks_ioc@2788 123451ca-8445-de46-9d55-352943316053

  1. … 9 more files in changeset.
make example job outputs concise

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_staging@2693 123451ca-8445-de46-9d55-352943316053

  1. … 24 more files in changeset.
Fixed pregelix tests to use INCApplicationEntryPoint. Fixed BTree example to use INCApplicationEntryPoint. Fixed POMs to have versions for all plugins

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_hyracks_ioc@2458 123451ca-8445-de46-9d55-352943316053

  1. … 19 more files in changeset.
minor fix: 1. remove unecessary parameters; 2. passing frame size from config to runtime

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_staging@2050 123451ca-8445-de46-9d55-352943316053

    • -0
    • +37
    ./resources/cluster/cluster.properties
  1. … 15 more files in changeset.
add Pregelix codebase

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_staging@1960 123451ca-8445-de46-9d55-352943316053

    • -0
    • +597
    ./java/edu/uci/ics/pregelix/core/join/JoinTest.java
    • -0
    • +92
    ./java/edu/uci/ics/pregelix/core/util/TestUtils.java
    • -0
    • +1
    ./resources/cluster/data.properties
    • -0
    • +18
    ./resources/hadoop/conf/core-site.xml
    • -0
    • +18
    ./resources/hadoop/conf/hdfs-site.xml
    • -0
    • +94
    ./resources/hadoop/conf/log4j.properties
    • -0
    • +25
    ./resources/hadoop/conf/mapred-site.xml
    • -0
    • +2
    ./resources/hyracks-deployment.properties
    • -0
    • +94
    ./resources/log4j.properties
    • -0
    • +66
    ./resources/logging.properties
  1. … 185 more files in changeset.