Clone Tools
  • last updated 20 mins ago
Constraints: committers
Constraints: files
Constraints: dates
Fixing Methods signature

re-enabled test

modified test case to use adaptor alias instead of class name

Merge branch 'master' into raman/master_feeds_adaptor_test

fix to support non-default pregelix cc http ports

fix parsing of actual 64-bit values

corrected the external adaptor test case to use the right library

compatibility for for bash versions <4.0

add (optional) CC_HTTPPORT and JOB_HISTORY_SIZE to conf

fix for linux setting

added test for installation and use of an external adaptor

fix client dyn-opt setting

1. make startcc/nc scripts flexible for different physical memory size; 2. add dynamic optimization option in the Client

support heterogenous cluster

  1. … 23 more files in changeset.
use Counters as partial value to simplify HadoopCountersAggregator

add new example for Counters usage

add support for Hadoop Counters via job.setCounterAggregatorClass

The PregelixJob.setCounterAggregatorClass sets up a (user-specified)

global aggregator and an iterationComplete hook to save Counter values.

The user-specified Counter-based aggregator (must extend

HadoopCountersAggregator) is saved to HDFS in each iteration and should

be restart/snapshot-aware.

The usage for setting up counters is to make a call to

job.setCounterAggregatorClass. After job completion, the Counters may

be retrieved from HDFS using BspUtils.getCounters(job).

Note that there is currently only one spot for iterationComplete hooks

and this behavior occupies it.

DmlRecoveryTest added

add an "iteration complete" hook for aggregation/reporting across iterations

This commit allows the user to specify a class which will be called upon

completion of each pregelix iteration. This allows us to perform a user-

specified action between iterations.

As an example, a PerIterationGlobalAggregatesHook is provided which

saves the complete set of global aggregator states from every iteration,

allowing the user to observe aggregates from all iterations.

The default hook does nothing.

The hook instance is attached directly to the PregelixJob so that it can

be retrieved by the Driver's caller.

Merge branch 'master' of

dml recovery added

fix NPE when no custom aggregator is set

api for specifying update state for activate() and voteToHalt()

allow global aggregators to be specified in xml

explicitly setting the aggregator in the PregelixJob constructors would

override any values read in from the conf's resources.

Instead, this commit doesn't set the conf explicitly and instead

specifies an array of aggregator class names which will always be in

place when `getGlobalAggregatorClasses` is called.

reenable feed jobspec alteration + logging to metadata (bug filed)

clean up azure adapters

Merge branch 'master' into zheilbron/asterix_msr_demo



Merge branch 'master' into zheilbron/hyracks_msr_demo



remove commented, unused import

change unary function type computer to support ANY type