Clone Tools
  • last updated 25 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Change folder structure for Java repackage

Change only the folders, not the files, for our package name change.

This will break the build, and needs to be followed by a change to

the package name in all of the source files. However performing

the folder move and file change in two steps lets Git understand

that the files are the same, and lets us track revisions across

those files.

Change-Id: I08aff75e25ac7c6298c32cf3402febbc4a318c2a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/307

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Chris Hillery <ceej@lambda.nu>

    • -293
    • +0
    ./HyracksClientInterfaceFunctions.java
    • -130
    • +0
    ./HyracksClientInterfaceRemoteProxy.java
    • -176
    • +0
    ./impl/ActivityClusterGraphBuilder.java
    • -22
    • +0
    ./impl/IConnectorDescriptorVisitor.java
    • -22
    • +0
    ./impl/IOperatorDescriptorVisitor.java
    • -141
    • +0
    ./impl/JobActivityGraphBuilder.java
    • -113
    • +0
    ./impl/JobSpecificationActivityClusterGraphGeneratorFactory.java
  1. … 3880 more files in changeset.
Add Shutdown via API to Hyracks

This change adds a method to HyracksConnection called stopCluster().

When the CC recieves a message from this, it asks all NC tasks to close

and acknowledge that they have recieved the message and are closing.

If all NCs have closed, or a 10 second timeout elapses, the CC then

exits with a 0 return code if all NCs closed, or a 1 if some did

not acknowledge the shutdown request.

Change-Id: Iaf3d395dc7964e114d4929830f40063f58e0d5da

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/76

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Vinayak Borkar <vinayakb@gmail.com>

    • -1
    • +12
    ./HyracksClientInterfaceFunctions.java
    • -0
    • +16
    ./HyracksClientInterfaceRemoteProxy.java
  1. … 19 more files in changeset.
Several major changes in hyracks: -- reduced CC/NC communications for reporting partition request and availability; partition request/availability are only reported for the case of send-side materialized (without pipelining) policies in case of task re-attempt. -- changed buffer cache to dynamically allocate memory based on needs instead of pre-allocating -- changed each network channel to lazily allocate memory based on needs, and changed materialized connectors to lazily allocate files based on needs -- changed several major CCNCCFunctions to use non-java serde -- added a sort-based group-by operator which pushes group-by aggregations into an external sort -- make external sort a stable sort

1,3,and 4 is to reduce the job overhead.

2 is to reduce the unecessary NC resource consumptions such as memory and files.

5 and 6 are improvements to runtime operators.

One change in algebricks:

-- implemented a rule to push group-by aggregation into sort, i.e., using the sort-based gby operator

Several important changes in pregelix:

-- remove static states in vertex

-- direct check halt bit without deserialization

-- optimize the sort algorithm by packing yet-another 2-byte normalized key into the tPointers array

Change-Id: Id696f9a9f1647b4a025b8b33d20b3a89127c60d6

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/35

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <westmann@gmail.com>

    • -0
    • +1
    ./impl/JobSpecificationActivityClusterGraphGeneratorFactory.java
  1. … 276 more files in changeset.
merge from master

    • -1
    • +1
    ./HyracksClientInterfaceFunctions.java
    • -1
    • +1
    ./HyracksClientInterfaceRemoteProxy.java
    • -0
    • +14
    ./impl/JobSpecificationActivityClusterGraphGeneratorFactory.java
  1. … 7 more files in changeset.
add/update license headers

    • -1
    • +1
    ./HyracksClientInterfaceFunctions.java
    • -1
    • +1
    ./HyracksClientInterfaceRemoteProxy.java
    • -1
    • +1
    ./impl/ActivityClusterGraphBuilder.java
    • -1
    • +1
    ./impl/IConnectorDescriptorVisitor.java
    • -1
    • +1
    ./impl/IOperatorDescriptorVisitor.java
    • -0
    • +14
    ./impl/JobActivityGraphBuilder.java
    • -0
    • +14
    ./impl/JobSpecificationActivityClusterGraphGeneratorFactory.java
  1. … 2274 more files in changeset.
added job info

    • -0
    • +20
    ./HyracksClientInterfaceFunctions.java
    • -0
    • +8
    ./HyracksClientInterfaceRemoteProxy.java
    • -0
    • +5
    ./impl/JobSpecificationActivityClusterGraphGeneratorFactory.java
  1. … 6 more files in changeset.
reintegrate fullstack_dynamic_deployment

    • -1
    • +60
    ./HyracksClientInterfaceFunctions.java
    • -0
    • +24
    ./HyracksClientInterfaceRemoteProxy.java
  1. … 199 more files in changeset.
Merged fullstack_lsm_staging upto r3336

git-svn-id: https://hyracks.googlecode.com/svn/trunk/fullstack@3339 123451ca-8445-de46-9d55-352943316053

    • -67
    • +1
    ./HyracksClientInterfaceFunctions.java
    • -23
    • +2
    ./HyracksClientInterfaceRemoteProxy.java
    • -1
    • +1
    ./impl/JobSpecificationActivityClusterGraphGeneratorFactory.java
  1. … 893 more files in changeset.
cross merge fullstack_release_candidate into trunk

git-svn-id: https://hyracks.googlecode.com/svn/trunk/fullstack@3208 123451ca-8445-de46-9d55-352943316053

    • -0
    • +74
    ./HyracksClientInterfaceFunctions.java
    • -0
    • +7
    ./HyracksClientInterfaceRemoteProxy.java
    • -0
    • +3
    ./impl/JobSpecificationActivityClusterGraphGeneratorFactory.java
  1. … 896 more files in changeset.
Merge fullstack_asterix_stabilization with fullstack_hyracks_result_distribution.

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_hyracks_result_distribution@3170 123451ca-8445-de46-9d55-352943316053

    • -67
    • +1
    ./HyracksClientInterfaceFunctions.java
    • -23
    • +2
    ./HyracksClientInterfaceRemoteProxy.java
    • -1
    • +1
    ./impl/JobSpecificationActivityClusterGraphGeneratorFactory.java
  1. … 185 more files in changeset.
Merged fullstack_asterix_stabilization -r 2933:3157

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_hyracks_ioc@3164 123451ca-8445-de46-9d55-352943316053

    • -0
    • +74
    ./HyracksClientInterfaceFunctions.java
    • -0
    • +7
    ./HyracksClientInterfaceRemoteProxy.java
    • -0
    • +3
    ./impl/JobSpecificationActivityClusterGraphGeneratorFactory.java
  1. … 1161 more files in changeset.
Merge fullstack_asterix_stabilization into fullstack_hyracks_result_distribution.

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_hyracks_result_distribution@3124 123451ca-8445-de46-9d55-352943316053

    • -0
    • +3
    ./impl/JobSpecificationActivityClusterGraphGeneratorFactory.java
  1. … 706 more files in changeset.
Reverting the merge of fullstack_hyracks_result_distribution branch until all the tests pass.

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_asterix_stabilization@3033 123451ca-8445-de46-9d55-352943316053

    • -74
    • +0
    ./HyracksClientInterfaceFunctions.java
    • -7
    • +0
    ./HyracksClientInterfaceRemoteProxy.java
  1. … 132 more files in changeset.
create a rewriter during each initialize call

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_job_rewriter@2999 123451ca-8445-de46-9d55-352943316053

    • -1
    • +1
    ./impl/JobSpecificationActivityClusterGraphGeneratorFactory.java
  1. … 1 more file in changeset.
Remove the usage of serialized descriptor.

We do not have the necessity to pass the serialized record descriptor to

clients, so there is no need to pass it to DatasetDirectoryService to

store it in the record since we serialize all the results in the Hyracks

operator before sending the results now.

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_hyracks_result_distribution@2978 123451ca-8445-de46-9d55-352943316053

    • -26
    • +0
    ./HyracksClientInterfaceFunctions.java
  1. … 16 more files in changeset.
rewrite hyracks activity cluster graphs to eliminate one-to-one connectors

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_job_rewriter@2964 123451ca-8445-de46-9d55-352943316053

    • -0
    • +3
    ./impl/JobSpecificationActivityClusterGraphGeneratorFactory.java
  1. … 5 more files in changeset.
Fix an issue that arose during the merge of missing to merge a conflicted hunk.

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_hyracks_result_distribution@2849 123451ca-8445-de46-9d55-352943316053

Implement the client side of the RPC chain to fetch the job status from the client.

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_hyracks_result_distribution@2842 123451ca-8445-de46-9d55-352943316053

    • -0
    • +27
    ./HyracksClientInterfaceFunctions.java
  1. … 10 more files in changeset.
Style and documentation fixes.

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_hyracks_result_distribution@2834 123451ca-8445-de46-9d55-352943316053

  1. … 4 more files in changeset.
Remove the job id parameter to fetch the dataset directory service info.

This is removed because, in contrast to our original plan we will not

have support for per job dataset directory service. There will be a single

directory service for all the jobs, i.e. for the whole hyracks instance.

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_hyracks_result_distribution@2832 123451ca-8445-de46-9d55-352943316053

    • -3
    • +2
    ./HyracksClientInterfaceRemoteProxy.java
  1. … 2 more files in changeset.
Implement the client side of RPCs to give the clients the ability to retrieve serialized record descriptor.

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_hyracks_result_distribution@2831 123451ca-8445-de46-9d55-352943316053

    • -2
    • +19
    ./HyracksClientInterfaceFunctions.java
  1. … 7 more files in changeset.
Use DatasetDirectoryRecords instead of NetworkAddresses everywhere we access that information in the dataset stack.

Also pass the result id through every method that works either on fetching

the metadata for results or the results themselves.

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_hyracks_result_distribution@2534 123451ca-8445-de46-9d55-352943316053

    • -6
    • +15
    ./HyracksClientInterfaceFunctions.java
  1. … 9 more files in changeset.
Make all the necessary changes or additions to report the data transfer info across the new result distribution socket for the node controller.

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_hyracks_result_distribution@2520 123451ca-8445-de46-9d55-352943316053

  1. … 5 more files in changeset.
Connect the whole get dataset directory service info and get result locations path from the client through the ClusterController by implementing the necessary RPC interfaces.

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_hyracks_result_distribution@2477 123451ca-8445-de46-9d55-352943316053

    • -0
    • +47
    ./HyracksClientInterfaceFunctions.java
    • -0
    • +8
    ./HyracksClientInterfaceRemoteProxy.java
  1. … 1 more file in changeset.
Eliminated Application deployment code. Eliminated hyracks-cli. Fixed hyracks examples to work with the new application model

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_hyracks_ioc@2433 123451ca-8445-de46-9d55-352943316053

    • -67
    • +1
    ./HyracksClientInterfaceFunctions.java
    • -23
    • +2
    ./HyracksClientInterfaceRemoteProxy.java
    • -1
    • +1
    ./impl/JobSpecificationActivityClusterGraphGeneratorFactory.java
  1. … 104 more files in changeset.
Truncated old project trunks

git-svn-id: https://hyracks.googlecode.com/svn/trunk@2424 123451ca-8445-de46-9d55-352943316053

    • -195
    • +0
    ./HyracksClientInterfaceFunctions.java
    • -96
    • +0
    ./HyracksClientInterfaceRemoteProxy.java
    • -176
    • +0
    ./impl/ActivityClusterGraphBuilder.java
    • -22
    • +0
    ./impl/IConnectorDescriptorVisitor.java
    • -22
    • +0
    ./impl/IOperatorDescriptorVisitor.java
    • -127
    • +0
    ./impl/JobActivityGraphBuilder.java
    • -88
    • +0
    ./impl/JobSpecificationActivityClusterGraphGeneratorFactory.java
  1. … 1460 more files in changeset.
Added flag to decide task cluster scheduling

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_staging@2059 123451ca-8445-de46-9d55-352943316053

    • -0
    • +1
    ./impl/JobSpecificationActivityClusterGraphGeneratorFactory.java
  1. … 3 more files in changeset.
Added code to propagate ConnectorPolicyAssignmentPolicy from the JobSpecification

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_staging@1990 123451ca-8445-de46-9d55-352943316053

    • -0
    • +1
    ./impl/JobSpecificationActivityClusterGraphGeneratorFactory.java
  1. … 2 more files in changeset.
Copied hyracks trunk into fullstack

git-svn-id: https://hyracks.googlecode.com/svn/branches/fullstack_staging@1958 123451ca-8445-de46-9d55-352943316053

    • -0
    • +195
    ./HyracksClientInterfaceFunctions.java
    • -0
    • +96
    ./HyracksClientInterfaceRemoteProxy.java
    • -0
    • +176
    ./impl/ActivityClusterGraphBuilder.java
    • -0
    • +22
    ./impl/IConnectorDescriptorVisitor.java
    • -0
    • +22
    ./impl/IOperatorDescriptorVisitor.java
    • -0
    • +127
    ./impl/JobActivityGraphBuilder.java
    • -0
    • +88
    ./impl/JobSpecificationActivityClusterGraphGeneratorFactory.java
  1. … 1447 more files in changeset.
Merged hyracks_asterix_stabilization upto rev 1913

git-svn-id: https://hyracks.googlecode.com/svn/trunk@1924 123451ca-8445-de46-9d55-352943316053

    • -36
    • +27
    ./HyracksClientInterfaceFunctions.java
    • -10
    • +10
    ./HyracksClientInterfaceRemoteProxy.java
    • -0
    • +176
    ./impl/ActivityClusterGraphBuilder.java
    • -0
    • +22
    ./impl/IConnectorDescriptorVisitor.java
    • -0
    • +22
    ./impl/IOperatorDescriptorVisitor.java
    • -0
    • +127
    ./impl/JobActivityGraphBuilder.java
    • -0
    • +88
    ./impl/JobSpecificationActivityClusterGraphGeneratorFactory.java
    • -0
    • +47
    ./impl/PlanUtils.java
  1. … 892 more files in changeset.