Clone Tools
  • last updated 18 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Change folder structure for Java repackage

Change only the folders, not the files, for our package name change.

This will break the build, and needs to be followed by a change to

the package name in all of the source files. However performing

the folder move and file change in two steps lets Git understand

that the files are the same, and lets us track revisions across

those files.

Change-Id: I08aff75e25ac7c6298c32cf3402febbc4a318c2a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/307

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Chris Hillery <ceej@lambda.nu>

  1. … 3879 more files in changeset.
VariableSizeFrame(VSizeFrame) support for Hyracks.

This patch replaced Frame/Accessor/Appender with the new API which

supports BigObject.

The ExternalSorter/TopKSorter/ExternalGroupSorter

have been implemented to support big object.

The Groupby && Join should work with BigObject also. But it will break the

memory budget when it encounter a big object. I will fix the memory

problem later in a separate CR.

The design about the frame allocation is

here:https://docs.google.com/presentation/d/15h9iQf5OYsgGZoQTbGHkj1yS2G9q2fd0s1lDAD1EJq0/edit?usp=sharing

Suggest review order:

Patch 12: It includes all of the sorting operators.

Patch 13: It applys the new IFrame API to all Hyracks codes.

Patch 14: Some bug fixes to pass all Asterix's tests.

Patch 15: Skip it!

Patch 16: Some bug fixes to the Asterix's tests in small frame setting.

Later Patch: address the comments

Change-Id: I2e08692078683f6f2cf17387e39037ad851fc05b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/234

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -4
    • +17
    ./ics/hyracks/test/support/TestTaskContext.java
  1. … 218 more files in changeset.
- Fixed Type Casting issue - Reorganized duplicated internal class in the DelimitedDataParser and DelimitedDataParserFactory - Prevented a user from creating an inverted index on a dataset with a variable-length PK

Change-Id: Ic5606501223b8d860b49a258ff49afacd7d76b9a

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/191

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <westmann@gmail.com>

  1. … 65 more files in changeset.
Added LSM component-level filters for all indexes.

Change-Id: I898cf885c9f88feae85c99799a00fd8ec036efea

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/81

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 129 more files in changeset.
Several major changes in hyracks: -- reduced CC/NC communications for reporting partition request and availability; partition request/availability are only reported for the case of send-side materialized (without pipelining) policies in case of task re-attempt. -- changed buffer cache to dynamically allocate memory based on needs instead of pre-allocating -- changed each network channel to lazily allocate memory based on needs, and changed materialized connectors to lazily allocate files based on needs -- changed several major CCNCCFunctions to use non-java serde -- added a sort-based group-by operator which pushes group-by aggregations into an external sort -- make external sort a stable sort

1,3,and 4 is to reduce the job overhead.

2 is to reduce the unecessary NC resource consumptions such as memory and files.

5 and 6 are improvements to runtime operators.

One change in algebricks:

-- implemented a rule to push group-by aggregation into sort, i.e., using the sort-based gby operator

Several important changes in pregelix:

-- remove static states in vertex

-- direct check halt bit without deserialization

-- optimize the sort algorithm by packing yet-another 2-byte normalized key into the tPointers array

Change-Id: Id696f9a9f1647b4a025b8b33d20b3a89127c60d6

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/35

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <westmann@gmail.com>

  1. … 276 more files in changeset.
LSM Rtree Optimization

  1. … 54 more files in changeset.
fix issue 130 thanks to patch from Till

Ensure batchSize is at least 1 in all cases

Ensure batchSize > 0 even when running with very many CPU cores

Merge branch 'master' into pouria/fix-memory

Conflicts:

hyracks/hyracks-api/src/main/java/edu/uci/ics/hyracks/api/context/IHyracksCommonContext.java

hyracks/hyracks-client/src/main/java/edu/uci/ics/hyracks/client/dataset/DatasetClientContext.java

hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/edu/uci/ics/hyracks/storage/am/lsm/invertedindex/ondisk/OnDiskInvertedIndex.java

hyracks/hyracks-test-support/src/main/java/edu/uci/ics/hyracks/test/support/TestTaskContext.java

    • -4
    • +3
    ./ics/hyracks/test/support/TestTaskContext.java
  1. … 3 more files in changeset.
Fixing Methods signature

    • -1
    • +1
    ./ics/hyracks/test/support/TestTaskContext.java
  1. … 3 more files in changeset.
added ability to trigger an application state dump through the rest api

  1. … 21 more files in changeset.
fix IIndexAccessor interface, add a boolean exclusiveMode parameter for the createSearchCursor method

  1. … 38 more files in changeset.
Optimized search and split of the btree for monotonically increasing keys.

    • -0
    • +75
    ./ics/hyracks/storage/am/btree/OrderedIndexSortedInsertTest.java
  1. … 4 more files in changeset.
Adding required method to TestCtx

    • -0
    • +5
    ./ics/hyracks/test/support/TestTaskContext.java
Modified tests parameters.

Added another debugging parameters.

Added debugging parameters.

try to fix memory leak issue

    • -0
    • +4
    ./ics/hyracks/test/support/TestTaskContext.java
  1. … 7 more files in changeset.
Implemented k-buffering for lsm indexes. Also add a fix for issues 589 and 594.

  1. … 91 more files in changeset.
revise an index test for the new exception

merge from zheilbron/hyracks_msr

  1. … 291 more files in changeset.
let inserting duplicate keys become a NoOp instead of throwing exceptions which fail the job

  1. … 11 more files in changeset.
Merged master

  1. … 81 more files in changeset.
Delete the artifacts of a loaded component if it turn out to be an empty load.

Addressed code review comments.

  1. … 7 more files in changeset.
Pass a boolean argument to the bulkload to decides if checking for an empty index is needed.

  1. … 23 more files in changeset.
Prevented bulkloading an lsm index if it is not empty.

  1. … 29 more files in changeset.
add/update license headers

  1. … 2273 more files in changeset.
activating/deactivating an index multiple times now throws error

  1. … 5 more files in changeset.
Orchestrated the use of executors and thread factory and added buffer cache core dump.

  1. … 17 more files in changeset.