Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Added LSM component-level filters for all indexes.

Change-Id: I898cf885c9f88feae85c99799a00fd8ec036efea

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/81

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 129 more files in changeset.
Several major changes in hyracks: -- reduced CC/NC communications for reporting partition request and availability; partition request/availability are only reported for the case of send-side materialized (without pipelining) policies in case of task re-attempt. -- changed buffer cache to dynamically allocate memory based on needs instead of pre-allocating -- changed each network channel to lazily allocate memory based on needs, and changed materialized connectors to lazily allocate files based on needs -- changed several major CCNCCFunctions to use non-java serde -- added a sort-based group-by operator which pushes group-by aggregations into an external sort -- make external sort a stable sort

1,3,and 4 is to reduce the job overhead.

2 is to reduce the unecessary NC resource consumptions such as memory and files.

5 and 6 are improvements to runtime operators.

One change in algebricks:

-- implemented a rule to push group-by aggregation into sort, i.e., using the sort-based gby operator

Several important changes in pregelix:

-- remove static states in vertex

-- direct check halt bit without deserialization

-- optimize the sort algorithm by packing yet-another 2-byte normalized key into the tPointers array

Change-Id: Id696f9a9f1647b4a025b8b33d20b3a89127c60d6

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/35

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <westmann@gmail.com>

  1. … 276 more files in changeset.
LSM Rtree Optimization

  1. … 54 more files in changeset.
fix issue 130 thanks to patch from Till

Ensure batchSize is at least 1 in all cases

Ensure batchSize > 0 even when running with very many CPU cores

Merge branch 'master' into pouria/fix-memory

Conflicts:

hyracks/hyracks-api/src/main/java/edu/uci/ics/hyracks/api/context/IHyracksCommonContext.java

hyracks/hyracks-client/src/main/java/edu/uci/ics/hyracks/client/dataset/DatasetClientContext.java

hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/edu/uci/ics/hyracks/storage/am/lsm/invertedindex/ondisk/OnDiskInvertedIndex.java

hyracks/hyracks-test-support/src/main/java/edu/uci/ics/hyracks/test/support/TestTaskContext.java

  1. … 3 more files in changeset.
Fixing Methods signature

  1. … 3 more files in changeset.
added ability to trigger an application state dump through the rest api

  1. … 21 more files in changeset.
fix IIndexAccessor interface, add a boolean exclusiveMode parameter for the createSearchCursor method

  1. … 38 more files in changeset.
Optimized search and split of the btree for monotonically increasing keys.

    • -0
    • +75
    ./hyracks/storage/am/btree/OrderedIndexSortedInsertTest.java
  1. … 4 more files in changeset.
Adding required method to TestCtx

Modified tests parameters.

Added another debugging parameters.

Added debugging parameters.

try to fix memory leak issue

  1. … 7 more files in changeset.
Implemented k-buffering for lsm indexes. Also add a fix for issues 589 and 594.

  1. … 91 more files in changeset.
revise an index test for the new exception

merge from zheilbron/hyracks_msr

  1. … 291 more files in changeset.
let inserting duplicate keys become a NoOp instead of throwing exceptions which fail the job

  1. … 11 more files in changeset.
Merged master

  1. … 81 more files in changeset.
Delete the artifacts of a loaded component if it turn out to be an empty load.

Addressed code review comments.

  1. … 7 more files in changeset.
Pass a boolean argument to the bulkload to decides if checking for an empty index is needed.

  1. … 23 more files in changeset.
Prevented bulkloading an lsm index if it is not empty.

  1. … 29 more files in changeset.
add/update license headers

  1. … 2273 more files in changeset.
activating/deactivating an index multiple times now throws error

  1. … 5 more files in changeset.
Orchestrated the use of executors and thread factory and added buffer cache core dump.

  1. … 17 more files in changeset.
Fix for issue 450.

  1. … 13 more files in changeset.
added IVirtualBufferCacheProvider interface; fixed bug in VBC; add reference counts to VBC open/close methods; give unique virtual file names in lsm indices

    • -0
    • +25
    ./hyracks/test/support/TestVirtualBufferCacheProvider.java
  1. … 43 more files in changeset.
Passing bloom filter parameters to the LSM indexes.

  1. … 54 more files in changeset.