Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Change folder structure for Java repackage

Change only the folders, not the files, for our package name change.

This will break the build, and needs to be followed by a change to

the package name in all of the source files. However performing

the folder move and file change in two steps lets Git understand

that the files are the same, and lets us track revisions across

those files.

Change-Id: I08aff75e25ac7c6298c32cf3402febbc4a318c2a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/307

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Chris Hillery <ceej@lambda.nu>

    • -20
    • +0
    ./api/ILSMComponentFilterFactory.java
    • -40
    • +0
    ./api/ILSMComponentFilterFrame.java
    • -19
    • +0
    ./api/ILSMComponentFilterFrameFactory.java
    • -32
    • +0
    ./api/ILSMComponentFilterManager.java
    • -21
    • +0
    ./api/ILSMIOOperationCallbackFactory.java
    • -19
    • +0
    ./api/ILSMIOOperationCallbackProvider.java
    • -21
    • +0
    ./api/ILSMIOOperationScheduler.java
    • -23
    • +0
    ./api/ILSMIOOperationSchedulerProvider.java
  1. … 3879 more files in changeset.
improve the buffer cache perf. with 1) a better hash function for fileid-pageid, 2) reduce synchronization in clock page replacement policy.

Change-Id: I296c589a556a9afa7f27c6f560fa07fc4e2c1861

Reviewed-on: https://asterix-gerrit.ics.uci.edu/342

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

  1. … 2 more files in changeset.
Introducing data replication API to LSM indexes

Change-Id: I80565fc9d74e30440d2df5917911904ba8f33c25

Reviewed-on: https://asterix-gerrit.ics.uci.edu/322

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -0
    • +23
    ./api/ILSMIndexReplicationJob.java
    • -0
    • +2
    ./impls/AbstractMemoryLSMComponent.java
    • -0
    • +42
    ./impls/LSMIndexReplicationJob.java
    • -0
    • +10
    ./impls/MultitenantVirtualBufferCache.java
  1. … 25 more files in changeset.
VariableSizeFrame(VSizeFrame) support for Hyracks.

This patch replaced Frame/Accessor/Appender with the new API which

supports BigObject.

The ExternalSorter/TopKSorter/ExternalGroupSorter

have been implemented to support big object.

The Groupby && Join should work with BigObject also. But it will break the

memory budget when it encounter a big object. I will fix the memory

problem later in a separate CR.

The design about the frame allocation is

here:https://docs.google.com/presentation/d/15h9iQf5OYsgGZoQTbGHkj1yS2G9q2fd0s1lDAD1EJq0/edit?usp=sharing

Suggest review order:

Patch 12: It includes all of the sorting operators.

Patch 13: It applys the new IFrame API to all Hyracks codes.

Patch 14: Some bug fixes to pass all Asterix's tests.

Patch 15: Skip it!

Patch 16: Some bug fixes to the Asterix's tests in small frame setting.

Later Patch: address the comments

Change-Id: I2e08692078683f6f2cf17387e39037ad851fc05b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/234

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -14
    • +13
    ./dataflow/LSMIndexInsertUpdateDeleteOperatorNodePushable.java
  1. … 219 more files in changeset.
1. Fix the "writerCount!=0 during component flushing" issue 2. Fix the duplicate LSM disk component file name issue by avoiding duplicate timestamps for different components.

Note that this change includes https://asterix-gerrit.ics.uci.edu/#/c/268/.

Change-Id: I805eab33603f52e19a1b76f1c315f9b75b6e3c03

Reviewed-on: https://asterix-gerrit.ics.uci.edu/278

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

    • -8
    • +37
    ./impls/AbstractLSMIndexFileManager.java
    • -3
    • +7
    ./impls/AbstractMemoryLSMComponent.java
    • -2
    • +7
    ./impls/MultitenantVirtualBufferCache.java
  1. … 11 more files in changeset.
Add a flag for LSM-based indices to indicate whether force pages to disk devices during flush and merge.

Change-Id: I988716c03cffe30b008e144d3a478ee25e367212

Reviewed-on: https://asterix-gerrit.ics.uci.edu/240

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

    • -4
    • +6
    ./dataflow/AbstractLSMIndexDataflowHelper.java
    • -1
    • +4
    ./dataflow/AbstractLSMIndexDataflowHelperFactory.java
    • -1
    • +0
    ./impls/AbstractMemoryLSMComponent.java
  1. … 73 more files in changeset.
Fixing LSMHarness issue.

The following commits from your working branch will be included:

commit 4a2a16f16df99cbf29ac53bf7009e2dc07bdbb26

Author: hubailmor@gmail.com <mhubail@uci.edu>

Date: Fri Apr 24 17:24:02 2015 -0700

Fixing LSMHarness issue

Change-Id: I7c1b3e8283fc3a661c80202cc896a42a1b322416

Reviewed-on: https://asterix-gerrit.ics.uci.edu/251

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -4
    • +12
    ./impls/AbstractMemoryLSMComponent.java
- Fixed Type Casting issue - Reorganized duplicated internal class in the DelimitedDataParser and DelimitedDataParserFactory - Prevented a user from creating an inverted index on a dataset with a variable-length PK

Change-Id: Ic5606501223b8d860b49a258ff49afacd7d76b9a

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/191

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <westmann@gmail.com>

  1. … 63 more files in changeset.
Added a new method to switch LSM component state Change-Id: If4647e31c1a81e29093f84a26059b020508db7f6 Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/208 Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu> Reviewed-by: abdullah alamoudi <bamousaa@gmail.com> Reviewed-by: Ian Maxon <imaxon@uci.edu>

    • -0
    • +4
    ./impls/AbstractMemoryLSMComponent.java
Added a method to check current LSM component status

The following commits from your working branch will be included:

commit caf3da2933ce7218492bf2c05cb9553d4686cac3

Author: hubailmor@gmail.com <mhubail@uci.edu>

Date: Tue Nov 11 12:45:32 2014 -0800

Added a method to check current LSM component status

Change-Id: I3a5a5a5bdc1eb02bfc78f94afc574ce8f2d12659

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/170

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Changes to fix issue 803

commit b19cde423b858bf4964cba3c8b76b17a1d6b0300

Author: Young-Seok <kisskys@gmail.com>

Date: Mon Sep 29 17:03:20 2014 -0700

Change-Id: Icbb1b71bfa263e67ad52cf61e831c614c630ea4b

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/142

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -1
    • +2
    ./impls/AbstractMemoryLSMComponent.java
- Added Tokenize Operator in addition to the bulkload operator changes that were made by Zachary Heilbron. The tokenize operator is only added to the logical plan when bulk-loading the data. - Each secondary index is now updated in the separate branch by using the replicate operator. - Sink Operator now accepts multiple inputs. - Fixed the bulk-load so that it correctly produces auto-generated PK.

Change-Id: Ifb591754dba5eb4a9207edaa4e658f4cc745893a

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/78

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -3
    • +3
    ./impls/LSMComponentFilterManager.java
  1. … 52 more files in changeset.
Changes to allow having the no-merge policy as an option in asterix.

Change-Id: I573b6a09185d51df1ec115edc38a89bd029574d5

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/107

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -1
    • +10
    ./impls/ConstantMergePolicyFactory.java
    • -0
    • +58
    ./impls/NoMergePolicyFactory.java
    • -1
    • +10
    ./impls/PrefixMergePolicyFactory.java
  1. … 16 more files in changeset.
Adding virtual pages to BufferCache for lsm-on-hdfs

Change-Id: Ifc69b80fc485f4b3057d717a314f0e203e557b3f

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/104

Reviewed-by: Sattam Alsubaiee <salsubaiee@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +26
    ./impls/MultitenantVirtualBufferCache.java
  1. … 8 more files in changeset.
Added LSM component-level filters for all indexes.

Change-Id: I898cf885c9f88feae85c99799a00fd8ec036efea

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/81

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -0
    • +34
    ./api/ILSMComponentFilter.java
    • -0
    • +20
    ./api/ILSMComponentFilterFactory.java
    • -0
    • +40
    ./api/ILSMComponentFilterFrame.java
    • -0
    • +19
    ./api/ILSMComponentFilterFrameFactory.java
    • -0
    • +32
    ./api/ILSMComponentFilterManager.java
    • -3
    • +13
    ./dataflow/AbstractLSMIndexDataflowHelper.java
    • -1
    • +10
    ./dataflow/AbstractLSMIndexDataflowHelperFactory.java
    • -0
    • +104
    ./frames/LSMComponentFilterFrame.java
    • -0
    • +36
    ./frames/LSMComponentFilterFrameFactory.java
    • -2
    • +10
    ./impls/AbstractMemoryLSMComponent.java
  1. … 116 more files in changeset.
Adding external indexes

In Hyracks side, this change include the following:

1. The addition of three indexes:

a) external b-tree index

b) external r-tree index

c) external b-tree with buddy b-tree index

2. creating an additional logical operator in algebricks for performing lookup operations over external data and modify the different visitors to work with this operator

3. Added copyright header to all new files

Change-Id: Iecfbd86f06aff3caaf3a9652b63420666745ebb9

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/69

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Zachary Heilbron <zheilbron@gmail.com>

Reviewed-by: Sattam Alsubaiee <salsubaiee@gmail.com>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

    • -0
    • +73
    ./api/ITwoPCIndex.java
    • -0
    • +23
    ./impls/AbortRecoverLSMIndexFileManager.java
    • -0
    • +106
    ./impls/AbstractLSMIndexFileManager.java
    • -0
    • +338
    ./impls/ExternalIndexHarness.java
  1. … 58 more files in changeset.
fixed issue 731, 740, and more

commit 8911cc529e72e2bb544d9b472d6e10f173d173af

Author: Young-Seok <kisskys@gmail.com>

Date: Sun May 18 11:28:28 2014 -0700

another fix for picking available index for leftouterjoin plan

commit 9bce43087615fee53613467a027833dd53e190f9

Merge: c8e85ac efab69f

Author: Young-Seok <kisskys@gmail.com>

Date: Sun May 11 22:22:10 2014 -0700

merged master to kisskys/left-outer-join-issue branch

commit c8e85aca31545c13b2a02ff6dc259943e2cf66ad

Author: Young-Seok <kisskys@gmail.com>

Date: Sun May 11 20:17:17 2014 -0700

changes for left-outer-join to pick available indexes

Change-Id: Ib0fc186bc9388802f95445edee92c428b3bb69cc

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/34

Reviewed-by: Inci Cetindil <icetindil@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -2
    • +2
    ./dataflow/LSMTreeIndexCompactOperatorDescriptor.java
    • -2
    • +2
    ./dataflow/LSMTreeIndexInsertUpdateDeleteOperatorDescriptor.java
  1. … 55 more files in changeset.
Fixed an issue in the prefix merge policy where the components are looked at in the wrong order.

Make sure the validty bit in the metadata page is flushed to disk when marking a component to be valid.

Pages are decided to be dirty when the write latches are released. Optimized the scan and update cursor.

  1. … 21 more files in changeset.
Merge remote-tracking branch 'origin/raman/master_feeds' into zheilbron/hyracks_msr_demo

Conflicts:

hyracks/hyracks-control/hyracks-control-cc/src/main/java/edu/uci/ics/hyracks/control/cc/job/JobRun.java

hyracks/hyracks-storage-am-common/src/main/java/edu/uci/ics/hyracks/storage/am/common/impls/AbstractTreeIndex.java

  1. … 3 more files in changeset.
Fix for asterix issue 630.

    • -0
    • +33
    ./impls/LSMIOOperationTask.java
  1. … 14 more files in changeset.
Merge master into salsubaiee/master_issue_92.

    • -4
    • +4
    ./dataflow/AbstractLSMIndexDataflowHelperFactory.java
  1. … 11 more files in changeset.
Allowed user to specify the merging policy

    • -0
    • +27
    ./api/ILSMMergePolicyFactory.java
    • -6
    • +10
    ./dataflow/AbstractLSMIndexDataflowHelperFactory.java
    • -0
    • +48
    ./impls/ConstantMergePolicyFactory.java
    • -36
    • +0
    ./impls/ConstantMergePolicyProvider.java
    • -0
    • +86
    ./impls/PrefixMergePolicy.java
    • -0
    • +50
    ./impls/PrefixMergePolicyFactory.java
  1. … 13 more files in changeset.
Support for full compation.

    • -0
    • +68
    ./dataflow/LSMIndexCompactOperatorNodePushable.java
    • -0
    • +54
    ./dataflow/LSMTreeIndexCompactOperatorDescriptor.java
  1. … 9 more files in changeset.
fixed recovery bugs

    • -6
    • +11
    ./impls/BlockingIOOperationCallbackWrapper.java
Changed the location of clearing the component holder.

  1. … 4 more files in changeset.
Allowed merge subset of the disk components correctly. Added new merge policy.

    • -10
    • +55
    ./impls/LSMIndexSearchCursor.java
  1. … 25 more files in changeset.
checkpoint towards fixing LSN related issues(issue 591, 609, and 614) and more

    • -1
    • +1
    ./api/ILSMIOOperationCallbackFactory.java
    • -6
    • +6
    ./dataflow/AbstractLSMIndexDataflowHelper.java
    • -4
    • +4
    ./dataflow/AbstractLSMIndexDataflowHelperFactory.java
  1. … 52 more files in changeset.
Merge branch 'master' into dev/hyracks_msr

Conflicts:

hyracks/hyracks-storage-am-lsm-common/src/main/java/edu/uci/ics/hyracks/storage/am/lsm/common/impls/LSMHarness.java