Clone Tools
  • last updated 28 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
More index operator cleanup.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1885 123451ca-8445-de46-9d55-352943316053

    • -2
    • +5
    ./dataflow/LSMInvertedIndexBulkLoadOperatorDescriptor.java
    • -98
    • +0
    ./dataflow/LSMInvertedIndexBulkLoadOperatorNodePushable.java
    • -1
    • +1
    ./dataflow/LSMInvertedIndexDataflowHelper.java
    • -1
    • +1
    ./dataflow/LSMInvertedIndexSearchOperatorNodePushable.java
    • -1
    • +1
    ./impls/LSMInvertedIndexOpContext.java
  1. … 52 more files in changeset.
First phase of cleaning up index operators for better code sharing.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1884 123451ca-8445-de46-9d55-352943316053

    • -5
    • +7
    ./api/IInvertedIndexOperatorDescriptor.java
    • -87
    • +14
    ./dataflow/AbstractLSMInvertedIndexOperatorDescriptor.java
    • -10
    • +10
    ./dataflow/LSMInvertedIndexBulkLoadOperatorDescriptor.java
    • -10
    • +10
    ./dataflow/LSMInvertedIndexCreateOperatorDescriptor.java
    • -54
    • +0
    ./dataflow/LSMInvertedIndexCreateOperatorNodePushable.java
    • -28
    • +13
    ./dataflow/LSMInvertedIndexDataflowHelper.java
    • -9
    • +8
    ./dataflow/LSMInvertedIndexSearchOperatorDescriptor.java
  1. … 12 more files in changeset.
Switched inverted-index operators to lsm-inverted-index. Minor cleaning for better code sharing.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1883 123451ca-8445-de46-9d55-352943316053

    • -3
    • +3
    ./api/IInvertedIndexOperatorDescriptor.java
    • -157
    • +0
    ./dataflow/AbstractInvertedIndexOperatorDescriptor.java
    • -0
    • +157
    ./dataflow/AbstractLSMInvertedIndexOperatorDescriptor.java
    • -56
    • +0
    ./dataflow/InvertedIndexBulkLoadOperatorDescriptor.java
    • -96
    • +0
    ./dataflow/InvertedIndexBulkLoadOperatorNodePushable.java
    • -52
    • +0
    ./dataflow/InvertedIndexCreateOperatorDescriptor.java
    • -53
    • +0
    ./dataflow/InvertedIndexCreateOperatorNodePushable.java
    • -52
    • +0
    ./dataflow/InvertedIndexDataflowHelper.java
    • -33
    • +0
    ./dataflow/InvertedIndexDataflowHelperFactory.java
    • -64
    • +0
    ./dataflow/InvertedIndexSearchOperatorDescriptor.java
    • -163
    • +0
    ./dataflow/InvertedIndexSearchOperatorNodePushable.java
    • -0
    • +56
    ./dataflow/LSMInvertedIndexBulkLoadOperatorDescriptor.java
    • -0
    • +98
    ./dataflow/LSMInvertedIndexBulkLoadOperatorNodePushable.java
    • -0
    • +52
    ./dataflow/LSMInvertedIndexCreateOperatorDescriptor.java
    • -0
    • +54
    ./dataflow/LSMInvertedIndexCreateOperatorNodePushable.java
  1. … 27 more files in changeset.
Properly Using op callbacks in the lsm inverted index.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1882 123451ca-8445-de46-9d55-352943316053

    • -17
    • +16
    ./impls/LSMInvertedIndexAccessor.java
    • -1
    • +9
    ./impls/LSMInvertedIndexOpContext.java
    • -0
    • +5
    ./impls/LSMInvertedIndexRangeSearchCursor.java
    • -4
    • +12
    ./impls/LSMInvertedIndexSearchCursor.java
    • -4
    • +6
    ./impls/LSMInvertedIndexSearchCursorInitialState.java
  1. … 2 more files in changeset.
Adapted inverted index lifecycle calls to latest interfaces.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1881 123451ca-8445-de46-9d55-352943316053

    • -8
    • +6
    ./dataflow/InvertedIndexBulkLoadOperatorNodePushable.java
    • -4
    • +1
    ./dataflow/InvertedIndexCreateOperatorNodePushable.java
    • -1
    • +1
    ./dataflow/InvertedIndexDataflowHelper.java
    • -6
    • +4
    ./dataflow/InvertedIndexSearchOperatorNodePushable.java
Forgot a file.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1874 123451ca-8445-de46-9d55-352943316053

    • -0
    • +34
    ./ondisk/FixedSizeElementInvertedListBuilderFactory.java
Fixed incorrect sharing of inverted-list builder causing the multi-threaded tests to break. Finished multi-threading tests.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1873 123451ca-8445-de46-9d55-352943316053

    • -0
    • +20
    ./api/IInvertedListBuilderFactory.java
    • -1
    • +0
    ./ondisk/FixedSizeElementInvertedListCursor.java
    • -4
    • +6
    ./ondisk/OnDiskInvertedIndexFactory.java
    • -0
    • +1
    ./ondisk/OnDiskInvertedIndexRangeSearchCursor.java
  1. … 6 more files in changeset.
Fixed a few multi-threading issues in the lsm-inverted-index. Still some more to go.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1871 123451ca-8445-de46-9d55-352943316053

    • -2
    • +1
    ./impls/LSMInvertedIndexRangeSearchCursor.java
    • -1
    • +1
    ./impls/LSMInvertedIndexSearchCursor.java
    • -8
    • +16
    ./inmemory/InMemoryInvertedListCursor.java
    • -3
    • +2
    ./ondisk/OnDiskInvertedIndexRangeSearchCursor.java
  1. … 11 more files in changeset.
Checkpointing lsm-inverted-index multi thread test. Still needs some debugging.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1870 123451ca-8445-de46-9d55-352943316053

    • -1
    • +3
    ./impls/LSMInvertedIndexRangeSearchCursor.java
  1. … 15 more files in changeset.
Added tests for gram-based indexes, and hashed tokens.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1868 123451ca-8445-de46-9d55-352943316053

    • -2
    • +5
    ./impls/LSMInvertedIndexSearchCursor.java
    • -3
    • +3
    ./inmemory/InMemoryInvertedListCursor.java
    • -1
    • +1
    ./ondisk/OnDiskInvertedIndexRangeSearchCursor.java
    • -2
    • +3
    ./search/TOccurrenceSearcherSuffixProbeOnly.java
    • -2
    • +3
    ./search/TOccurrenceSearcherSuffixScanOnly.java
    • -18
    • +15
    ./tokenizers/DelimitedUTF8StringBinaryTokenizerFactory.java
    • -1
    • +0
    ./tokenizers/HashedUTF8NGramTokenFactory.java
    • -0
    • +42
    ./tokenizers/NGramUTF8StringBinaryTokenizerFactory.java
  1. … 14 more files in changeset.
Improved and fixed a few bugs in the lsm inverted index tests.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1867 123451ca-8445-de46-9d55-352943316053

    • -0
    • +5
    ./search/ConjunctiveSearchModifier.java
    • -0
    • +5
    ./search/EditDistanceSearchModifier.java
  1. … 9 more files in changeset.
Another bugfix.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1866 123451ca-8445-de46-9d55-352943316053

    • -0
    • +3
    ./ondisk/OnDiskInvertedIndexRangeSearchCursor.java
Small bugfix related to pinning/unpinning papes properly.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1865 123451ca-8445-de46-9d55-352943316053

    • -2
    • +4
    ./ondisk/OnDiskInvertedIndexRangeSearchCursor.java
Fixed deletion protocol in the lsm-inverted-index. The in-memory deleted-keys BTree now only contains keys referring to on-disk components. Deletions that refer to documents in the in-memory inverted index are physically removed from there, and no entry to the deleted-keys BTree is made. This behavior seems necessary to avoid a pathological case of 'lost deletes' to on-disk components (more details in comments of the code).

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1864 123451ca-8445-de46-9d55-352943316053

    • -2
    • +2
    ./impls/LSMInvertedIndexRangeSearchCursor.java
    • -1
    • +1
    ./impls/LSMInvertedIndexSearchCursor.java
  1. … 14 more files in changeset.
Merged hyracks_lsm_tree r1846:r1858.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1859 123451ca-8445-de46-9d55-352943316053

    • -10
    • +1
    ./dataflow/AbstractInvertedIndexOperatorDescriptor.java
    • -8
    • +6
    ./dataflow/InvertedIndexBulkLoadOperatorDescriptor.java
    • -15
    • +5
    ./dataflow/InvertedIndexBulkLoadOperatorNodePushable.java
    • -8
    • +6
    ./dataflow/InvertedIndexCreateOperatorDescriptor.java
    • -7
    • +5
    ./dataflow/InvertedIndexSearchOperatorDescriptor.java
    • -11
    • +1
    ./dataflow/InvertedIndexSearchOperatorNodePushable.java
  1. … 3 more files in changeset.
Dealing with deleted keys in search cursors. Tests to follow.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1858 123451ca-8445-de46-9d55-352943316053

    • -2
    • +2
    ./impls/LSMInvertedIndexRangeSearchCursor.java
    • -7
    • +40
    ./impls/LSMInvertedIndexSearchCursor.java
Minor bugfix.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1856 123451ca-8445-de46-9d55-352943316053

    • -2
    • +4
    ./impls/LSMInvertedIndexRangeSearchCursorInitialState.java
Significantly simplified LSMInvertedIndexSearchCursor in preparation to dealing with deletes.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1855 123451ca-8445-de46-9d55-352943316053

    • -96
    • +0
    ./impls/LSMInvertedIndexCursorInitialState.java
    • -7
    • +1
    ./impls/LSMInvertedIndexRangeSearchCursor.java
    • -10
    • +15
    ./impls/LSMInvertedIndexRangeSearchCursorInitialState.java
    • -75
    • +44
    ./impls/LSMInvertedIndexSearchCursor.java
    • -0
    • +116
    ./impls/LSMInvertedIndexSearchCursorInitialState.java
Minor cleanup for better code sharing.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1854 123451ca-8445-de46-9d55-352943316053

    • -47
    • +4
    ./impls/LSMInvertedIndexRangeSearchCursor.java
  1. … 1 more file in changeset.
Added basic lsm-inverted-index delete test that validates the index using a range search cursor (sort-merges multiple components and removes deleted entries). Still need to remove deleted entries during regular inverted index searches.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1853 123451ca-8445-de46-9d55-352943316053

    • -2
    • +57
    ./impls/LSMInvertedIndexRangeSearchCursor.java
    • -6
    • +27
    ./impls/LSMInvertedIndexRangeSearchCursorInitialState.java
  1. … 3 more files in changeset.
Added delete test for in-memory inverted index (only adding key to LSM buddy BTree can lead to false positives, practically impossible to test). Fixed a bug in the counting BTree cursor when dealing with empty pages due to deletes.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1852 123451ca-8445-de46-9d55-352943316053

    • -1
    • +1
    ./inmemory/InMemoryInvertedIndexAccessor.java
    • -1
    • +2
    ./inmemory/InMemoryInvertedListCursor.java
  1. … 11 more files in changeset.
First steps to getting delete working.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1851 123451ca-8445-de46-9d55-352943316053

    • -11
    • +13
    ./impls/LSMInvertedIndexAccessor.java
    • -13
    • +22
    ./impls/LSMInvertedIndexOpContext.java
    • -6
    • +16
    ./inmemory/InMemoryInvertedIndex.java
    • -5
    • +7
    ./inmemory/InMemoryInvertedIndexAccessor.java
    • -6
    • +9
    ./inmemory/InMemoryInvertedIndexOpContext.java
    • -78
    • +0
    ./util/InvertedIndexInsertTupleIterator.java
    • -0
    • +78
    ./util/InvertedIndexTokenizingTupleIterator.java
  1. … 4 more files in changeset.
Implemented lsm-inverted-index merge. Added test.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1850 123451ca-8445-de46-9d55-352943316053

    • -4
    • +4
    ./impls/LSMInvertedIndexMergeOperation.java
    • -28
    • +11
    ./impls/LSMInvertedIndexOpContext.java
    • -0
    • +1
    ./ondisk/OnDiskInvertedIndexRangeSearchCursor.java
  1. … 14 more files in changeset.
Implemented lsm inverted index range search cursor that is used in merges.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1847 123451ca-8445-de46-9d55-352943316053

    • -229
    • +48
    ./impls/LSMInvertedIndexRangeSearchCursor.java
    • -0
    • +95
    ./impls/LSMInvertedIndexRangeSearchCursorInitialState.java
  1. … 6 more files in changeset.
Made changes to inverted index ops according to Zach's latest changes to index lifecycle management.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1846 123451ca-8445-de46-9d55-352943316053

    • -1
    • +10
    ./dataflow/AbstractInvertedIndexOperatorDescriptor.java
    • -6
    • +8
    ./dataflow/InvertedIndexBulkLoadOperatorDescriptor.java
    • -2
    • +12
    ./dataflow/InvertedIndexBulkLoadOperatorNodePushable.java
    • -6
    • +8
    ./dataflow/InvertedIndexCreateOperatorDescriptor.java
    • -5
    • +7
    ./dataflow/InvertedIndexSearchOperatorDescriptor.java
    • -1
    • +11
    ./dataflow/InvertedIndexSearchOperatorNodePushable.java
Added range search cursor for on-disk inverted index in preparation for implementing merge.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1842 123451ca-8445-de46-9d55-352943316053

    • -0
    • +13
    ./impls/LSMInvertedIndexAccessor.java
    • -1
    • +1
    ./impls/LSMInvertedIndexRangeSearchCursor.java
    • -0
    • +14
    ./inmemory/InMemoryInvertedIndexAccessor.java
    • -0
    • +132
    ./ondisk/OnDiskInvertedIndexRangeSearchCursor.java
  1. … 2 more files in changeset.
Minor cleanup.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1841 123451ca-8445-de46-9d55-352943316053

    • -45
    • +26
    ./impls/LSMInvertedIndexSearchCursor.java
Simplified a bunch of inverted-index interfaces. Added lsm inverted index search test.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1840 123451ca-8445-de46-9d55-352943316053

    • -3
    • +2
    ./api/IInvertedIndexSearchModifier.java
    • -12
    • +24
    ./impls/LSMInvertedIndexAccessor.java
    • -5
    • +0
    ./inmemory/InMemoryInvertedIndexAccessor.java
    • -9
    • +3
    ./search/ConjunctiveSearchModifier.java
    • -9
    • +4
    ./search/EditDistanceSearchModifier.java
    • -10
    • +5
    ./search/JaccardSearchModifier.java
  1. … 4 more files in changeset.
Added bulk loader for lsm inverted index and test (no multi-bulk-load test yet).

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1835 123451ca-8445-de46-9d55-352943316053

  1. … 1 more file in changeset.
Added lsm inverted index insert test based on new testing framework.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1834 123451ca-8445-de46-9d55-352943316053

    • -17
    • +59
    ./impls/LSMInvertedIndexAccessor.java
    • -0
    • +24
    ./impls/LSMInvertedIndexCursorInitialState.java
    • -6
    • +6
    ./impls/LSMInvertedIndexFileManager.java
    • -0
    • +100
    ./impls/LSMInvertedIndexMergeOperation.java
    • -3
    • +5
    ./impls/LSMInvertedIndexOpContext.java
    • -4
    • +1
    ./impls/LSMInvertedIndexRangeSearchCursor.java
    • -8
    • +8
    ./impls/LSMInvertedIndexSearchCursor.java
  1. … 14 more files in changeset.