Clone Tools
  • last updated 24 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Added tests for gram-based indexes, and hashed tokens.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1868 123451ca-8445-de46-9d55-352943316053

    • -2
    • +5
    ./impls/LSMInvertedIndexSearchCursor.java
    • -3
    • +3
    ./inmemory/InMemoryInvertedListCursor.java
    • -1
    • +1
    ./ondisk/OnDiskInvertedIndexRangeSearchCursor.java
    • -2
    • +3
    ./search/TOccurrenceSearcherSuffixProbeOnly.java
    • -2
    • +3
    ./search/TOccurrenceSearcherSuffixScanOnly.java
    • -18
    • +15
    ./tokenizers/DelimitedUTF8StringBinaryTokenizerFactory.java
    • -1
    • +0
    ./tokenizers/HashedUTF8NGramTokenFactory.java
    • -0
    • +42
    ./tokenizers/NGramUTF8StringBinaryTokenizerFactory.java
  1. … 14 more files in changeset.
Improved and fixed a few bugs in the lsm inverted index tests.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1867 123451ca-8445-de46-9d55-352943316053

    • -0
    • +5
    ./search/ConjunctiveSearchModifier.java
    • -0
    • +5
    ./search/EditDistanceSearchModifier.java
  1. … 9 more files in changeset.
Another bugfix.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1866 123451ca-8445-de46-9d55-352943316053

    • -0
    • +3
    ./ondisk/OnDiskInvertedIndexRangeSearchCursor.java
Small bugfix related to pinning/unpinning papes properly.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1865 123451ca-8445-de46-9d55-352943316053

    • -2
    • +4
    ./ondisk/OnDiskInvertedIndexRangeSearchCursor.java
Fixed deletion protocol in the lsm-inverted-index. The in-memory deleted-keys BTree now only contains keys referring to on-disk components. Deletions that refer to documents in the in-memory inverted index are physically removed from there, and no entry to the deleted-keys BTree is made. This behavior seems necessary to avoid a pathological case of 'lost deletes' to on-disk components (more details in comments of the code).

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1864 123451ca-8445-de46-9d55-352943316053

    • -2
    • +2
    ./impls/LSMInvertedIndexRangeSearchCursor.java
    • -1
    • +1
    ./impls/LSMInvertedIndexSearchCursor.java
  1. … 14 more files in changeset.
Merged hyracks_lsm_tree r1846:r1858.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1859 123451ca-8445-de46-9d55-352943316053

    • -10
    • +1
    ./dataflow/AbstractInvertedIndexOperatorDescriptor.java
    • -8
    • +6
    ./dataflow/InvertedIndexBulkLoadOperatorDescriptor.java
    • -15
    • +5
    ./dataflow/InvertedIndexBulkLoadOperatorNodePushable.java
    • -8
    • +6
    ./dataflow/InvertedIndexCreateOperatorDescriptor.java
    • -7
    • +5
    ./dataflow/InvertedIndexSearchOperatorDescriptor.java
    • -11
    • +1
    ./dataflow/InvertedIndexSearchOperatorNodePushable.java
  1. … 3 more files in changeset.
Dealing with deleted keys in search cursors. Tests to follow.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1858 123451ca-8445-de46-9d55-352943316053

    • -2
    • +2
    ./impls/LSMInvertedIndexRangeSearchCursor.java
    • -7
    • +40
    ./impls/LSMInvertedIndexSearchCursor.java
Minor bugfix.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1856 123451ca-8445-de46-9d55-352943316053

    • -2
    • +4
    ./impls/LSMInvertedIndexRangeSearchCursorInitialState.java
Significantly simplified LSMInvertedIndexSearchCursor in preparation to dealing with deletes.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1855 123451ca-8445-de46-9d55-352943316053

    • -96
    • +0
    ./impls/LSMInvertedIndexCursorInitialState.java
    • -7
    • +1
    ./impls/LSMInvertedIndexRangeSearchCursor.java
    • -10
    • +15
    ./impls/LSMInvertedIndexRangeSearchCursorInitialState.java
    • -75
    • +44
    ./impls/LSMInvertedIndexSearchCursor.java
    • -0
    • +116
    ./impls/LSMInvertedIndexSearchCursorInitialState.java
Minor cleanup for better code sharing.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1854 123451ca-8445-de46-9d55-352943316053

    • -47
    • +4
    ./impls/LSMInvertedIndexRangeSearchCursor.java
  1. … 1 more file in changeset.
Added basic lsm-inverted-index delete test that validates the index using a range search cursor (sort-merges multiple components and removes deleted entries). Still need to remove deleted entries during regular inverted index searches.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1853 123451ca-8445-de46-9d55-352943316053

    • -2
    • +57
    ./impls/LSMInvertedIndexRangeSearchCursor.java
    • -6
    • +27
    ./impls/LSMInvertedIndexRangeSearchCursorInitialState.java
  1. … 3 more files in changeset.
Added delete test for in-memory inverted index (only adding key to LSM buddy BTree can lead to false positives, practically impossible to test). Fixed a bug in the counting BTree cursor when dealing with empty pages due to deletes.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1852 123451ca-8445-de46-9d55-352943316053

    • -1
    • +1
    ./inmemory/InMemoryInvertedIndexAccessor.java
    • -1
    • +2
    ./inmemory/InMemoryInvertedListCursor.java
  1. … 11 more files in changeset.
First steps to getting delete working.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1851 123451ca-8445-de46-9d55-352943316053

    • -11
    • +13
    ./impls/LSMInvertedIndexAccessor.java
    • -13
    • +22
    ./impls/LSMInvertedIndexOpContext.java
    • -6
    • +16
    ./inmemory/InMemoryInvertedIndex.java
    • -5
    • +7
    ./inmemory/InMemoryInvertedIndexAccessor.java
    • -6
    • +9
    ./inmemory/InMemoryInvertedIndexOpContext.java
    • -78
    • +0
    ./util/InvertedIndexInsertTupleIterator.java
    • -0
    • +78
    ./util/InvertedIndexTokenizingTupleIterator.java
  1. … 4 more files in changeset.
Implemented lsm-inverted-index merge. Added test.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1850 123451ca-8445-de46-9d55-352943316053

    • -4
    • +4
    ./impls/LSMInvertedIndexMergeOperation.java
    • -28
    • +11
    ./impls/LSMInvertedIndexOpContext.java
    • -0
    • +1
    ./ondisk/OnDiskInvertedIndexRangeSearchCursor.java
  1. … 14 more files in changeset.
Implemented lsm inverted index range search cursor that is used in merges.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1847 123451ca-8445-de46-9d55-352943316053

    • -229
    • +48
    ./impls/LSMInvertedIndexRangeSearchCursor.java
    • -0
    • +95
    ./impls/LSMInvertedIndexRangeSearchCursorInitialState.java
  1. … 6 more files in changeset.
Made changes to inverted index ops according to Zach's latest changes to index lifecycle management.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1846 123451ca-8445-de46-9d55-352943316053

    • -1
    • +10
    ./dataflow/AbstractInvertedIndexOperatorDescriptor.java
    • -6
    • +8
    ./dataflow/InvertedIndexBulkLoadOperatorDescriptor.java
    • -2
    • +12
    ./dataflow/InvertedIndexBulkLoadOperatorNodePushable.java
    • -6
    • +8
    ./dataflow/InvertedIndexCreateOperatorDescriptor.java
    • -5
    • +7
    ./dataflow/InvertedIndexSearchOperatorDescriptor.java
    • -1
    • +11
    ./dataflow/InvertedIndexSearchOperatorNodePushable.java
Added range search cursor for on-disk inverted index in preparation for implementing merge.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1842 123451ca-8445-de46-9d55-352943316053

    • -0
    • +13
    ./impls/LSMInvertedIndexAccessor.java
    • -1
    • +1
    ./impls/LSMInvertedIndexRangeSearchCursor.java
    • -0
    • +14
    ./inmemory/InMemoryInvertedIndexAccessor.java
    • -0
    • +132
    ./ondisk/OnDiskInvertedIndexRangeSearchCursor.java
  1. … 2 more files in changeset.
Minor cleanup.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1841 123451ca-8445-de46-9d55-352943316053

    • -45
    • +26
    ./impls/LSMInvertedIndexSearchCursor.java
Simplified a bunch of inverted-index interfaces. Added lsm inverted index search test.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1840 123451ca-8445-de46-9d55-352943316053

    • -3
    • +2
    ./api/IInvertedIndexSearchModifier.java
    • -12
    • +24
    ./impls/LSMInvertedIndexAccessor.java
    • -5
    • +0
    ./inmemory/InMemoryInvertedIndexAccessor.java
    • -9
    • +3
    ./search/ConjunctiveSearchModifier.java
    • -9
    • +4
    ./search/EditDistanceSearchModifier.java
    • -10
    • +5
    ./search/JaccardSearchModifier.java
  1. … 4 more files in changeset.
Added bulk loader for lsm inverted index and test (no multi-bulk-load test yet).

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1835 123451ca-8445-de46-9d55-352943316053

  1. … 1 more file in changeset.
Added lsm inverted index insert test based on new testing framework.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1834 123451ca-8445-de46-9d55-352943316053

    • -17
    • +59
    ./impls/LSMInvertedIndexAccessor.java
    • -0
    • +24
    ./impls/LSMInvertedIndexCursorInitialState.java
    • -6
    • +6
    ./impls/LSMInvertedIndexFileManager.java
    • -0
    • +100
    ./impls/LSMInvertedIndexMergeOperation.java
    • -3
    • +5
    ./impls/LSMInvertedIndexOpContext.java
    • -4
    • +1
    ./impls/LSMInvertedIndexRangeSearchCursor.java
    • -8
    • +8
    ./impls/LSMInvertedIndexSearchCursor.java
  1. … 14 more files in changeset.
Almost done with flush operation of lsm inverted index.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1833 123451ca-8445-de46-9d55-352943316053

    • -0
    • +23
    ./api/IInvertedIndexFileNameMapper.java
    • -180
    • +124
    ./impls/LSMInvertedIndex.java
    • -65
    • +67
    ./impls/LSMInvertedIndexFileManager.java
    • -0
    • +74
    ./impls/LSMInvertedIndexFlushOperation.java
    • -6
    • +31
    ./impls/LSMInvertedIndexOpContext.java
    • -0
    • +4
    ./inmemory/InMemoryInvertedIndexAccessor.java
    • -5
    • +9
    ./ondisk/OnDiskInvertedIndexFactory.java
Added component finalizer for lsm inverted index. Started to implement corresponding index file manager.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1832 123451ca-8445-de46-9d55-352943316053

    • -107
    • +0
    ./impls/InvertedIndexComponentFinalizer.java
    • -0
    • +61
    ./impls/LSMInvertedIndexComponentFinalizer.java
    • -40
    • +56
    ./impls/LSMInvertedIndexFileManager.java
  1. … 13 more files in changeset.
Started working on the lifecycle stuff for the LSM inverted index.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1827 123451ca-8445-de46-9d55-352943316053

    • -165
    • +180
    ./impls/LSMInvertedIndex.java
    • -119
    • +133
    ./impls/LSMInvertedIndexFileManager.java
    • -3
    • +3
    ./ondisk/OnDiskInvertedIndexFactory.java
Cleaned up more old tests.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1826 123451ca-8445-de46-9d55-352943316053

  1. … 3 more files in changeset.
Added basic search test for in-memory inverted index based on new framework. Fixed a few bugs in the in-memory inverted list cursor.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1824 123451ca-8445-de46-9d55-352943316053

    • -0
    • +5
    ./inmemory/InMemoryInvertedIndexAccessor.java
    • -3
    • +28
    ./inmemory/InMemoryInvertedListCursor.java
  1. … 3 more files in changeset.
Added first inverted-index search test based on new testing framework. Found and fixed a few bugs.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1823 123451ca-8445-de46-9d55-352943316053

    • -3
    • +4
    ./ondisk/FixedSizeElementInvertedListBuilder.java
    • -1
    • +1
    ./ondisk/FixedSizeElementInvertedListCursor.java
  1. … 12 more files in changeset.
Added validation of on-disk inverted index. Added bulk-load test for on-disk inverted index, and insert test for in-memory inverted index based on a new testing framework.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1821 123451ca-8445-de46-9d55-352943316053

    • -0
    • +28
    ./api/IInvertedIndexAccessor.java
    • -1
    • +1
    ./impls/LSMInvertedIndexRangeSearchCursor.java
    • -8
    • +20
    ./inmemory/InMemoryInvertedIndexAccessor.java
    • -2
    • +3
    ./inmemory/InMemoryInvertedListCursor.java
    • -2
    • +2
    ./ondisk/FixedSizeElementInvertedListCursor.java
  1. … 16 more files in changeset.
Added prefix-search capability for sorted sets of CheckTuples to enable comparing inverted indexes against a baseline inverted index.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1819 123451ca-8445-de46-9d55-352943316053

    • -35
    • +16
    ./inmemory/InMemoryInvertedIndex.java
    • -10
    • +8
    ./inmemory/InMemoryInvertedIndexAccessor.java
    • -7
    • +11
    ./inmemory/InMemoryInvertedIndexOpContext.java
    • -0
    • +78
    ./util/InvertedIndexInsertTupleIterator.java
  1. … 4 more files in changeset.
Started implementation of InvertedIndexTestContext.

git-svn-id: https://hyracks.googlecode.com/svn/branches/hyracks_inverted_index_updates_new@1818 123451ca-8445-de46-9d55-352943316053

    • -0
    • +10
    ./inmemory/InMemoryInvertedIndex.java
  1. … 23 more files in changeset.