Checkout Tools
  • last updated 7 hours ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates

Changeset 1832803 is being indexed.

OAK-7522: Enable collection of simple operation stats for DataStore garbage collection

Added timers for mark & sweep

OAK-7522: Enable collection of simple operation stats for DataStore garbage collection

Added simple metrics for dsgc

    • ?
    ./plugins/blob/OperationsStatsMBean.java
    • ?
    ./plugins/blob/OperationStatsCollector.java
  1. … 4 more files in changeset.
OAK-7473 - Remove hardcoded value for maxLastModifiedInterval
OAK-7209: Race condition can resurrect blobs during blob GC

Calling globalMerge() to retrieve and merge locally all blob ids files from DataStore

Credits to Csaba Varga for the patch

  1. … 1 more file in changeset.
OAK-7223: Files could be kept partially in case of disconnection from backends

- Created a utility method in FileIOUtils#copyInputStreamToFile which delegates to IOUtils but deletes file on exception thus removing any potential partial files created.

- Using the new method in FileCache load.

  1. … 5 more files in changeset.
OAK-7186: avoid use of guava Iterators.emptyIterator()
OAK-7119: Restrict de-serialization mechanism for older serialized cache map in DataStoreCacheUtils to the classes required

- Restricted to whitelist of Map, HashMap, Long, Number, String

  1. … 1 more file in changeset.
OAK-7093: ActiveDelete synchronization with BlobTracker leaves temp files

- Fixed by not calling the method which returns the iterator over records file as not needed

- Added a simple check to a test to assert empty tmp dir

  1. … 1 more file in changeset.
OAK-6066: Migration of binaries relies on implementation details of the TarMK
  1. … 2 more files in changeset.
OAK-6843: Log correct path while initializing the DataStore

- Logging the 'path' if configured

OAK-6550: Make BlobTracker snapshot process smarter

- Logging skipped snapshot for better debugging

OAK-6827: Consistency check fails with active deletions

- consistencyCheck now filers out missing candidates tracked with active deletions

- GC when complete ejects out active deleted tracked ids which are not reported even by mark which essentially means that the those dangling/unused references have been eventually cleared out by version gc

  1. … 1 more file in changeset.
OAK-6827: Consistency check fails with active deletions

Active deleted tracker to keep track of deleted files and provides methods to filter and reconcile

  1. … 1 more file in changeset.
OAK-6550: Make BlobTracker snapshot process smarter

- Enabled snapshot after removal

- And skipping regular snapshot if happened within the last interval

  1. … 1 more file in changeset.
OAK-5983: BlobGC should log the amount of space reclaimed after GC run is done

- Fixing some test failures

OAK-5983: BlobGC should log the amount of space reclaimed after GC run is done

- Logging the total size that's cleaned up only for cases where the length is encoded in the ids.

  1. … 2 more files in changeset.
OAK-5546: [BlobGC] Adapt time to delete blobs based on lucene indexing activity

- Fixed minor typo

OAK-6801: Set minRecordLength to the recommended 4096 in OakFileDataStore

- Setting the default to 4096

OAK-6800: Minor typo in message returned by consistency check through mbean
OAK-5546: [BlobGC] Adapt time to delete blobs based on lucene indexing activity

- Using the minimum of oldest checkpoint timestamp and the mark phase start time as the reference time to calculate max age of blobs to be considered for deletion

- Added test

- Updated NodeStoreService to pass whiteboard instance to the GC object

  1. … 4 more files in changeset.
OAK-6670 - Move FSBlobSerializer to oak-blob-plugins module
    • ?
    ./plugins/blob/serializer/FSBlobSerializer.java
  1. … 4 more files in changeset.
OAK-6669 - Blob serializer to serialize BlobStoreBlob
    • ?
    ./plugins/blob/serializer/BlobIdSerializer.java
  1. … 2 more files in changeset.
OAK-6630: Remove older cached FileDataStore relying on JR caching

- Remove older code relying on CachingFDS from JR2

  1. … 2 more files in changeset.
OAK-6621: Initialize a default sensible stats provider in UploadStagingCache

- Initializing DefaultStatsProvider in case none provided

  1. … 1 more file in changeset.
OAK-6413: FileCache getIfPresent doesn't update cache hit/miss counters

Directly calling CacheLirs.getIfPresent() to properly update hit/miss counter

  1. … 1 more file in changeset.
OAK-6283: FileCache should ignore when file evicted with replacement

Ignore deletion from cache when evicted with REPLACE

  1. … 1 more file in changeset.
OAK-5935: AbstractSharedCachingDataStore#getRecordIfStored should use the underlying cache.get

- The call now returns a FileCachedDataRecord for cases where the cache did not have any entry but the backend has and lazily loads the input stream forcing load through the cache so that the cache is loaded

- Added test lazyLoadStream

- Fixed an issue with the test which would keep a stale reference to a file

  1. … 3 more files in changeset.
OAK-6095 : Get rid of dependency from oak-segment-tar to oak-core tests
    • ?
    ./plugins/blob/migration/BlobMigrator.java
    • ?
    ./plugins/blob/migration/BlobMigration.java
  1. … 17 more files in changeset.
OAK-6073 : Get rid of dependency from oak-blob-azure from oak-core
    • ?
    ./plugins/blob/cloud/CloudBlobStore.java
    • ?
    ./plugins/blob/BlobGarbageCollector.java
    • ?
    ./plugins/blob/ReferenceCollector.java
    • ?
    ./plugins/blob/GarbageCollectionRepoStats.java
    • ?
    ./plugins/blob/datastore/OakCachingFDS.java
    • ?
    ./plugins/blob/datastore/BlobTracker.java
    • ?
    ./plugins/blob/DataStoreCacheUpgradeUtils.java
    • ?
    ./plugins/blob/SharedDataStore.java
  1. … 785 more files in changeset.