Checkout Tools
  • last updated 2 hours ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates

Changeset 1842677 is being indexed.

OAK-7798: Return stream for downloaded files rather than directly from backend

- Returning stream from a downloaded file rather than directly from backend

  1. … 1 more file in changeset.
OAK-7753: Enable collection of stats for DataStore checkConsistency operation

Added statistics for checkConsistency operation

  1. … 3 more files in changeset.
OAK-7727: Aggregate references from repositories with same repositoryId

- A different references file is created and then aggregated to account for all references available

  1. … 3 more files in changeset.
OAK-7738: Add method to check for existence of metadata in SharedDataStore

- Added a new method to the ShareDataStore and implementations

  1. … 11 more files in changeset.
OAK-7737: Add edge cases checks and tests for metadata related methods S3/FileDataStore

- Superseded by SharedDataStoreTest which covers all SharedDataStore tests

OAK-7737: Add edge cases checks and tests for metadata related methods S3/FileDataStore

- Added tests for metadata operations

    • ?
    ./blob/datastore/SharedDataStoreTest.java
  1. … 4 more files in changeset.
OAK-7692: [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded

Patch from Alexander Klimetschek

  1. … 1 more file in changeset.
OAK-7569: Direct Binary Access

Blob APIs and S3/Azure implementation for direct binary upload/download. Modified patch from Matt Ryan

  1. … 23 more files in changeset.
OAK-7649: oak-run command to execute datastore garbage collection

- Enable 'datastore' command with operations --check-consistency & --collect-garbage

  1. … 6 more files in changeset.
OAK-7646: UploadStagingCacheTest#testConcurrentSameAddRequest fails intermittently

- Changed MeterStats#mark() to MeterStats#mark(1) which curiously fixes the issue.

- Added a slight delay in starting the 2nd thread

  1. … 1 more file in changeset.
OAK-7511: get rid of JSR 305 dependency - use jetbrains nullability annotations instead - oak-blob-plugins
  1. … 20 more files in changeset.
OAK-7638: Race condition when simultaneous request to stage file for async upload

- Fixed the entry condition to check for file existence and create if required

  1. … 2 more files in changeset.
OAK-7584: Add stats for DataStore GC performance

Added stats for candidates found, blobs deleted, size deleted

  1. … 2 more files in changeset.
OAK-7566: Rename metrics for DataStore garbage collection

- Renamed some metrics

- Removed a redundant counter FINISH_SUCCESS

  1. … 4 more files in changeset.
OAK-7522: Enable collection of simple operation stats for DataStore garbage collection

Added simple metrics for dsgc

  1. … 9 more files in changeset.
OAK-7209: Race condition can resurrect blobs during blob GC

Calling globalMerge() to retrieve and merge locally all blob ids files from DataStore

Credits to Csaba Varga for the patch

  1. … 1 more file in changeset.
OAK-7223: Files could be kept partially in case of disconnection from backends

- Created a utility method in FileIOUtils#copyInputStreamToFile which delegates to IOUtils but deletes file on exception thus removing any potential partial files created.

- Using the new method in FileCache load.

  1. … 4 more files in changeset.
OAK-7119: Restrict de-serialization mechanism for older serialized cache map in DataStoreCacheUtils to the classes required

- Restricted to whitelist of Map, HashMap, Long, Number, String

  1. … 1 more file in changeset.
OAK-6066: Migration of binaries relies on implementation details of the TarMK
  1. … 3 more files in changeset.
OAK-6827: Consistency check fails with active deletions

More test coverage

  1. … 1 more file in changeset.
OAK-6827: Consistency check fails with active deletions

Active deleted tracker to keep track of deleted files and provides methods to filter and reconcile

  1. … 2 more files in changeset.
OAK-6550: Make BlobTracker snapshot process smarter

- Enabled snapshot after removal

- And skipping regular snapshot if happened within the last interval

  1. … 1 more file in changeset.
OAK-5983: BlobGC should log the amount of space reclaimed after GC run is done

- Logging the total size that's cleaned up only for cases where the length is encoded in the ids.

  1. … 3 more files in changeset.
OAK-5546: [BlobGC] Adapt time to delete blobs based on lucene indexing activity

- Using the minimum of oldest checkpoint timestamp and the mark phase start time as the reference time to calculate max age of blobs to be considered for deletion

- Added test

- Updated NodeStoreService to pass whiteboard instance to the GC object

    • ?
    ./blob/BlobGCCheckpointRefTest.java
  1. … 4 more files in changeset.
OAK-6799 - OSGi service tests for Blob GC

- Tests for BlobGC Mbean initialization for OSGi *NodeStoreServices

    • ?
    ./blob/AbstractBlobGCRegistrationTest.java
  1. … 2 more files in changeset.
OAK-6798 - Basic Blob GC test for in-memory NodeStore and BlobStore

- Added a BlobGCTest with a in-memory test extension to MemoryNodeStore and an in-memory test BlobStore

OAK-6670 - Move FSBlobSerializer to oak-blob-plugins module
    • ?
    ./blob/serializer/FSBlobSerializerTest.java
  1. … 4 more files in changeset.
OAK-6669 - Blob serializer to serialize BlobStoreBlob
    • ?
    ./blob/serializer/BlobIdSerializerTest.java
  1. … 2 more files in changeset.
OAK-6630: Remove older cached FileDataStore relying on JR caching

- Remove older code relying on CachingFDS from JR2

  1. … 3 more files in changeset.
OAK-6621: Initialize a default sensible stats provider in UploadStagingCache

- Initializing DefaultStatsProvider in case none provided

  1. … 1 more file in changeset.