Checkout Tools
  • last updated 2 hours ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
OAK-8780 - Remove the synchronized block on the FSBackend

Slightly modified patch from Ankita Agarwal

Moved the copy of file to a temp file outside the synchronized block

  1. … 1 more file in changeset.
fix svn:eol-style
  1. … 4 more files in changeset.
OAK-8280: Add support to request ignoring domain override for signed URIs.

This change introduces BlobUploadOptions so options can be requested for

both upload and download URIs, and adds the option for both URI requests

to ignore the domain override if one is set.

    • ?
    ./directaccess/DataRecordUploadOptions.java
  1. … 22 more files in changeset.
OAK-8104: Update to Azure SDK v8.6.0 to fix signed URI C-D headers.

The purpose of this commit is to fix Content-Disposition header support for signed download URIs. The filename* portion of the header specification was disabled in OAK-8013 due to a bug in the Azure SDK pre-8.5.0. This commit adds filename* portion support back into Oak, which addresses the following issues:

- OAK-8104 - Fix the Content-Disposition header correctly (required upgrading the Azure SDK version to 8.6.0)

- OAK-8607 - Undo the workarounds implemented in OAK-8013.

  1. … 5 more files in changeset.
OAK-8576: Extend instrumentation for data stores.

This commit adds instrumentation for other methods in data stores, including

support for metrics for SharedDataStore implementations and for direct

binary access functions, as well as improving existing instrumentation for

uploads and downloads. It also adds error counts to aid in identifying

and diagnosing failure conditions in production environments.

  1. … 9 more files in changeset.
OAK-8676: deprecate FileIOUtils.FileLineDifferenceIterator
  1. … 5 more files in changeset.
OAK-8666: deprecate FileIOUtils.BurnOnCloseFileIterator
  1. … 4 more files in changeset.
OAK-8298: Add tracking of blob ids added via direct upload

OAK-8314: BlobIds created with direct uploads lack length suffix

Suffixing length to the ids being returned for for directly uploded binaries

  1. … 1 more file in changeset.
OAK-8115: Allow disabling of BlobTracker when not required

- BlobTracker can be disabled by setting config property snapshotIntervalSecs to 0 or below as this prop needs a sensible non-zero value anyways to be effective

  1. … 7 more files in changeset.
OAK-8013: Disable filename* part of Content-Disposition specification for direct binary access.

  1. … 2 more files in changeset.
OAK-7818: Includes chained exception in AzureBlobStoreBackend#completeHttpUpload exception handling.

  1. … 1 more file in changeset.
OAK-7727: Aggregate references from repositories with same repositoryId

- A different references file is created and then aggregated to account for all references available

  1. … 5 more files in changeset.
OAK-7738: Add method to check for existence of metadata in SharedDataStore

- Added a new method to the ShareDataStore and implementations

  1. … 11 more files in changeset.
OAK-7737: Add edge cases checks and tests for metadata related methods S3/FileDataStore

- Added tests for metadata operations

  1. … 4 more files in changeset.
OAK-7692: [DirectBinaryAccess] Upload token HMAC signature must be base64 encoded

Patch from Alexander Klimetschek

  1. … 1 more file in changeset.
OAK-7569: Direct Binary Access

Blob APIs and S3/Azure implementation for direct binary upload/download. Modified patch from Matt Ryan

    • ?
    ./directaccess/DataRecordAccessProvider.java
    • ?
    ./directaccess/DataRecordUpload.java
    • ?
    ./directaccess/DataRecordUploadToken.java
    • ?
    ./directaccess/ConfigurableDataRecordAccessProvider.java
    • ?
    ./directaccess/DataRecordDownloadOptions.java
    • ?
    ./directaccess/DataRecordUploadException.java
  1. … 19 more files in changeset.
OAK-7511: get rid of JSR 305 dependency - use jetbrains nullability annotations instead - oak-blob-plugins
  1. … 22 more files in changeset.
OAK-7209: Race condition can resurrect blobs during blob GC

Calling globalMerge() to retrieve and merge locally all blob ids files from DataStore

Credits to Csaba Varga for the patch

  1. … 1 more file in changeset.
OAK-7186: avoid use of guava Iterators.emptyIterator()
  1. … 1 more file in changeset.
OAK-7093: ActiveDelete synchronization with BlobTracker leaves temp files

- Fixed by not calling the method which returns the iterator over records file as not needed

- Added a simple check to a test to assert empty tmp dir

  1. … 1 more file in changeset.
OAK-6843: Log correct path while initializing the DataStore

- Logging the 'path' if configured

OAK-6550: Make BlobTracker snapshot process smarter

- Logging skipped snapshot for better debugging

OAK-6827: Consistency check fails with active deletions

Active deleted tracker to keep track of deleted files and provides methods to filter and reconcile

  1. … 1 more file in changeset.
OAK-6550: Make BlobTracker snapshot process smarter

- Enabled snapshot after removal

- And skipping regular snapshot if happened within the last interval

  1. … 1 more file in changeset.
OAK-5983: BlobGC should log the amount of space reclaimed after GC run is done

- Logging the total size that's cleaned up only for cases where the length is encoded in the ids.

  1. … 3 more files in changeset.
OAK-6801: Set minRecordLength to the recommended 4096 in OakFileDataStore

- Setting the default to 4096

OAK-6630: Remove older cached FileDataStore relying on JR caching

- Remove older code relying on CachingFDS from JR2

  1. … 2 more files in changeset.
OAK-6073 : Get rid of dependency from oak-blob-azure from oak-core
  1. … 785 more files in changeset.