Checkout Tools
  • last updated 2 hours ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
OAK-8780 - Remove the synchronized block on the FSBackend

Fix the failing tests

OAK-8780 - Remove the synchronized block on the FSBackend

Slightly modified patch from Ankita Agarwal

Moved the copy of file to a temp file outside the synchronized block

fix svn:eol-style
  1. … 3 more files in changeset.
OAK-8280: Add support to request ignoring domain override for signed URIs.

This change introduces BlobUploadOptions so options can be requested for

both upload and download URIs, and adds the option for both URI requests

to ignore the domain override if one is set.

  1. … 18 more files in changeset.
OAK-8104: Update to Azure SDK v8.6.0 to fix signed URI C-D headers.

The purpose of this commit is to fix Content-Disposition header support for signed download URIs. The filename* portion of the header specification was disabled in OAK-8013 due to a bug in the Azure SDK pre-8.5.0. This commit adds filename* portion support back into Oak, which addresses the following issues:

- OAK-8104 - Fix the Content-Disposition header correctly (required upgrading the Azure SDK version to 8.6.0)

- OAK-8607 - Undo the workarounds implemented in OAK-8013.

  1. … 3 more files in changeset.
OAK-8807: Update FileCache to use non-cache segment blocking method

- Building the cache without the CacheLoader and using the get method with a valueLoader

OAK-8781: Add debug logs before and after cache purge to get the uploadCache size info

- Added size in the logs when logging purge start and finish

OAK-8786: Remove temporary file on upload immediately after a sync upload

- Delete temporary file after sync upload

OAK-8576: Extend instrumentation for data stores - fix svn:eol-style
OAK-8576: Extend instrumentation for data stores.

This commit adds instrumentation for other methods in data stores, including

support for metrics for SharedDataStore implementations and for direct

binary access functions, as well as improving existing instrumentation for

uploads and downloads. It also adds error counts to aid in identifying

and diagnosing failure conditions in production environments.

  1. … 5 more files in changeset.
OAK-8676: deprecate FileIOUtils.FileLineDifferenceIterator
  1. … 4 more files in changeset.
OAK-8666: deprecate FileIOUtils.BurnOnCloseFileIterator
  1. … 4 more files in changeset.
fix svn:eol-style
OAK-8612: Make the Azure Data Store compatible with Guava 15 and 26
OAK-8599: Fix direct binary upload init to return null if disabled

  1. … 3 more files in changeset.
OAK-8552: Add data store config to skip exists check on download URI generation

This change adds a new optional configuration parameter for AzureDataStore and S3DataStore. The new parameter, named "presignedHttpDownloadURIVerifyExists", has a default value of true which means the createHttpDownloadURI() method will ensure the requested blob exists in blob storage before creating the signed download URI. If this parameter is set to false, the exists check will be skipped, resulting in higher signed download URI generation performance but potentially returning a URI to a nonexistent blob in some cases (see OAK-7998).

  1. … 7 more files in changeset.
OAK-8578: Introduce API to check whether blob inlined in Id

OAK-8552: Minimize network calls required when creating a direct download URI

Add a new method #isInlined to Blob class to check if the external blob is id inlined

  1. … 8 more files in changeset.
OAK-8578:Reverting commit r1865962
  1. … 13 more files in changeset.
OAK-8552: Minimize network calls required when creating a direct download URI

Add a new method #isInlined to Blob class to check if the external blob is id inlined

  1. … 13 more files in changeset.
OAK-8575: Remove unneeded existence check in cloud data store getRecord() calls

  1. … 1 more file in changeset.
OAK-8298: Add tracking of blob ids added via direct upload

OAK-8520: Return existing DataRecord when completeUpload called for existing binary

  1. … 2 more files in changeset.
OAK-7998: Verify existence of binary before generating direct download URI

This change adds code and tests to ensure that when generating a direct

download URI we make sure the binary actually exists first. If caching

is being used it is possible that a client requests a direct download

URI for a blob that was already added the standard (i.e. non-direct)

way but is still in the cache and hasn't actually been uploaded to

cloud storage yet - and therefore cannot be downloaded by direct

download URI.

This change also adds some documentation changes to reflect this

possible situation.

  1. … 3 more files in changeset.
OAK-8463: Fix merge in consistency check to allow paths suffixed to blob ids

- Fix merge to ignore paths if present

OAK-8447: Handle all available references in the DataStore when doing consistency check

- Fixed check for SharedDataStore

OAK-8447: Handle all available references in the DataStore when doing consistency check

close (another) FileReader when done

OAK-8447: Handle all available references in the DataStore when doing consistency check

skip handling specific to SharedDataStore when the BlobStore is not a SharedDataStore

OAK-8446: Enable datastore consistency check after completion of garbage collection

close FileReader when done

OAK-8447: Handle all available references in the DataStore when doing consistency check

- Now if available all reference files will be accounted for when doing consistency check

OAK-8446: Enable datastore consistency check after completion of garbage collection

- Enables datastore consistency in a low cost way after GC

  1. … 3 more files in changeset.