Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
HADOOP-16005. NativeAzureFileSystem does not support setXAttr.

Contributed by Clemens Wolff.

  1. … 5 more files in changeset.
HADOOP-16785. Improve wasb and abfs resilience on double close() calls.

This hardens the wasb and abfs output streams' resilience to being invoked

in/after close().

wasb:

Explicity raise IOEs on operations invoked after close,

rather than implicitly raise NPEs.

This ensures that invocations which catch and swallow IOEs will perform as

expected.

abfs:

When rethrowing an IOException in the close() call, explicitly wrap it

with a new instance of the same subclass.

This is needed to handle failures in try-with-resources clauses, where

any exception in closed() is added as a suppressed exception to the one

thrown in the try {} clause

*and you cannot attach the same exception to itself*

Contributed by Steve Loughran.

Change-Id: Ic44b494ff5da332b47d6c198ceb67b965d34dd1b

  1. … 5 more files in changeset.
HADOOP-16785. Improve wasb and abfs resilience on double close() calls.

This hardens the wasb and abfs output streams' resilience to being invoked

in/after close().

wasb:

Explicity raise IOEs on operations invoked after close,

rather than implicitly raise NPEs.

This ensures that invocations which catch and swallow IOEs will perform as

expected.

abfs:

When rethrowing an IOException in the close() call, explicitly wrap it

with a new instance of the same subclass.

This is needed to handle failures in try-with-resources clauses, where

any exception in closed() is added as a suppressed exception to the one

thrown in the try {} clause

*and you cannot attach the same exception to itself*

Contributed by Steve Loughran.

Change-Id: Ic44b494ff5da332b47d6c198ceb67b965d34dd1b

  1. … 5 more files in changeset.
HADOOP-16699. Add verbose TRACE logging to ABFS.

Contributed by Sneha Vijayarajan,

Change-Id: Ic616a10406e6e9f11616c9cc05d8630ebbedaf65

HADOOP-16778. ABFS: Backport HADOOP-16660 ABFS: Make RetryCount in ExponentialRetryPolicy Configurable to Branch-2. Contributed by Sneha Vijayarajan.

    • -7
    • +4
    ./services/ExponentialRetryPolicy.java
  1. … 3 more files in changeset.
HADOOP-16734. Backport HADOOP-16455- "ABFS: Implement FileSystem.access() method" to branch-2. Contributed by Bilahari T H.

    • -1
    • +3
    ./constants/FileSystemConfigurations.java
  1. … 3 more files in changeset.
HADOOP-16660. ABFS: Make RetryCount in ExponentialRetryPolicy Configurable.

Contributed by Sneha Vijayarajan.

    • -7
    • +4
    ./services/ExponentialRetryPolicy.java
  1. … 3 more files in changeset.
HADOOP-16660. ABFS: Make RetryCount in ExponentialRetryPolicy Configurable.

Contributed by Sneha Vijayarajan.

    • -7
    • +4
    ./services/ExponentialRetryPolicy.java
  1. … 3 more files in changeset.
HADOOP-16455. ABFS: Implement FileSystem.access() method.

Contributed by Bilahari T H.

    • -1
    • +2
    ./constants/FileSystemConfigurations.java
  1. … 3 more files in changeset.
HADOOP-16455. ABFS: Implement FileSystem.access() method.

Contributed by Bilahari T H.

    • -1
    • +2
    ./constants/FileSystemConfigurations.java
  1. … 3 more files in changeset.
HADOOP-16612. Track Azure Blob File System client-perceived latency

Contributed by Jeetesh Mangwani.

This add the ability to track the end-to-end performance of ADLS Gen 2 REST APIs by measuring latency in the Hadoop ABFS driver.

The latency information is sent back to the ADLS Gen 2 REST API endpoints in the subsequent requests.

    • -330
    • +514
    ./AzureBlobFileSystemStore.java
    • -0
    • +1
    ./constants/FileSystemConfigurations.java
    • -0
    • +1
    ./constants/HttpHeaderConfigurations.java
    • -0
    • +34
    ./contracts/services/AbfsPerfLoggable.java
    • -0
    • +133
    ./services/AbfsPerfInfo.java
    • -0
    • +319
    ./services/AbfsPerfTracker.java
  1. … 5 more files in changeset.
HADOOP-16612. Track Azure Blob File System client-perceived latency

Contributed by Jeetesh Mangwani.

This add the ability to track the end-to-end performance of ADLS Gen 2 REST APIs by measuring latency in the Hadoop ABFS driver.

The latency information is sent back to the ADLS Gen 2 REST API endpoints in the subsequent requests.

    • -330
    • +514
    ./AzureBlobFileSystemStore.java
    • -0
    • +1
    ./constants/FileSystemConfigurations.java
    • -0
    • +1
    ./constants/HttpHeaderConfigurations.java
    • -0
    • +34
    ./contracts/services/AbfsPerfLoggable.java
    • -0
    • +133
    ./services/AbfsPerfInfo.java
    • -0
    • +319
    ./services/AbfsPerfTracker.java
  1. … 5 more files in changeset.
HADOOP-16652. Backport of HADOOP-16587: Make ABFS AAD endpoints configurable

    • -0
    • +45
    ./constants/AuthConfigurations.java
    • -3
    • +9
    ./oauth2/RefreshTokenBasedTokenProvider.java
HADOOP-16630 : Backport of Hadoop-16548 : Disable Flush() over config

    • -0
    • +1
    ./constants/FileSystemConfigurations.java
  1. … 2 more files in changeset.
HADOOP-16587. Make ABFS AAD endpoints configurable.

Contributed by Bilahari T H.

This also addresses HADOOP-16498: AzureADAuthenticator cannot authenticate

in China.

Change-Id: I2441dd48b50b59b912b0242f7f5a4418cf94a87c

    • -0
    • +45
    ./constants/AuthConfigurations.java
    • -3
    • +9
    ./oauth2/RefreshTokenBasedTokenProvider.java
HADOOP-16587. Make ABFS AAD endpoints configurable.

Contributed by Bilahari T H.

This also addresses HADOOP-16498: AzureADAuthenticator cannot authenticate

in China.

Change-Id: I2441dd48b50b59b912b0242f7f5a4418cf94a87c

    • -0
    • +45
    ./constants/AuthConfigurations.java
    • -3
    • +9
    ./oauth2/RefreshTokenBasedTokenProvider.java
HADOOP-16578 : Avoid FileSystem API calls when FileSystem already exists

  1. … 1 more file in changeset.
HADOOP-16578 : Avoid FileSystem API calls when FileSystem already exists

  1. … 1 more file in changeset.
HADOOP-16578 : Avoid FileSystem API calls when FileSystem already exists

  1. … 1 more file in changeset.
HADOOP-16548 : Disable Flush() over config

    • -0
    • +1
    ./constants/FileSystemConfigurations.java
  1. … 2 more files in changeset.
HADOOP-16548 : Disable Flush() over config

    • -0
    • +1
    ./constants/FileSystemConfigurations.java
  1. … 2 more files in changeset.
HADOOP-15691 Add PathCapabilities to FileSystem and FileContext.

Contributed by Steve Loughran.

This complements the StreamCapabilities Interface by allowing applications to probe for a specific path on a specific instance of a FileSystem client

to offer a specific capability.

This is intended to allow applications to determine

* Whether a method is implemented before calling it and dealing with UnsupportedOperationException.

* Whether a specific feature is believed to be available in the remote store.

As well as a common set of capabilities defined in CommonPathCapabilities,

file systems are free to add their own capabilities, prefixed with

fs. + schema + .

The plan is to identify and document more capabilities -and for file systems which add new features, for a declaration of the availability of the feature to always be available.

Note

* The remote store is not expected to be checked for the feature;

It is more a check of client API and the client's configuration/knowledge

of the state of the remote system.

* Permissions are not checked.

Change-Id: I80bfebe94f4a8bdad8f3ac055495735b824968f5

  1. … 39 more files in changeset.
HADOOP-16371: Option to disable GCM for SSL connections when running on Java 8.

Contributed by Sahil Takiar.

This moves the SSLSocketFactoryEx class from hadoop-azure into hadoop-common

as the DelegatingSSLSocketFactory and binds the S3A connector to it so that

it can avoid using those HTTPS algorithms which are underperformant on Java 8.

Change-Id: Ie9e6ac24deac1aa05e136e08899620efa7d22abd

    • -3
    • +3
    ./constants/FileSystemConfigurations.java
  1. … 11 more files in changeset.
HADOOP-16315. ABFS: transform full UPN for named user in AclStatus

Contributed by Da Zhou

Change-Id: Ibc78322415fcbeff89c06c8586c53f5695550290

  1. … 1 more file in changeset.
HADOOP-16315. ABFS: transform full UPN for named user in AclStatus

Contributed by Da Zhou

Change-Id: Ibc78322415fcbeff89c06c8586c53f5695550290

  1. … 1 more file in changeset.
HADOOP-16315. ABFS: transform full UPN for named user in AclStatus

Contributed by Da Zhou

Change-Id: Ibc78322415fcbeff89c06c8586c53f5695550290

  1. … 1 more file in changeset.
HADOOP-16479. ABFS FileStatus.getModificationTime returns localized time instead of UTC.

Contributed by Bilahari T H

Change-Id: I532055baaadfd7c324710e4b25f60cdf0378bdc0

  1. … 1 more file in changeset.
HADOOP-16479. ABFS FileStatus.getModificationTime returns localized time instead of UTC.

Contributed by Bilahari T H

Change-Id: I532055baaadfd7c324710e4b25f60cdf0378bdc0

  1. … 1 more file in changeset.
HADOOP-16479. ABFS FileStatus.getModificationTime returns localized time instead of UTC.

Contributed by Bilahari T H

Change-Id: I532055baaadfd7c324710e4b25f60cdf0378bdc0

  1. … 1 more file in changeset.
HADOOP-16404. ABFS default blocksize change(256MB from 512MB)

Contributed by: Arun Singh

    • -1
    • +1
    ./constants/FileSystemConfigurations.java
  1. … 1 more file in changeset.