Clone Tools
  • last updated 11 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
HADOOP-16612. Track Azure Blob File System client-perceived latency

Contributed by Jeetesh Mangwani.

This add the ability to track the end-to-end performance of ADLS Gen 2 REST APIs by measuring latency in the Hadoop ABFS driver.

The latency information is sent back to the ADLS Gen 2 REST API endpoints in the subsequent requests.

    • -330
    • +514
    ./AzureBlobFileSystemStore.java
    • -0
    • +1
    ./constants/FileSystemConfigurations.java
    • -0
    • +1
    ./constants/HttpHeaderConfigurations.java
    • -0
    • +34
    ./contracts/services/AbfsPerfLoggable.java
    • -0
    • +133
    ./services/AbfsPerfInfo.java
    • -0
    • +319
    ./services/AbfsPerfTracker.java
  1. … 5 more files in changeset.
HADOOP-16652. Backport of HADOOP-16587: Make ABFS AAD endpoints configurable

    • -0
    • +45
    ./constants/AuthConfigurations.java
    • -3
    • +9
    ./oauth2/RefreshTokenBasedTokenProvider.java
HADOOP-16630 : Backport of Hadoop-16548 : Disable Flush() over config

    • -0
    • +1
    ./constants/FileSystemConfigurations.java
  1. … 2 more files in changeset.
HADOOP-16587. Make ABFS AAD endpoints configurable.

Contributed by Bilahari T H.

This also addresses HADOOP-16498: AzureADAuthenticator cannot authenticate

in China.

Change-Id: I2441dd48b50b59b912b0242f7f5a4418cf94a87c

    • -0
    • +45
    ./constants/AuthConfigurations.java
    • -3
    • +9
    ./oauth2/RefreshTokenBasedTokenProvider.java
HADOOP-16587. Make ABFS AAD endpoints configurable.

Contributed by Bilahari T H.

This also addresses HADOOP-16498: AzureADAuthenticator cannot authenticate

in China.

Change-Id: I2441dd48b50b59b912b0242f7f5a4418cf94a87c

    • -0
    • +45
    ./constants/AuthConfigurations.java
    • -3
    • +9
    ./oauth2/RefreshTokenBasedTokenProvider.java
HADOOP-16578 : Avoid FileSystem API calls when FileSystem already exists

  1. … 1 more file in changeset.
HADOOP-16578 : Avoid FileSystem API calls when FileSystem already exists

  1. … 1 more file in changeset.
HADOOP-16578 : Avoid FileSystem API calls when FileSystem already exists

  1. … 1 more file in changeset.
HADOOP-16548 : Disable Flush() over config

    • -0
    • +1
    ./constants/FileSystemConfigurations.java
  1. … 2 more files in changeset.
HADOOP-16548 : Disable Flush() over config

    • -0
    • +1
    ./constants/FileSystemConfigurations.java
  1. … 2 more files in changeset.
HADOOP-15691 Add PathCapabilities to FileSystem and FileContext.

Contributed by Steve Loughran.

This complements the StreamCapabilities Interface by allowing applications to probe for a specific path on a specific instance of a FileSystem client

to offer a specific capability.

This is intended to allow applications to determine

* Whether a method is implemented before calling it and dealing with UnsupportedOperationException.

* Whether a specific feature is believed to be available in the remote store.

As well as a common set of capabilities defined in CommonPathCapabilities,

file systems are free to add their own capabilities, prefixed with

fs. + schema + .

The plan is to identify and document more capabilities -and for file systems which add new features, for a declaration of the availability of the feature to always be available.

Note

* The remote store is not expected to be checked for the feature;

It is more a check of client API and the client's configuration/knowledge

of the state of the remote system.

* Permissions are not checked.

Change-Id: I80bfebe94f4a8bdad8f3ac055495735b824968f5

  1. … 39 more files in changeset.
HADOOP-16371: Option to disable GCM for SSL connections when running on Java 8.

Contributed by Sahil Takiar.

This moves the SSLSocketFactoryEx class from hadoop-azure into hadoop-common

as the DelegatingSSLSocketFactory and binds the S3A connector to it so that

it can avoid using those HTTPS algorithms which are underperformant on Java 8.

Change-Id: Ie9e6ac24deac1aa05e136e08899620efa7d22abd

    • -3
    • +3
    ./constants/FileSystemConfigurations.java
  1. … 11 more files in changeset.
HADOOP-16315. ABFS: transform full UPN for named user in AclStatus

Contributed by Da Zhou

Change-Id: Ibc78322415fcbeff89c06c8586c53f5695550290

  1. … 1 more file in changeset.
HADOOP-16315. ABFS: transform full UPN for named user in AclStatus

Contributed by Da Zhou

Change-Id: Ibc78322415fcbeff89c06c8586c53f5695550290

  1. … 1 more file in changeset.
HADOOP-16315. ABFS: transform full UPN for named user in AclStatus

Contributed by Da Zhou

Change-Id: Ibc78322415fcbeff89c06c8586c53f5695550290

  1. … 1 more file in changeset.
HADOOP-16479. ABFS FileStatus.getModificationTime returns localized time instead of UTC.

Contributed by Bilahari T H

Change-Id: I532055baaadfd7c324710e4b25f60cdf0378bdc0

  1. … 1 more file in changeset.
HADOOP-16479. ABFS FileStatus.getModificationTime returns localized time instead of UTC.

Contributed by Bilahari T H

Change-Id: I532055baaadfd7c324710e4b25f60cdf0378bdc0

  1. … 1 more file in changeset.
HADOOP-16479. ABFS FileStatus.getModificationTime returns localized time instead of UTC.

Contributed by Bilahari T H

Change-Id: I532055baaadfd7c324710e4b25f60cdf0378bdc0

  1. … 1 more file in changeset.
HADOOP-16404. ABFS default blocksize change(256MB from 512MB)

Contributed by: Arun Singh

    • -1
    • +1
    ./constants/FileSystemConfigurations.java
  1. … 1 more file in changeset.
HADOOP-16404. ABFS default blocksize change(256MB from 512MB)

Contributed by: Arun Singh

    • -1
    • +1
    ./constants/FileSystemConfigurations.java
  1. … 1 more file in changeset.
HADOOP-16404. ABFS default blocksize change(256MB from 512MB)

Contributed by: Arun Singh

    • -1
    • +1
    ./constants/FileSystemConfigurations.java
  1. … 1 more file in changeset.
HADOOP-16340. ABFS driver continues to retry on IOException responses from REST operations.

Contributed by Robert Levas.

This makes the HttpException constructor protected rather than public, so it is possible

to implement custom subclasses of this exception -exceptions which will not be retried.

Change-Id: Ie8aaa23a707233c2db35948784908b6778ff3a8f

HADOOP-16340. ABFS driver continues to retry on IOException responses from REST operations.

Contributed by Robert Levas.

This makes the HttpException constructor protected rather than public, so it is possible

to implement custom subclasses of this exception -exceptions which will not be retried.

Change-Id: Ie8aaa23a707233c2db35948784908b6778ff3a8f

HADOOP-16376. ABFS: Override access() to no-op.

Contributed by Da Zhou.

Change-Id: Ia0024bba32250189a87eb6247808b2473c331ed0

HADOOP-16376. ABFS: Override access() to no-op.

Contributed by Da Zhou.

Change-Id: Ia0024bba32250189a87eb6247808b2473c331ed0

HADOOP-16376. ABFS: Override access() to no-op.

Contributed by Da Zhou.

Change-Id: Ia0024bba32250189a87eb6247808b2473c331ed0

Revert "HADOOP-16050: s3a SSL connections should use OpenSSL"

This reverts commit b067f8acaa79b1230336900a5c62ba465b2adb28.

Change-Id: I584b050a56c0e6f70b11fa3f7db00d5ac46e7dd8

    • -3
    • +3
    ./constants/FileSystemConfigurations.java
  1. … 10 more files in changeset.
HADOOP-16269. ABFS: add listFileStatus with StartFrom.

Author: Da Zhou

  1. … 3 more files in changeset.
HADOOP-16269. ABFS: add listFileStatus with StartFrom.

Author: Da Zhou

  1. … 3 more files in changeset.
HADOOP-16269. ABFS: add listFileStatus with StartFrom.

Author: Da Zhou

  1. … 3 more files in changeset.