Clone Tools
  • last updated 16 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
HADOOP-16785. followup to abfs close() fix.

Adds one extra test to the ABFS close logic, to explicitly

verify that the close sequence of FilterOutputStream is

not going to fail.

This is just a due-diligence patch, but it helps ensure

that no regressions creep in in future.

Contributed by Steve Loughran.

Change-Id: Ifd33a8c322d32513411405b15f50a1aebcfa6e48

HADOOP-16785. followup to abfs close() fix.

Adds one extra test to the ABFS close logic, to explicitly

verify that the close sequence of FilterOutputStream is

not going to fail.

This is just a due-diligence patch, but it helps ensure

that no regressions creep in in future.

Contributed by Steve Loughran.

Change-Id: Ifd33a8c322d32513411405b15f50a1aebcfa6e48

HADOOP-16005. NativeAzureFileSystem does not support setXAttr.

Contributed by Clemens Wolff.

    • -0
    • +62
    ./hadoop/fs/azure/NativeAzureFileSystemBaseTest.java
    • -0
    • +104
    ./hadoop/fs/azurebfs/ITestAzureBlobFileSystemAttributes.java
  1. … 5 more files in changeset.
HADOOP-16785. Improve wasb and abfs resilience on double close() calls.

This hardens the wasb and abfs output streams' resilience to being invoked

in/after close().

wasb:

Explicity raise IOEs on operations invoked after close,

rather than implicitly raise NPEs.

This ensures that invocations which catch and swallow IOEs will perform as

expected.

abfs:

When rethrowing an IOException in the close() call, explicitly wrap it

with a new instance of the same subclass.

This is needed to handle failures in try-with-resources clauses, where

any exception in closed() is added as a suppressed exception to the one

thrown in the try {} clause

*and you cannot attach the same exception to itself*

Contributed by Steve Loughran.

Change-Id: Ic44b494ff5da332b47d6c198ceb67b965d34dd1b

    • -0
    • +20
    ./hadoop/fs/azure/ITestFileSystemOperationExceptionHandling.java
  1. … 3 more files in changeset.
HADOOP-16785. Improve wasb and abfs resilience on double close() calls.

This hardens the wasb and abfs output streams' resilience to being invoked

in/after close().

wasb:

Explicity raise IOEs on operations invoked after close,

rather than implicitly raise NPEs.

This ensures that invocations which catch and swallow IOEs will perform as

expected.

abfs:

When rethrowing an IOException in the close() call, explicitly wrap it

with a new instance of the same subclass.

This is needed to handle failures in try-with-resources clauses, where

any exception in closed() is added as a suppressed exception to the one

thrown in the try {} clause

*and you cannot attach the same exception to itself*

Contributed by Steve Loughran.

Change-Id: Ic44b494ff5da332b47d6c198ceb67b965d34dd1b

    • -0
    • +20
    ./hadoop/fs/azure/ITestFileSystemOperationExceptionHandling.java
  1. … 3 more files in changeset.
HADOOP-16778. ABFS: Backport HADOOP-16660 ABFS: Make RetryCount in ExponentialRetryPolicy Configurable to Branch-2. Contributed by Sneha Vijayarajan.

    • -1
    • +1
    ./hadoop/fs/azurebfs/TestAbfsConfigurationFieldsValidation.java
    • -0
    • +67
    ./hadoop/fs/azurebfs/oauth2/RetryTestTokenProvider.java
  1. … 3 more files in changeset.
HADOOP-16734. Backport HADOOP-16455- "ABFS: Implement FileSystem.access() method" to branch-2. Contributed by Bilahari T H.

    • -0
    • +320
    ./hadoop/fs/azurebfs/ITestAzureBlobFileSystemCheckAccess.java
  1. … 10 more files in changeset.
HADOOP-16660. ABFS: Make RetryCount in ExponentialRetryPolicy Configurable.

Contributed by Sneha Vijayarajan.

    • -1
    • +1
    ./hadoop/fs/azurebfs/TestAbfsConfigurationFieldsValidation.java
    • -0
    • +67
    ./hadoop/fs/azurebfs/oauth2/RetryTestTokenProvider.java
  1. … 3 more files in changeset.
HADOOP-16660. ABFS: Make RetryCount in ExponentialRetryPolicy Configurable.

Contributed by Sneha Vijayarajan.

    • -1
    • +1
    ./hadoop/fs/azurebfs/TestAbfsConfigurationFieldsValidation.java
    • -0
    • +67
    ./hadoop/fs/azurebfs/oauth2/RetryTestTokenProvider.java
  1. … 3 more files in changeset.
HADOOP-16455. ABFS: Implement FileSystem.access() method.

Contributed by Bilahari T H.

    • -0
    • +314
    ./hadoop/fs/azurebfs/ITestAzureBlobFileSystemCheckAccess.java
  1. … 10 more files in changeset.
HADOOP-16455. ABFS: Implement FileSystem.access() method.

Contributed by Bilahari T H.

    • -0
    • +314
    ./hadoop/fs/azurebfs/ITestAzureBlobFileSystemCheckAccess.java
  1. … 10 more files in changeset.
HADOOP-16687. ABFS: Fix testcase added for HADOOP-16138 for namespace enabled account. (#1701)

HADOOP-16612. Track Azure Blob File System client-perceived latency

Contributed by Jeetesh Mangwani.

This add the ability to track the end-to-end performance of ADLS Gen 2 REST APIs by measuring latency in the Hadoop ABFS driver.

The latency information is sent back to the ADLS Gen 2 REST API endpoints in the subsequent requests.

    • -0
    • +408
    ./hadoop/fs/azurebfs/services/TestAbfsPerfTracker.java
  1. … 16 more files in changeset.
HADOOP-16612. Track Azure Blob File System client-perceived latency

Contributed by Jeetesh Mangwani.

This add the ability to track the end-to-end performance of ADLS Gen 2 REST APIs by measuring latency in the Hadoop ABFS driver.

The latency information is sent back to the ADLS Gen 2 REST API endpoints in the subsequent requests.

    • -0
    • +408
    ./hadoop/fs/azurebfs/services/TestAbfsPerfTracker.java
  1. … 16 more files in changeset.
HADOOP-16640. WASB: Override getCanonicalServiceName() to return URI

(cherry picked from commit 9a8edb0aeddd7787b2654f6e2a8465c325e048a2)

    • -1
    • +29
    ./hadoop/fs/azure/ITestWasbUriAndConfiguration.java
  1. … 1 more file in changeset.
HADOOP-16640. WASB: Override getCanonicalServiceName() to return URI

    • -1
    • +29
    ./hadoop/fs/azure/ITestWasbUriAndConfiguration.java
  1. … 1 more file in changeset.
HADOOP-16640. WASB: Override getCanonicalServiceName() to return URI

(cherry picked from commit 9a8edb0aeddd7787b2654f6e2a8465c325e048a2)

    • -16
    • +45
    ./hadoop/fs/azure/ITestWasbUriAndConfiguration.java
  1. … 1 more file in changeset.
HADOOP-16630 : Backport of Hadoop-16548 : Disable Flush() over config

  1. … 6 more files in changeset.
HADOOP-16578 : Avoid FileSystem API calls when FileSystem already exists

  1. … 1 more file in changeset.
HADOOP-16578 : Avoid FileSystem API calls when FileSystem already exists

  1. … 1 more file in changeset.
HADOOP-16578 : Avoid FileSystem API calls when FileSystem already exists

  1. … 1 more file in changeset.
HADOOP-16548 : Disable Flush() over config

  1. … 6 more files in changeset.
HADOOP-16548 : Disable Flush() over config

  1. … 6 more files in changeset.
HADOOP-16138. hadoop fs mkdir / of nonexistent abfs container raises NPE (#1302). Contributed by Gabor Bota.

Change-Id: I2f637865c871e400b95fe7ddaa24bf99fa192023

    • -0
    • +65
    ./hadoop/fs/azurebfs/ITestAzureBlobFileSystemCLI.java
  1. … 1 more file in changeset.
HADOOP-16371: Option to disable GCM for SSL connections when running on Java 8.

Contributed by Sahil Takiar.

This moves the SSLSocketFactoryEx class from hadoop-azure into hadoop-common

as the DelegatingSSLSocketFactory and binds the S3A connector to it so that

it can avoid using those HTTPS algorithms which are underperformant on Java 8.

Change-Id: Ie9e6ac24deac1aa05e136e08899620efa7d22abd

    • -8
    • +8
    ./hadoop/fs/azurebfs/TestAbfsConfigurationFieldsValidation.java
  1. … 14 more files in changeset.
HADOOP-16315. ABFS: transform full UPN for named user in AclStatus

Contributed by Da Zhou

Change-Id: Ibc78322415fcbeff89c06c8586c53f5695550290

  1. … 2 more files in changeset.
HADOOP-16315. ABFS: transform full UPN for named user in AclStatus

Contributed by Da Zhou

Change-Id: Ibc78322415fcbeff89c06c8586c53f5695550290

  1. … 2 more files in changeset.
HADOOP-16315. ABFS: transform full UPN for named user in AclStatus

Contributed by Da Zhou

Change-Id: Ibc78322415fcbeff89c06c8586c53f5695550290

  1. … 2 more files in changeset.
HADOOP-16479. ABFS FileStatus.getModificationTime returns localized time instead of UTC.

Contributed by Bilahari T H

Change-Id: I532055baaadfd7c324710e4b25f60cdf0378bdc0

    • -0
    • +18
    ./hadoop/fs/azurebfs/ITestAzureBlobFileSystemFileStatus.java
  1. … 1 more file in changeset.
HADOOP-16479. ABFS FileStatus.getModificationTime returns localized time instead of UTC.

Contributed by Bilahari T H

Change-Id: I532055baaadfd7c324710e4b25f60cdf0378bdc0

    • -0
    • +18
    ./hadoop/fs/azurebfs/ITestAzureBlobFileSystemFileStatus.java
  1. … 1 more file in changeset.