Clone Tools
  • last updated 20 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
HADOOP-14630 Contract Tests to verify create, mkdirs and rename under a file is forbidden

Contributed by Steve Loughran.

Not all stores do complete validation here; in particular the S3A

Connector does not: checking up the entire directory tree to see if a path matches

is a file significantly slows things down.

This check does take place in S3A mkdirs(), which walks backwards up the list of

parent paths until it finds a directory (success) or a file (failure).

In practice production applications invariably create destination directories

before writing 1+ file into them -restricting check purely to the mkdirs()

call deliver significant speed up while implicitly including the checks.

Change-Id: I2c9df748e92b5655232e7d888d896f1868806eb0

    • -12
    • +23
    ./metrics/TestNativeAzureFileSystemMetricsSystem.java
  1. … 10 more files in changeset.
HADOOP-16005. NativeAzureFileSystem does not support setXAttr.

Contributed by Clemens Wolff.

    • -0
    • +62
    ./NativeAzureFileSystemBaseTest.java
  1. … 6 more files in changeset.
HADOOP-16785. Improve wasb and abfs resilience on double close() calls.

This hardens the wasb and abfs output streams' resilience to being invoked

in/after close().

wasb:

Explicity raise IOEs on operations invoked after close,

rather than implicitly raise NPEs.

This ensures that invocations which catch and swallow IOEs will perform as

expected.

abfs:

When rethrowing an IOException in the close() call, explicitly wrap it

with a new instance of the same subclass.

This is needed to handle failures in try-with-resources clauses, where

any exception in closed() is added as a suppressed exception to the one

thrown in the try {} clause

*and you cannot attach the same exception to itself*

Contributed by Steve Loughran.

Change-Id: Ic44b494ff5da332b47d6c198ceb67b965d34dd1b

    • -0
    • +20
    ./ITestFileSystemOperationExceptionHandling.java
  1. … 4 more files in changeset.
HADOOP-16785. Improve wasb and abfs resilience on double close() calls.

This hardens the wasb and abfs output streams' resilience to being invoked

in/after close().

wasb:

Explicity raise IOEs on operations invoked after close,

rather than implicitly raise NPEs.

This ensures that invocations which catch and swallow IOEs will perform as

expected.

abfs:

When rethrowing an IOException in the close() call, explicitly wrap it

with a new instance of the same subclass.

This is needed to handle failures in try-with-resources clauses, where

any exception in closed() is added as a suppressed exception to the one

thrown in the try {} clause

*and you cannot attach the same exception to itself*

Contributed by Steve Loughran.

Change-Id: Ic44b494ff5da332b47d6c198ceb67b965d34dd1b

    • -0
    • +20
    ./ITestFileSystemOperationExceptionHandling.java
  1. … 4 more files in changeset.
HADOOP-16640. WASB: Override getCanonicalServiceName() to return URI

(cherry picked from commit 9a8edb0aeddd7787b2654f6e2a8465c325e048a2)

    • -1
    • +29
    ./ITestWasbUriAndConfiguration.java
  1. … 1 more file in changeset.
HADOOP-16640. WASB: Override getCanonicalServiceName() to return URI

    • -1
    • +29
    ./ITestWasbUriAndConfiguration.java
  1. … 1 more file in changeset.
HADOOP-16640. WASB: Override getCanonicalServiceName() to return URI

(cherry picked from commit 9a8edb0aeddd7787b2654f6e2a8465c325e048a2)

    • -16
    • +45
    ./ITestWasbUriAndConfiguration.java
  1. … 1 more file in changeset.
HADOOP-16105. WASB in secure mode does not set connectingUsingSAS.

Contributed by Steve Loughran.

(cherry picked from commit 9cb2f470b759bbe7609a00e8f8f72779e2daae80)

    • -1
    • +60
    ./ITestWasbUriAndConfiguration.java
  1. … 3 more files in changeset.
HADOOP-16105. WASB in secure mode does not set connectingUsingSAS.

Contributed by Steve Loughran.

(cherry picked from commit 9cb2f470b759bbe7609a00e8f8f72779e2daae80)

    • -1
    • +60
    ./ITestWasbUriAndConfiguration.java
  1. … 3 more files in changeset.
HADOOP-16105. WASB in secure mode does not set connectingUsingSAS.

Contributed by Steve Loughran.

    • -1
    • +60
    ./ITestWasbUriAndConfiguration.java
  1. … 3 more files in changeset.
HADOOP-16104. Wasb tests to downgrade to skip when test a/c is namespace enabled. Contributed by Masatake Iwasaki.

(cherry picked from commit aa3ad3660506382884324c4b8997973f5a68e29a)

    • -0
    • +1
    ./contract/NativeAzureFileSystemContract.java
  1. … 2 more files in changeset.
HADOOP-16104. Wasb tests to downgrade to skip when test a/c is namespace enabled. Contributed by Masatake Iwasaki.

    • -0
    • +1
    ./contract/NativeAzureFileSystemContract.java
  1. … 2 more files in changeset.
HADOOP-16074. WASB: Update container not found error code.

Contributed by Da Zhou.

(cherry picked from commit ba9efe06fadb9631763f3e623d8748bbe59ff748)

  1. … 1 more file in changeset.
HADOOP-16074. WASB: Update container not found error code.

Contributed by Da Zhou.

  1. … 1 more file in changeset.
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki.

    • -43
    • +26
    ./ITestWasbRemoteCallHelper.java
    • -47
    • +10
    ./metrics/ITestAzureFileSystemInstrumentation.java
  1. … 333 more files in changeset.
HADOOP-15957. WASB: Add asterisk wildcard support for PageBlobDirSet.

Contributed by Da Zhou.

(cherry picked from commit 7ccb640a66bb5bb9f657a2db85bcc8ae0ded4892)

    • -0
    • +170
    ./TestKeyPageBlobDirectories.java
  1. … 1 more file in changeset.
HADOOP-15957. WASB: Add asterisk wildcard support for PageBlobDirSet.

Contributed by Da Zhou.

    • -0
    • +170
    ./TestKeyPageBlobDirectories.java
  1. … 1 more file in changeset.
HADOOP-15773. Fixing checkstyle and other issues raised by Yetus.

    • -1
    • +1
    ./ITestNativeFileSystemStatistics.java
  1. … 9 more files in changeset.
HADOOP-15773. Fixing checkstyle and other issues raised by Yetus.

    • -1
    • +1
    ./ITestNativeFileSystemStatistics.java
  1. … 9 more files in changeset.
HADOOP-15744. AbstractContractAppendTest fails against HDFS on HADOOP-15407 branch. Contributed by Steve Loughran.

    • -23
    • +0
    ./contract/ITestAzureNativeContractAppend.java
  1. … 2 more files in changeset.
HADOOP-15744. AbstractContractAppendTest fails against HDFS on HADOOP-15407 branch. Contributed by Steve Loughran.

    • -23
    • +0
    ./contract/ITestAzureNativeContractAppend.java
  1. … 2 more files in changeset.
HADOOP-15677. WASB: Add support for StreamCapabilities. Contributed by Thomas Marquardt.

(cherry picked from commit 1eef0acfe5725d08d3a4eb76c5138076ff1857f1)

  1. … 1 more file in changeset.
HADOOP-15677. WASB: Add support for StreamCapabilities. Contributed by Thomas Marquardt.

  1. … 1 more file in changeset.
HADOOP-15547/ WASB: improve listStatus performance. Contributed by Thomas Marquardt.

    • -0
    • +196
    ./ITestListPerformance.java
    • -2
    • +2
    ./ITestNativeAzureFileSystemConcurrencyLive.java
  1. … 7 more files in changeset.
HADOOP-15664. ABFS: Reduce test run time via parallelization and grouping. Contributed by Da Zhou.

    • -0
    • +99
    ./ITestNativeFileSystemStatistics.java
    • -67
    • +13
    ./NativeAzureFileSystemBaseTest.java
  1. … 7 more files in changeset.
HADOOP-15664. ABFS: Reduce test run time via parallelization and grouping. Contributed by Da Zhou.

    • -0
    • +99
    ./ITestNativeFileSystemStatistics.java
    • -66
    • +12
    ./NativeAzureFileSystemBaseTest.java
  1. … 7 more files in changeset.
HADOOP-15663. ABFS: Simplify configuration. Contributed by Da Zhou.

    • -1
    • +2
    ./ITestFileSystemOperationExceptionMessage.java
    • -26
    • +0
    ./ITestWasbUriAndConfiguration.java
    • -2
    • +2
    ./metrics/TestRollingWindowAverage.java
  1. … 19 more files in changeset.
HADOOP-15663. ABFS: Simplify configuration. Contributed by Da Zhou.

    • -1
    • +2
    ./ITestFileSystemOperationExceptionMessage.java
    • -26
    • +0
    ./ITestWasbUriAndConfiguration.java
    • -2
    • +2
    ./metrics/TestRollingWindowAverage.java
  1. … 19 more files in changeset.
HADOOP-15680. ITestNativeAzureFileSystemConcurrencyLive times out. Contributed by Andras Bokor.

(cherry picked from commit e8d138ca7c1b695688515d816ac693437c87df62)

    • -1
    • +1
    ./ITestNativeAzureFileSystemConcurrencyLive.java
HADOOP-15680. ITestNativeAzureFileSystemConcurrencyLive times out. Contributed by Andras Bokor.

    • -1
    • +1
    ./ITestNativeAzureFileSystemConcurrencyLive.java