hadoop

Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
HADOOP-16645. S3A Delegation Token extension point to use StoreContext.

Contributed by Steve Loughran.

This is part of the ongoing refactoring of the S3A codebase, with the

delegation token support (HADOOP-14556) no longer given a direct reference

to the owning S3AFileSystem. Instead it gets a StoreContext and a new

interface, DelegationOperations, to access those operations offered by S3AFS

which are specifically needed by the DT bindings.

The sole operation needed is listAWSPolicyRules(), which is used to allow

S3A FS and the S3Guard metastore to return the AWS policy rules needed to

access their specific services/buckets/tables, allowing the AssumedRole

delegation token to be locked down.

As further restructuring takes place, that interface's implementation

can be moved to wherever the new home for those operations ends up.

Although it changes the API of an extension point, that feature (S3

Delegation Tokens) has not shipped; backwards compatibility is not a

problem except for anyone who has implemented DT support against trunk.

To those developers: sorry.

Change-Id: I770f58b49ff7634a34875ba37b7d51c94d7c21da

HDFS-15066. HttpFS: Implement setErasureCodingPolicy , unsetErasureCodingPolicy , getErasureCodingPolicy. Contributed by hemanthboyina.

HDFS-14788. Use dynamic regex filter to ignore copy of source files in Distcp.

Contributed by Mukund Thakur.

Change-Id: I781387ddce95ee300c12a160dc9a0f7d602403c3

YARN-9956. Improved connection error message for YARN ApiServerClient. Contributed by Prabhu Joseph

YARN-10026. Pull out common code pieces from ATS v1.5 and v2. Contributed by Adam Antal

YARN-10035. Add ability to filter the Cluster Applications API request by name. Contributed by Adam Antal

  1. … 6 more files in changeset.
HADOOP-16772. Extract version numbers to head of pom.xml (addendum) (#1774). Contributed by Tamas Penzes.

Follow up task of HADOOP-16729, extract even more version numbers for branch-3.2.

Change-Id: I0546fc8ec8f68baa4bcfa9db0df4d8293b078913

HDFS-15090. RBF: MountPoint Listing Should Return Flag Values Of Destination. Contributed by Ayush Saxena.

MAPREDUCE-7255. Fix typo in MapReduce documentaion example (#1793)

(cherry picked from commit b343e1533b1f4ebbbf10b063c35d79d53a741321)

MAPREDUCE-7255. Fix typo in MapReduce documentaion example (#1793)

(cherry picked from commit b343e1533b1f4ebbbf10b063c35d79d53a741321)

MAPREDUCE-7255. Fix typo in MapReduce documentaion example (#1793)

HDFS-15089. RBF: SmallFix for RBFMetrics in doc (#1786)

HADOOP-16778. ABFS: Backport HADOOP-16660 ABFS: Make RetryCount in ExponentialRetryPolicy Configurable to Branch-2. Contributed by Sneha Vijayarajan.

HADOOP-16778. ABFS: Backport HADOOP-16660 ABFS: Make RetryCount in ExponentialRetryPolicy Configurable to Branch-2. Contributed by Sneha Vijayarajan.

HADOOP-16734. Backport HADOOP-16455- "ABFS: Implement FileSystem.access() method" to branch-2. Contributed by Bilahari T H.

HADOOP-16734. Backport HADOOP-16455- "ABFS: Implement FileSystem.access() method" to branch-2. Contributed by Bilahari T H.

HDFS-15091. Cache Admin and Quota Commands Should Check SuperUser Before Taking Lock. Contributed by Ayush Saxena.

HADOOP-16789. In TestZKFailoverController, restore changes from HADOOP-11149 that were dropped by HDFS-6440. Contributed by Jim Brennan.

HDFS-15068. DataNode could meet deadlock if invoke refreshVolumes when register. Contributed by Aiphago.

Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>

(cherry picked from commit 037ec8cfb1406ea3a8225a1b6306c2e78440353b)

HDFS-15068. DataNode could meet deadlock if invoke refreshVolumes when register. Contributed by Aiphago.

Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>

(cherry picked from commit 037ec8cfb1406ea3a8225a1b6306c2e78440353b)

HDFS-15068. DataNode could meet deadlock if invoke refreshVolumes when register. Contributed by Aiphago.

Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>

HADOOP-16751. DurationInfo text parsing/formatting should be moved out of hotpath.

Contributed by Rajesh Balamohan

Change-Id: Icc3dcfa81aa69164f2c088f9b533d231138cbb8b

HADOOP-16784. Update the year to 2020. Contributed by Ayush Saxena.

HADOOP-16784. Update the year to 2020. Contributed by Ayush Saxena.

HADOOP-16784. Update the year to 2020. Contributed by Ayush Saxena.

HADOOP-16784. Update the year to 2020. Contributed by Ayush Saxena.

HADOOP-16784. Update the year to 2020. Contributed by Ayush Saxena.

HADOOP-16784. Update the year to 2020. Contributed by Ayush Saxena.

HADOOP-16777. Add Tez to LimitedPrivate of ClusterStorageCapacityExceededException

Contributed by Wang Yan.

Change-Id: I92dfe7079ba8ebe89d70255bb845309be0603a8e

HADOOP-16775. DistCp reuses the same temp file within the task for different files.

Contributed by Amir Shenavandeh.

This avoids overwrite consistency issues with S3 and other stores -though

given S3's copy operation is O(data), you are still best of using -direct

when distcp-ing to it.

Change-Id: I8dc9f048ad0cc57ff01543b849da1ce4eaadf8c3