hadoop

Clone Tools
  • last updated 11 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
HADOOP-16759. Filesystem openFile() builder to take a FileStatus param (#1761). Contributed by Steve Loughran

* Enhanced builder + FS spec

* s3a FS to use this to skip HEAD on open

* and to use version/etag when opening the file

works with S3AFileStatus FS and S3ALocatedFileStatus

  1. … 3 more files in changeset.
YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R.

HDFS-15092. TestRedudantBlocks#testProcessOverReplicatedAndRedudantBlock sometimes fails. Contributed by Fei Hui.

HDFS-15126. TestDatanodeRegistration#testForcedRegistration fails intermittently. Contributed by Ahmed Hussein.

HADOOP-16346. Stabilize S3A OpenSSL support.

Introduces `openssl` as an option for `fs.s3a.ssl.channel.mode`.

The new option is documented and marked as experimental.

For details on how to use this, consult the peformance document

in the s3a documentation.

This patch is the successor to HADOOP-16050 "S3A SSL connections

should use OpenSSL" -which was reverted because of

incompatibilities between the wildfly OpenSSL client and the AWS

HTTPS servers (HADOOP-16347). With the Wildfly release moved up

to 1.0.7.Final (HADOOP-16405) everything should now work.

Related issues:

* HADOOP-15669. ABFS: Improve HTTPS Performance

* HADOOP-16050: S3A SSL connections should use OpenSSL

* HADOOP-16371: Option to disable GCM for SSL connections when running on Java 8

* HADOOP-16405: Upgrade Wildfly Openssl version to 1.0.7.Final

Contributed by Sahil Takiar

Change-Id: I80a4bc5051519f186b7383b2c1cea140be42444e

HDFS-15125. Pull back HDFS-11353, HDFS-13993, HDFS-13945, and HDFS-14324 to branch-2.10. Contributed by Jim Brennan.

HADOOP-16808. Use forkCount and reuseForks parameters instead of forkMode in the config of maven surefire plugin. Contributed by Xieming Li.

HADOOP-16808. Use forkCount and reuseForks parameters instead of forkMode in the config of maven surefire plugin. Contributed by Xieming Li.

(cherry picked from commit f6d20daf404fab28b596171172afa4558facb504)

HADOOP-16808. Use forkCount and reuseForks parameters instead of forkMode in the config of maven surefire plugin. Contributed by Xieming Li.

(cherry picked from commit f6d20daf404fab28b596171172afa4558facb504)

HADOOP-16808. Use forkCount and reuseForks parameters instead of forkMode in the config of maven surefire plugin. Contributed by Xieming Li.

(cherry picked from commit f6d20daf404fab28b596171172afa4558facb504)

HADOOP-16808. Use forkCount and reuseForks parameters instead of forkMode in the config of maven surefire plugin. Contributed by Xieming Li.

(cherry picked from commit f6d20daf404fab28b596171172afa4558facb504)

HADOOP-16808. Use forkCount and reuseForks parameters instead of forkMode in the config of maven surefire plugin. Contributed by Xieming Li.

(cherry picked from commit f6d20daf404fab28b596171172afa4558facb504)

HADOOP-16753. Refactor HAAdmin. Contributed by Xieming Li.

HADOOP-16785. followup to abfs close() fix.

Adds one extra test to the ABFS close logic, to explicitly

verify that the close sequence of FilterOutputStream is

not going to fail.

This is just a due-diligence patch, but it helps ensure

that no regressions creep in in future.

Contributed by Steve Loughran.

Change-Id: Ifd33a8c322d32513411405b15f50a1aebcfa6e48

HADOOP-16785. followup to abfs close() fix.

Adds one extra test to the ABFS close logic, to explicitly

verify that the close sequence of FilterOutputStream is

not going to fail.

This is just a due-diligence patch, but it helps ensure

that no regressions creep in in future.

Contributed by Steve Loughran.

Change-Id: Ifd33a8c322d32513411405b15f50a1aebcfa6e48

YARN-7913. Improve error handling when application recovery fails with exception. Contributed by Wilfred Spiegelenburg

YARN-9525. IFile format is not working against s3a remote folder. Contributed by Adam Antal

YARN-9462. TestResourceTrackerService.testNodeRemovalGracefully fails sporadically. Contributed by Prabhu Joseph

YARN-10081. Exception message from ClientRMProxy#getRMAddress is misleading. Contributed by Ravuri Sushma Sree

YARN-8148. Update decimal values for queue capacities shown on queue status CLI. Contributed by Prabhu Joseph

YARN-8148. Update decimal values for queue capacities shown on queue status CLI. Contributed by Prabhu Joseph

YARN-8148. Update decimal values for queue capacities shown on queue status CLI. Contributed by Prabhu Joseph

HDFS-15112. RBF: Do not return FileNotFoundException when a subcluster is unavailable.

HADOOP-16621. [pb-upgrade] Remove Protobuf classes from signatures of Public APIs. Contributed by Vinayakumar B. (#1803)

YARN-9970. Refactor TestUserGroupMappingPlacementRule#verifyQueueMapping. Contributed by Manikandan R

YARN-10070. Fix NPE if no queue mapping defined for proxy user when using application tag based placement.

Contributed by Kinga Marton. Reviewed by Adam Antal.

MAPREDUCE-7247. Modify HistoryServerRest.html content,change The job attempt id‘s datatype from string to int (#1684)

(cherry picked from commit b8518e6dabc1cf73fa7c43b3f8904d2e2beb2bf3)

MAPREDUCE-7247. Modify HistoryServerRest.html content,change The job attempt id‘s datatype from string to int (#1684)

MAPREDUCE-7247. Modify HistoryServerRest.html content,change The job attempt id‘s datatype from string to int (#1684)

(cherry picked from commit b8518e6dabc1cf73fa7c43b3f8904d2e2beb2bf3)

MAPREDUCE-7247. Modify HistoryServerRest.html content,change The job attempt id‘s datatype from string to int (#1684)

(cherry picked from commit b8518e6dabc1cf73fa7c43b3f8904d2e2beb2bf3)