hadoop

Clone Tools
  • last updated 17 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
HDDS-1451 : SCMBlockManager findPipeline and createPipeline are not lock protected. (#799)

* HDDS-1451 : SCMBlockManager findPipeline and createPipeline are not lock protected.

* HDDS-1451 : Address review comments.

YARN-9546. Add configuration option for YARN Native services AM classpath. Contributed by Gergely Pollak.

fix review comments

Fixed checkstyle issues.

HDDS-1499. OzoneManager Cache. (#798)

HADOOP-16085. S3Guard: use object version or etags to protect against inconsistent read after replace/overwrite.

Contributed by Ben Roling.

S3Guard will now track the etag of uploaded files and, if an S3

bucket is versioned, the object version.

You can then control how to react to a mismatch between the data

in the DynamoDB table and that in the store: warn, fail, or, when

using versions, return the original value.

This adds two new columns to the table: etag and version.

This is transparent to older S3A clients -but when such clients

add/update data to the S3Guard table, they will not add these values.

As a result, the etag/version checks will not work with files uploaded by older clients.

For a consistent experience, upgrade all clients to use the latest hadoop version.

  1. … 41 more files in changeset.
SUBMARINE-58. Submarine client needs to generate fat jar. Contributed by Zac Zhou.

    • -0
    • +183
    /hadoop-submarine/hadoop-submarine-all/pom.xml
  1. … 27 more files in changeset.
HDDS-1535. Space tracking for Open Containers : Handle Node Startup. Contributed by Supratim Deka

HDDS-1422. Exception during DataNode shutdown. Contributed by Arpit A… (#725)

* HDDS-1422. Exception during DataNode shutdown. Contributed by Arpit Agarwal.

Change-Id: I6db6bdd19839a45e5341ed7e745cd38f68af8378

* Suppress spurious findbugs warning.

* Remove log file that got committed in error

HDFS-14500. NameNode StartupProgress should not allow new steps in an already-completed phase. Contributed by Erik Krogen.

(cherry-picked from 55e0c134f002f74cb4a0360b6682a1b6796d1598)

(cherry-picked from c26cf22d6b3949a4798d8133669b451660a0f050)

(cherry-picked from 9295e95174ad8627874b81a9d43183384a81fb16)

HDFS-14500. NameNode StartupProgress should not allow new steps in an already-completed phase. Contributed by Erik Krogen.

(cherry-picked from 55e0c134f002f74cb4a0360b6682a1b6796d1598)

HDFS-14500. NameNode StartupProgress should not allow new steps in an already-completed phase. Contributed by Erik Krogen.

(cherry-picked from 55e0c134f002f74cb4a0360b6682a1b6796d1598)

(cherry-picked from c26cf22d6b3949a4798d8133669b451660a0f050)

HDFS-14500. NameNode StartupProgress should not allow new steps in an already-completed phase. Contributed by Erik Krogen.

HDFS-14500. NameNode StartupProgress should not allow new steps in an already-completed phase. Contributed by Erik Krogen.

(cherry-picked from 55e0c134f002f74cb4a0360b6682a1b6796d1598)

(cherry-picked from c26cf22d6b3949a4798d8133669b451660a0f050)

(cherry-picked from 9295e95174ad8627874b81a9d43183384a81fb16)

YARN-9505. Add container allocation latency for Opportunistic Scheduler. Contributed by Abhishek Modi.

HDFS-14323. Distcp fails in Hadoop 3.x when 2.x source webhdfs url has special characters in hdfs file path. Contributed by Srinivasu Majeti.

Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>

(cherry picked from commit 3e5e5b028ad7e199d08e524fe7cddeee5db51a6d)

(cherry picked from commit 85f3664bb84f393f8006d3ab781d8ce5ebb1ab62)

HDFS-14323. Distcp fails in Hadoop 3.x when 2.x source webhdfs url has special characters in hdfs file path. Contributed by Srinivasu Majeti.

Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>

(cherry picked from commit 3e5e5b028ad7e199d08e524fe7cddeee5db51a6d)

HDFS-14323. Distcp fails in Hadoop 3.x when 2.x source webhdfs url has special characters in hdfs file path. Contributed by Srinivasu Majeti.

Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>

HADOOP-16248. MutableQuantiles leak memory under heavy load.

Contributed by Alexis Daboville,

(cherry picked from commit 4cb3da6ac703b10de59a8ca371fe957a4a338601)

HADOOP-16248. MutableQuantiles leak memory under heavy load.

Contributed by Alexis Daboville,

(cherry picked from commit 4cb3da6ac703b10de59a8ca371fe957a4a338601)

HADOOP-16248. MutableQuantiles leak memory under heavy load.

Contributed by Alexis Daboville,

(cherry picked from commit 4cb3da6ac703b10de59a8ca371fe957a4a338601)

HADOOP-16248. MutableQuantiles leak memory under heavy load.

Contributed by Alexis Daboville,

HADOOP-16248. MutableQuantiles leak memory under heavy load.

Contributed by Alexis Daboville,

(cherry picked from commit 4cb3da6ac703b10de59a8ca371fe957a4a338601)

HADOOP-16248. MutableQuantiles leak memory under heavy load.

Contributed by Alexis Daboville,

(cherry picked from commit 4cb3da6ac703b10de59a8ca371fe957a4a338601)

HDDS-1550. MiniOzoneChaosCluster is not shutting down all the threads during shutdown. Contributed by Mukul Kumar Singh.

HDDS-1527. HDDS Datanode start fails due to datanode.id file read error. Contributed by Siddharth Wagle.

This closes #822.

YARN-9554. Fixed TimelineEntity DAO serialization handling. Contributed by Prabhu Joseph

HDFS-14447. RBF: Router should support RefreshUserMappingsProtocol. Contributed by Shen Yinjie.

HDFS-14447. RBF: Router should support RefreshUserMappingsProtocol. Contributed by Shen Yinjie.

HDFS-14447. RBF: Router should support RefreshUserMappingsProtocol. Contributed by Shen Yinjie.