hadoop

Clone Tools
  • last updated 21 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
SUBMARINE-62. PS_LAUNCH_CMD CLI description is wrong in RunJobCli. Contributed by Adam Antal

(cherry picked from commit be784de2d4c8d7ae2724cf348925a0fbdbe0c503)

SUBMARINE-89. Add submarine-src profile to generate source package. Contributed by Sunil Govindan.

(cherry picked from commit cd17cc286860cfe143d24b001e83f9d1b141e48e)

SUBMARINE-88. rat.sh regex pattern not working issue while using lower version of grep in ubuntu16.04. Contributed by Zhankun Tang.

(cherry picked from commit 96d818a2f7a0cd62f252234ba1a27c8c70bfed1f)

SUBMARINE-82. Fix english grammar mistakes in documentation. Contributed by Szilard Nemeth.

(cherry picked from commit 799115967d6e1a4074d0186b06b4eb97251a19df)

SUBMARINE-87. Add an apache rat check script for submarine. Contributed by Zhankun Tang.

(cherry picked from commit 08363db09326d421bd38b72b86a24cb75c339043)

    • -0
    • +24
    /hadoop-submarine/dev-support/checks/rat.sh
Preparing for submarine-0.2.1 development

Preparing for submarine-0.3.0 development

SUBMARINE-86. Bump TonY version to pick up a docker related fix & Update documentation. Contributed by Keqiu Hu.

(cherry picked from commit 2b303e9d5f2108fee621dd0c6e5582a861c401e4)

SUBMARINE-83. Refine the documents of submarine targeting 0.2.0 release. Contributed by Zhankun Tang.

(cherry picked from commit 03aa70fe19e06a0b3d637acc7b5db4c955f0a693)

HDFS-14353. Erasure Coding: metrics xmitsInProgress become to negative. Contributed by maobaolong.

HDDS-1487. Bootstrap React framework for Recon UI (#831)

HDDS-1487. Bootstrap React framework for Recon UI.

Contributed by Vivek Ratnavel Subramanian.

    • -14
    • +76
    /hadoop-ozone/ozone-recon/pom.xml
  1. … 15 more files in changeset.
HDDS-1451 : SCMBlockManager findPipeline and createPipeline are not lock protected. (#799)

* HDDS-1451 : SCMBlockManager findPipeline and createPipeline are not lock protected.

* HDDS-1451 : Address review comments.

YARN-9546. Add configuration option for YARN Native services AM classpath. Contributed by Gergely Pollak.

HDDS-1499. OzoneManager Cache. (#798)

HADOOP-16085. S3Guard: use object version or etags to protect against inconsistent read after replace/overwrite.

Contributed by Ben Roling.

S3Guard will now track the etag of uploaded files and, if an S3

bucket is versioned, the object version.

You can then control how to react to a mismatch between the data

in the DynamoDB table and that in the store: warn, fail, or, when

using versions, return the original value.

This adds two new columns to the table: etag and version.

This is transparent to older S3A clients -but when such clients

add/update data to the S3Guard table, they will not add these values.

As a result, the etag/version checks will not work with files uploaded by older clients.

For a consistent experience, upgrade all clients to use the latest hadoop version.

  1. … 41 more files in changeset.
SUBMARINE-58. Submarine client needs to generate fat jar. Contributed by Zac Zhou.

    • -0
    • +183
    /hadoop-submarine/hadoop-submarine-all/pom.xml
  1. … 27 more files in changeset.
HDDS-1422. Exception during DataNode shutdown. Contributed by Arpit A… (#725)

* HDDS-1422. Exception during DataNode shutdown. Contributed by Arpit Agarwal.

Change-Id: I6db6bdd19839a45e5341ed7e745cd38f68af8378

* Suppress spurious findbugs warning.

* Remove log file that got committed in error

YARN-9505. Add container allocation latency for Opportunistic Scheduler. Contributed by Abhishek Modi.

HDFS-14323. Distcp fails in Hadoop 3.x when 2.x source webhdfs url has special characters in hdfs file path. Contributed by Srinivasu Majeti.

Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>

HADOOP-16248. MutableQuantiles leak memory under heavy load.

Contributed by Alexis Daboville,

HDDS-1527. HDDS Datanode start fails due to datanode.id file read error. Contributed by Siddharth Wagle.

This closes #822.

YARN-9554. Fixed TimelineEntity DAO serialization handling. Contributed by Prabhu Joseph

HDDS-1522. Provide intellij runConfiguration for Ozone components. Contributed by Elek, Marton.

    • -0
    • +18
    /hadoop-ozone/dev-support/intellij/log4j.properties
    • -0
    • +66
    /hadoop-ozone/dev-support/intellij/ozone-site.xml
HADOOP-16294: Enable access to input options by DistCp subclasses.

Adding a protected-scope getter for the DistCpOptions, so that a subclass does

not need to save its own copy of the inputOptions supplied to its constructor,

if it wishes to override the createInputFileListing method with logic similar

to the original implementation, i.e. calling CopyListing#buildListing with a path and input options.

Author: Andrew Olson

HADOOP-16307. Intern User Name and Group Name in FileStatus.

Author: David Mollitor

HDDS-1531. Disable the sync flag by default during chunk writes in Datanode (#820). Contributed by Shashikant Banerjee.

SUBMARINE-56. Update documentation to describe single-node PyTorch integration. Contributed by Szilard Nemeth.

HADOOP-16247. NPE in FsUrlConnection. Contributed by Karthik Palanisamy.

HDFS-14390. Provide kerberos support for AliasMap service used by Provided storage. Contributed by Ashvin Agrawal

YARN-9552. FairScheduler: NODE_UPDATE can cause NoSuchElementException. Contributed by Peter Bacsko.