hadoop

Clone Tools
  • last updated 19 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
YARN-9756: Create metric that sums total memory/vcores preempted per round. Contributed by Manikandan R (manirajv06).

(cherry picked from commit d562050cec83a2bc2ffb6d109ed3d64b394b870d)

YARN-9438. launchTime not written to state store for running applications

(cherry picked from commit 9568656cd21d9c02168e18ce35c6726077bbf3a1)

(cherry picked from commit 0c498de6e87c6bdc959afa31deb03d0907e0e1a1)

(cherry picked from commit f73842780e3e5daff2db4d30771a194f3bc47d6d)

YARN-9775. RMWebServices /scheduler-conf GET returns all hadoop configurations for ZKConfigurationStore. Contributed by Prabhu Joseph

(cherry picked from commit 8660e48ca15098e891c560beb3181c22ef3f80ff)

(cherry picked from commit e4249c320257586384035ea3fc286fe54cc699a1)

(cherry picked from commit 6baa0d1e4d78c15833257d98bdd2b38806faed9f)

HDFS-13977. NameNode can kill itself if it tries to send too many txns to a QJM simultaneously. Contributed by Erik Krogen.

Revert "HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow."

This reverts commit 8755de108d0e711cbfa784a20bd89e28fe79266b.

HDFS-14311. Multi-threading conflict at layoutVersion when loading block pool storage. Contributed by Yicong Cai.

HDFS-13101. Yet another fsimage corruption related to snapshot. Contributed by Shashikant Banerjee.

Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>

HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow.

HDFS-14725. Backport HDFS-12914 to branch-2 (Block report leases cause missing blocks until next report). Contributed by He Xiaoqiao.

Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>

YARN-9442. container working directory has group read permissions. Contributed by Jim Brennan.

(cherry picked from commit 2ac029b949f041da2ee04da441c5f9f85e1f2c64)

Conflicts:

hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c

(cherry picked from commit cec71691be76577718b22f936aea9e2b2cd100ea)

Conflicts:

hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c

(cherry picked from commit db88224e8f9d164ac811fcca9efe4a350cebecd1)

Fix JDK7 compatibility in branch-2.

HDFS-13898 Addendum: add a missing test helper method

HDFS-14423. Percent (%) and plus (+) characters no longer work in WebHDFS.

Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>

(cherry picked from commit da0006fe0473e353ee2d489156248a01aa982dfd)

Conflicts:

hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java

(cherry picked from commit d7ca016d63d89e5c8377a035f93485a7c77c3430)

Conflicts:

hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java

hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java

hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java

hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsUrl.java

Revert "HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao."

This reverts commit 567e1178d88ccfc258ce2ade4f8af66cc5a4daa7.

[SBN read] Addendum: Misc fix of issues, javadoc, whitespace unused backported fields etc

HADOOP-16315. ABFS: transform full UPN for named user in AclStatus

Contributed by Da Zhou

Change-Id: Ibc78322415fcbeff89c06c8586c53f5695550290

HDFS-14696. Backport HDFS-11273 to branch-2 (Move TransferFsImage#doGetUrl function to a Util class) (#1251) Contributed by Siyao Meng.

HADOOP-16479. ABFS FileStatus.getModificationTime returns localized time instead of UTC.

Contributed by Bilahari T H

Change-Id: I532055baaadfd7c324710e4b25f60cdf0378bdc0

HDFS-14034. Support getQuotaUsage API in WebHDFS. Contributed by Chao Sun.

[SBN read] Addendum: Misc fix to be Java 7 compatible

HDFS-14631. The DirectoryScanner doesn't fix the wrongly placed replica. Contributed by Jinglun.

HDFS-14313. Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du. Contributed by Lisheng Sun.

HADOOP-16494. Add SHA-512 checksum to release artifact to comply with the release distribution policy.

This closes #1243

(cherry picked from commit 34dd9ee36674be670013d4fc3d9b8f5b36886812)

HDFS-14707. Add JAVA_LIBRARY_PATH to HTTPFS startup options in branch-2.

Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>

YARN-9559. Create AbstractContainersLauncher for pluggable ContainersLauncher logic. (Contributed by Jonathan Hung)

(cherry picked from commit f51702d5398531835b24d812f6f95094a0e0493e)

(cherry picked from commit 8d357343c4bc9f18e25543583f8f217b8a2f621b)

(cherry picked from commit f943bff2546330bc9eb0914d1a0fa1ef4c709b72)

(cherry picked from commit 6109ecf16d6f0ebf1e314e163b5c41ad75758be1)

HDFS-14697. Backport of HDFS-14513. FSImage which is saving should be clean while NameNode shutdown. Contributed by He Xiaoqiao.

YARN-8045. Reduce log output from container status calls. Contributed by Craig Condit

(cherry picked from commit 144a55f0e3ba302327baf2e98d1e07b953dcbbfd)

HDFS-14462 Ensure WebHDFS client throws the correct exception during writes. Contributed by Simbarashe Dzinamarira.

(cherry picked from e7a0b8aa83c1fb933d409c514d2155e986e4e25b)

(cherry picked from ba4ca7d996cafa1b1786249cae2b207f0eaa1070)

(cherry picked from da548cf263c699932e5a6b60e00893f857713bcf)

(cherry picked from 055f824bcbeb82ee7eb49576ca7b9446dccea401)

HADOOP-15237. In KMS docs there should be one space between KMS_LOG and NOTE.

Contributed by Snigdhanjali Mishra.

Change-Id: I3abaa658c8786f8afa802ccbb629551313431b0e

YARN-9596: QueueMetrics has incorrect metrics when labelled partitions are involved. Contributed by Muhammad Samir Khan.

(cherry picked from commit c04ccf296dab6eaa53f008342e50f8729935a50d)