hadoop

Clone Tools
  • last updated 18 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
HDFS-14303. Addendum: check block directory logic not correct when there is only meta file, print no meaning warn log. Contributed by qiang Liu.

HDFS-14633. The StorageType quota and consume in QuotaFeature is not handled for rename. Contributed by Jinglun.

(cherry picked from commit 62d71fbac3789c7d484bc76ced9ec7fa6ff94de1)

YARN-9756: Create metric that sums total memory/vcores preempted per round. Contributed by Manikandan R (manirajv06).

(cherry picked from commit d562050cec83a2bc2ffb6d109ed3d64b394b870d)

Revert "HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow."

This reverts commit 22321f1d3dfae5bf03b8724f5d74d292d551580f.

HDFS-14311. Multi-threading conflict at layoutVersion when loading block pool storage. Contributed by Yicong Cai.

(cherry picked from commit bdab03cf711f75e88f80c21b3193fd5070cb6364)

HDFS-13101. Yet another fsimage corruption related to snapshot. Contributed by Shashikant Banerjee.

Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>

(cherry picked from commit ab7509efe7375949db3b819b53e9461624cd2b1e)

HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow.

(cherry picked from commit 8755de108d0e711cbfa784a20bd89e28fe79266b)

HDFS-14723. Add helper method FSNamesystem#setBlockManagerForTesting() in branch-2. Contributed by Wei-Chiu Chuang.

Reviewed-by: Chao Sun <sunchao@apache.org>

Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>

Reviewed-by: Erik Krogen <xkrogen@apache.org>

HDFS-14725. Backport HDFS-12914 to branch-2 (Block report leases cause missing blocks until next report). Contributed by He Xiaoqiao.

Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>

(cherry picked from commit b020c5ccfd38e295f04e4c26a556c9e523f175e6)

YARN-9442. container working directory has group read permissions. Contributed by Jim Brennan.

(cherry picked from commit 2ac029b949f041da2ee04da441c5f9f85e1f2c64)

Conflicts:

hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c

(cherry picked from commit cec71691be76577718b22f936aea9e2b2cd100ea)

Conflicts:

hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c

(cherry picked from commit db88224e8f9d164ac811fcca9efe4a350cebecd1)

Revert "HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao."

This reverts commit 94604e9f267f73df82c2b83bae6f72bdbb0c766b.

HDFS-14631. The DirectoryScanner doesn't fix the wrongly placed replica. Contributed by Jinglun.

(cherry picked from 97600cd9160eef9391ae23c79509e6e8853cab32)

HADOOP-16494. Add SHA-512 checksum to release artifact to comply with the release distribution policy.

This closes #1243

(cherry picked from commit 34dd9ee36674be670013d4fc3d9b8f5b36886812)

(cherry picked from commit 786df37429b984c21d0e855f969cf3ec04b1e8cb)

YARN-8045. Reduce log output from container status calls. Contributed by Craig Condit

(cherry picked from commit 144a55f0e3ba302327baf2e98d1e07b953dcbbfd)

YARN-9596: QueueMetrics has incorrect metrics when labelled partitions are involved. Contributed by Muhammad Samir Khan.

(cherry picked from commit c04ccf296dab6eaa53f008342e50f8729935a50d)

HDFS-14464. Remove unnecessary log message from DFSInputStream. Contributed by Chao Sun.

(cherry picked from commit 3941b3c708f0c0ebae6a57752e3b1a820a819aa4)

HADOOP-16435. RpcMetrics should not retained forever. Contributed by Zoltan Haindrich.

(cherry picked from commit cbfa3f3e988b8a3f142abadfe4f18201b6438ac9)

(cherry picked from commit 46c00aca2176400d1ba77f8af7756303b5826888)

HDFS-14429. Block remain in COMMITTED but not COMPLETE caused by Decommission. Contributed by Yicong Cai.

(cherry picked from commit 27ead06a13a66fe863ae5b4e8d3ae81fef8c6aaf)

HDFS-14547. Improve memory efficiency of quotas when storage type quotas are not set. Contributed by Jinglun.

(cherry-picked from cf7b9a5bd5636e8c4853692abc0d0f20251dba02)

HADOOP-16386. FindBugs warning in branch-2: GlobalStorageStatistics defines non-transient non-serializable instance field map.

Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>

(cherry picked from commit 62fdfaf0931aeee79ecbd99c870d8c131208c3f7)

HDFS-12748. NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY. Contributed by Weiwei Yang.

(cherry picked from commit 6c8cfd135eea15e3dfd76dd33af24a9a8c38fd8d)

(cherry picked from commit 3f112e7347fa7b75eab3a417c9d6520ca8b3053e)

Revert "Backport HDFS-3246,HDFS-14111 ByteBuffer pread interface to branch-2.9"

This reverts commit 52004ce08195fd2b2d0994a9b3faf560bc42fc0d.

HDFS-14483 Backport HDFS-3246,HDFS-14111 ByteBuffer pread interface to branch-2.9""

This is a revert of a revert, i.e. a reapplication of the HDFS-14483

patch. Original commit was missing the JIRA number.

Revert "Revert "Backport HDFS-3246,HDFS-14111 ByteBuffer pread interface to branch-2.9""

This reverts commit b26ad8a4c38d27894bd42d3fba7abff80bd895f4.

HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao.

(cherry picked from commit 567e1178d88ccfc258ce2ade4f8af66cc5a4daa7)

HDFS-13694. Making md5 computing being in parallel with image loading.

(cherry picked from commit 9c907294860a48f0d1676a31bda15795b7a6771a)

YARN-9655. AllocateResponse in FederationInterceptor lost applicationPriority. Contributed by hunshenshi.

(cherry picked from commit 78a6e4099459e2fa9e9552887107c1f1223dba36)

Backport HDFS-3246,HDFS-14111 ByteBuffer pread interface to branch-2.9

Signed-off-by: stack <stack@apache.org>

HDFS-14585 Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch2.9 Reapply but include JIRA # in the commit message this time.

Revert "Revert "Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch-2.9""

This reverts commit 1dd54163fcb40b60efc7ca153d443129e424df7d.

Revert "Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch-2.9"

This reverts commit bccb0fd846b7cc69e0d83140193eaa7fc0ac081d.

HDFS-14541. When evictableMmapped or evictable size is zero, do not throw NoSuchElementException.

(cherry picked from commit 0966407ad6fddb7adf60dba846d1f7c8a13b8ec1)