hadoop

Clone Tools
  • last updated 19 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
HDFS-13101. Yet another fsimage corruption related to snapshot. Contributed by Shashikant Banerjee.

Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>

(cherry picked from commit ab7509efe7375949db3b819b53e9461624cd2b1e)

HDFS-14687. Standby Namenode never come out of safemode when EC files are being written. Contributed by Surendra Singh Lilhore.

Reviewed-by: Siyao Meng <smeng@cloudera.com>

Reviewed-by: Wei-Chiu CHuang <weichiu@apache.org>

Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>

(cherry picked from commit b8db5b9a9812023754ed1b3e5b428e161f0add50)

(cherry picked from commit 6b01effd011ab1f88bdbb9f2dbf9bbb0765984e6)

YARN-9442. container working directory has group read permissions. Contributed by Jim Brennan.

(cherry picked from commit 2ac029b949f041da2ee04da441c5f9f85e1f2c64)

Conflicts:

hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c

(cherry picked from commit cec71691be76577718b22f936aea9e2b2cd100ea)

Conflicts:

hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c

HDFS-14313. Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du. Contributed by Lisheng Sun.

YARN-9559. Create AbstractContainersLauncher for pluggable ContainersLauncher logic. (Contributed by Jonathan Hung)

(cherry picked from commit f51702d5398531835b24d812f6f95094a0e0493e)

(cherry picked from commit 8d357343c4bc9f18e25543583f8f217b8a2f621b)

(cherry picked from commit f943bff2546330bc9eb0914d1a0fa1ef4c709b72)

HDFS-14697. Backport of HDFS-14513. FSImage which is saving should be clean while NameNode shutdown. Contributed by He Xiaoqiao.

(cherry picked from ee1547a8071fbdebb72d409ddfcd46e6c63f82aa)

(cherry picked from 69e3ac5cbb2547416b27dbcfb458099c3a951101)

YARN-8045. Reduce log output from container status calls. Contributed by Craig Condit

(cherry picked from commit 144a55f0e3ba302327baf2e98d1e07b953dcbbfd)

HDFS-14462 Ensure WebHDFS client throws the correct exception during writes. Contributed by Simbarashe Dzinamarira.

(cherry picked from e7a0b8aa83c1fb933d409c514d2155e986e4e25b)

(cherry picked from ba4ca7d996cafa1b1786249cae2b207f0eaa1070)

(cherry picked from da548cf263c699932e5a6b60e00893f857713bcf)

HDFS-14034. Support getQuotaUsage API in WebHDFS. Contributed by Chao Sun.

Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>

(cherry picked from 3ae775d74029b6ae82263739f598ceb25c597dcd)

(cherry picked from d38b617baaf10cb35f3a8eba904b930a142a071e)

(cherry picked from 90447baeb0c82d2602282c7369de2b603e16a93f)

YARN-9596: QueueMetrics has incorrect metrics when labelled partitions are involved. Contributed by Muhammad Samir Khan.

HDFS-14370. Add exponential backoff to the edit log tailer to avoid spinning on empty edit tail requests. Contributed by Erik Krogen.

(cherry picked from 827dbb11e24be294b40088a8aa46086ba8ca4ba8)

(cherry picked from 016aa139406d1a151fbec3fb8912eb08e0f638d7)

(cherry picked from f6ce2f4a50898b16556089fe18a6717a15570d85)

YARN-9668. UGI conf doesn't read user overridden configurations on RM and NM startup. (Contributed by Jonanthan Hung)

Addendum to fix 3.0 build

YARN-9668. UGI conf doesn't read user overridden configurations on RM and NM startup. (Contributed by Jonanthan Hung)

HDFS-12748. NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY. Contributed by Weiwei Yang.

HDFS-12703. Exceptions are fatal to decommissioning monitor. Contributed by He Xiaoqiao.

(cherry picked from commit 3d396786cf6eaab49c1c9b8b2a4652c2e440b9e3)

(cherry picked from commit 950aa74d5f284942e0ffc2f40bac0286c578d854)

(cherry picked from commit fe40fbbd4d4858c55d081dffed9d68a4f0b8e923)

HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao.

Revert "HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao."

This reverts commit 974dd2b4b6103374969fd7cfeb2cee50d4112c6a.

HDFS-14547. Improve memory efficiency of quotas when storage type quotas are not set. Contributed by Jinglun.

(cherry-picked from 4632708148ed7dec75810feddb22ad98e4670483)

(cherry-picked from 491dc7cc9d6836f34d2647b12dcefa86cc97a927)

(cherry-picked from a59553b5e6bc6fa59ed31306708d899342013191)

HDFS-13694. Making md5 computing being in parallel with image loading.

(cherry picked from commit 9c907294860a48f0d1676a31bda15795b7a6771a)

YARN-9655. AllocateResponse in FederationInterceptor lost applicationPriority. Contributed by hunshenshi.

HDFS-14541. When evictableMmapped or evictable size is zero, do not throw NoSuchElementException.

(cherry picked from commit 0966407ad6fddb7adf60dba846d1f7c8a13b8ec1)

HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao.

Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>

Co-authored-by: He Xiaoqiao <hexiaoqiao@apache.org>

(cherry picked from commit 026cc2647d4e3c1cffdd154995a5f1f07956c2a0)

HDFS-14303. check block directory logic not correct when there is only meta file, print no meaning warn log. Contributed by qiang Liu.

(cherry picked from commit a5d76a4965b686ec5532add511c64d5f87c88865)

(cherry picked from commit f33bcbb687e4dd4ab10d5dad89b1b8eae5788efb)

HDFS-14537. Journaled Edits Cache is not cleared when formatting the JN. Contributed by Ranith Sardar.

HDFS-14101. Random failure of testListCorruptFilesCorruptedBlock. Contributed by Zsolt Venczel, Nikhil Navadia.

Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>

Co-authored-by: Nikhil Navadia <nikhil.navadiya@cloudera.com>

(cherry picked from commit 7c00756aff6bf8d5e1d05748ba32b679e397b53f)

(cherry picked from commit 4ea3b04bf36af56612c6b47d9ff8aa49cbbbcc9e)

YARN-9584. Should put initializeProcessTrees method call before get pid. Contributed by Wanqiang Ji.

(cherry picked from commit 67414a1a80039e70e0afc1de171831a6e981f37a)

HDFS-13730. BlockReaderRemote.sendReadResult throws NPE. Contributed by Yuanbo Liu.

(cherry picked from commit 62ad9885ea8c75c134de43a3a925c76b253658e1)

(cherry picked from commit 45543ffd6af723bb83bd44a99f12165363c70abb)

(cherry picked from commit fc8983a5b8c94746d3e58cc203974aa09c1b640b)

HDFS-11950. Disable libhdfs zerocopy test on Mac. Contributed by Akira Ajisaka.

Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>

(cherry picked from commit a95e87d827381a7f87c01f0befc2475f9478e63b)

(cherry picked from commit f68099aa1316f34591c757bcf4198a3a3c7f691a)

(cherry picked from commit 3d1d8623a11b72c7939d6500537be5fd6565d4d6)

HDFS-14465. When the Block expected replications is larger than the number of DataNodes, entering maintenance will never exit. Contributed by Yicong Cai.

(cherry picked from commit 7db922c6474cb29e0e9bad320fcda4769366340a)

(cherry picked from commit 0eb42e1e05b780dab718bb3adfdd55f527dd5545)

(cherry picked from commit b3e0430c3bee30a52ec08786982522bfcf0bb911)

HADOOP-16369. Fix zstandard shortname misspelled as zts. Contributed by Jonathan Eagles.

(cherry picked from commit 54f9f75a443d7d167a7aa7d04a87e3f5af049887)