hadoop

Clone Tools
  • last updated 20 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
HADOOP-16255. Add ChecksumFs.rename(path, path, boolean). Contributed by Jungtaek Lim.

(cherry picked from commit 14ff6171a5879f63c1188b07cff8cbe135b9f802)

Revert "MAPREDUCE-7277. IndexCache totalMemoryUsed differs from cache contents. Contributed by Jon Eagles (jeagles)."

This reverts commit 2e749f509c420a931e1b9a819ac8136df78f43f3.

MAPREDUCE-7277. IndexCache totalMemoryUsed differs from cache contents. Contributed by Jon Eagles (jeagles).

(cherry picked from commit 568f185bf8c27887d3f4c2559b301275d4bfa486)

MAPREDUCE-7272. TaskAttemptListenerImpl excessive log messages. Contributed by Ahmed Hussein (ahussein)

(cherry picked from commit 48a8d7691ead49911a8ff56ba678a093342cabdd)

HDFS-14758. Make lease hard limit configurable and reduce the default. Contributed by hemanthboyina.

(cherry picked from commit 9b8a78d97bfd825ce840c6033371c7f10e49a5b8)

(cherry picked from commit f1840669cfc92c033521bf95989479523e2a649d)

(cherry picked from commit bf09bfa2db7c0790f72d6d7d357e0b80c9eff1f2)

(cherry picked from commit 28619362288fbaf9c3757ee3ef770f575f7a6916)

(cherry picked from commit e2ea7089ee4376fcddec6c9c5b52e11dfdaf5d4d)

Conflicts:

hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsClientConfigKeys.java

hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml

HADOOP-15787. TestIPC.testRTEDuringConnectionSetup fails with jdk8u242. Contributed by Zsolt Venczel.

(cherry picked from commit a463cf75a0ab1f0dbb8cfa16c39a4e698bc1a625)

(cherry picked from commit 0b19c8e0860bf558d995951ab9ac4cd8a1674759)

Conflicts:

hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestIPC.java

HADOOP-16808. Use forkCount and reuseForks parameters instead of forkMode in the config of maven surefire plugin. Contributed by Xieming Li.

(cherry picked from commit f6d20daf404fab28b596171172afa4558facb504)

HADOOP-16784. Update the year to 2020. Contributed by Ayush Saxena.

HDFS-15012. NN fails to parse Edit logs after applying HDFS-13101. Contributed by Shashikant Banerjee.

(cherry picked from commit fdd96e46d1f89f0ecdb9b1836dc7fca9fbb954fd)

HADOOP-16754. Fix docker failed to build yetus/hadoop

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>

HADOOP-16610. Upgrade to yetus 0.11.1 and use emoji vote on github pre commit (#1527). Contributed by Duo Zhang.

Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>

(cherry picked from commit 3cecb2a46974d446a91435ac49c70742e3cbb6bd)

(cherry picked from commit 440b7abc049a7ea60cc2f72f33a792bdae523e03)

YARN-9914. Use separate configs for free disk space checking for full and not-full disks. Contributed by Jim Brennan

HDFS-13901. INode access time is ignored because of race between open and rename. Contributed by Jinglun.

(cherry picked from commit a805d80ebe3b1683049119d3d19e8b085141e3cf)

(cherry picked from commit 2b351536e22aeb990016cf387ec27df2d1275a5e)

(cherry picked from commit 6a769b994a54785bf94b3ca6edfbd7eadd072955)

HDFS-12749. DN may not send block report to NN after NN restart. Contributed by Xiaoqiao He.

(cherry picked from commit c4e27ef7735acd6f91b73d2ecb0227f8dd75a2e4)

HADOOP-16662. Remove unnecessary InnerNode check in NetworkTopology#add(). Contributed by Lisheng Sun.

HADOOP-16609. Add Jenkinsfile for all active branches (#1557)

* Backport HADOOP-16035. Jenkinsfile for Hadoop

* Use Yetus 0.10.0

(cherry picked from commit ae8ca70fc0f83384800a249625285b657c3b3057)

(cherry picked from commit eda4f5ed7ca5f4d21856de1be019282b8e17bc4e)

YARN-9855. Fix ApplicationReportProto submitTime id in branch-2.8/branch-2.7

HADOOP-16582. LocalFileSystem's mkdirs() does not work as expected under viewfs. Contributed by Kihwal Lee

(cherry picked from commit d4205dce176287e863f567b333e0d408bf51ae6d)

HADOOP-16555. Update commons-compress to 1.19. (#1425) Contributed by YiSheng Lien.

(cherry picked from commit 1843c4688a3040c48f6834d166494cbf68b5e4fd)

Conflicts:

LICENSE-binary

(cherry picked from commit d39ebbfebc9da982cadfbe92f3fb6677e744622d)

(cherry picked from commit 7d7b415409901bbb0095af4e9d56aade158c1438)

(cherry picked from commit 699297e0e6178bb802d86fe0f4b269fa35e717c7)

Revert "HDFS-14633. The StorageType quota and consume in QuotaFeature is not handled for rename. Contributed by Jinglun."

This reverts commit 12ec7c301ff19b004fd94e3f0af7238f8050bccc.

HDFS-14633. The StorageType quota and consume in QuotaFeature is not handled for rename. Contributed by Jinglun.

(cherry picked from commit 62d71fbac3789c7d484bc76ced9ec7fa6ff94de1)

HDFS-14633. The StorageType quota and consume in QuotaFeature is not handled for rename. Contributed by Jinglun.

(cherry picked from commit 62d71fbac3789c7d484bc76ced9ec7fa6ff94de1)

YARN-9756: Create metric that sums total memory/vcores preempted per round. Contributed by Manikandan R (manirajv06).

(cherry picked from commit d562050cec83a2bc2ffb6d109ed3d64b394b870d)

HDFS-14725. Backport HDFS-12914 to branch-2 (Block report leases cause missing blocks until next report). Contributed by He Xiaoqiao.

Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>

Revert "HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow."

This reverts commit dbc7e4d74b3494e049bef6baf16efc88dca30fc3.

HDFS-14311. Multi-threading conflict at layoutVersion when loading block pool storage. Contributed by Yicong Cai.

(cherry picked from commit bdab03cf711f75e88f80c21b3193fd5070cb6364)

(cherry picked from commit 0e7afe054ad67052359ab62757d6241d955b94d5)

HDFS-13101. Yet another fsimage corruption related to snapshot. Contributed by Shashikant Banerjee.

Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>

(cherry picked from commit ab7509efe7375949db3b819b53e9461624cd2b1e)

(cherry picked from commit 8a924a81fa4140ab99468eda1452915ea60b6ce0)

Conflicts:

hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithSnapshot.java

HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow.

(cherry picked from commit 8755de108d0e711cbfa784a20bd89e28fe79266b)

(cherry picked from commit 22321f1d3dfae5bf03b8724f5d74d292d551580f)

HDFS-14723. Add helper method FSNamesystem#setBlockManagerForTesting() in branch-2. Contributed by Wei-Chiu Chuang.

Reviewed-by: Chao Sun <sunchao@apache.org>

Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>

Reviewed-by: Erik Krogen <xkrogen@apache.org>

(cherry picked from commit f27d040ce329894d14ce7caf42a64816df0a9455)

Conflicts:

hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java

YARN-9442. container working directory has group read permissions. Contributed by Jim Brennan.