hadoop

Clone Tools
  • last updated 12 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Revert "HDFS-13179. TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently. Contributed by Ahmed Hussein."

This reverts commit ed59412434027b133f2774af69228510f09a0264.

HDFS-15386. ReplicaNotFoundException keeps happening in DN after removing multiple DN's data directories (#2052)

Contributed by Toshihiro Suzuki.

(cherry picked from commit 545a0a147c5256c44911ba57b4898e01d786d836)

Conflicts:

hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java

(cherry picked from commit e5a02a7499cb124e166fc2765c082efe3b92fe29)

HADOOP-14698. Make copyFromLocals -t option available for put as well. Contributed by Andras Bokor.

(cherry picked from commit 1f674e26ba056ae2bffd54ddda267302e5d2519b)

Revert "HADOOP-14557. Document HADOOP-8143 (Change distcp to have -pb on by default)."

This reverts commit 44350fdf495f5cf1bb15b1fe6f6e9587d3de0a59.

It is related to the rollback of HADOOP-8143.

Change-Id: If48e3dd670c920ada702dc36461ff398fe9d35cc

Revert "HADOOP-8143. Change distcp to have -pb on by default."

This reverts commit dd65eea74b1f9dde858ff34df8111e5340115511.

Change-Id: I74180cf59d5bbad8c9f66cb331535addcbea863e

HADOOP-16840. AliyunOSS: getFileStatus throws FileNotFoundException in versioning bucket. Contributed by wujinhu.

(cherry picked from commit 6dfe00c71eb3721e9be3fc42349a81c4b013ada1)

HADOOP-14206. TestSFTPFileSystem#testFileExists failure: Invalid encoding for signature. Contributed by Jim Brennan.

(cherry picked from commit 999096d82e6cc24023a77a69509af6820913f942)

YARN-10161. TestRouterWebServicesREST is corrupting STDOUT. Contributed by Jim Brennan.

(cherry picked from commit a43510e21d01e6c78e98e7ad9469cbea70a66466)

HDFS-13179. TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently. Contributed by Ahmed Hussein.

(cherry picked from commit 1839c467f60cbb8592d446694ec3d7710cda5142)

Revert "HDFS-15062. Add LOG when sendIBRs failed. Contributed by Fei Hui."

This reverts commit 0b75c5238b468e0a486a7b7b0d3cf9fc74bc7b22.

HDFS-15062. Add LOG when sendIBRs failed. Contributed by Fei Hui.

(cherry picked from commit 52d7b745c6d95e799542d6409dac30d0418ce8a8)

HADOOP-16758. Refine testing.md to tell user better how to use auth-keys.xml (#1753)

Contributed by Mingliang Liu

HADOOP-16735. Make it clearer in config default that EnvironmentVariableCredentialsProvider supports AWS_SESSION_TOKEN. Contributed by Mingliang Liu

This closes #1733

HADOOP-16705. MBeanInfoBuilder puts unnecessary memory pressure on the system with a debug log.

MBeanInfoBuilder's get() method DEBUG logs all the MBeanAttributeInfo attributes that it gathered. This can have a high memory churn that can be easily avoided.

(cherry picked from commit c73334a924d2009d136a8ee62278aaac53418a24)

HDFS-14931. hdfs crypto commands limit column width. Contributed by Eric Badger

(cherry picked from commit 9ef6ed9c1c83b9752e772ece7a716a33045752bf)

YARN-9914. Use separate configs for free disk space checking for full and not-full disks. Contributed by Jim Brennan

(cherry picked from commit eef34f2d87a75e16b2cca870d99a5e1e28c31d9b)

HADOOP-16598. Backport "HADOOP-16558 [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes" to all active branches

Signed-off-by: stack <stack@apache.org>

  1. … 28 more files in changeset.
Revert "HDFS-14633. The StorageType quota and consume in QuotaFeature is not handled for rename. Contributed by Jinglun."

This reverts commit 5369a3bd94ee6b49d3aacdee1ca238be0c25e370.

HDFS-14633. The StorageType quota and consume in QuotaFeature is not handled for rename. Contributed by Jinglun.

(cherry picked from commit 62d71fbac3789c7d484bc76ced9ec7fa6ff94de1)

HDFS-14633. The StorageType quota and consume in QuotaFeature is not handled for rename. Contributed by Jinglun.

(cherry picked from commit 62d71fbac3789c7d484bc76ced9ec7fa6ff94de1)

YARN-9756: Create metric that sums total memory/vcores preempted per round. Contributed by Manikandan R (manirajv06).

(cherry picked from commit d562050cec83a2bc2ffb6d109ed3d64b394b870d)

YARN-9775. RMWebServices /scheduler-conf GET returns all hadoop configurations for ZKConfigurationStore. Contributed by Prabhu Joseph

(cherry picked from commit 8660e48ca15098e891c560beb3181c22ef3f80ff)

(cherry picked from commit e4249c320257586384035ea3fc286fe54cc699a1)

(cherry picked from commit 6baa0d1e4d78c15833257d98bdd2b38806faed9f)

HDFS-13101. Yet another fsimage corruption related to snapshot. Contributed by Shashikant Banerjee.

Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>

(cherry picked from commit ab7509efe7375949db3b819b53e9461624cd2b1e)

HDFS-14687. Standby Namenode never come out of safemode when EC files are being written. Contributed by Surendra Singh Lilhore.

Reviewed-by: Siyao Meng <smeng@cloudera.com>

Reviewed-by: Wei-Chiu CHuang <weichiu@apache.org>

Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>

(cherry picked from commit b8db5b9a9812023754ed1b3e5b428e161f0add50)

(cherry picked from commit 6b01effd011ab1f88bdbb9f2dbf9bbb0765984e6)

YARN-9442. container working directory has group read permissions. Contributed by Jim Brennan.

(cherry picked from commit 2ac029b949f041da2ee04da441c5f9f85e1f2c64)

Conflicts:

hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c

(cherry picked from commit cec71691be76577718b22f936aea9e2b2cd100ea)

Conflicts:

hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c

HDFS-14313. Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du. Contributed by Lisheng Sun.

YARN-9559. Create AbstractContainersLauncher for pluggable ContainersLauncher logic. (Contributed by Jonathan Hung)

(cherry picked from commit f51702d5398531835b24d812f6f95094a0e0493e)

(cherry picked from commit 8d357343c4bc9f18e25543583f8f217b8a2f621b)

(cherry picked from commit f943bff2546330bc9eb0914d1a0fa1ef4c709b72)

HDFS-14697. Backport of HDFS-14513. FSImage which is saving should be clean while NameNode shutdown. Contributed by He Xiaoqiao.

(cherry picked from ee1547a8071fbdebb72d409ddfcd46e6c63f82aa)

(cherry picked from 69e3ac5cbb2547416b27dbcfb458099c3a951101)

YARN-8045. Reduce log output from container status calls. Contributed by Craig Condit

(cherry picked from commit 144a55f0e3ba302327baf2e98d1e07b953dcbbfd)

HDFS-14462 Ensure WebHDFS client throws the correct exception during writes. Contributed by Simbarashe Dzinamarira.

(cherry picked from e7a0b8aa83c1fb933d409c514d2155e986e4e25b)

(cherry picked from ba4ca7d996cafa1b1786249cae2b207f0eaa1070)

(cherry picked from da548cf263c699932e5a6b60e00893f857713bcf)