hadoop

Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
HADOOP-15691 Add PathCapabilities to FileSystem and FileContext.

Contributed by Steve Loughran.

This complements the StreamCapabilities Interface by allowing applications to probe for a specific path on a specific instance of a FileSystem client

to offer a specific capability.

This is intended to allow applications to determine

* Whether a method is implemented before calling it and dealing with UnsupportedOperationException.

* Whether a specific feature is believed to be available in the remote store.

As well as a common set of capabilities defined in CommonPathCapabilities,

file systems are free to add their own capabilities, prefixed with

fs. + schema + .

The plan is to identify and document more capabilities -and for file systems which add new features, for a declaration of the availability of the feature to always be available.

Note

* The remote store is not expected to be checked for the feature;

It is more a check of client API and the client's configuration/knowledge

of the state of the remote system.

* Permissions are not checked.

Change-Id: I80bfebe94f4a8bdad8f3ac055495735b824968f5

  1. … 25 more files in changeset.
HDDS-2165. Freon fails if bucket does not exists

Closes #1503

HDDS-1738. Add nullable annotation for OMResponse classes

Closes #1499

  1. … 4 more files in changeset.
HDDS-2171. Dangling links in test report due to incompatible realpath

Closes #1515

fix checkstyle issue

YARN-6715. Fix documentation about NodeHealthScriptRunner. Contributed by Peter Bacsko

YARN-9808. Zero length files in container log output haven't got a header. Contributed by Adam Antal

HDDS-2174. Delete GDPR Encryption Key from metadata when a Key is deleted

Signed-off-by: Anu Engineer <aengineer@apache.org>

HDDS-2168. TestOzoneManagerDoubleBufferWithOMResponse sometimes fails with out of memory error (#1509)

HDDS-2172.Ozone shell should remove description about REST protocol support. Contributed by Siddharth Wagle.

HDFS-14808. EC: Improper size values for corrupt ec block in LOG. Contributed by Ayush Saxena.

HDFS-14808. EC: Improper size values for corrupt ec block in LOG. Contributed by Ayush Saxena.

(cherry picked from commit 66400c1cbb2b4b2f08f7db965c8b7237072bdcc4)

HDFS-14808. EC: Improper size values for corrupt ec block in LOG. Contributed by Ayush Saxena.

(cherry picked from commit 66400c1cbb2b4b2f08f7db965c8b7237072bdcc4)

(cherry picked from commit 2bb4ac18ec0ea8804b424b593615bb06e21a4b6c)

HDFS-14655. [SBN Read] Namenode crashes if one of The JN is down. Contributed by Ayush Saxena.

HDFS-14655. [SBN Read] Namenode crashes if one of The JN is down. Contributed by Ayush Saxena.

HDFS-14655. [SBN Read] Namenode crashes if one of The JN is down. Contributed by Ayush Saxena.

YARN-9730. Support forcing configured partitions to be exclusive based on app node label

(cherry picked from commit 73a044a63822303f792183244e25432528ecfb1e)

(cherry picked from commit dd094d79023f6598e47146166aa8c213e03d41b7)

(cherry picked from commit 10bdcb6f1da3b86146efa479c0bbc8d1da505789)

  1. … 8 more files in changeset.
YARN-9730. Support forcing configured partitions to be exclusive based on app node label

  1. … 8 more files in changeset.
YARN-9730. Support forcing configured partitions to be exclusive based on app node label

(cherry picked from commit 73a044a63822303f792183244e25432528ecfb1e)

  1. … 8 more files in changeset.
YARN-9730. Support forcing configured partitions to be exclusive based on app node label

(cherry picked from commit 73a044a63822303f792183244e25432528ecfb1e)

(cherry picked from commit dd094d79023f6598e47146166aa8c213e03d41b7)

  1. … 8 more files in changeset.
HDDS-2170. Add Object IDs and Update ID to Volume Object (#1510)

HDFS-14843. Double Synchronization in BlockReportLeaseManager. Contributed by David Mollitor.

HDFS-14837. Review of Block.java. Contributed by David Mollitor.

HDFS-14868. RBF: Fix typo in TestRouterQuota. Contributed by Jinglun.

Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>

Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>

HDDS-2167. Hadoop31-mr acceptance test is failing due to the shading

Closes #1507

HDFS-13660. DistCp job fails when new data is appended in the file while the DistCp copy job is running

This uses the length of the file known at the start of the copy to determine the amount of data to copy.

* If a file is appended to during the copy, the original bytes are copied.

* If a file is truncated during a copy, or the attempt to read the data fails with a truncated stream,

distcp will now fail. Until now these failures were not detected.

Contributed by Mukund Thakur.

Change-Id: I576a49d951fa48d37a45a7e4c82c47488aa8e884

(cherry picked from commit 51c64b357d4bd1a0038e61df3d4b8ea0a3ad7449)

HDFS-13660. DistCp job fails when new data is appended in the file while the DistCp copy job is running

This uses the length of the file known at the start of the copy to determine the amount of data to copy.

* If a file is appended to during the copy, the original bytes are copied.

* If a file is truncated during a copy, or the attempt to read the data fails with a truncated stream,

distcp will now fail. Until now these failures were not detected.

Contributed by Mukund Thakur.

Change-Id: I576a49d951fa48d37a45a7e4c82c47488aa8e884

HDFS-13660. DistCp job fails when new data is appended in the file while the DistCp copy job is running

This uses the length of the file known at the start of the copy to determine the amount of data to copy.

* If a file is appended to during the copy, the original bytes are copied.

* If a file is truncated during a copy, or the attempt to read the data fails with a truncated stream,

distcp will now fail. Until now these failures were not detected.

Contributed by Mukund Thakur.

Change-Id: I576a49d951fa48d37a45a7e4c82c47488aa8e884

(cherry picked from commit 51c64b357d4bd1a0038e61df3d4b8ea0a3ad7449)

package-info fix + removing FWIO

package-info fix + removing FWIO