hadoop-distcp

Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Preparing for 3.3.0 Release

  1. … 110 more files in changeset.
Revert "HADOOP-14557. Document HADOOP-8143 (Change distcp to have -pb on by default)."

This reverts commit 44350fdf495f5cf1bb15b1fe6f6e9587d3de0a59.

It is related to the rollback of HADOOP-8143.

Change-Id: If48e3dd670c920ada702dc36461ff398fe9d35cc

Revert "HADOOP-14557. Document HADOOP-8143 (Change distcp to have -pb on by default)."

This reverts commit 44350fdf495f5cf1bb15b1fe6f6e9587d3de0a59.

It is related to the rollback of HADOOP-8143.

Change-Id: If48e3dd670c920ada702dc36461ff398fe9d35cc

Revert "HADOOP-14557. Document HADOOP-8143 (Change distcp to have -pb on by default)."

This reverts commit 44350fdf495f5cf1bb15b1fe6f6e9587d3de0a59.

It is related to the rollback of HADOOP-8143.

Change-Id: If48e3dd670c920ada702dc36461ff398fe9d35cc

Revert "HADOOP-14557. Document HADOOP-8143 (Change distcp to have -pb on by default)."

This reverts commit 44350fdf495f5cf1bb15b1fe6f6e9587d3de0a59.

It is related to the rollback of HADOOP-8143.

Change-Id: If48e3dd670c920ada702dc36461ff398fe9d35cc

Revert "HADOOP-14557. Document HADOOP-8143 (Change distcp to have -pb on by default)."

This reverts commit 44350fdf495f5cf1bb15b1fe6f6e9587d3de0a59.

It is related to the rollback of HADOOP-8143.

Change-Id: If48e3dd670c920ada702dc36461ff398fe9d35cc

Revert "HADOOP-14557. Document HADOOP-8143 (Change distcp to have -pb on by default)."

This reverts commit 44350fdf495f5cf1bb15b1fe6f6e9587d3de0a59.

It is related to the rollback of HADOOP-8143.

Change-Id: If48e3dd670c920ada702dc36461ff398fe9d35cc

Revert "HADOOP-8143. Change distcp to have -pb on by default."

This reverts commit dd65eea74b1f9dde858ff34df8111e5340115511.

Change-Id: I74180cf59d5bbad8c9f66cb331535addcbea863e

Revert "HADOOP-8143. Change distcp to have -pb on by default."

This reverts commit dd65eea74b1f9dde858ff34df8111e5340115511.

Change-Id: I74180cf59d5bbad8c9f66cb331535addcbea863e

Revert "HADOOP-8143. Change distcp to have -pb on by default."

This reverts commit dd65eea74b1f9dde858ff34df8111e5340115511.

Change-Id: I74180cf59d5bbad8c9f66cb331535addcbea863e

Revert "HADOOP-8143. Change distcp to have -pb on by default."

This reverts commit dd65eea74b1f9dde858ff34df8111e5340115511.

Change-Id: I74180cf59d5bbad8c9f66cb331535addcbea863e

Revert "HADOOP-8143. Change distcp to have -pb on by default."

This reverts commit dd65eea74b1f9dde858ff34df8111e5340115511.

Change-Id: I74180cf59d5bbad8c9f66cb331535addcbea863e

Revert "HADOOP-8143. Change distcp to have -pb on by default."

This reverts commit dd65eea74b1f9dde858ff34df8111e5340115511.

Change-Id: I74180cf59d5bbad8c9f66cb331535addcbea863e

HADOOP-14254. Add a Distcp option to preserve Erasure Coding attributes. Contributed by Ayush Saxena.

Preparing for 3.3.1 development

  1. … 110 more files in changeset.
Preparing for 3.1.4 release

Change-Id: Ibde183eb466eda4210c0e3ee77dcd27672d337cd

  1. … 96 more files in changeset.
Preparing for 3.1.5 development

Change-Id: Iabc64aba7392e3b6f9e4e18109fcaa2cfc01d1f9

  1. … 96 more files in changeset.
HADOOP-16988. Remove source code from branch-2. (aajisaka via jhung)

This closes #1959

    • -103
    • +0
    ./src/main/java/org/apache/hadoop/tools/DiffInfo.java
  1. … 10832 more files in changeset.
HADOOP-16932. distcp copy calls getFileStatus() needlessly and can fail against S3 (#1936)

Contributed by Steve Loughran.

This strips out all the -p preservation options which have already been

processed when uploading a file before deciding whether or not to query

the far end for the status of the (existing/uploaded) file to see if any

other attributes need changing.

This will avoid 404 caching-related issues in S3, wherein a newly created

file can have a 404 entry in the S3 load balancer's cache from the

probes for the file's existence prior to the upload.

It partially addresses a regression caused by HADOOP-8143,

"Change distcp to have -pb on by default" that causes a resurfacing

of HADOOP-13145, "In DistCp, prevent unnecessary getFileStatus call when

not preserving metadata"

Change-Id: Ibc25d19e92548e6165eb8397157ebf89446333f7

HADOOP-16932. distcp copy calls getFileStatus() needlessly and can fail against S3 (#1936)

Contributed by Steve Loughran.

This strips out all the -p preservation options which have already been

processed when uploading a file before deciding whether or not to query

the far end for the status of the (existing/uploaded) file to see if any

other attributes need changing.

This will avoid 404 caching-related issues in S3, wherein a newly created

file can have a 404 entry in the S3 load balancer's cache from the

probes for the file's existence prior to the upload.

It partially addresses a regression caused by HADOOP-8143,

"Change distcp to have -pb on by default" that causes a resurfacing

of HADOOP-13145, "In DistCp, prevent unnecessary getFileStatus call when

not preserving metadata"

Preparing for 3.4.0 development

  1. … 110 more files in changeset.
HADOOP-16775. DistCp reuses the same temp file within the task for different files.

Contributed by Amir Shenavandeh.

This avoids overwrite consistency issues with S3 and other stores

Change-Id: Ic4d05ef3397e963ba28fd9f775bb362b0da36ad9

HADOOP-16909 Typo in distcp counters.

Contributed by Sebastian Nagel.

  1. … 1 more file in changeset.
HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency (#1635). Contributed by Vinayakumar B.

  1. … 424 more files in changeset.
HADOOP-16808. Use forkCount and reuseForks parameters instead of forkMode in the config of maven surefire plugin. Contributed by Xieming Li.

  1. … 1 more file in changeset.
HADOOP-16808. Use forkCount and reuseForks parameters instead of forkMode in the config of maven surefire plugin. Contributed by Xieming Li.

(cherry picked from commit f6d20daf404fab28b596171172afa4558facb504)

  1. … 1 more file in changeset.
HADOOP-16808. Use forkCount and reuseForks parameters instead of forkMode in the config of maven surefire plugin. Contributed by Xieming Li.

(cherry picked from commit f6d20daf404fab28b596171172afa4558facb504)

  1. … 1 more file in changeset.
HADOOP-16808. Use forkCount and reuseForks parameters instead of forkMode in the config of maven surefire plugin. Contributed by Xieming Li.

(cherry picked from commit f6d20daf404fab28b596171172afa4558facb504)

  1. … 1 more file in changeset.
HADOOP-16808. Use forkCount and reuseForks parameters instead of forkMode in the config of maven surefire plugin. Contributed by Xieming Li.

(cherry picked from commit f6d20daf404fab28b596171172afa4558facb504)

  1. … 1 more file in changeset.
HADOOP-16808. Use forkCount and reuseForks parameters instead of forkMode in the config of maven surefire plugin. Contributed by Xieming Li.

(cherry picked from commit f6d20daf404fab28b596171172afa4558facb504)

  1. … 1 more file in changeset.