acmurthy in lucene

Merge -r 611886:611887 from trunk to branch-0.15 to fix HADOOP-2574.
  1. … 22 more files in changeset.
HADOOP-2574. Fixed mapred_tutorial.xml to correct minor errors with the WordCount examples.

HADOOP-1876. Updating hadoop-default.html to reflect changes to hadoop-default.xml.

HADOOP-2077. Added version and build information to STARTUP_MSG for all hadoop daemons to aid error-reporting, debugging etc.

HADOOP-1876. Persist statuses of completed jobs in HDFS so that the JobClient can query and get information about decommissioned jobs and also across JobTracker restarts. Contributed by Alejandro Abdelnur.

HADOOP-1965. Interleave sort/spill in teh map-task along with calls to the method. This is done by splitting the 'io.sort.mb' buffer into two and using one half for collecting map-outputs and the other half for sort/spill. Contributed by Amar Kamat.

HADOOP-2131. Allow finer-grained control over speculative-execution. Now users can set it for maps and reduces independently. Contributed by Amareshwari Sri Ramadasu.

HADOOP-2487. Added an option to get statuses for all submitted/run jobs. This information can be used to develop tools for analysing jobs. Contributed by Amareshwari Sri Ramadasu.

HADOOP-2268. Fix org.apache.hadoop.mapred.jobcontrol classes to use the List/Map interfaces rather than concrete ArrayList/HashMap classes internally. Contributed by Adrian Woodhead.

HADOOP-1281. Ensure running tasks of completed map TIPs (e.g. speculative tasks) are killed as soon as the TIP completed.

HADOOP-1660. Add the cwd of the map/reduce task to the java.library.path of the child-jvm to support loading of native libraries distributed via the DistributedCache.

Removed support for deprecated mapred.child.heap.size and fixed some indentation issues in TaskRunner.

HADOOP-2390. Added documentation for user-controls for intermediate map-outputs & final job-outputs and native-hadoop libraries.

    • ?
    • ?
  1. … 29 more files in changeset.
HADOOP-2511. Fix a javadoc warning in org.apache.hadoop.util.Shell introduced by HADOOP-2344.

Added missing attribution of HADOOP-2344 for Amar Kamat.

HADOOP-2344. Enhance the utility for executing shell commands to read the

stdout/stderr streams while waiting for the command to finish (to free up

the buffers). Also, this patch throws away stderr of the DF utility.


org.apache.hadoop.fs.ShellCommand for org.apache.hadoop.util.Shell

org.apache.hadoop.util.ShellUtil for org.apache.hadoop.util.Shell.ShellCommandExecutor

    • ?
HADOOP-2466. Change FileInputFormat.computeSplitSize to a protected non-static method to allow sub-classes to provide alternate implementations. Contributed by Alejandro Abdelnur.

Scheduling HADOOP-2247 for release 0.16.0.
Merge -r 606267:606266 from trunk to branch-0.15 to revert HADOOP-2247.
Merge -r 606266:606267 from trunk to branch-0.15 to fix HADOOP-2247.
HADOOP-2247. Fine-tune the strategies for killing mappers and reducers due to failures while fetching map-outputs. Now the map-completion times and number of currently running reduces are taken into account by the JobTracker before killing the mappers, while the progress made by the reducer and the number of fetch-failures vis-a-vis total number of fetch-attempts are taken into account before teh reducer kills itself. Contributed by Amar Kamat.

Merge -r 605867:605868 from trunk to branch-0.15 to fix HADOOP-2437.
HADOOP-2437. Fix the LocalDirAllocator to choose the seed for the round-robin disk selections randomly. This helps in spreading data across multiple partitions much better.

HADOOP-2457. Added a 'forrest.home' property to the 'docs' target in build.xml.

HADOOP-2352. Remove AC_CHECK_LIB for libz and liblzo to ensure that doesn't have a dependency on them.

HADOOP-2215. Fix documentation in cluster_setup.html & mapred_tutorial.html reflect that mapred.tasktracker.tasks.maximum has been superceeded by mapred.tasktracker.{map|reduce}.tasks.maximum. Contributed by Amareshwari Sri Ramadasu.

HADOOP-2453. Fix the configuration for wordcount-simple example in Hadoop Pipes which currently produces an XML parsing error. Contributed by Amareshwari Sri Ramadasu.

Merge -r 605470:605471 from trunk to branch-0.15 to fix HADOOP-2227.
HADOOP-2227. Use the LocalDirAllocator uniformly for handling all of the temporary storage required for a given task. It also implies that mapred.local.dir.minspacestart is handled by checking if there is enough free-space on any one of the available disks. Contributed by Amareshwari Sri Ramadasu.

Reverting HADOOP-2245 since it can be significantly improved, specifically it copies code into JobControlTestUtils from TestJobControl, and yet doesn't remove the original code. It also introduced new compiler warnings which somehow weren't caught by the patch-process.