Don't mix Java versions in a Hadoop cluster

During the Hadoop project we ran into an issue where the mapping would stall out, slowly but surely. The mappers never actually died, they just kept taking longer and longer to get to each tenth of a percent, then each hundredth of a percent, etc.
After messing about for longer than I would have liked we discovered this was due to having half the cluster running Java 5 and the other half running Java 6. An entire network of either is fine, but apparently mixing the versions is a big no-no. After we homogenized the JRE, everything purred along prettily.
The moral of the story: Don't mix and match Java versions within a Hadoop cluster