title: Is Reduce always required to remove duplicates in MapReduce?
body: <p>Why is Reduce required in MapReduce? If a job such as counting the number of words in a book will result in the same outcome if performed by a single process or MapReduced over a farm of servers, what possibility exists that duplicates will have to be removed? I'm assuming the Reduce step, at least in this example, would simply SUM the outcome from each worker process and deliver the total count of words in the book. I don't understand where duplicates of anything come into the picture. </p>