Take the 2-minute tour ×

Can someone please tell me what is the difference in dataflow of Hadoop Standalone and Pseudodistributed mode. Infact I am trying to run an example of matrix multiplication presented by John Norstad. It runs fine in hadoop standalone mode but does not work properly in pseudodistributed mode. I am unable to fix the problem so please tell me the principle difference between hadoop standalone and pseudodistributed mode which can be helpful for fixing the stated problem.Thanks



share|improve this question
What error do you get in pseudo-distributed mode? –  Chris Shain Jan 12 '12 at 14:43
it does not give any run time error but the answers of matrix multiplication are not correct in some cases. –  waqas Jan 12 '12 at 14:52

1 Answer 1

In standalone mode everything (namenode, datanode, tasktracker, jobtracker) is running in one JVM on one machine. In pseudo-distributed mode, everything is running each in it's own JVM, but still on one machine. In terms of the client interface there shouldn't be any difference, but I wouldn't be surprised if the serialization requirements are more strict in pseudo-distributed mode.

My reasoning for the above is that in pseudo-distributed mode, everything must be serialized to pass data between JVMs. In standalone mode, it isn't strictly necessary for everything to be serializable (since everything is in one JVM, you have shared memory), but I don't remember if the code is written to take advantage of that fact, since that's not a normal use case for Hadoop.

EDIT: Given that you are not seeing an error, I think it sounds like a problem in the way the MapReduce job is coded. Perhaps he relies on something like shared memory among the reducers? If so, that would work in standalone mode but not in pseudo-distributed mode (or truly distributed mode, for that matter).

share|improve this answer
and this serialization will be handled by keys or what??? –  waqas Jan 12 '12 at 14:54
As I am not expert in this field. Can you please suggest me that what factors I need to check which shows that it is relying on shared memory? –  waqas Jan 12 '12 at 15:00
Are there any static variables in the mapper class, the reducer class, or any custom classes that are used by them? –  Chris Shain Jan 12 '12 at 15:02
no there are no static variables in mapper or reducer classes. but yeah there are some global vaiables defined as static which are being used in these classes. –  waqas Jan 12 '12 at 15:15
That's almost certainly the issue. You may need to re-work the jobs for proper distributed operation. –  Chris Shain Jan 12 '12 at 17:03

Your Answer


