Because if you use your map of reality to look at other people's map of reality, you will distort their map.
Imagine you use your map of reality to look at other people's map of reality you will distort their map