Buyers and Sellers Talking

As an agent, part of my job is to be an intermediary. My client communicates through me, I then relay a message to the other agent involved and they then communicate that information to their client. Buyers and sellers typically don’t talk to each other directly and, honestly, I prefer it that way. But how would a buyer and seller communicate directly if there are agents involved? And what can go wrong if they do