The term robber baron was first used in the 12th and 13th centuries to describe rich men who lived in large castles along major transportation rivers, like the Rhine in Europe. They would send ships out onto these rivers to stop anybody who was traveling through who did not know better and collect a tax from them on any money or goods on these ships. They were not officially allowed to collect these taxes. It was only the Holy Roman Emperor who could decide what taxes to be charged, how much and who could charge it.