it was said , however , that his relationship with madam soraya , puteri paramita 's mother , was a special one , and that after his relationship with her , he never took another mistress , even after they broke up as soon as he was crowned sultan and she moved to europe with her child to escape judgmental eyes and tongues .