However, his increasing womanising and reckless behaviour during the 1920s and 1930s worried Prime Minister Baldwin and his hard working, strict father King George V. Edward was also uninterested with royal duties and disliked the British establishment. George V was disappointed by Edward's failure to settle down in life and was furious with his many affairs with married women, and was reluctant to see him inherit the Crown. George V favoured his son Albert (later King George VI) who was a family man and loyal to his beloved wife.