Sir William Baillie and his wife Lady Baillie extended the house in 1822 and the building was in continuous ownership by the Baillie family until the 1950s. At its height the building had 39 rooms and offices, 11 domestic servants and servants quarters. The grounds included stables, fodder store, a horse driven mill, a tack room for keeping carriages and later motor cars and tennis courts and bowling green. The Baillie family played an active role in the community, giving significant sums of money for the construction of public buildings and other developments, including the Baillie Institutes (community centres) built in Harthill, Whitburn and Blackburn. The Baillie Institute building in Harthill is now the Pentecostal church. Sir William and Lady Baillie were also responsible for the establishment of the Lady Baillie Sabbath School in Whitburn. Once a year the family would open their mansion to the public and on several other days each summer made it available for Sunday school outings.