In 1940 the cotton market collapsed and the family had to sell their home. Phillips' father went to work in Abergavenny, leaving his wife to run the boarding house in London. After the war the family finances improved and they were able to holiday annually in France and Germany. His parents began to buy short leasehold properties as investments and although these did not yield the return that they wished his mother did buy the freehold of one house, which would later become her son's studio and home.