 While Atherstone's career continued declining over the next decade, Earle retired from the music hall and turned to teaching.He and Earle began arguing regarding his accusations of Earle carrying on an affair as well as his resentment towards her success as a professional schoolteacher.Earle threw him out and sent him to live with his two sons Thomas Frederick and William Gordon.