His father became a publican by the time Joe was two years old. The family had moved to Whittington Moor, where Fred Davis had took over the Travellers Rest pub. As a young child, Davis was sent to live with his maternal grandparents Thomas and Sarah Clark. They lived at Newbold, Derbyshire. This was still in the registration and postal district of Whittington Moor. The full name of the township was Newbold and Dunston. Joe was recorded as living there at the time of the 1911 United Kingdom census, just 13 days before his tenth birthday. He went to School at Newbold while living with his grand parents.