Linley's parents were William (1704–1792), a carpenter, and Maria (1701–1792). He was the couple's eldest child and was born on 17 January 1733 in the village of Badminton in Gloucestershire. When Linley was 11 years old, in 1744, the family moved to Bath, Somerset where he served an apprenticeship with Thomas Chilcot, the organist at Bath Abbey. Before becoming Chilcot's apprentice, Linley had initially assisted by pumping the organ. He may also have received tuition from Domenico Paradisi who was giving harpsichord lessons in London from around 1747. William Boyce also tutored him in London.