Wragge was originally named William, but this was changed to Clement (Lindley was the name of his great uncle). Both of his parents died when he was young: his mother at five months and his father, Clement Ingleby Wragge, at five years following a fall from his horse. He was raised for a number of years by his grandmother, Emma Wragge (formerly Ingleby) at Oakamoor, Staffordshire who taught him the rudiments of cosmology and meteorology. Emma's husband George had died in 1849 and had managed the Oakamoor works of the Cheadle Brass Wire Company before it was sold to Thomas Bolton in 1852. Wragge became an avid naturalist at a young age, being surrounded by the beauty of the Churnet valley. He was educated initially at the Church school in Oakamoor, and then his formal education was at Uttoxeter Thomas Alleyne's Grammar School. Wragge hated being a boarder at Uttoxeter and ran away, but was returned to the school where he excelled. Upon the death of his grandmother in 1865 his uncles George and William decided that he should he move to London to live with his Aunt Fanny and her family in Teddington. He was considered by his aunt to be spoilt and he rebelled against the harsh treatment. There he later boarded at the Belvedere school in Upper Norwood and at the end of his education he improved his Latin in Cornwall. He then followed in the footsteps of his father, studying law at Lincoln's Inn. He also attended St Bartholomew's Hospital alongside medical students to watch operations. Wragge travelled on the continent of Europe extensively with his Uncle William of Cheltenham. His second cousin was Clement Mansfield Ingleby, a partner in the family law firm Ingleby, Wragge, and Ingleby (which later became known as Wragge & Co of Birmingham), and later a literary scholar. At the age of 21 Wragge came into the inheritance left to him by his parents and a legacy and family silver left to him by his aunt on his mother's side of the family. He decided to take eight months break from Lincoln's Inn to visit the Egypt and the Levant.