Patrick Dove came from a family of distinguished clergymen and landowners. He was descended from Sir William Dove of Upton, near Peterborough (son of Bishop Thomas Dove). His branch of the family had connections with the Royal Navy going back to 1716 when they moved to Devon where the family still live. His father, Lieutenant Henry Dove RN, retired from active service with the Navy at the end of the Napoleonic wars in 1815, taking up an appointment at Deal, Kent connected with the Cinque Ports. Edward was educated in England and in France, where he was expelled from the Academy for leading a rebellion against the master. He greatly desired a career in the navy, but his father would not allow this and in 1830 Edward was sent to Scotland to learn farming. Following this he spent time in Paris, in Spain and in London, where he impressed a friend by his "enormous energy, both physical and mental".