Kyle was born in Stirling, Scotland on 2 February 1838.  He completed an apprenticeship with an Edinburgh pharmacy in 1854 and became assistant to Dr Stevenson Macadam, lecturer in chemistry to Surgeons' Hall, Edinburgh.  He made his first scientific discovery at the age of 18.  Moving to the field of industrial chemistry, he was head of the chemical laboratory of Glasgow University and then manager of an animal charcoal manufacturer in Greenock.  He was a fellow of the Royal Society of Arts.
He emigrated to Argentina in July 1862.  When President-Marshall Solano Lopez of Paraguay invaded Corrientes Province in 1865 there broke out the War of the Triple Alliance and Kyle joined the medical corps of the Argentine Army as a pharmacist with the rank of lieutenant.  He participated in the siege of Uruguaiana (where the defenders were reduced to living on lump sugar), the three-day battle of the Boquerón and in the Battle of Tuyutí, the bloodiest international battle in the history of South America.  He served on board the hospital ship Pavón and returned to Buenos Aires in December 1866 in charge of a convoy of wounded soldiers.  His wartime experiences led him to take a foundational interest in the Argentine Red Cross Society, of which he was made an honorary member in 1896.