Ankrah joined the Gold Coast Regiment in 1939. On the outbreak of World War II, Ankrah was mobilized into the Royal West African Frontier Force. While his Brigade was in East Africa in 1940, he was transferred to the Record Office in Accra with the rank of Warrant Officer Class II and made second-in-command. In October 1946, he went to the Marshfield Officer Cadets Training Unit in the United Kingdom and graduated in February 1947 as the first African officer in the Gold Coast Army. He was commissioned a lieutenant in 1947 and became the first African camp commandant at the Army Headquarters. He was later made the first Ghanaian Chief Instructor of the Education Unit. He was promoted Major in 1956 and became the first African to command an all-African company, the Charlie Company of the First Battalion at Tamale, Ghana. He later became Lieutenant Colonel and took over the whole battalion. He rose to the rank of colonel by 1960, at a time when there were few Ghanaian officers at that level. During the United Nations Operation in the Congo, he was the Brigade Commander of the force-based at Luluabourg, Kasai in the present-day Democratic Republic of Congo. He was the only Ghanaian awarded the Military Cross in Leopoldville for acts of unsurpassed gallantry in Congo in 1961. The citation read: