Sharma taught at colleges in Arrah (1943) and Bhagalpur (July 1944 to November 1946) before coming to Patna College, Patna University in 1946. He became the head of the Department of History at Patna University from 1958 to 1973. He became a university professor in 1958. He served as professor and dean of the History Department at Delhi University from 1973 to 1978. He got the Jawaharlal Fellowship in 1969. He was the founding chairperson of Indian Council of Historical Research from 1972 to 1977. He has been a visiting fellow at the School of Oriental and African Studies (1959–64); University Grants Commission National Fellow (1958–81); visiting professor of history in University of Toronto (1965–66); President of Indian History Congress in 1975 and recipient of Jawaharlal Nehru Award in 1989. He became the deputy-chairperson of UNESCO's International Association for Study of Central Asia from 1973 to 1978; he has served as an important member of the National Commission of History of Sciences in India and a member of the University Grants Commission.