Kaji graduated from Keio University, Department of French Studies in 1945. After joining the assistant director department of Nikkei Photo Studio, he mainly worked as an assistant director for , , and Buichi Saito. In 1965, he made his directorial debut in the TV film "Beyond the Mountains". Kaji became director in 1966 with his first film debut, "Miss Oyuki". In 1969, Kaji became left and joined variety of television studios.