After the death of Mutsu, he played an important role as Saionji's close adviser, and when Saionji joined the third Ito cabinet formed in January 1898 as the minister of education, Takekoshi was assigned the post of secretary to the minister together with the additional post by the imperial order, the councilor of the Ministry of Education.
陸奥の死後は西園寺の側近としても活躍し、明治31年（1898年）1月に成立した第3次伊藤内閣に西園寺が文部大臣として入閣すると、竹越は大臣秘書官兼文部省勅任参事官に任命された。