Takanaga was the third son (some say forth) of Kazunobu NAITO the lord of Tanagura Domain in Mutsu Province (and subsequently the lord of Murakami Domain in Echigo Province).
陸奥国棚倉藩主（のち越後国村上藩主）内藤弌信の三男（四男との説もある）。