According to the family tree of the Shimai family, the family was descended from the Northern House of the Fujiwara clan and the descendants declared themselves Toji (the Fujiwara clan) for generations; however, they changed their family name to Shimai in the generation of Jiro Uemon Shigehisa.
島井家の系図には、藤原北家の血筋をひき、代々藤氏を名乗ったが、次郎右衛門茂久の代に、島井姓に改めたと書いてある。