It was said that when Zhang was at a prefectural post, he would send his subordinates to secretly investigate who are the good and who are the evil people of the locale, allowing him to punish the evil and discover the good. He was also praised for suppressing the local bullies and helping the weak. However, it was also said that he divorced his wife and married a Lady Li as his new wife. Lady Li was said to be promiscuous and jealous, but was greatly favored by Zhang. When they went to Xiang Prefecture, there was a young man who sold pens for a living and was good at singing and dancing. Lady Li carried on an affair with the young man, and, falsely declaring that Zhang had previously carried on an affair with the young man's mother, encouraging Zhang to adopt the young man as a son, to be named Zhang Shenji () — an act opposed by Zhang's son, by his former wife, Zhang Shenwei (), to no avail. It was also said that Lady Li favored witchcraft, engaging many witches, and interfered with Zhang's administration, and from that point on, Zhang's reputation began to suffer.