However, he feels he cannot accept the call because of the shame that his family bears due to the antics of his Uncle James and (he suspects) his father.
He is hesitant to accept the phone call because he feels responsible for the negative reputation of his family, which stems from his uncle's behavior and possibly his father's involvement.