seems like you're so busy anyway and then that's just one more thing to have to worry about so
"It's just another worry on my mind that you are so occupied anyway."
"I'm not really worried about you being so busy."