I wish you'd come in when it was just a chip on your shoulder.
A "weight on your chest" and a "chip on your shoulder" are both figurative expressions that refer to problems or stresses that people silently bear. While not usually directly compared, a literal "weight on your chest" (pictured) is apparently more difficult to deal with because its heaver, and the "chip" can become the "weight" if left unaddressed.