There are a million things that can distract you while driving.  Let's be real about this:  there are a lot of lousy drivers out there.  Can you be distracted by a sign?  Yes - but if you are looking at the sign and not paying attention to what's happening on the road - that's on you.