Throw out the baby with the bath water
This idiom comes from a German proverb, das Kind mit dem Bade ausschütten. In 1512,  Narrenbeschwörung (Appeal to Fools) by Thomas Murner includes a woodcut picture showing a woman tossing a baby out with waste water.