You don't want the residue on your contacts. There is also no need to use antibacterial soap because it is no more effective than regular soap.
[header] How to take out hard contacts [title] Wash your hands with soap and warm water. [step] You must wash your hands with the right kind of soap before removing your contacts. Avoid using deodorant or moisturizing soap.