Filter water using a charcoal filter.
Instead of coffee grounds, use activated charcoal.  Rinse the charcoal well before placing the charcoal in a coffee filter.  Place the coffee filter into the coffee maker and proceed as if making coffee, instead it will be clean water.