You don't need to open the windows if it is freezing cold outside. This will allow it to air out a bit, taking away any musty smells.
[header] How to impress your parents by cleaning the house [title] Dilute the musty smell in your house. [step] Open all the windows in your parents room. If weather permits.