It's wrong to pretend to go to work.
Gordon has taken a day off from work to golf and is about to leave the house in his golf outfit, when his wife asks where he's going like that.
Gordon wants to answer his wife's question.
Gordon tells his wife that he's going golfing since he's taken a day off work today.
Gordon tells his wife that he's going to work, it's just casual day, so he's taking advantage.
Gordon's wife is happy to see Gordon utilizing some time off work to go do something relaxing.
Gordon's wife later finds out he lied, and he had taken a day off work and is now suspicious of Gordon and insecure about him.