The England prop referred to Lee, who is from the Traveller community, as "Gypsy boy" during England's win at Twickenham.
The Six Nations accepted Marler's private and public apology and imposed no punishment.
"As far as I'm concerned that should be the end of the matter," said Ritchie.
World Rugby has since begun an investigation into the process behind clearing Marler and could yet enforce sanctions on the 25-year-old - but Ritchie has backed the Six Nations' disciplinary rulings.
"Let's take some perspective and let's not deflect from what we did in terms of the Grand Slam," added Ritchie.
"What Joe said was entirely inappropriate. He recognised that, he apologised very quickly and has apologised publicly since then.
"It's pretty clear that nobody made a complaint about it, and I think Samson Lee's reaction to it has been evident and he accepted the apology.
"[England coach] Eddie Jones and I spoke about it and when Eddie talks to someone that person is left in no doubt as to the appropriate nature of his behaviour, and that it should not be repeated.
"There was no citing officer complaint, there was no referee complaint and the Six Nations looked at the matter fully as well and came to a conclusion."
Campaigners from the Traveller community - and figures inside the sport - had been critical of Marler for making the comment and called for a ban.
The Harlequins player apologised to Lee, who said he accepted the comment as "banter".
Wales head coach Warren Gatland also described the incident as "banter", but subsequently apologised following criticism.
The Welsh Rugby Union's questioning of the punishment caused England head coach Jones to accuse them of trying to derail his side's Grand Slam bid.
The decision by Six Nations bosses not to punish Joe Marler over remarks to Wales' Samson Lee should end the matter, says RFU chief Ian Ritchie.