Someone inviting you up to their room (for sex or not) does not constitute sexual harassment. From your description he didn't pressure you, and he never became aggressive but rather embarrassed after you declined his poorly planned invitation.

Conflating your experience with so many other women just belittles the real issue of harassment.