I know Rob left early, but our memory is of us as a five, and I wouldn't have liked it if it was just us four."
Even though Rob left early, we remember being a group of five, and I would not have been happy with it if it had only been the four of us.