You're driving at a skeptical argument to suit your cause/view. Not all boys who were abused were altar servers. Thus, that already is a damaging blow to your argument. If you want to make homosexuality a pristine inviolable contact of sexuality where no man/woman with those tendencies would ever do that to a child, that's on you. 

Definitions are the rulebooks we apply in life. Contours of lines drawn on the road to separate traffic for everyone's safety. Lines drawn in the sand, so-to-speak. The word "homosexual" means orientation to someone of the same sex. That can be by choice or by tendency. Thus, someone who prefers children of the same gender falls under that definition. 

As I said before, my argument isn't all homosexuals go after or have an orientation towards children of the same sex. So, it isn't generalized. Straight men are just as much required to not engage in hetero relations as a Priest. So the rules are the same.