It is a "crime" because the right to enter is not based on need, or on ability, or on any universally applied criteria, but strictly based on religion.  Your second statement, "America can exercise its authority who it wishes to allow to come to its borders" is correct.  This is certainly not the first time America has exercised its authority in criminal fashion.  In World War II the USA prohibited Jewish refugees escaping from Nazi Germany from entering the country.  Popular opinion supported the move. At least in that case they weren't the ones who created the refugees in the first place, as they are today.