While Semites are as you describe -- various ethnicities speaking various languages - including Arabic -- the common and widely accepted use and understanding of that term is specifically anti-Jewish, both in terms of racial and religious connotation.

Islam is no more an "ideology" than Judaism in this case, so your distinction is a false one. And the definition of Islamophobia as an "unfounded or irrational fear" is again not the common understanding of the word. Islamophobia refers to anti-Muslim bigotry, hatred etc in the same way that anti-Semitism refers to anti-Jewish bigotry, hatred etc. 

Split hairs all you like but your definitions do not represent the common usage and understanding of those terms.