ALBANY, Dec. 17— Governor Cuomo spoke out today against ethnic slights and stereotypes that he said had been inflicted on himself and other Italian-Americans.

''The conservatives who attack me with great regularity - think about it - whenever they get really annoyed at something I'm doing and they want to write a piece that's negative, it almost always mentions 'The Godfather,' always mentions ethnicity, always mentions ethnicity,'' the Governor said.

Mr. Cuomo's comments, in a discussion with reporters, were primarily aimed against the use of the word ''Mafia'' to describe organized crime.

The word was used repeatedly yesterday and today in relation to the slaying on Monday of Paul Castellano, the reputed organized-crime leader.

'An Ugly Stereotype'

The Governor wound up speaking for 45 minutes, describing what he sees as an overall stereotyping of Italian-Americans. ''As I have gotten older, this has become more significant to me,'' Mr. Cuomo said.

He said the word Mafia was ''an Italian word and every time you say it, you suggest to people that organized crime is Italian - it's an ugly stereotype.''

''I've lived with this notion all my life,'' he added later.

Mr. Cuomo asked if other ethnic groups - such as blacks, Jews, Russian immigrants - could not also be linked to organized crime.

''Why don't you go to Rikers Island, conclude that 94 percent of them are black, and think up a name that would create that association in people's minds?'' he said.

At times, Mr. Cuomo indicated that the Mafia was only one element of organized crime. But at one point, the Governor said, ''You're telling me that Mafia is an organization, and I'm telling you that's a lot of baloney.''

Told that Rudolph W. Giuliani, the United States Attorney for the Southern District of New York, used the term ''Mafia,'' Mr. Cuomo said, ''You call him up today and say the Governor of the State of New York is surprised that he would use the word 'Mafia' to describe organized crime.''

Reached by telephone at his Manhattan office, Mr. Giuliani said that there was indeed a group known as the Mafia that ''organizes itself around the fact that it is Italian or Italian-American.''

''You can't go so far as to suggest that the Mafia doesn't exist,'' he said.

He said he agreed with the Governor that the term ''Mafia'' should not be applied to all organized crime, and he noted that there were crime groups of Chinese, Israelis, Cubans and others.

Fighting the Stereotype

In the discussion today, Mr. Cuomo said that President-elect Jimmy Carter had offered him various positions, but that he had said: ''I don't want anything, Mr. President. The one thing I would love to see is an Italian as Attorney General.' ''

He said such an appointment was one way to reduce the ethnic group's ''Mafia'' stereotype.

Late today, fearful of how the Governor's comments might be interpreted, the administration sought to clarify them. Officials said the 94 percent figure for blacks on Rikers Island was ''just picked at random as a number.'' And the Governor himself said he wanted to make clear that ''Italian-Americans share the problem of stereotyping with Jews and Poles and blacks and Irish.''