a method and apparatus for performing authentication in a communication system . the method includes the steps of detecting an unexpected authentication message , updating a counter value , comparing the counter value to a threshold value , and performing an encryption key update operation in response to the comparison between the counter value and the threshold value . the apparatus includes a base station , a switching center coupled to the base station , and an authentication center responsive to the switching center . the switching center detects an unexpected authentication message received by the base station . the authentication center updates a counter when the switching center detects the unexpected authentication message and compares the updated counter with a threshold value . the authentication center initiates an encryption key update operation when the updated counter value exceeds the threshold value .