Introduction: With the development of medical technology, clinical alarms from various medical devices, which are rapidly increasing, are becoming a new problem in intensive care units. The aim of this study was to evaluate alarm fatigue in Polish nurses employed in Intensive Care Units and identify the factors associated with alarm fatigue. Methods: A cross-sectional study. The study used the nurses’ alarm fatigue questionnaire by Torabizadeh. The study covered 400 Intensive Care Unit nurses. The data were collected from February to June 2021. Results: The overall mean score of alarm fatigue was 25.8 ± 5.8. Participation in training programs related to the use of monitoring devices available in the ward, both regularly (ß = −0.21) and once (ß = −0.17), negatively correlated with nurses’ alarm fatigue. On the other hand, alarm fatigue was positively associated with 12 h shifts [vs. 8 h shifts and 24 h shifts] (ß = 0.11) and employment in Intensive Cardiac Surveillance Units—including Cardiac Surgery [vs. other Intensive Care Units] (ß = 0.10). Conclusion: Monitoring device alarms constitute a significant burden on Polish Intensive Care Unit nurses, in particular those who do not take part in training on the operation of monitoring devices available in their ward. It is necessary to improve Intensive Care Unit personnel’s awareness of the consequences of overburdening and alarm fatigue, as well as to identify fatigue-related factors.