Square-wave bursting is an activity pattern common to a variety of neuronal and endocrine cell models that has been linked to central pattern generation for respiration and other physiological functions. Many of the reduced mathematical models that exhibit square-wave bursting yield transitions to an alternative pseudo-plateau bursting pattern with small parameter changes. This susceptibility to activity change could represent a problematic feature in settings where the release events triggered by spike production are necessary for function. In this work, we analyze how model bursting and other activity patterns vary with changes in a timescale associated with the conductance of a fast inward current. Specifically, using numerical simulations and dynamical systems methods, such as fast-slow decomposition and bifurcation and phase-plane analysis, we demonstrate and explain how the presence of a slow negative feedback associated with a gradual reduction of a fast inward current in these models helps to maintain the presence of spikes within the active phases of bursts. Therefore, although such a negative feedback is not necessary for burst production, we find that its presence generates a robustness that may be important for function.Supplementary InformationThe online version contains supplementary material available at 10.1007/s10827-023-00846-y.