Abbott next courageously led his men at the Battle of Fredericksburg. Due to the absence of Colonel Lee, Captain George Macy was acting colonel and Abbott was acting major of the regiment. In preliminary street fighting on the eve of the battle, Abbott was one of the few officers in the 20th Massachusetts Infantry not to be killed or wounded, when he and his men engaged in fierce combat the 21st Mississippi Infantry Regiment commanded by Abbott's Harvard classmate and friend, Lieutenant Lane Brandon. Abbott's men drove Confederate forces away from the river so Union Army engineers could complete pontoon bridges for a crossing in force. Thirty of the 60 men that Abbott had led into combat had been killed or wounded. Overall, 113 of the 335 men in the regiment became casualties that day. The next day, Abbott participated with his regiment in the disastrous Union Army assault on Marye's Heights where waves of Union attackers were cut down by Confederate artillery and infantry protected by a stone wall. Abbott survived unscathed, although a bullet did hit his scabbard. Sixty men and three officers were killed in a matter of minutes in the attack on Marye's Heights, bringing the losses of the 20th Massachusetts Infantry in the battle to 168 men and 8 officers of 335 men and officers engaged. Abbott in large part blamed Republican political leadership for the losses because they had removed Major General George McClellan, a fellow Democrat, from command of the Army of the Potomac.