Broyles entered coaching in 1947 as an assistant coach under head coach Bob Woodruff at Baylor University. In 1950, Broyles followed Woodruff when the latter took the head coach position at the University of Florida. In 1951, he left Florida and returned to Georgia Tech as an offensive coordinator under coach Bobby Dodd. Broyles sought the head coaching position at Northwestern University in 1954, and ultimately left Georgia Tech in 1957 when he was offered the position of head coach at the University of Missouri. Broyles stayed at Missouri only one season when he was offered the head coaching job at Arkansas. During his nineteen years as head coach there, he was offered other major coaching and leadership positions, but remained at Arkansas.