In 1954, he was the International League's All-Star shortstop, then he batted a career-best .301 in 1955. But he was not able to break into the Dodger lineup, which featured future Baseball Hall of Famer Pee Wee Reese as shortstop and anchor of its infield. After 3 years at Triple-A, Fernández finally was recalled to Brooklyn in July 1956.
Fernández's impressive performance with the International League earned him an All-Star selection in 1954, followed by a remarkable .301 batting average in 1955. However, he struggled to secure a spot in the Dodger lineup, which was led by future Hall of Famer Pee Wee Reese as shortstop and infield anchor. After three years of development at Triple-A, Fernández's perseverance paid off when he was finally promoted to Brooklyn in July 1956.