Ramírez began his minor league career in 2003 with the Tampa Yankees of the single-A advanced Florida State League. After he had a 2–8 record, a 5.21 earned run average (ERA), 21 strikeouts, and innings pitched in 14 starts, he was promoted to the Trenton Thunder of the Double-A Eastern League. With the Thunder, he had a 1–1 record, a 1.69 ERA, 21 strikeouts, and innings pitched in four games (three starts), and this earned him a promotion to the Columbus Clippers of the triple-A International League. In two games (one start) with Columbus, he had an 0–1 record, a 4.50 ERA, five strikeouts, and six innings pitched. He finished his first minor league season with a 3–10 record, a 4.43 ERA, 96 strikeouts, and innings pitched in 20 games (18 starts).