Ramirez was born in Santo Domingo, Dominican Republic to Aristides and Onelcida Ramirez and spent 13 years living there. As a child, Ramirez was obsessed with baseball. When he was eight years old, his grandmother got him a Dodgers uniform with the number 30 on the back, which he considers to be one of his most prized possessions. In 1985, he moved to the Washington Heights neighborhood of New York City with his parents. He often played ball at the nearby Snake Hill, the same place Lou Gehrig played during his childhood. Despite living just a short distance from Yankee Stadium, Ramirez rooted for the Toronto Blue Jays, who had his Dominican heroes George Bell and Tony Fernández on its roster; he attended games when the Blue Jays were in town.