Rivera was born on December 22, 1935, in Crystal City, Texas, to Spanish-speaking, migrant farmworkers, Florencio and Josefa Rivera. At eleven years old, Rivera was in a car accident in Bay City, Michigan. After the accident, Rivera decided to write his first story about the wreck and called it "The Accident". In an interview with Juan D. Bruce-Novoa, Rivera explains: "I felt a sensation I still get when I write. I wanted to capture something I would never forget and it happened to be the sensation of having a wreck". Rivera continued writing throughout high school, creative pieces as well as essays. He dreamed of being a sportswriter as an adult, inspired by what he read most, sports articles and adventure stories. In the same article, Rivera explains the reality of growing up with ambitions to be a writer in a migrant worker family. He explains that "When people asked what I wanted to be, I'd tell them a writer. They were surprised or indifferent. If people don't read, what is a writer?". His grandfather was his main supporter, though, and provided him with supplies and encouragement.