Midway through his second season, Miller was traded to the Cleveland Browns, and finished the 1975 season with only 13 carries. Miller earned a starting role in Browns' backfield for the 1976 season, a job he would hold through 1979. In 1976 Miller ran for 613 yards and four touchdowns (4.0 avg.) as a fullback blocking for All-Pro running back Greg Pruitt. The Browns finished the season with a record of 9-5, but failed to make the playoffs despite ranking 9th in the league in rushing.