Ellison spent most of the remainder of his career shuttling between cowboy pictures and more varied roles, primarily in B movies like Mr. District Attorney in the Carter Case and The Undying Monster. He had a supporting role in 1941's "Charley's Aunt" (which starred Jack Benny) and played the romantic lead in 1943's "The Gang's All Here", a Twentieth Century Fox musical in which he seemed somewhat lost among the vivid antics of Carmen Miranda, Charlotte Greenwood, and Edward Everett Horton (and was the only principal not to sing a note). He also co-starred with Tom Conway and Frances Dee in Val Lewton's production of "I Walked with a Zombie", directed by Jacques Tourneur.