Crabbe was born on February 7, 1908 in Oakland, California to Edward Clinton Simmons Crabbe and Lucy Agnes McNamara Crabbe. His father was a real estate broker. He was named Clarence Linden Crabbe after his grandfather. Clarence had a brother named Edward Clinton Simmons Crabbe II who was a year younger. He was named for their father. Clarence was nicknamed "Buster" and Edward II was nicknamed "Buddy". Both boys kept these nicknames for the rest of their lives.