Allin's earliest musical influences were 1960s British Invasion bands including the Beatles, the Rolling Stones, and the Dave Clark Five. In the early 1970s, Alice Cooper became an important influence on Allin. Allin's earliest recorded musical endeavors were as a drummer, he also wrote most of his songs on an acoustic guitar. In his mid-teens, he and his older brother Merle, who plays bass guitar, formed their first band, Little Sister's Date, which lasted a little over a year. The group covered songs by Aerosmith, Kiss, and other popular rock bands of the time period. Both Allin and Merle gained a strong interest in punk rock. The Ramones and the Stooges were major influences on Allin.