The song is in the key of B major and begins with a "simple two-chord" riff (F–E), which Buckley believes hooks the listener instantly. Doggett calls the riff "reminiscent"–albeit "in very different circumstances"–to the title track of "Aladdin Sane". According to Buckley, Bowie was looking to emulate something of the glitzy nostalgia of "On Broadway", which he was playing on piano in the studio when he came up with "Golden Years". Doggett writes: "The magical ingredients were percussive: the rattling of sticks against the hi-hat cymbal from the start, the startling clack of woodblocks, the sudden drum fills." These combined elements "channel" the spirit of Elvis Presley in the verses with a "haughtier, more strident tone" in the chorus.