Smith's approach to writing lyrics reflected her background as a poet, drawing inspiration from sources such as the French Symbolism movement and the works of Charles Baudelaire, William Blake, and Smith's lifelong idol Arthur Rimbaud. CMJ writer Steve Klinge found that the lyrics on "Horses" recalled the "revolutionary spirit" of Rimbaud and the energy of Beat poetry. Author Nick Johnstone suggested that Smith's style of writing lyrics from a gender-ambiguous perspective was informed by her teenage readings of Rimbaud's work, which convinced Smith that "each person contains as much female as male".