Both of Jordan's biographers relate a story told by Sid Boyum, which places the inspiration for the house in a meeting between Alex Jordan Jr. and Frank Lloyd Wright, at some unspecified time between 1914 and 1923. Jordan Sr. supposedly drove with Boyum to Taliesin to show Wright the plans for a building, the Villa Maria in Madison. Jordan idolized the famous architect and hoped for his approval. Wright looked at the plans and told Jordan: "I wouldn't hire you to design a cheese crate or a chicken coop. You're not capable." Fuming, on the drive back on Highway 23, Jordan pointed to a spire of rock and told Boyum: "I'm going to put up a Japanese house on one of those pinnacle rocks and advertise it."