Looking for someone to take his passport photo, he encountered photographer and filmmaker Rudy Burckhardt in Switzerland in 1934, and the two remained inseparable for the rest of Denby's life. The following year, they returned to New York City, New York, and rented a loft for eighteen dollars a month in a five-story walk-up building on West 21st Street in Chelsea. Denby's friendship with painter Willem de Kooning, who lived one floor below in the adjacent building, began shortly thereafter when de Kooning's kitten turned up on the fire-escape outside of Denby's window one evening.