not the lifeless city townhouse where he had earlier made his flight but his old house where his wife had lived , where browns son had joined them , spending his first few years of life in their house .