i began to grow increasingly intolerant of people who wanted to join me and have a say in how the house or the mill should be built , yet paid no rent and could walk away whenever it suited them , leaving me to pick up the pieces .