Sometimes they build their colonies around the roots of plants. This isn’t generally a problem. However, there might be a conflict of interest if they decide to live in the plant you have in your living room or next to the swing in your garden.