Upon his release, he emigrated to the United States where he clung to various priests as an organist. He felt called to the religious life and joined the Franciscan order in Pulaski, Wisconsin, but was expelled and moved to Manitowoc, Wisconsin, where he worked in various menial jobs. He was organist at the independent Sweetest Heart of Mary Church in Detroit, Michigan (which Vilatte consecrated in 1893) but later quarreled with and wrote in newspapers against the pastor, , and left.
After his release from prison, he moved to the United States and worked as an organist for various priests. He felt drawn to religious life and joined the Franciscan order in Wisconsin, but was expelled and then went on to work various jobs before becoming the organist at a church in Michigan.