After Zwan's breakup, Sweeney played guitar with Bonnie "Prince" Billy for several tours in 2004. January 2005 saw the release of their collaboration "Superwolf" on Drag City. Sweeney and Bonnie toured behind the Superwolf record in the U.S. and Europe. 2005 also saw Sweeney producing Brooklyn, NY heavy metal band Early Man's debut album, "Closing In". The following year found Sweeney working in both the country music and hip-hop scenes, providing guitar work, along with Mike Campbell and Smokey Hormel on the Johnny Cash album, "" and appearing with Yo La Tengo bass player James McNew and Def Jux founder El-P, as guest artists on the first track of rapper Cage's album "Hell's Winter". He also played guitar on the Dixie Chicks "Taking the Long Way" album. In addition, he became a member of apocalyptic Christian folk legends Current 93 touring group, playing shows in Europe in support of the "Black Ships Ate The Sky" album. He also recorded with long-time friend Andrew W.K. for W.K.'s "Close Calls With Brick Walls" album.