The pair were caught buying cocaine near the Champs Elysees avenue at around 03:00 (02:00 GMT) on Saturday morning, police sources said.
Williams, 35, plays for Racing 92 and O'Connor, 26, is at Toulon.
Last October two other New Zealanders playing for Racing 92 were cleared of doping charges.
The side are current champions of French rugby.
Two dealers are also in custody.
Racing 92 said in a statement (in French) that it had suspended Williams "temporarily".
"If the investigation confirmed the possession of cocaine and the transaction, it would not only be against the law but also a serious breach of our ethics," the club said.
Williams earned 77 New Zealand caps between 2002 and 2012.
O'Connor has made 44 appearances for Australia between 2008 and 2013.
Two former rugby internationals, New Zealander Ali Williams and Australian James O'Connor, have been arrested in Paris on drugs charges.