Media reports claimed some players questioned Sam Burgess' World Cup selection over Luther Burrell.
The RFU confirmed a pre-tournament row occurred between Danny Cipriani and coach Mike Catt and is investigating claims kit man Dave Tennison urged players to buy particular shares.
"The trust has gone," Brown told the Daily Telegraph.
"I don't think anyone was good enough in an England shirt to be piping up saying: 'This was wrong, that was wrong' and that sort of thing.
"If they are going to say something, I think they should put their name to it.
"It will come out who it was at the end of the day so it will be interesting. When it does I will be one of the first people to speak to them."
England failed to make the tournament's knock-out stages for the first time after Pool A defeats by Wales and Australia.
A five-man panel is reviewing England's showing in the tournament, with head coach Stuart Lancaster under scrutiny.
England are due to reassemble in January before their Six Nations opener away to Scotland on 6 February.
Brown said that relationships within the squad would be rebuilt before the trip to Murrayfield.
"It is going to be hard for me to call anyone team-mates until we meet up," he added.
"It is going to be tough. But it will all get sorted out with whoever is in charge or whoever is there. It always does in team environments."
England full-back Mike Brown says he has lost all trust in his team-mates after leaks from the dressing room.