Aslef's Simon Weller said there was an "utter lack of trust" between his union's members and Southern.
Paul Cox, of the RMT union, questioned Aslef's leadership after the latest result, accusing its bosses of being "out of touch with reality".
The RMT is due to hold talks with Southern later, ahead of a planned strike this weekend.
The RMT talks follow the rejection of a proposed agreement to the long-running row over driver-only operated trains by members of drivers' union Aslef.
The narrow 51.8% vote against a settlement set back hopes of ending the deadlock which has caused travel misery for passengers.
On 8 April - Grand National weekend - 2,000 RMT members for Southern rail, Merseyrail and Northern rail plan to walk out for 24 hours.
Speaking on BBC Surrey earlier, Mr Cox said: "What is surprising is Aslef are going to go back and start talks again.
"Their members are quite clearly telling them they have problems with this driver-only-operated operation.
"They want them to do something about it, not to keep fiddling around and keep tweaking a deal that's been rejected twice."
Mr Weller said: "We are trying to ensure there are second people on the train, safety trained, safety critical, to make sure that everyone has a decent service."
The year-long row is over the Southern giving drivers responsibility for operating the doors, and changing the role of guards to on board supervisors which is a less safety-critical role.
The rail firm claims it always aims to keep the supervisor on board as a second member of staff, apart from in exceptional circumstances.
The deal negotiations have focused on reducing those exceptions.
A breakdown of trust has caused train drivers to reject a second deal with Southern rail, a union has said.