The body of Michael Collings, 53, was recovered but Ken Cresswell, 57, John Shaw, 61, and Chris Huxtable, 34, were trapped when it came down in February.
Their bodies were not found until earlier this month.
John Cosgrove, 65, of Rhoose, Vale of Glamorgan, was 98ft (30m) away when the incident happened.
"All of a sudden, I heard a noise, like a rumbling noise. At first I thought it was one of the scrap lorries getting loaded up with scrap metal," he said.
"And then it got louder and then the Portakabins started shaking violently.
"And it finished and I opened the door and then I knew something serious had happened because I couldn't see a thing in front of me, so I shut the door straight away.
"The dust - probably 50 years worth of dust. I just stepped outside and I just saw one of the lads.
"I don't know, I don't know who to this day, he was on the floor and I think he was crying."
Mr Cosgrove met Mr Huxtable, of Swansea, when they worked together to dismantle a battleship two years ago and described him as a "smashing lad".
After the collapse, Mr Cosgrove and his colleagues helped search the rubble before the recovery operation was later taken over by a different firm.
He said that was difficult for him and his colleagues because they wanted to be the ones to find their friends.
"We wanted to be there. We wanted to get our pals out," he said.
"We felt cheated because they were our pals under there and we wanted to get them out. The guard of honours for our pals - the last two weeks - we wanted to be at that guard of honour, but unfortunately not."
The remaining section of the Didcot A plant was brought down using explosives in July.
An investigation by Thames Valley Police and the Health and Safety Executive is continuing to establish the circumstances of the men's deaths.
A Didcot Power Station worker has told how he escaped the partial collapse of its boiler house which killed four colleagues because he was on a break.