Ross Allum scored twice in two minutes to give the SPFL newcomers a surprise lead but the Premiership side hit back to prevail 4-2.
Highland League Cove Rangers dished out a 3-0 lesson to Montrose, leapfrogging the League Two side in Group D.
Cowdenbeath picked up their first points of the competition with a 2-0 win over Arbroath.
Last season's Lowland League champions were 2-0 up after 19 minutes thanks to striker Allum.
However, Ali Crawford and Alex D'Acol made it four goals inside nine minutes to restore parity at Meadowbank.
Crawford hammered in a long-range free-kick and D'Acol was on hand steer home a deflected Louis Longridge strike.
Teenage defender Jordan McGregor netted his first senior goal on 66 minutes, diverting in a low delivery from Longridge and Dougie Imrie slammed a penalty high into the net after a foul on Grant Gillespie.
The Accies are now second in Group G, three points behind leaders Ayr United, who have completed their four matches.
Hamilton, who have a better goal difference than Ayr, host Livingston on Saturday, while St Mirren are at home to City hoping a big win could take them to the top of the section.
Cowdenbeath moved off the foot of Group C thanks to first-half goals from Craig Johnston and Bulgarian teenager Nikolay Todorov, who signed on loan from Hearts earlier in the day.
Hopes of an Arbroath comeback were extinguished when David McGurn saved a Steven Doris penalty after the interval.
Montrose suffered a chastening loss at the home of the Highland League champions, with Stuart Walker, Mitch Megginson and Daniel Park on target.
Hamilton Academical recovered from 2-0 down to avoid an embarrassing League Cup result at Edinburgh City.