After setting a new Irish indoor 1500m record last weekend, the Portaferry runner led the pack chasing leader Meseret Defar early on before exiting.
"To say I'm disappointed is an understatement," said Mageean on Twitter after Sunday's race.
"I will dust myself off for there is so much more to come."
Twice Olympic champion Defar and a couple of early pacemakers broke out from the off and the Ethiopian had nearly 26 seconds to spare from American runner-up Abbey D'Agostino at the finish as she won in 8:30.83.
While a fast time, Defar's mark was still seven seconds outside her personal best set nine years ago, which then represented a world record.
Mageean set her indoor 3000m best of 8:55.09 last year.
The Portaferry athlete's new Irish 1500m mark of 4:08.66 in Karlsruhe earlier this month qualified her for next month's World Indoor Championships in Oregon, although she has still to confirm that she will compete in the United States.
Mageean has already booked her place in the 1500m at the Rio Olympics after the IAAF revised the qualifying standard for the event before Christmas.
Ciara Mageean's build-up to the Olympics endured a setback as she dropped out of the 3000m at the New Balance Indoor Grand Prix in Boston.