Banks labelled "too big to fail" must have a so called "living will" that would allow them to close down without the help of public money.
Bank of America, Bank of New York Mellon, JPMorgan Chase, State Street and Wells Fargo have until 1 October to submit improved plans.
JP Morgan said it was "disappointed with the conclusion".
"The most important thing is that we work with our regulators to understand their feedback in more detail.  And we are fully committed to meeting their expectations," said JP Morgan's chief financial officer Marianne Lake.
The Federal Deposit Insurance Corporation (FDIC) and the Federal Reserve board jointly rejected the banks' plans.
If they fail to come up with improved plans they could face "more stringent" requirements.
The FDIC said that Goldman Sach's plan "was not credible", while the Federal Reserve came to the same conclusion about Morgan Stanley's plans.
Regulators have been criticised for not providing enough information on how the plans have been judged.
On Tuesday, the Government Oversight Committee (GAO), a bi-partisan group, said financial regulators were not providing banks with enough information about how the plans were reviewed.
"Without greater disclosure, companies lack information they could use to assess and enhance their plans," the GAO report said.
The GAO recommended that "the FDIC and the Federal Reserve publicly disclose information about their assessment frameworks".
US regulators have rejected the plans of five big US banks for closing their operations in the event of a crisis.