The UK government is due to publish its Wales Bill soon, transferring new powers and setting out how to make the devolution settlement clearer.
Last week, First Minister Carwyn Jones warned he would reject any attempt to weaken the powers of the assembly.
On Thursday, Mr Crabb told the Western Mail newspaper the chances of an agreement were now "very, very low".
Mr Jones's comments were in response to a report by academics that described the proposals as "convoluted" and "unstable".
The disagreement centres on a move to a Scottish-style reserved powers model, where a list of subjects remains under UK government control, leaving all other matters to be considered devolved.
Wales currently has conferred powers, where the responsibilities of the assembly are listed.
Mr Crabb said:  "Carwyn and some of the academics and some more nationalist lawyers have been talking about a separate legal jurisdiction for Wales.
"The vast, vast majority of the body of law that affects Wales and the interests of Wales is done on an England and Wales basis.
"To somehow try to unpick that now is a massive, massive exercise - hugely costly."
Mr Crabb said the UK government was "having to work really, really hard with the Welsh government in a very pragmatic way but I feel very pessimistic about the chances of reaching an agreement with the Welsh government on this".
A spokesman for Mr Jones said: "We are still willing to work with the UK government on the draft Wales Bill, but we've been absolutely clear with them that what they're offering at the moment won't be acceptable to the people of Wales, or to the national assembly.
"The current proposals threaten to actually roll-back the devolution settlement - there's no mandate for that whatsoever.
"Let's be clear, if we are not satisfied that the Bill meets the needs of Wales, we will not support it."
Welsh Secretary Stephen Crabb has said he is "pessimistic" about agreeing with Welsh ministers on further devolution.