Spectacular byelection results make exciting news, but they can tell unreliable stories. Byelections tend to be poor predictors of general elections. Seats lost in the glare of a midterm spotlight are often soon regained. But there are exceptions to every rule. I think the Crewe and Nantwich byelection is one of them.

The most striking fact about Thursday's result - the first Conservative byelection win from Labour in 30 years - is that it is not out of sync with the other evidence. On the contrary, it is an unexpectedly emphatic confirmation of what we already know. The May 1 local elections showed the Tories 20 points ahead of Labour. The May opinion polls have consistently reported large Tory leads - the Guardian's ICM poll this week put the gap at 14 points. Gordon Brown's personal ratings are in freefall. Against that backcloth any result in Crewe other than a healthy Tory win would have been surprising.

Only one thing of importance has happened in the weeks since the local elections that might have got Labour out of jail this week. Alistair Darling's £2.7bn giveaway was very important indeed. It was a decisive correction of the 10p tax rate abolition shambles that he inherited from Brown's final budget. Ministers genuinely expected an electoral dividend from the largesse. Instead there was none at all. The citizens of Crewe and Nantwich simply pocketed their tax rebate and voted Tory. If anything, the mini-budget probably lost Labour votes rather than gaining them.

This surely marks a watershed. The combined evidence from the local elections and Thursday's byelection suggests that people have now stopped listening to what Labour is saying. It is now Labour, not the Tories, who are struggling to get the public's permission to be heard. But unless Labour can get the voters to start listening again, there will be more and worse to come.

Labour governments have been in this position before - and the precedents are not good. Harold Wilson lost a fistful of byelections by double-digit swings in seats such as Dudley or Meriden in 1968, and was swept out of office two years later by Ted Heath. Jim Callaghan had terrible byelection losses at Ashfield and Birmingham Stechford in 1977 and, despite his personal popularity, was drubbed by Margaret Thatcher two years after that. On that basis, Crewe loudly tolls the knell for a third Labour defeat, this time at David Cameron's hands, in 2010.

None of this is set in stone, however. Sometimes, of course, a government can recover from even a humiliating byelection reverse. Think back to 1990 when Labour, then in opposition, captured Mid Staffordshire from the Tories with a swing of 21% that dwarfs even Edward Timpson's 18% swing in Crewe this week. Two years after that byelection, the Tories bounced back to win a fourth successive general election. They did so, however, by replacing their leader.

Gordon Brown's leadership was not the sole reason why Labour lost in Crewe this week, or why it was swept aside in the local elections, or why it is trailing so terribly in the polls. But it is an absolutely integral, undeniable major factor in all three. Until Crewe, the best Tory byelection performance in this parliament was to add a paltry 2% to their vote in Cheadle in 2005, when Michael Howard was still leader. Under Cameron, the byelection record has so far been dismal. Yet on Thursday Timpson added 6,000 new voters to the Tory camp. Something has to explain this striking change in the electoral market since 2007. That something is the experience of Brown as the Labour party leader.

So far, very few MPs have gone public about their lack of confidence in Brown. But, make no mistake, such views are now the norm among increasingly large numbers of consenting backbenchers in private. These backbenchers have finally been pummelled out of their comfort zone by the events of this spring. They now fear Labour cannot win the next general election under Brown's leadership. They say and believe that he has to go. They do not believe either that Brown will change or that - even if he did - voters would any longer pay attention to it. The question that now consumes these MPs is not whether Gordon Brown will step down - but how and when.

In reality there are only two ways of getting rid of any leader. One is to overthrow him. The other is to force him to quit. The trouble with the first option is no cabinet minister can challenge for the leadership without resigning, while any backbencher who attempted to challenge would be dismissed as a wrecker. Few Labour MPs have the taste or the nerve for this. Moreover the process would be bloody and protracted, and its eventual outcome is uncertain. For all these reasons it seems unlikely to happen.

The second option - a coordinated effort to persuade Brown to step down - is another matter. This has some of the advantages that a direct challenge lacks. It is private, it is direct, and it could be quick. Perhaps, just as when MPs attempted to get Michael Foot to quit in March 1983, an initiative may come from the backbenches. But the pivotal players are in the cabinet - as they were in Thatcher's overthrow in 1990.

This does not mean that a majority of the cabinet is about to issue Brown with an ultimatum. But events were beginning to move in that direction yesterday. The key ministers in any such process are those in the middle who have managed to steer clear of the Blair-Brown polarisation. If some or all of Alistair Darling, Jack Straw, Alan Johnson, Des Browne and Geoff Hoon were to call on him to step down, it would be hard for Brown to resist. It would be harder still if these ministers made clear they would resign if he did not. But who is going to be Labour's Geoffrey Howe?

The incipient fall of any leader is always a vertiginous and volatile political moment. The dilemmas are shocking, cruel and tragic. Fear and conscience can easily get the upper hand. But, as John Major calmly points out in his memoirs, events such as those of 1990 are ultimately driven not by panic but by a rational calculation. The undeniable risk of embarking on them is simply outweighed by the even more undeniable risk of not doing so. Labour confronts such a moment today. To do nothing would defy political logic.