
Flights of fancy, and other irrational approaches, are common.  The crucial
thing is not to sit around just having fantasies; they aren't of any use
unless they make you do some experiments.  I've known a lot of scientists
whose fantasies lead them on to creative work; usually they won't admit
out loud what the fantasy was, prior to the consumption of a few beers.

(Simple example: Warren Jelinek noticed an extremely heavy band on a DNA
electrophoresis gel of human ALU fragments.  He got very excited, hoping that
he'd seen some essential part of the control mechanism for eukaryotic
genes.  This fantasy led him to sequence samples of the band and carry out
binding assays.  The result was a well-conserved, 400 or so bp, sequence
that occurs about 500,000 times in the human genome.  Unfortunately for
Warren's fantasy, it turns out to be a transposon that is present in
so many copies because it replicates itself and copies itself back into
the genome.  On the other hand, the characteristics of transposons were
much elucidated; the necessity of a cellular reverse transcriptase was
recognized; and the standard method of recognizing human DNA was created.
Other species have different sets of transposons.  Fortunately for me,
Warren and I used to eat dinner at T.G.I. Fridays all the time.)


I'm not sure that it's random.  But there is no known rational mechanism
for generating a rich set of interesting hypotheses.  If you are really
working in an unknown area, it is unlikely that you will have much sense
of what might or might not be true; under those circumstances, the best
thing to do is just follow whatever instincts you have.  If they are wrong,
you will find out soon enough; but at least, you will find out _something_.
If you try to do experiments at random, with no prior conceptions at all
in mind, you will probably get nowhere.


Unfortunately, the critical function does sometimes become hostage to
non-rational forces.  Then we get varieties of pathological science:
Lysenko, Mirsky's opposition to DNA-as-gene, cold fusion, and so forth.


I don't agree that this follows.  In fact, this is _exactly_ the point at
which I disagree with Feyerabend.  It is a most important part of the
culture of science that one keeps one's jealousies out of the refereeing
process.  Failures there are aplenty, but, on the whole, things work out.

Another point: there are a couple of senses of the phrase ``experimental
design''.  I'd say that the less rational part is in experimental _choice_,
not design.  Alexander Fleming (Proc. Royal Soc., 1922) chose to look for
bacteriophage in his own mucus for strange reasons (Phage had previously
been found in locust diarrhea; Fleming probably thought runny bottom, runny
nose, what the hell, it's worth a try.) but his method of looking for phage
was well-designed to detect anything phage-like; in fact, he found lysozyme.


It is not clear to me what you mean by rational vs. irrational.  Perhaps
you can give a few examples of surprising experiments that were tried out
for perfectly rational reasons, or interesting new theories that were first
advanced from logical grounds.  The main examples I can think of are from
modern high-energy physics which is not typical of science as a whole.