I got tired of wondering what timing dependencies might still be lurking
in the parallel regression tests, and wrote a little Perl script to find
out by making variant versions of the parallel_schedule file.

Essentially, the script forces each test in a parallel group to run
before all the other ones in its group, and also after all the other
ones.  This isn't a completely bulletproof check: you could imagine
that test A might be unhappy about some intermediate state created by
test B, while not being unhappy with either the starting or ending
states.  But it's a lot better than guess-and-hope.

The script successfully exposed the problem reported earlier today by
Robert Hentosh (create_index test depends on create_misc having run),
and didn't find any other problems, which I guess is a good sign.
(I've now committed a fix for that mistake, BTW.)

I'm not sure if the script has any long-term usefulness (anyone feel it
deserves to get into CVS in src/tools/?).  But I'll attach it anyway
just so it gets into the pghackers archives.

To use it you'd do something like

mkdir scheds
./sched_variants parallel_schedule scheds/sch
for f in scheds/sch*
do
	echo $f
	/bin/sh ./pg_regress --schedule=$f
	ff=`basename $f`
	mv regression.out scheds/regression.out.$ff
	mv regression.diffs scheds/regression.diffs.$ff
done


			regards, tom lane


#! /usr/bin/perl

# Generate variants of parallel_schedule file to verify that there are
# no order dependencies between tests executed in parallel.

# Usage: sched_variants input_file output_prefix

# Output files are named with 'output_prefix' suffixed .1, .2, etc.

die "Usage: sched_variants input_file output_prefix\n" if ($#ARGV != 1);

$infile = $ARGV[0];
$outprefix = $ARGV[1];

$outcount = 0;			# number of output files created

# We scan the input file repeatedly.  On each pass we generate two
# output files, one where the k'th entry of each parallel test set
# has been extracted and forced to run first, and one where it's been
# forced to run last.  The number of passes needed is the same as the
# largest number of tests in a parallel test set.

$k = 1;				# test index we are currently hacking

$more = 1;			# true if we need another pass

while ($more) {
    $more = 0;			# until proven differently

    open(INFILE, $infile) || die "$infile: $!\n";

    $outcount++;
    $outbefore = $outprefix . "." . $outcount;
    open(OUTBEFORE, "> $outbefore") || die "$outbefore: $!\n";

    $outcount++;
    $outafter = $outprefix . "." . $outcount;
    open(OUTAFTER, "> $outafter") || die "$outafter: $!\n";

    while (<INFILE>) {
	if (! /^test:/) {
	    # comment line
	    print OUTBEFORE $_;
	    print OUTAFTER $_;
	    next;
	}
	@tests = split;
	shift(@tests);		# remove test:
	if ($#tests < $k-1 || $#tests == 0) {
	    # too few tests in this set, just repeat as-is
	    print OUTBEFORE $_;
	    print OUTAFTER $_;
	    next;
	}
	if ($#tests >= $k) {
	    $more = 1;		# need more passes to process this set
	}
	@thistest = splice(@tests, $k-1, 1);
	print OUTBEFORE "test: @thistest\n";
	print OUTBEFORE "test: @tests\n";

	print OUTAFTER "test: @tests\n";
	print OUTAFTER "test: @thistest\n";
    }

    close OUTBEFORE;
    close OUTAFTER;
    close INFILE;

    $k++;
}

print "$outcount test files generated.\n";

exit 0;
Tue, 22 Oct 2002 16:31:59 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Silly little tool for making parallel_schedule variants