Dann,

> > Thus, the best test
> > team is a bunch of people doing unplanned things with the
> > software, on a wide variety of platforms...
>
> That is the worst possible test plan.  It totally lacks organization and
> there is no hint to define when the feature set has been covered.  Ad
> hoc testing is a useful addition, but it cannot replace all the standard
> tests that have been used by the industry for decades.

Them's fighting words, Mister.   We have a test team ... a good test team ... 
I'm one of them.  I have 7.4 CVS on 2 servers right now, and have loaded them 
with things I doubt Tom ever expected.   You're directly insulting most of 
the people on this list.  What's your motivation, exactly?

Further, results argue against you.   PostgreSQL's "ad-hoc" testing beats the 
formal testing of many large companies -- such as Microsoft.   I 
professionally admin both PostgreSQL and MS SQL Server (which MS puts through 
a multi-million dollar testing proccess) ... guess which has less bugs, and 
less critical bugs?

You're barging on this list, criticizing its members, pointing out problems 
where they don't exist, and not offering any help.   Are you being paid by 
someone who doesn't like PostgreSQL, or do you just enjoy making enemies?

> > Are you volunteering to create it?  Step right up.
>
> No.  And as an outsider, I rather doubt if any procedures I developed
> would be taken very seriously. 

That's a pretty weak-ass excuse.  If you're not offering to create or at least 
*work on* an expanded test suite, why are you wasting our time?

According to you, your company has a elaborate testing procedure with numerous 
staff and (I would assume) some pretty comprehensive testing software.  How 
about a donation?

> A.  Combine:
<snip>
> B.  Automate:
<snip>

These parts sound pretty good.  How about *you* create a test suite that does 
this?

> C.  Assign:
> 	1.  Criteria for acceptance of a build for release
> 	2.  Authority for acceptance of a build for release
> 	3.  Delegation rules for issue resolution
> 	4.  Procedures for issue resolution

This sounds very corporate, and not in keeping with our OSS community. Our bug 
and patch submission process works pretty good right now; why monkey with 
what works?  An OSS project that tries to assign people duties and jobs they 
don't want is  a deserted and dead OSS project.

You may know lots about automated testing, Dann, but you know squat-all about 
Open Source and about community relations.  

-- 
Josh Berkus
Aglio Database Solutions
San Francisco

Sat, 21 Jun 2003 11:24:18 -0700
Josh Berkus <josh@agliodbs.com>
Re: [DEFAULT] Automated testing WAS: Two weeks to feature freeze