This's great.  I have tested Postgres and MySQL with the benchmark
shipped with mysql and (of course) MySQL out perform Postgres.

I wonder does anyone know where can we download the ANSI SQL benchmark
(AN3AP) suite?  I'd like to run this benchmark test myself, since this
is the only benchmark I have read so far that Postgres out perform
MySQL.

Thanx in advance.

>
> Did someone read bout this?
>
> http://www.angelfire.com/nv/aldev/pgsql/GreatBridge.html
>


--
Limin Liu




This's great.  I have tested Postgres and MySQL with the benchmark
shipped with mysql and (of course) MySQL out perform Postgres.
I wonder does anyone know where can we download the ANSI SQL benchmark
(AN3AP) suite?  I'd like to run this benchmark test myself, since
this is the only benchmark I have read so far that Postgres out perform
MySQL.
Thanx in advance.
>
> Did someone read bout this?
>
> http://www.angelfire.com/nv/aldev/pgsql/GreatBridge.html
>
 
-- 
Limin Liu
Mon, 13 Nov 2000 10:24:21 -0800
Limin Liu <limin@pumpkinnet.com>
(download ANSI SQL benchmark?) Re: Postgres article
(bcc to -hackers)

The Benchmark Factory software, recently acquired by Quest Software,
can be downloaded for a 30-day trial (limited # of users) at:

http://www.benchmarkfactory.com/emaildatabase/FormBenchmarkFactoryTrial.asp?product=BenchmarkFactory

(note long URL above might have wrapped)

That's the package we used - also the one that eWeek (formerly
PCWeek) and other trade magazines use.  See
http://www.greatbridge.com/about/press.php?content_id=4 for more
info on how we conducted the test.

Regards,
Ned

Limin Liu wrote:

> This's great.  I have tested Postgres and MySQL with the benchmark
> shipped with mysql and (of course) MySQL out perform Postgres.
>
> I wonder does anyone know where can we download the ANSI SQL
> benchmark (AN3AP) suite?  I'd like to run this benchmark test
> myself, since this is the only benchmark I have read so far that
> Postgres out perform MySQL.
>
> Thanx in advance.
>
> >
> > Did someone read bout this?
> >
> > http://www.angelfire.com/nv/aldev/pgsql/GreatBridge.html
> >
>
>
> --
> Limin Liu
>
>


Mon, 20 Nov 2000 22:35:40 -0500
Ned Lilly <ned@greatbridge.com>
Re: [HACKERS] (download ANSI SQL benchmark?) Re: Postgres article
Mon, 20 Nov 2000 20:56:17 -0800
Don Baccus <dhogaza@pacifier.com>
Re: [HACKERS] (download ANSI SQL benchmark?) Re: Postgres
  article
Don Baccus writes:
 > I also hope that the PG crew, and Great Bridge, never stoop so low
 > as to ship benchmarks wired to "prove" PG's superiority.

I thought that Great Bridge's August benchmarks were rather skewed.
They only used one particular test from the AS3AP suite.  That was the
basis for their headline figure of 4-5 times the performance of the
competition.

I was however impressed by the TPC-C results.  MySQL and Interbase
were unable to complete them.  PostgreSQL showed almost identical
performance over a range of loads to Proprietary 1 (version 8.1.5, on
Linux) and Proprietary 2 (version 7.0, on NT).
-- 
Pete Forman                 -./\.- Disclaimer: This post is originated
Western Geophysical           -./\.-  by myself and does not represent
pete.forman@westgeo.com         -./\.-  the opinion of Baker Hughes or
http://www.crosswinds.net/~petef  -./\.-  its divisions.

Tue, 21 Nov 2000 10:19:53 +0000 (GMT)
Pete Forman <pete.forman@westgeo.com>
Re: (download ANSI SQL benchmark?) Re: Postgres
  article
At 10:19 AM 11/21/00 +0000, Pete Forman wrote:
>Don Baccus writes:
> > I also hope that the PG crew, and Great Bridge, never stoop so low
> > as to ship benchmarks wired to "prove" PG's superiority.
>
>I thought that Great Bridge's August benchmarks were rather skewed.
>They only used one particular test from the AS3AP suite.  That was the
>basis for their headline figure of 4-5 times the performance of the
>competition.
>
>I was however impressed by the TPC-C results.  MySQL and Interbase
>were unable to complete them.  PostgreSQL showed almost identical
>performance over a range of loads to Proprietary 1 (version 8.1.5, on
>Linux) and Proprietary 2 (version 7.0, on NT).

Great Bridge didn't do the benchmarking, they hired a third party to
do so.  And that third party didn't, AFAIK, cherry-pick tests in order
to "prove" PG's superiority.

The report itself mentioned the testing group's surprise over MySQL's
poor showing in the simple, non-TPC-C test.  I'm sure it was tossed
in so they could answer the question "how much does it cost you to
use a transaction-based system rather than MySQL", since avoiding that
overhead is the big argument that the MySQL makes in favor of their
product.  I'm sure the hope was there that the answer would be "not all
that much", instead the answer was "gee, you're not that fast after
all".

Clearly the real target of the benchmark effort was Oracle.  However
inadequate the benchmarking effort might've been (they're all inadequate,
after all) the fact is that Great Bridge at least did run a set of
standard benchmarks.

The MySQL folk have always cherry-picked their benchmarks, long lied
about features in PG, do their benchmarking using default values
for PG's shared buffer etc WITHOUT TELLING PEOPLE while at the same
time installing MySQL with larger-than-default memory usage limits (the
group hired by GB used MySQL's default installation, but EXPLICITLY SAID
SO in the report), etc.

The GB-financed benchmarks weren't perfect, but they weren't dishonest.
The MySQL folks have done things over the years that have been out-and-out
dishonest, IMO.



- Don Baccus, Portland OR <dhogaza@pacifier.com>
  Nature photos, on-line guides, Pacific Northwest
  Rare Bird Alert Service and other goodies at
  http://donb.photo.net.

Tue, 21 Nov 2000 06:33:35 -0800
Don Baccus <dhogaza@pacifier.com>
Re: (download ANSI SQL benchmark?) Re: Postgres
  article
Pete Forman <pete.forman@westgeo.com> writes:
> I thought that Great Bridge's August benchmarks were rather skewed.
> They only used one particular test from the AS3AP suite.

AFAIK there was nothing particularly sinister about that --- they
didn't have time to run a large number of different tests, so they
chose ones that seemed most important.  They certainly didn't try
a bunch of tests and then publish only the most favorable; the two
tests used were selected at the beginning of the project, before
anyone knew what the results would look like.

			regards, tom lane

Tue, 21 Nov 2000 10:29:06 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: [HACKERS] (download ANSI SQL benchmark?) Re: Postgres article 
Don Baccus <dhogaza@pacifier.com> writes:
> Great Bridge didn't do the benchmarking, they hired a third party to
> do so.  And that third party didn't, AFAIK, cherry-pick tests in order
> to "prove" PG's superiority.

In fairness, the third party was Xperts Inc, who have long done a lot
of programming-related work for Landmark Communications; so there's a
pretty close working relationship, it's not exactly arms-length.

I think what may be more worth noting is that that benchmarking project
was started as part of Landmark's "due diligence" investigation while
deciding whether they wanted to bet a company on Postgres.  They didn't
go into it with the notion of proving Postgres superior; they went into
it to find out if they were betting on a dog.  They were very pleasantly
surprised (as was the core group, when we first saw the results!).
Naturally, their marketing guys said "hey, let's clean this up and
publish it".  There's a certain amount of after-the-fact selection here,
since you'd certainly never have seen the results if they hadn't been
favorable to Postgres; but there was no attempt to skew the results in
Postgres' favor.  If anything, the opposite.

> The MySQL folk have always cherry-picked their benchmarks, long lied
> about features in PG, do their benchmarking using default values
> for PG's shared buffer etc WITHOUT TELLING PEOPLE while at the same
> time installing MySQL with larger-than-default memory usage limits (the
> group hired by GB used MySQL's default installation, but EXPLICITLY SAID
> SO in the report), etc.

The revised results that are on GB's site now include curves for MySQL
*with* tuning per advice from the MySQL folk.

			regards, tom lane

Tue, 21 Nov 2000 10:58:35 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: [HACKERS] (download ANSI SQL benchmark?) Re: Postgres article 
At 10:58 AM 11/21/00 -0500, Tom Lane wrote:

>> The MySQL folk have always cherry-picked their benchmarks, long lied
>> about features in PG, do their benchmarking using default values
>> for PG's shared buffer etc WITHOUT TELLING PEOPLE while at the same
>> time installing MySQL with larger-than-default memory usage limits (the
>> group hired by GB used MySQL's default installation, but EXPLICITLY SAID
>> SO in the report), etc.
>
>The revised results that are on GB's site now include curves for MySQL
>*with* tuning per advice from the MySQL folk.

That's good.  Have the MySQL folk made any effort to reciprocate?



- Don Baccus, Portland OR <dhogaza@pacifier.com>
  Nature photos, on-line guides, Pacific Northwest
  Rare Bird Alert Service and other goodies at
  http://donb.photo.net.

Tue, 21 Nov 2000 09:51:54 -0800
Don Baccus <dhogaza@pacifier.com>
Re: (download ANSI SQL benchmark?) Re: Postgres
  article