
> > > > Have you seen my example ? If calculated in float4 the result of
> > > > 1.00000000000001*1000.0-1000.0 would be 0.0, no ? 
> > > 
> > > So?  If you are storing one input as float4, then you cannot rationally
> > > say that you know the result to better than 6 digits, because you don't
> > > know the input to better than 6 digits.  Claiming that 1000.00000000001
> > > is a more accurate answer for the product than 1000.0 is simply wishful
> > > thinking on your part: nothing to the right of the sixth digit actually
> > > means a darn thing, because you don't know whether the input was really
> > > exactly 1000, or should have been perhaps 1000.001.
> > 
> > I still see 1E-10 as a better answer to above calculation than your 0,
> > and my snapshot 9/11 does return that 1E-10.
> 
> Well, then you'd be wrong. Numerical analysis says you _can't_ get more
> information out than went in to the _least_ precise part of a calculation.
> What your suggesting is the equivalent of wanting to put up a shelf, so
> you estimate the length of the wall by eyeballing it, then measure the
> wood for the shelf with a micrometer, to be sure it fits 
> exactly right.
 
> We teach this in intro science classes all the time: if you calculate with
> 3.14 as an approximation to pi, you better not report the circumference
> of a circle as 2.45678932 cm, I'll take off points!

What if he must display 9 digits and says the result is approximately 2.45678932
would that be worse than 2.46000000 ? 
That is what I am trying to say. Probably the standard is meant as a hint for db 
users, that such results will be approximate, not where the first digit sits that 
is not exact any more.

For above calculation pg will in the future return 0.00000000000000000000 as an
answer to 1.00000000000001*1000.0-1000.0 when used in my example context, while
it currently returns 0.000000000010 ... 
You both are saying, that 0.00000000000000000000 is a better answer. 

Andreas

Thu, 19 Sep 2002 18:00:37 +0200
"Zeugswetter Andreas SB SD" <ZeugswetterA@spardat.at>
Re: Proposal for resolving casting issues
On Thu, Sep 19, 2002 at 06:00:37PM +0200, Zeugswetter Andreas SB SD wrote:
> 
> What if he must display 9 digits and says the result is approximately 2.45678932
> would that be worse than 2.46000000 ? 

Yup. Trailing zeros are not significant. That's why scientific notation is nice:
you don't fill in all those insignificant placeholders.

> 
> For above calculation pg will in the future return 0.00000000000000000000 as an
> answer to 1.00000000000001*1000.0-1000.0 when used in my example context, while
> it currently returns 0.000000000010 ... 
> You both are saying, that 0.00000000000000000000 is a better answer. 

That's right. And correct, as well.

Ross

Thu, 19 Sep 2002 11:05:39 -0500
"Ross J. Reedstrom" <reedstrm@rice.edu>
Re: Proposal for resolving casting issues
"Zeugswetter Andreas SB SD" <ZeugswetterA@spardat.at> writes:
> For above calculation pg will in the future return 0.00000000000000000000 as an
> answer to 1.00000000000001*1000.0-1000.0 when used in my example context, while
> it currently returns 0.000000000010 ... 
> You both are saying, that 0.00000000000000000000 is a better answer. 

Not exactly: we are saying it is not a worse answer.  There's no reason
to prefer one over the other, because they are both within the range
of uncertainty given the inherent uncertainty in the float4 input.

If you want exact results, you should be using exact datatypes.

			regards, tom lane

Thu, 19 Sep 2002 12:22:25 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: Proposal for resolving casting issues 