Hi again...

I've discovered (or perhaps re-discovered) what seems to be a memory leak
involving temp tables. It's so bad that I make a script (which runs as a
daemon) close the backend connection and reconnect each time it runs the
offending command sequences.  Without the reset I've seen the backend grow
to over 100 megs in size in a matter of a couple of minutes.

I've created a sample case that reproduces the error that I will attach
with this message.  Basically, I create a 50 column temp table (with no
rows in it) and then run updates on each column in succession.  The
backend gets large pretty quick - I'm seeing about 12Megs after running
the enclosed script which does an update on all 50 columns 3 times (150
updates).

Thanks...

- K

Kristofer Munn * KMI * 973-509-9414 * AIM KrMunn * http://www.munn.com/
Sun, 16 Jan 2000 23:40:44 -0500 (EST)
Kristofer Munn <kmunn@munn.com>
Temp Table Memory Leak
Kristofer Munn <kmunn@munn.com> writes:
> I've created a sample case that reproduces the error that I will attach
> with this message.  Basically, I create a 50 column temp table (with no
> rows in it) and then run updates on each column in succession.  The
> backend gets large pretty quick - I'm seeing about 12Megs after running
> the enclosed script which does an update on all 50 columns 3 times (150
> updates).

I confirm the leak in 6.5.* --- but I see no leak in current sources.

			regards, tom lane

Mon, 17 Jan 2000 00:01:27 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: [HACKERS] Temp Table Memory Leak 
> Hi again...
> 
> I've discovered (or perhaps re-discovered) what seems to be a memory leak
> involving temp tables. It's so bad that I make a script (which runs as a
> daemon) close the backend connection and reconnect each time it runs the
> offending command sequences.  Without the reset I've seen the backend grow
> to over 100 megs in size in a matter of a couple of minutes.
> 
> I've created a sample case that reproduces the error that I will attach
> with this message.  Basically, I create a 50 column temp table (with no
> rows in it) and then run updates on each column in succession.  The
> backend gets large pretty quick - I'm seeing about 12Megs after running
> the enclosed script which does an update on all 50 columns 3 times (150
> updates).

Is this with 6.5.* or 7.0.  If it is 6.5.*, can you try it with our
current tree for testing purposes.  I think you will find it is fixed.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Mon, 17 Jan 2000 00:04:39 -0500 (EST)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: [HACKERS] Temp Table Memory Leak
On Mon, 17 Jan 2000, Tom Lane wrote:
> 
> I confirm the leak in 6.5.* --- but I see no leak in current sources.

Good news for 7.0 but... if anyone has a patch I could (safely) apply to
6.5.3 for this problem, it would be much appreciated.

- K

Kristofer Munn * KMI * 973-509-9414 * AIM KrMunn * http://www.munn.com/


Mon, 17 Jan 2000 00:06:42 -0500 (EST)
Kristofer Munn <kmunn@munn.com>
Re: [HACKERS] Temp Table Memory Leak 
> Kristofer Munn <kmunn@munn.com> writes:
> > I've created a sample case that reproduces the error that I will attach
> > with this message.  Basically, I create a 50 column temp table (with no
> > rows in it) and then run updates on each column in succession.  The
> > backend gets large pretty quick - I'm seeing about 12Megs after running
> > the enclosed script which does an update on all 50 columns 3 times (150
> > updates).
> 
> I confirm the leak in 6.5.* --- but I see no leak in current sources.

Great.  Now the big question is should we backpatch, and if so do we
want a 6.5.4.  I know you(Tom) have put a number of patches into the
6.5.* branch, and we are at least 2 months away from our next release.

Comments?

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Mon, 17 Jan 2000 00:32:27 -0500 (EST)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: [HACKERS] Temp Table Memory Leak
Bruce Momjian <pgman@candle.pha.pa.us> writes:
>> I confirm the leak in 6.5.* --- but I see no leak in current sources.

> Great.  Now the big question is should we backpatch, and if so do we
> want a 6.5.4.

Do you have a low-risk patch for this?  I recall that we did some
fairly extensive changes involving not only temp tables but the regular
relation cache.  Extracting a patch that could be trusted seems like
it might be tough.

> I know you(Tom) have put a number of patches into the 6.5.* branch,
> and we are at least 2 months away from our next release.

I have been throwing low-risk/high-reward fixes into REL6_5 when I
could, with the thought that we might want to do another 6.5.* release.
But I'm undecided on whether we should or not.  It seems like we are
close enough to 7.0 beta cycle that we should focus our effort there.

			regards, tom lane

Mon, 17 Jan 2000 00:44:33 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: [HACKERS] Temp Table Memory Leak 
On Mon, 17 Jan 2000, Bruce Momjian wrote:

> > Kristofer Munn <kmunn@munn.com> writes:
> > > I've created a sample case that reproduces the error that I will attach
> > > with this message.  Basically, I create a 50 column temp table (with no
> > > rows in it) and then run updates on each column in succession.  The
> > > backend gets large pretty quick - I'm seeing about 12Megs after running
> > > the enclosed script which does an update on all 50 columns 3 times (150
> > > updates).
> > 
> > I confirm the leak in 6.5.* --- but I see no leak in current sources.
> 
> Great.  Now the big question is should we backpatch, and if so do we
> want a 6.5.4.  I know you(Tom) have put a number of patches into the
> 6.5.* branch, and we are at least 2 months away from our next release.
> 
> Comments?

I'm all for it...I think that snce ppl have been consciously making an
effort to backpatch as appropriate (aren't CVS branches great? *grin*), we
should try and provide periodic releases, as appropriate ...

Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 


Mon, 17 Jan 2000 02:07:40 -0400 (AST)
The Hermit Hacker <scrappy@hub.org>
Re: [HACKERS] Temp Table Memory Leak
On Mon, 17 Jan 2000, Tom Lane wrote:

> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> >> I confirm the leak in 6.5.* --- but I see no leak in current sources.
> 
> > Great.  Now the big question is should we backpatch, and if so do we
> > want a 6.5.4.
> 
> Do you have a low-risk patch for this?  I recall that we did some
> fairly extensive changes involving not only temp tables but the regular
> relation cache.  Extracting a patch that could be trusted seems like
> it might be tough.
> 
> > I know you(Tom) have put a number of patches into the 6.5.* branch,
> > and we are at least 2 months away from our next release.
> 
> I have been throwing low-risk/high-reward fixes into REL6_5 when I
> could, with the thought that we might want to do another 6.5.* release.
> But I'm undecided on whether we should or not.  It seems like we are
> close enough to 7.0 beta cycle that we should focus our effort there.

past experience tends to be that even when we beta 7.0 on the 1st of Feb,
which I haven't heard anyone suggest changing yet, we're still talking
another month, maybe two, before release...

I think it would be nice to put out a 6.5.4 about the same time as we go
beta ... 

Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 


Mon, 17 Jan 2000 02:09:46 -0400 (AST)
The Hermit Hacker <scrappy@hub.org>
Re: [HACKERS] Temp Table Memory Leak 
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> >> I confirm the leak in 6.5.* --- but I see no leak in current sources.
> 
> > Great.  Now the big question is should we backpatch, and if so do we
> > want a 6.5.4.
> 
> Do you have a low-risk patch for this?  I recall that we did some
> fairly extensive changes involving not only temp tables but the regular
> relation cache.  Extracting a patch that could be trusted seems like
> it might be tough.

I remember now.  That entire code is changed to do the replacement
before getting to actual cache.

> 
> > I know you(Tom) have put a number of patches into the 6.5.* branch,
> > and we are at least 2 months away from our next release.
> 
> I have been throwing low-risk/high-reward fixes into REL6_5 when I
> could, with the thought that we might want to do another 6.5.* release.
> But I'm undecided on whether we should or not.  It seems like we are
> close enough to 7.0 beta cycle that we should focus our effort there.
> 

Seems we can not fix this in 6.5.* without the risk of more bugs.  I
agree on focusing on 7.0.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Mon, 17 Jan 2000 01:15:25 -0500 (EST)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: [HACKERS] Temp Table Memory Leak