Can anyone comment on this work?  Karel?

> 
>   I start detail study of PG's memory management (because, I want remove
> prepared query-cache to shmem (more is in my old discussion with Jan)).
> 
>  I see current code in the aset.c and I found small non-effective memory 
> usage.
> 
> 
>  Description:
> 
>  The postgresql use blocks for allocation. These blocks are inward
> organized/split via chunks. 
> 
>  If a palloc() wants memory:
>    1) try use some chunk in a freelist of chunks
>    2) try use free space in an actual block
>    3) if wanted space is large - allocate specific one-block-for-one-chunk
>    4) if previous options are not possible allocate new block
>  
>  A problem:
> 
>    - if use option 4) and already exist (old) block (but space in this block 
> is less than wanted space) current algorithm _skip_ and not _use_ this small
> space in old block. For a detail see the 'else' on line 327 in aset.c.
> 
>   I test it and average is 8-10b per a block (but max is 1000b) - large is 
> this space if a palloc() wants bigger space. 
> 
>  A solution:
> 
>   Create from this non-used residual space chunk and remove it into free 
>   chunk list. 
>  
> 
>  Comments?
> 
> 						Karel
> 
> 
> -----> A tested patch (hmm, we are freeze, possible for 7.0.?):
> 
> *** aset.orig.c	Thu Apr 13 18:33:45 2000
> --- aset.c	Thu Apr 20 18:45:50 2000
> ***************
> *** 323,326 ****
> --- 323,346 ----
>   		else
>   		{
> + 			int 	oldfree = set->blocks->endptr - set->blocks->freeptr;
> + 		
> + 			/*
> + 			 * Try create from residual space in block free chunk
> + 			 */
> + 			if (oldfree > MAXALIGN(1) + ALLOC_CHUNKHDRSZ) {
> + 				
> + 				int x_fidx = AllocSetFreeIndex(oldfree - ALLOC_CHUNKHDRSZ );
> + 				
> + 				chunk = (AllocChunk) (set->blocks->freeptr);
> + 				chunk->size = oldfree - ALLOC_CHUNKHDRSZ;
> + 				
> + 				/* put chunk into freelist */
> + 				chunk->aset = (void *) set->freelist[x_fidx];
> + 				set->freelist[x_fidx] = chunk;
> + 				
> + 				/* unset free space in block */                
> + 				set->blocks->freeptr = set->blocks->endptr;
> + 			}
> + 		
>   			/* Get size of prior block */
>   			blksize = set->blocks->endptr - ((char *) set->blocks);
> 
> 
> 
> 


-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Mon, 12 Jun 2000 20:07:56 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: memory management suggestion
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Can anyone comment on this work?  Karel?

I thought it was a good idea, except for the one possible bug about
whether small wasted chunks go into the right freelist or not.

I think Karel submitted an updated version of the patch later, no?

			regards, tom lane

Mon, 12 Jun 2000 22:41:52 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: memory management suggestion 
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > Can anyone comment on this work?  Karel?
> 
> I thought it was a good idea, except for the one possible bug about
> whether small wasted chunks go into the right freelist or not.
> 
> I think Karel submitted an updated version of the patch later, no?

I see a patch, and your comment on it, but nothing after that.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Tue, 13 Jun 2000 03:18:01 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: memory management suggestion

> Can anyone comment on this work?  Karel?


 Yes. Tom & Jan already comment it. I prepare it for 7.1 with
Tom's advices.

					Karel


Tue, 13 Jun 2000 10:09:20 +0200 (CEST)
Karel Zak <zakkr@zf.jcu.cz>
Re: memory management suggestion
> 
> > Can anyone comment on this work?  Karel?
> 
> 
>  Yes. Tom & Jan already comment it. I prepare it for 7.1 with
> Tom's advices.

Is it already applied to 7.1?

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Tue, 13 Jun 2000 04:09:23 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: memory management suggestion

On Tue, 13 Jun 2000, Bruce Momjian wrote:

> > 
> > > Can anyone comment on this work?  Karel?
> > 
> > 
> >  Yes. Tom & Jan already comment it. I prepare it for 7.1 with
> > Tom's advices.
> 
> Is it already applied to 7.1?

 No. I said it bad, I want say "I *will* prepare..." :-)

 Well, I write it next week. Now I work on contrib/pg_lodump and the
others things...

 Bruce, (it is probably already discussed, but refresh me..) for what time
is planned 7.1?

							Karel



Tue, 13 Jun 2000 10:49:34 +0200 (CEST)
Karel Zak <zakkr@zf.jcu.cz>
Re: memory management suggestion
> 
> On Tue, 13 Jun 2000, Bruce Momjian wrote:
> 
> > > 
> > > > Can anyone comment on this work?  Karel?
> > > 
> > > 
> > >  Yes. Tom & Jan already comment it. I prepare it for 7.1 with
> > > Tom's advices.
> > 
> > Is it already applied to 7.1?
> 
>  No. I said it bad, I want say "I *will* prepare..." :-)
> 
>  Well, I write it next week. Now I work on contrib/pg_lodump and the
> others things...

Oh, OK.  Just checking.

> 
>  Bruce, (it is probably already discussed, but refresh me..) for what time
> is planned 7.1?

August.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Tue, 13 Jun 2000 05:01:41 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: memory management suggestion
Where are we on this?


> 
>   I start detail study of PG's memory management (because, I want remove
> prepared query-cache to shmem (more is in my old discussion with Jan)).
> 
>  I see current code in the aset.c and I found small non-effective memory 
> usage.
> 
> 
>  Description:
> 
>  The postgresql use blocks for allocation. These blocks are inward
> organized/split via chunks. 
> 
>  If a palloc() wants memory:
>    1) try use some chunk in a freelist of chunks
>    2) try use free space in an actual block
>    3) if wanted space is large - allocate specific one-block-for-one-chunk
>    4) if previous options are not possible allocate new block
>  
>  A problem:
> 
>    - if use option 4) and already exist (old) block (but space in this block 
> is less than wanted space) current algorithm _skip_ and not _use_ this small
> space in old block. For a detail see the 'else' on line 327 in aset.c.
> 
>   I test it and average is 8-10b per a block (but max is 1000b) - large is 
> this space if a palloc() wants bigger space. 
> 
>  A solution:
> 
>   Create from this non-used residual space chunk and remove it into free 
>   chunk list. 
>  
> 
>  Comments?
> 
> 						Karel
> 
> 
> -----> A tested patch (hmm, we are freeze, possible for 7.0.?):
> 
> *** aset.orig.c	Thu Apr 13 18:33:45 2000
> --- aset.c	Thu Apr 20 18:45:50 2000
> ***************
> *** 323,326 ****
> --- 323,346 ----
>   		else
>   		{
> + 			int 	oldfree = set->blocks->endptr - set->blocks->freeptr;
> + 		
> + 			/*
> + 			 * Try create from residual space in block free chunk
> + 			 */
> + 			if (oldfree > MAXALIGN(1) + ALLOC_CHUNKHDRSZ) {
> + 				
> + 				int x_fidx = AllocSetFreeIndex(oldfree - ALLOC_CHUNKHDRSZ );
> + 				
> + 				chunk = (AllocChunk) (set->blocks->freeptr);
> + 				chunk->size = oldfree - ALLOC_CHUNKHDRSZ;
> + 				
> + 				/* put chunk into freelist */
> + 				chunk->aset = (void *) set->freelist[x_fidx];
> + 				set->freelist[x_fidx] = chunk;
> + 				
> + 				/* unset free space in block */                
> + 				set->blocks->freeptr = set->blocks->endptr;
> + 			}
> + 		
>   			/* Get size of prior block */
>   			blksize = set->blocks->endptr - ((char *) set->blocks);
> 
> 
> 
> 


-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Fri, 29 Sep 2000 22:47:54 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: memory management suggestion

On Fri, 29 Sep 2000, Bruce Momjian wrote:

> Where are we on this?
> 
> >   Create from this non-used residual space chunk and remove it into free 
> >   chunk list. 


 This is really old story. I hope that Tom think of this and has it's 
in his care. A problem is that standard day of this planet has 
24 hours for all people (incl. Tom :-), right?

					Karel


Sun, 1 Oct 2000 10:13:12 +0200 (CEST)
Karel Zak <zakkr@zf.jcu.cz>
Re: memory management suggestion
Karel Zak <zakkr@zf.jcu.cz> writes:
>  This is really old story. I hope that Tom think of this and has it's 
> in his care. A problem is that standard day of this planet has 
> 24 hours for all people (incl. Tom :-), right?

Yup.  I've got one more area to wrap up in the new-features business
(I really want to make UNION/INTERSECT/EXCEPT work with subqueries in
FROM) and then it's back to mopup on memory management etc.

Fortunately, since Vadim requested a postponement of the beta schedule,
there's still time ...

			regards, tom lane

Sun, 01 Oct 2000 11:19:37 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: memory management suggestion 
Tom, guess I can delete this email now?  :-)

> 
>   I start detail study of PG's memory management (because, I want remove
> prepared query-cache to shmem (more is in my old discussion with Jan)).
> 
>  I see current code in the aset.c and I found small non-effective memory 
> usage.
> 
> 
>  Description:
> 
>  The postgresql use blocks for allocation. These blocks are inward
> organized/split via chunks. 
> 
>  If a palloc() wants memory:
>    1) try use some chunk in a freelist of chunks
>    2) try use free space in an actual block
>    3) if wanted space is large - allocate specific one-block-for-one-chunk
>    4) if previous options are not possible allocate new block
>  
>  A problem:
> 
>    - if use option 4) and already exist (old) block (but space in this block 
> is less than wanted space) current algorithm _skip_ and not _use_ this small
> space in old block. For a detail see the 'else' on line 327 in aset.c.
> 
>   I test it and average is 8-10b per a block (but max is 1000b) - large is 
> this space if a palloc() wants bigger space. 
> 
>  A solution:
> 
>   Create from this non-used residual space chunk and remove it into free 
>   chunk list. 
>  
> 
>  Comments?
> 
> 						Karel
> 
> 
> -----> A tested patch (hmm, we are freeze, possible for 7.0.?):
> 
> *** aset.orig.c	Thu Apr 13 18:33:45 2000
> --- aset.c	Thu Apr 20 18:45:50 2000
> ***************
> *** 323,326 ****
> --- 323,346 ----
>   		else
>   		{
> + 			int 	oldfree = set->blocks->endptr - set->blocks->freeptr;
> + 		
> + 			/*
> + 			 * Try create from residual space in block free chunk
> + 			 */
> + 			if (oldfree > MAXALIGN(1) + ALLOC_CHUNKHDRSZ) {
> + 				
> + 				int x_fidx = AllocSetFreeIndex(oldfree - ALLOC_CHUNKHDRSZ );
> + 				
> + 				chunk = (AllocChunk) (set->blocks->freeptr);
> + 				chunk->size = oldfree - ALLOC_CHUNKHDRSZ;
> + 				
> + 				/* put chunk into freelist */
> + 				chunk->aset = (void *) set->freelist[x_fidx];
> + 				set->freelist[x_fidx] = chunk;
> + 				
> + 				/* unset free space in block */                
> + 				set->blocks->freeptr = set->blocks->endptr;
> + 			}
> + 		
>   			/* Get size of prior block */
>   			blksize = set->blocks->endptr - ((char *) set->blocks);
> 
> 
> 
> 


-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Sun, 10 Dec 2000 15:34:22 -0500 (EST)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: memory management suggestion
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Tom, guess I can delete this email now?  :-)

Yes, it's done.

			regards, tom lane

Sun, 10 Dec 2000 17:26:55 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: memory management suggestion 