Tom doesn't like returning the server's logfile using a pgsql function

unless logfile rotation is implemented, so here it is.

This proposal doesn't include a daemon that triggers pg_logfile_rotate 
when appropriate (timestamp and/or length dependent), pg_autovacuum 
might be a good place to do that.

I'd also like to see a table pg_logfile which contains all logfile 
names, to be able to access older logfiles.


Comments?

Regards,
Andreas
Sun, 13 Jun 2004 18:43:20 +0200
Andreas Pflug <pgadmin@pse-consulting.de>
logfile rotation
Andreas Pflug <pgadmin@pse-consulting.de> writes:
> Tom doesn't like returning the server's logfile using a pgsql function
> unless logfile rotation is implemented, so here it is.

I'll repeat what I said in response to your other posting:

This uses a shared memory area with no lock, which seems a bad design;
but the alternative of having a lock is even worse, since the postmaster
would also have to take the lock.  We agreed long ago that the
postmaster should never depend on the correctness of any shared memory
data structure; but this patch would make it do so.

I really don't think this is an acceptable solution.

			regards, tom lane

Sun, 13 Jun 2004 13:03:27 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: logfile rotation 
Tom Lane wrote:

>I'll repeat what I said in response to your other posting:
>  
>
Hm? I never posted something with shared mem usage before, what do you mean?

>This uses a shared memory area with no lock, which seems a bad design;
>  
>

AFAICS there should be no lock necessary.


>We agreed long ago that the
>postmaster should never depend on the correctness of any shared memory
>data structure; but this patch would make it do so.
>  
>
I understand that, so what's the suggested way to store data common for 
all backends?

Regards,
Andreas


Sun, 13 Jun 2004 19:18:55 +0200
Andreas Pflug <pgadmin@pse-consulting.de>
Re: logfile rotation
Andreas Pflug wrote:

>
>> We agreed long ago that the
>> postmaster should never depend on the correctness of any shared memory
>> data structure; but this patch would make it do so.
>>  
>>
> I understand that, so what's the suggested way to store data common 
> for all backends?


Answering my own question, the distribution of the current logfile name 
could be done trough a file handle. So the only thing remaining in 
shared mem would be the "reopen logfile" flag, which seems 
nonproblematic. In case of garbled sharedmem, a backend would reopen the 
logfile, which does no harm at all, or continue writing the outdated 
logfile which would be annoying but not harmful either.

Acceptable?

Regards,
Andreas


Sun, 13 Jun 2004 20:47:47 +0200
Andreas Pflug <pgadmin@pse-consulting.de>
Re: logfile rotation
Andreas Pflug wrote:

> Andreas Pflug wrote:
>
>>
>>> We agreed long ago that the
>>> postmaster should never depend on the correctness of any shared memory
>>> data structure; but this patch would make it do so.
>>>  
>>>
>> I understand that, so what's the suggested way to store data common 
>> for all backends?
>
>
>
> Answering my own question, the distribution of the current logfile 
> name could be done trough a file handle. So the only thing remaining 
> in shared mem would be the "reopen logfile" flag, which seems 
> nonproblematic. In case of garbled sharedmem, a backend would reopen 
> the logfile, which does no harm at all, or continue writing the 
> outdated logfile which would be annoying but not harmful either.
>
> Acceptable?


Tom,
would you mind commenting on my suggestion so I can continue on that topic?

Regards,
Andreas


Wed, 16 Jun 2004 20:43:16 +0200
Andreas Pflug <pgadmin@pse-consulting.de>
Re: logfile rotation
Andreas Pflug <pgadmin@pse-consulting.de> writes:
>> Answering my own question, the distribution of the current logfile 
>> name could be done trough a file handle.

> would you mind commenting on my suggestion so I can continue on that topic?

There is no portable way to redistribute a file handle.

			regards, tom lane

Wed, 16 Jun 2004 14:45:33 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: logfile rotation 
Tom Lane wrote:

>Andreas Pflug <pgadmin@pse-consulting.de> writes:
>  
>
>>>Answering my own question, the distribution of the current logfile 
>>>name could be done trough a file handle.
>>>      
>>>
>
>  
>
>>would you mind commenting on my suggestion so I can continue on that topic?
>>    
>>
>
>There is no portable way to redistribute a file handle.
>  
>

Seems I didn't make clear enough what I mean.

I'd fopen a file handle  in the postmaster, and all subsequent processes 
will inherit that handle just as they do for stderr; no redistribution 
required.
The log filename is written to that file when pg_logfile_rotate is called:
fseek(fh, 0, SEEK_SET);
fprintf(fh, "%s", newlogfilname);
fflush();

and all subprocesses may retrieve the filename when required by

char buf[MAXPGPATH];
fseek(fh, 0, SEEK_SET);
fread(buf, 1, MAXPGPATH, fh);
buf[MAXPGPATH-1]=0; // prevent buffer overflow
logfile=fopen(buf, "a+");

Regards,
Andreas



Wed, 16 Jun 2004 21:03:52 +0200
Andreas Pflug <pgadmin@pse-consulting.de>
Re: logfile rotation

Actually, this is the current state of this issue.

---------------------------------------------------------------------------

Andreas Pflug wrote:
> Tom Lane wrote:
> 
> >Andreas Pflug <pgadmin@pse-consulting.de> writes:
> >  
> >
> >>>Answering my own question, the distribution of the current logfile 
> >>>name could be done trough a file handle.
> >>>      
> >>>
> >
> >  
> >
> >>would you mind commenting on my suggestion so I can continue on that topic?
> >>    
> >>
> >
> >There is no portable way to redistribute a file handle.
> >  
> >
> 
> Seems I didn't make clear enough what I mean.
> 
> I'd fopen a file handle  in the postmaster, and all subsequent processes 
> will inherit that handle just as they do for stderr; no redistribution 
> required.
> The log filename is written to that file when pg_logfile_rotate is called:
> fseek(fh, 0, SEEK_SET);
> fprintf(fh, "%s", newlogfilname);
> fflush();
> 
> and all subprocesses may retrieve the filename when required by
> 
> char buf[MAXPGPATH];
> fseek(fh, 0, SEEK_SET);
> fread(buf, 1, MAXPGPATH, fh);
> buf[MAXPGPATH-1]=0; // prevent buffer overflow
> logfile=fopen(buf, "a+");
> 
> Regards,
> Andreas
> 
> 

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Fri, 18 Jun 2004 23:09:01 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: logfile rotation
Bruce Momjian wrote:

>Actually, this is the current state of this issue.
>  
>

Right, please comment on this. To recall, it uses shared memory for a 
"switch to next logfile name" flag, which can't cause harm in case of 
shmem corruption, and a postmaster opened filehandle (kept open) to a 
dummy file containing the actual file name (pseudo code below).

Regards,
Andreas

>---------------------------------------------------------------------------
>
>Andreas Pflug wrote:
>  
>
>>Tom Lane wrote:
>>
>>    
>>
>>>Andreas Pflug <pgadmin@pse-consulting.de> writes:
>>> 
>>>
>>>      
>>>
>>>>>Answering my own question, the distribution of the current logfile 
>>>>>name could be done trough a file handle.
>>>>>     
>>>>>
>>>>>          
>>>>>
>>> 
>>>
>>>      
>>>
>>>>would you mind commenting on my suggestion so I can continue on that topic?
>>>>   
>>>>
>>>>        
>>>>
>>>There is no portable way to redistribute a file handle.
>>> 
>>>
>>>      
>>>
>>Seems I didn't make clear enough what I mean.
>>
>>I'd fopen a file handle  in the postmaster, and all subsequent processes 
>>will inherit that handle just as they do for stderr; no redistribution 
>>required.
>>The log filename is written to that file when pg_logfile_rotate is called:
>>fseek(fh, 0, SEEK_SET);
>>fprintf(fh, "%s", newlogfilname);
>>fflush();
>>
>>and all subprocesses may retrieve the filename when required by
>>
>>char buf[MAXPGPATH];
>>fseek(fh, 0, SEEK_SET);
>>fread(buf, 1, MAXPGPATH, fh);
>>buf[MAXPGPATH-1]=0; // prevent buffer overflow
>>logfile=fopen(buf, "a+");
>>
>>Regards,
>>Andreas
>>
>>
>>    
>>
>
>  
>



Sat, 19 Jun 2004 11:10:39 +0200
Andreas Pflug <pgadmin@pse-consulting.de>
Re: logfile rotation