
> The question is *which* seek APIs we need to support.  Are there any
> besides fseeko() and fgetpos()?

On AIX we have 
int fseeko64 (FILE* Stream, off64_t Offset, int Whence);
which is intended for large file access for programs that do NOT
#define _LARGE_FILES

It is functionality that is available if _LARGE_FILE_API is defined,
which is the default if _LARGE_FILES is not defined.

That would have been my preferred way of handling large files on AIX
in the two/three? places that need it (pg_dump/restore, psql and backend COPY).
This would have had the advantage that off_t is not 64 bit in all other places
where it is actually not needed, no ?

Andreas

Thu, 24 Oct 2002 11:23:13 +0200
"Zeugswetter Andreas SB SD" <ZeugswetterA@spardat.at>
Re: pg_dump and large files - is this a problem? 
Zeugswetter Andreas SB SD wrote:
> 
> > The question is *which* seek APIs we need to support.  Are there any
> > besides fseeko() and fgetpos()?
> 
> On AIX we have 
> int fseeko64 (FILE* Stream, off64_t Offset, int Whence);
> which is intended for large file access for programs that do NOT
> #define _LARGE_FILES
> 
> It is functionality that is available if _LARGE_FILE_API is defined,
> which is the default if _LARGE_FILES is not defined.
> 
> That would have been my preferred way of handling large files on AIX
> in the two/three? places that need it (pg_dump/restore, psql and backend COPY).
> This would have had the advantage that off_t is not 64 bit in all other places
> where it is actually not needed, no ?

OK, I am focusing on AIX now.  I don't think we can go down the road of
saying where large file support is needed or not needed.  I think for
each platform either we support large files or we don't.  Is there a way
to have off_t be 64 bits everywhere, and if it is, why wouldn't we just
enable that rather than poke around figuring out where it is needed?

Also, I have the open item:

	Fix AIX + Large File + Flex problem

Is there an AIX problem with Flex?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Thu, 24 Oct 2002 21:38:00 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: pg_dump and large files - is this a problem?
At 09:38 PM 24/10/2002 -0400, Bruce Momjian wrote:
>OK, I am focusing on AIX now.  I don't think we can go down the road of
>saying where large file support is needed or not needed.  I think for
>each platform either we support large files or we don't.

Rather than having a different patch file for each platform and refusing to 
code fseek/tell because we can't do SEEK_CUR, why not check for FSEEKO64 
and revert to a simple solution:

#ifdef HAVE_FSEEKO64
#define FSEEK fseeko64
#define FTELL ftello64
#define FILE_OFFSET off64_t
#else
#ifdef HAVE_FSEEKO
#define FSEEK fseeko
#define FTELL ftello
#define FILE_OFFSET off_t
#else
#if HAVE_FSEEK_BETTER_THAN_32_BIT
#define FSEEK FSEEK_BETTER_THAN_32_BIT
#define FTELL FTELL_BETTER_THAN_32_BIT
#define FILE_OFFSET FILE_OFFSET_BETTER_THAN_32_BIT
#else
#if sizeof(off_t) > sizeof(long)
#define IGNORE_FSEEK
#else
#define FSEEK fseek
#define FTELL ftell
#define FILE_OFFSET long
#end if...

Then use a correct checkSeek which also checks IGNORE_FSEEK.

AFAICT, this *will* do the job on all systems discussed. And we can 
certainly skip the HAVE_FSEEK_BETTER_THAN_32_BIT bit, but coding a trivial 
seek/tell pair for fsetpos/fgetpos is easy, even in a macro.



----------------------------------------------------------------
Philip Warner                    |     __---_____
Albatross Consulting Pty. Ltd.   |----/       -  \
(A.B.N. 75 008 659 498)          |          /(@)   ______---_
Tel: (+61) 0500 83 82 81         |                 _________  \
Fax: (+61) 0500 83 82 82         |                 ___________ |
Http://www.rhyme.com.au          |                /           \|
                                  |    --________--
PGP key available upon request,  |  /
and from pgp5.ai.mit.edu:11371   |/


Fri, 25 Oct 2002 12:10:11 +1000
Philip Warner <pjw@rhyme.com.au>
Re: pg_dump and large files - is this a problem?
Philip Warner wrote:
> Rather than having a different patch file for each platform and refusing to 
> code fseek/tell because we can't do SEEK_CUR, why not check for FSEEKO64 
> and revert to a simple solution:
> 
> #ifdef HAVE_FSEEKO64
> #define FSEEK fseeko64
> #define FTELL ftello64
> #define FILE_OFFSET off64_t

We can do this, but there is the problem of making the code pretty ugly.
Also, it is not immediately clear when off_t is something to be used by
fseek and when it is being used in file offsets that will never be
seeked.  I am concerned about perhaps making things worse than they are
now.

> #else
> #ifdef HAVE_FSEEKO
> #define FSEEK fseeko
> #define FTELL ftello
> #define FILE_OFFSET off_t
> #else
> #if HAVE_FSEEK_BETTER_THAN_32_BIT
> #define FSEEK FSEEK_BETTER_THAN_32_BIT
> #define FTELL FTELL_BETTER_THAN_32_BIT
> #define FILE_OFFSET FILE_OFFSET_BETTER_THAN_32_BIT
> #else
> #if sizeof(off_t) > sizeof(long)

Can't do sizeof() tests in cpp, which is where the #if is processed.

> #define IGNORE_FSEEK
> #else
> #define FSEEK fseek
> #define FTELL ftell
> #define FILE_OFFSET long
> #end if...
> 
> Then use a correct checkSeek which also checks IGNORE_FSEEK.
> 
> AFAICT, this *will* do the job on all systems discussed. And we can 
> certainly skip the HAVE_FSEEK_BETTER_THAN_32_BIT bit, but coding a trivial 
> seek/tell pair for fsetpos/fgetpos is easy, even in a macro.

I don't think we can assume that off_t can be passed to fset/getpos
unless we know the platform supports it, unless people think fpos_t
being integral and the same size as fpos_t is enough.

Also, I don't think these can be done a macro, perhaps
fseeko(...,SEEK_SET), but not the others, and not ftello.  See
port/fseeko.c for the reason.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Fri, 25 Oct 2002 00:07:37 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: pg_dump and large files - is this a problem?
At 12:07 AM 25/10/2002 -0400, Bruce Momjian wrote:
>I don't think we can assume that off_t can be passed to fset/getpos
>unless we know the platform supports it, unless people think fpos_t
>being integral and the same size as fpos_t is enough.

We don't need to. We would #define FILE_OFFSET as fpos_t in that case.

>Also, I don't think these can be done a macro, perhaps
>fseeko(...,SEEK_SET), but not the others, and not ftello.  See
>port/fseeko.c for the reason.

My understanding was that you could define a block and declare variables in 
a macro; just use a local for the temp storage of the in/out args. If this 
is the only thing stopping you adopting this approach, then I am very happy 
to try to code the macros properly.

However, I do get the impression that there is more resistance to the idea 
than just this.



----------------------------------------------------------------
Philip Warner                    |     __---_____
Albatross Consulting Pty. Ltd.   |----/       -  \
(A.B.N. 75 008 659 498)          |          /(@)   ______---_
Tel: (+61) 0500 83 82 81         |                 _________  \
Fax: (+61) 0500 83 82 82         |                 ___________ |
Http://www.rhyme.com.au          |                /           \|
                                  |    --________--
PGP key available upon request,  |  /
and from pgp5.ai.mit.edu:11371   |/


Fri, 25 Oct 2002 15:39:11 +1000
Philip Warner <pjw@rhyme.com.au>
Re: pg_dump and large files - is this a problem?