I'm just upgrading a database to v7.0, but I am rather unlucky in that in
each of 2 large (ca. 20000 rows) tables, 1 row is too big eg:

ERROR:  copy: line 57552, Tuple is too big: size 8152, max size 8140

Now, it would help me if I could actually get a clue as to which row this
is. The error message comes from hio.c:118 at which point one has a
HeapTuple. Is there a way of extracting a piece of the row in the tuple to
be able to identify it?

It seems that a HeapTuple starts with a HeapTupleHeader, so the data I
suppose starts at tuple->t_data[sizeof(HeapTupleHeaderData)] and goes on for
tuple->t_len-sizeof(HeapTupleHeaderData) bytes, but how is it represented?

Any pointers appreciated! (These are huge COPY statements, so after the
error I'm left with an empty table - I'd just like to chop the 12 bytes
off!)

Cheers,

Patrick

Mon, 15 May 2000 23:29:57 +0100
Patrick Welche <prlw1@newn.cam.ac.uk>
reading row in backend
Patrick Welche <prlw1@newn.cam.ac.uk> writes:
> ERROR:  copy: line 57552, Tuple is too big: size 8152, max size 8140

> Now, it would help me if I could actually get a clue as to which row this
> is.

Um ... doesn't that line number in the COPY source data help any?

			regards, tom lane

Mon, 15 May 2000 18:55:50 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: reading row in backend 
On Mon, May 15, 2000 at 06:55:50PM -0400, Tom Lane wrote:
> Patrick Welche <prlw1@newn.cam.ac.uk> writes:
> > ERROR:  copy: line 57552, Tuple is too big: size 8152, max size 8140
> 
> > Now, it would help me if I could actually get a clue as to which row this
> > is.
> 
> Um ... doesn't that line number in the COPY source data help any?

Unfortunately not - in fact the 2nd large tuple allegedly has a smaller
line number than the first :(

Am I barking up the wrong tree if I think that the HeapTuple contains the
actual data? The first number in it would the primary key...

Cheers,

Patrick

Tue, 16 May 2000 00:09:00 +0100
Patrick Welche <prlw1@newn.cam.ac.uk>
Re: reading row in backend
Patrick Welche <prlw1@newn.cam.ac.uk> writes:
>> Um ... doesn't that line number in the COPY source data help any?

> Unfortunately not - in fact the 2nd large tuple allegedly has a smaller
> line number than the first :(

Hmm, are you claiming that COPY is reporting a bogus line number?
If so, that needs to be looked into.

> Am I barking up the wrong tree if I think that the HeapTuple contains the
> actual data? The first number in it would the primary key...

Type HeapTuple is a pointer to a HeapTupleData, which is just
administrative overhead.  The t_data field of the HeapTupleData
points at the actual tuple (a HeapTupleHeaderData followed by
null-values bitmap and then the user data).  See include/access/htup.h

			regards, tom lane

Mon, 15 May 2000 19:22:30 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: reading row in backend 
On Mon, May 15, 2000 at 07:22:30PM -0400, Tom Lane wrote:
> Patrick Welche <prlw1@newn.cam.ac.uk> writes:
> >> Um ... doesn't that line number in the COPY source data help any?
> 
> > Unfortunately not - in fact the 2nd large tuple allegedly has a smaller
> > line number than the first :(
> 
> Hmm, are you claiming that COPY is reporting a bogus line number?
> If so, that needs to be looked into.

Yup:

ERROR:  copy: line 57552, Tuple is too big: size 8152, max size 8140
PQendcopy: resetting connection
ERROR:  copy: line 26714, Tuple is too big: size 8164, max size 8140
PQendcopy: resetting connection

% wc /home/prlw1/db.out
 3631449 19833180 186847533 /home/prlw1/db.out

and the second line is in a table that is over half way through the file.

I'll take a look after some sleep..

> > Am I barking up the wrong tree if I think that the HeapTuple contains the
> > actual data? The first number in it would the primary key...
> 
> Type HeapTuple is a pointer to a HeapTupleData, which is just
> administrative overhead.  The t_data field of the HeapTupleData
> points at the actual tuple (a HeapTupleHeaderData followed by
> null-values bitmap and then the user data).  See include/access/htup.h

I think my question really is, is the user data meant to be just text? When
I tried printing a block, I just got jibberish - it could well be the way I
tried to print it, but I don't know what to expect..

Cheers,

Patrick

Tue, 16 May 2000 00:37:02 +0100
Patrick Welche <prlw1@newn.cam.ac.uk>
Re: reading row in backend
Patrick Welche <prlw1@newn.cam.ac.uk> writes:
>> Hmm, are you claiming that COPY is reporting a bogus line number?
>> If so, that needs to be looked into.

> Yup:

> ERROR:  copy: line 57552, Tuple is too big: size 8152, max size 8140
> PQendcopy: resetting connection
> ERROR:  copy: line 26714, Tuple is too big: size 8164, max size 8140
> PQendcopy: resetting connection

> % wc /home/prlw1/db.out
>  3631449 19833180 186847533 /home/prlw1/db.out

> and the second line is in a table that is over half way through the file.

Looks reasonable enough to me.  The line numbers are within the data
being fed to that copy command --- copy has no way to know that it's
part of some larger script...

			regards, tom lane

Mon, 15 May 2000 20:44:03 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: reading row in backend 
On Mon, May 15, 2000 at 08:44:03PM -0400, Tom Lane wrote:
> Patrick Welche <prlw1@newn.cam.ac.uk> writes:
> >> Hmm, are you claiming that COPY is reporting a bogus line number?
> >> If so, that needs to be looked into.
> 
> > Yup:
> 
> > ERROR:  copy: line 57552, Tuple is too big: size 8152, max size 8140
> > PQendcopy: resetting connection
> > ERROR:  copy: line 26714, Tuple is too big: size 8164, max size 8140
> > PQendcopy: resetting connection
> 
> > % wc /home/prlw1/db.out
> >  3631449 19833180 186847533 /home/prlw1/db.out
> 
> > and the second line is in a table that is over half way through the file.
> 
> Looks reasonable enough to me.  The line numbers are within the data
> being fed to that copy command --- copy has no way to know that it's
> part of some larger script...

Ah! Thank you! So it's 57552 rows down from "COPY" !

OK

Cheers,

Patrick

Tue, 16 May 2000 10:42:05 +0100
Patrick Welche <prlw1@newn.cam.ac.uk>
Re: reading row in backend