Hi,

Now that postgresql doesn't have field size limits, it seems to
me they should be good for storing large blobs, even if it means
having to uuencode them to be non-binary or whatever. I don't
like the old large object implementation, I need to store very large
numbers of objects and unless this implementation has changed
in recent times it won't cut it.

So my question is, I assume TEXT is the best data type to store
large things in, what precisely is the range of characters that
I can store in TEXT? Is it only characters ascii <= 127, or is
it only printable characters, or everything except '\0' or what?


Wed, 10 Oct 2001 11:33:04 +1000
Chris Bitmead <chris@bitmead.com>
TOAST and TEXT
It should be noted that there is still a limit of about 1GB if I
remember correctly.
--
Rod Taylor

There are always four sides to every story: your side, their side, the
truth, and what really happened.

----- Original Message -----
From: "Chris Bitmead" <chris@bitmead.com>
To: <pgsql-hackers@postgresql.org>
Sent: Tuesday, October 09, 2001 9:33 PM
Subject: [HACKERS] TOAST and TEXT


> Hi,
>
> Now that postgresql doesn't have field size limits, it seems to
> me they should be good for storing large blobs, even if it means
> having to uuencode them to be non-binary or whatever. I don't
> like the old large object implementation, I need to store very large
> numbers of objects and unless this implementation has changed
> in recent times it won't cut it.
>
> So my question is, I assume TEXT is the best data type to store
> large things in, what precisely is the range of characters that
> I can store in TEXT? Is it only characters ascii <= 127, or is
> it only printable characters, or everything except '\0' or what?
>
>
> ---------------------------(end of
broadcast)---------------------------
> TIP 6: Have you searched our list archives?
>
> http://archives.postgresql.org
>


Tue, 9 Oct 2001 21:45:23 -0400
"Rod Taylor" <rbt@barchord.com>
Re: TOAST and TEXT
On Wed, Oct 10, 2001 at 11:33:04AM +1000, Chris Bitmead wrote:
> So my question is, I assume TEXT is the best data type to store
> large things in, what precisely is the range of characters that
> I can store in TEXT? Is it only characters ascii <= 127, or is
> it only printable characters, or everything except '\0' or what?

text accepts everything except \0, and also various funtions
take locale/charset info into account.

Use bytea, its for 0-255, binary data.  When your client
library does not support it, then base64 it in client side
and later decode() into place.

-- 
marko


Wed, 10 Oct 2001 04:48:42 +0200
Marko Kreen <marko@l-t.ee>
Re: TOAST and TEXT
Rod Taylor wrote:
> It should be noted that there is still a limit of about 1GB if I
> remember correctly.

    You're right, there is still a practical limit on the size of
    a text field. And it's usually much lower than 1GB.

    The problem is that first, the (encoded) data has to  be  put
    completely  into  the  querystring, passed to the backend and
    buffered there entirely in memory. Then it get's parsed,  and
    the  data  copied  into  a  const  node.  After rewriting and
    planning, a  heap  tuple  is  build,  containing  the  third,
    eventually fourth in memory copy of the data. After that, the
    toaster kicks in, allocates another chunk of that size to try
    to compress the data and finally slices it up for storage.

    So the limit depends on how much swapspace you have and where
    the per process virtual memory limit of your OS is.

    In practice, sizes of up to 10 MB are no problem. So  storing
    typical MP3s works.


Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #



_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com


Wed, 10 Oct 2001 08:52:54 -0400 (EDT)
Jan Wieck <JanWieck@Yahoo.com>
Re: TOAST and TEXT
Chris Bitmead <chris@bitmead.com> writes:
> ... I don't
> like the old large object implementation, I need to store very large
> numbers of objects and unless this implementation has changed
> in recent times it won't cut it.

Have you looked at 7.1?  AFAIK it has no particular problem with
lots of LOs.

Which is not to discourage you from going over to bytea fields instead,
if that model happens to be more convenient for your application.
But your premise above seems false.

			regards, tom lane

Thu, 11 Oct 2001 00:32:47 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: TOAST and TEXT 