Hello,

We are trying to import the TPC-H data into postgresql using the COPY
command and for the larger files we get an error due to insufficient
memory space.

We are using a linux system with Postgresql-7.3.4

Is it that Postgresql cannot handle such large files or is there some
other possible reason.

Thanks
Shalu Gupta
NC State University.

Wed, 21 Apr 2004 00:08:16 -0400 (EDT)
Shalu Gupta <sgupta5@unity.ncsu.edu>
TPC H data 
Shalu Gupta wrote:

>Hello,
>
>We are trying to import the TPC-H data into postgresql using the COPY
>command and for the larger files we get an error due to insufficient
>memory space.
>
>We are using a linux system with Postgresql-7.3.4
>
>Is it that Postgresql cannot handle such large files or is there some
>other possible reason.
>
>Thanks
>Shalu Gupta
>NC State University.
>
>  
>

Shalu,

I loaded the largest TPC-H table (lineitem, roughly 6 million rows) the 
other day into a completely untuned 7.5devel PostgreSQL instance running 
on RH 9, and it didn't raise a sweat. I delayed creating the indexes 
until after the load. Data load took roughly 10 minutes, index creation 
took a further 35 minutes (there are 13 of them).

HTH. (I'm just down the road from NCSU, would be happy to help out)

cheers

andrew



Mon, 26 Apr 2004 13:00:35 -0400
Andrew Dunstan <andrew@dunslane.net>
Re: [HACKERS] TPC H data
On Wed, 21 Apr 2004, Shalu Gupta wrote:

> Hello,
> 
> We are trying to import the TPC-H data into postgresql using the COPY
> command and for the larger files we get an error due to insufficient
> memory space.
> 
> We are using a linux system with Postgresql-7.3.4
> 
> Is it that Postgresql cannot handle such large files or is there some
> other possible reason.

what method(s) are you using to load the data?


Tue, 27 Apr 2004 14:47:01 -0600 (MDT)
"scott.marlowe" <scott.marlowe@ihs.com>
Re: [HACKERS] TPC H data
What scale factor TPC H are you importing?

additionally -  might be worth giving the specs of the machine you are 
doing this on.

(I seem to recall trying this with Pg 7.2 a while ago without this 
issue, mind you -  think I had ~1G of Ram and used the scale fact 1 
dataset, i.e 1G)

regards

Mark


Shalu Gupta wrote:

>Hello,
>
>We are trying to import the TPC-H data into postgresql using the COPY
>command and for the larger files we get an error due to insufficient
>memory space.
>
>We are using a linux system with Postgresql-7.3.4
>
>Is it that Postgresql cannot handle such large files or is there some
>other possible reason.
>
>Thanks
>Shalu Gupta
>NC State University.
>
>---------------------------(end of broadcast)---------------------------
>TIP 5: Have you checked our extensive FAQ?
>
>               http://www.postgresql.org/docs/faqs/FAQ.html
>
>
>  
>

Wed, 28 Apr 2004 20:10:53 +1200
Mark Kirkwood <markir@paradise.net.nz>
Re: TPC H data