> IMHO, you should copy into a temporary table and the do a select 
> distinct from it into the table that you want.

Which would be way too slow for normal operation :-(
We are talking about a "fast as possible" data load from a flat file
that may have duplicates (or even data errors, but that 
is another issue).

Andreas

Mon, 1 Oct 2001 16:39:36 +0200
"Zeugswetter Andreas SB SD" <ZeugswetterA@spardat.at>
Re: Bulkloading using COPY - ignore duplicates?
Zeugswetter Andreas SB SD wrote:

>>IMHO, you should copy into a temporary table and the do a select 
>>distinct from it into the table that you want.
>>
>
>Which would be way too slow for normal operation :-(
>We are talking about a "fast as possible" data load from a flat file
>that may have duplicates (or even data errors, but that 
>is another issue).
>
>Andreas
>
Then the IGNORE_DUPLICATE would definitely be the way to go, if speed is 
the question...









Zeugswetter Andreas SB SD wrote:


IMHO, you should copy into a temporary table and the do a select distinct from it into the table that you want.

Which would be way too slow for normal operation :-(We are talking about a "fast as possible" data load from a flat filethat may have duplicates (or even data errors, but that is another issue).Andreas

Then the IGNORE_DUPLICATE would definitely be the way to go, if speed is
the question...
Mon, 01 Oct 2001 09:42:56 -0500
Thomas Swan <tswan@olemiss.edu>
Re: Bulkloading using COPY - ignore duplicates?
Okay,

If I'm going to modify 'COPY INTO' to include 'ignore duplicates'
functionality it looks like I'll have to add to the COPY syntax. The
most obvious way is to add:

 WITH IGNORE DUPLICATES

to the syntax. I'm going to need my hand held a bit for this! The
grammar for COPY will need updating in gram.y and specifically the
'WITH' keyword will have 'IGNORE DUPLICATES' as well as 'NULL AS'.

Any pointers?

Thanks, Lee.

Mon, 1 Oct 2001 16:31:06 +0100 (BST)
Lee Kindness <lkindness@csl.co.uk>
Re: Bulkloading using COPY - ignore duplicates?
Lee Kindness writes:
 > If I'm going to modify 'COPY INTO' to include 'ignore duplicates'
 > functionality it looks like I'll have to add to the COPY syntax. The
 > most obvious way is to add:
 >  WITH IGNORE DUPLICATES

Or does it make more sense to add a 'COPY_IGNORE_DUPLICATES' SET
parameter? 

Lee.

Mon, 1 Oct 2001 16:50:41 +0100 (BST)
Lee Kindness <lkindness@csl.co.uk>
Re: Bulkloading using COPY - ignore duplicates?