
> You want to process all invoices to count them
> and to sum up the amounts on a per month/area/type
> basis. The initial data size is in GB, but the
> size of the expected result is in KB (namely 2 data
> for each 100 areas * 12 months * 4 types).

The key to handling large datasets for data mining is pre-aggregation based on the smallest time frame needed for details.

I'd suggest running these large queries and storing the results in other tables, and then writing a set of functions to work with those aggregate tables.

No sense in summing up the same set of static data more than once if you can help it.

Darren

Tue, 22 Jul 2003 13:52:20 -0400
"Darren King" <DarrenK@Routescape.com>
Re: suggestions to improve postgresql suitability for data-mining