Hi all,

I've been thinking about resource management and postgres. I want to
develop a user profile system (a-la oracle) which allows a DBA to
restrict/configure access to system resources. This would allow a DBA to
configure how much CPU time can be used per query/session for any user,
the number blocks that can be read/written by a user per query, and
perhaps some other things (see below).

Current, Postgres allows us to trivial calculate the following: CPU time
for a given query (and as such, CPU time for a session), file system
blocks read in/out for a given query (and therefore, for a given session),
and some other stuff. The question is, given access to this information,
how regularly should we check to see if the user has passed the set
limits? My concern is that if we check too regularly, we impact upon
performance. Also, should we be intelligent about it (would hate to error
out of a select when we believe that there is only a small number of
tuples left)?

There are some other resource management features which would be
useful: number of sessions per user, maximum memory usage per user,
maximum disk space for temp files, etc. Oracle supports things like
maximum time in parser, planner and executor, maximum memory blocks
accessed. I think that maximum sessions per user would be useful but the
other ones seem like more trouble than they're worth. Have I missed
any resources here?

Gavin


Sat, 21 Dec 2002 13:29:44 +1100 (EST)
Gavin Sherry <swm@linuxworld.com.au>
Resource management in 7.4
Gavin Sherry <swm@linuxworld.com.au> writes:
> I've been thinking about resource management and postgres. I want to
> develop a user profile system (a-la oracle) which allows a DBA to
> restrict/configure access to system resources. This would allow a DBA to
> configure how much CPU time can be used per query/session for any user,
> the number blocks that can be read/written by a user per query, and
> perhaps some other things (see below).

I've got really serious reservations about this whole idea.  I don't
like expending even one CPU cycle on it, and I don't like introducing a
potential cause of unnecessary query failure, and I don't believe that
the average DBA would be capable of configuring it intelligently.

To point out just one problem: in the current system design, the backend
that actually issues a write request is not necessarily, or even
probably, the one that dirtied the page.  And you can NOT refuse to
write a dirtied page because of some misbegotten notion about resource
limits; system reliability will go to zero if you do.

Another example is that the cost of verifying transaction completion is
actually paid by the first transaction to visit a tuple after the
tuple's authoring transaction completes.  Should a transaction be
penalized if it's foolish enough to do a seqscan shortly after someone
else does a mass insert or update?

In general, I think that per-user resource management policies would
force us to adopt inefficient algorithms that don't share overhead costs
across the whole community.  I'm not eager for that...

			regards, tom lane

Sat, 21 Dec 2002 00:20:56 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: Resource management in 7.4 
Personally I think that configuring things like that is definitely 
beyond the scope of an average administrator.

However, there is one thing which would be useful for many applications: 
It would be nice if there was a way to renice a connection. When it 
comes to reporting it would be nice to have a handle for slowing a 
backend down.

A patch for Linux would be quite easy ("SELECT nice_backend(int)") but I 
don't know how this fits into the Windows port and PostgreSQL on other 
platforms.

I think this would be a straight forward approach fixing most of the 
problems people might have with CPU usage.

Is this the right way to go?

    Regards,
        Hans

-- 
*Cybertec Geschwinde u Schoenig*
Ludo-Hartmannplatz 1/14, A-1160 Vienna, Austria
Tel: +43/1/913 68 09; +43/664/233 90 75
www.postgresql.at <http://www.postgresql.at>, cluster.postgresql.at 
<http://cluster.postgresql.at>, www.cybertec.at 
<http://www.cybertec.at>, kernel.cybertec.at <http://kernel.cybertec.at>



Sat, 21 Dec 2002 10:48:05 +0100
=?ISO-8859-1?Q?Hans-J=FCrgen_Sch=F6nig?= <postgres@cybertec.at>
Re: Resource management in 7.4
Tom Lane <tgl@sss.pgh.pa.us> wrote:


> Gavin Sherry <swm@linuxworld.com.au> writes:
> > I've been thinking about resource management and postgres. I want to
> > develop a user profile system (a-la oracle) which allows a DBA to
> > restrict/configure access to system resources. This would allow a DBA to
> > configure how much CPU time can be used per query/session for any user,
> > the number blocks that can be read/written by a user per query, and
> > perhaps some other things (see below).
>
> I've got really serious reservations about this whole idea.  I don't
> like expending even one CPU cycle on it, and I don't like introducing a
> potential cause of unnecessary query failure, and I don't believe that
> the average DBA would be capable of configuring it intelligently.
[snip]
> Another example is that the cost of verifying transaction completion is
> actually paid by the first transaction to visit a tuple after the
> tuple's authoring transaction completes.  Should a transaction be
> penalized if it's foolish enough to do a seqscan shortly after someone
> else does a mass insert or update?

Just want to give my $0.02. I believe the whole resource restriction
idea is only interessting when different people share the same data
e.g. several departments have access to a company-wide database for
analysis purposes. In such a case it could be useful to restrict the
resource usage of some users.

On the other hand, in a shared hosting environment, I don't think
anyone would really like to have resource limits. What does it help
if your online shop stops working because too many people are
ordering stuff? IMHO it would be much more useful to have resource
usage accounting in that case, so that users can be charged for
added database usage. That would probably also be easier to
implement because each user would have their own database and
only pg_xlog shared -- that part seems tricky as you said before.

Best Regards,
Michael Paesold







Sat, 21 Dec 2002 10:49:19 +0100
"Michael Paesold" <mpaesold@gmx.at>
Re: Resource management in 7.4 
=?ISO-8859-1?Q?Hans-J=FCrgen_Sch=F6nig?= <postgres@cybertec.at> writes:
> It would be nice if there was a way to renice a connection. When it 
> comes to reporting it would be nice to have a handle for slowing a 
> backend down.

> A patch for Linux would be quite easy ("SELECT nice_backend(int)")

No, it wouldn't.  Or at least, the side effects of just nice'ing a
backend would not be what you'd wish.  See past discussions (try
searching the archives for "priority inversion").

			regards, tom lane

Sat, 21 Dec 2002 11:40:03 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: Resource management in 7.4 