Well, O_DIRECT has finally made it into the Linux kernel.  It lets you 
open a file in such a way that reads and writes don't go to the buffer 
cache but straight to the disk.  Accesses must be aligned on
filesystem block boundaries.

Is there any case where PG would benefit from this?  I can see it
reducing memory pressure and duplication of data between PG shared
buffers and the block buffer cache.  OTOH, it does require that writes 
be batched up for decent performance, since each write has an implicit 
fsync() involved (just as with O_SYNC).

Anyone played with this on systems that already support it (Solaris?)

-Doug
-- 
In a world of steel-eyed death, and men who are fighting to be warm,
Come in, she said, I'll give you shelter from the storm.    -Dylan

25 Sep 2001 14:56:59 -0400
Doug McNaught <doug@wireboard.com>
O_DIRECT and performance
> Well, O_DIRECT has finally made it into the Linux kernel.  It lets you 
> open a file in such a way that reads and writes don't go to the buffer 
> cache but straight to the disk.  Accesses must be aligned on
> filesystem block boundaries.
> 
> Is there any case where PG would benefit from this?  I can see it
> reducing memory pressure and duplication of data between PG shared
> buffers and the block buffer cache.  OTOH, it does require that writes 
> be batched up for decent performance, since each write has an implicit 
> fsync() involved (just as with O_SYNC).
> 
> Anyone played with this on systems that already support it (Solaris?)

I have heard there are many cases there O_DIRECT on Solaris is slower
for databases than normal I/O.  I think bulk copy was faster but not
normal operation.  Probably not something we are going to get into soon.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Thu, 27 Sep 2001 20:40:17 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: O_DIRECT and performance
Red Hat Linux 7.2, postgresql 7.2b2 configured like this:

./configure --enable-syslog --prefix=/usr --sysconfdir=/etc --with-pam
--with-openssl --with-unixodbc --with-tcl --with-perl --with-python
--enable-locale --enable-recode --enable-multibyte --enable-nls

"make check" fails on initdb:

============== creating temporary installation        ==============
============== initializing database system           ==============

pg_regress: initdb failed
Examine ./log/initdb.log for the reason.


teg@halden postgresql-7.2b2]$ cat src/test/regress/log/initdb.log 
Running with noclean mode on. Mistakes will not be cleaned up.
/home/devel/teg/postgresql-7.2b2/src/test/regress/./tmp_check/install//usr/bin/pg_encoding: relocation error: /home/devel/teg/postgresql-7.2b2/src/test/regress/./tmp_check/install//usr/bin/pg_encoding: undefined symbol: pg_valid_server_encoding
initdb: pg_encoding failed

Perhaps you did not configure PostgreSQL for multibyte support or
the program was not successfully installed.
[teg@halden postgresql-7.2b2]$

-- 
Trond Eivind Glomsrï¿½d
Red Hat, Inc.

16 Nov 2001 12:17:06 -0500
teg@redhat.com (Trond Eivind =?iso-8859-1?q?Glomsr=F8d?=)
7.2b2 "make check" failure on Red Hat Linux 7.2
teg@redhat.com (Trond Eivind Glomsrï¿½d) writes:

> Red Hat Linux 7.2, postgresql 7.2b2 configured like this:
> 
> ./configure --enable-syslog --prefix=/usr --sysconfdir=/etc --with-pam
> --with-openssl --with-unixodbc --with-tcl --with-perl --with-python
> --enable-locale --enable-recode --enable-multibyte --enable-nls

When just using "configure" it works - apart from a false failure in
the regression tests (geometry), it seems to work (a couple cases of
"-0.121320343559642" vs. "-0.121320343559643" - a better way than raw
diff for comparing would be useful).

-- 
Trond Eivind Glomsrï¿½d
Red Hat, Inc.

16 Nov 2001 12:47:27 -0500
teg@redhat.com (Trond Eivind =?iso-8859-1?q?Glomsr=F8d?=)
Re: 7.2b2 "make check" failure on Red Hat Linux 7.2
teg@redhat.com (Trond Eivind =?iso-8859-1?q?Glomsr=F8d?=) writes:
> /home/devel/teg/postgresql-7.2b2/src/test/regress/./tmp_check/install//usr/bin/pg_encoding: relocation error: /home/devel/teg/postgresql-7.2b2/src/test/regress/./tmp_check/install//usr/bin/pg_encoding: undefined symbol: pg_valid_server_encoding
> initdb: pg_encoding failed

pg_encoding relies on libpq to supply the pg_valid_server_encoding()
subroutine, but that subroutine is only compiled into libpq in a
MULTIBYTE build.  I speculate that your executable was picking up
a non-MULTIBYTE libpq shared library from someplace.  Check ldconfig
and all that stuff...

FWIW, both multibyte and non-multibyte builds are working OK for me
in current sources.

			regards, tom lane

Fri, 16 Nov 2001 14:26:27 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: 7.2b2 "make check" failure on Red Hat Linux 7.2 
Tom Lane <tgl@sss.pgh.pa.us> writes:

> teg@redhat.com (Trond Eivind =?iso-8859-1?q?Glomsr=F8d?=) writes:
> > /home/devel/teg/postgresql-7.2b2/src/test/regress/./tmp_check/install//usr/bin/pg_encoding: relocation error: /home/devel/teg/postgresql-7.2b2/src/test/regress/./tmp_check/install//usr/bin/pg_encoding: undefined symbol: pg_valid_server_encoding
> > initdb: pg_encoding failed
> 
> pg_encoding relies on libpq to supply the pg_valid_server_encoding()
> subroutine, but that subroutine is only compiled into libpq in a
> MULTIBYTE build.  I speculate that your executable was picking up
> a non-MULTIBYTE libpq shared library from someplace.  Check ldconfig
> and all that stuff...

I have an existing installation of 7.1 on the system, that's why I did
"make check" in the build directory.

"--prefix=/usr" seems to be the "culprit" - without it, it regression
tests run just fine. 

-- 
Trond Eivind Glomsrï¿½d
Red Hat, Inc.

16 Nov 2001 16:37:53 -0500
teg@redhat.com (Trond Eivind =?iso-8859-1?q?Glomsr=F8d?=)
Re: 7.2b2 "make check" failure on Red Hat Linux 7.2
teg@redhat.com (Trond Eivind =?iso-8859-1?q?Glomsr=F8d?=) writes:
> Tom Lane <tgl@sss.pgh.pa.us> writes:
>> I speculate that your executable was picking up
>> a non-MULTIBYTE libpq shared library from someplace.  Check ldconfig
>> and all that stuff...

> I have an existing installation of 7.1 on the system, that's why I did
> "make check" in the build directory.

> "--prefix=/usr" seems to be the "culprit" - without it, it regression
> tests run just fine. 

The pg_regress script sets LD_LIBRARY_PATH to try to cause libpq and
the other shlibs to be picked up from the temp installation tree.
Perhaps this is wrong or insufficient on your platform.  It certainly
sounds like the dynamic linker is choosing the installed libpq over
the one that we want it to use.  Any thoughts on fixing that?

			regards, tom lane

Fri, 16 Nov 2001 16:46:57 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: 7.2b2 "make check" failure on Red Hat Linux 7.2 
Tom Lane <tgl@sss.pgh.pa.us> writes:

> teg@redhat.com (Trond Eivind =?iso-8859-1?q?Glomsr=F8d?=) writes:
> > Tom Lane <tgl@sss.pgh.pa.us> writes:
> >> I speculate that your executable was picking up
> >> a non-MULTIBYTE libpq shared library from someplace.  Check ldconfig
> >> and all that stuff...
> 
> > I have an existing installation of 7.1 on the system, that's why I did
> > "make check" in the build directory.
> 
> > "--prefix=/usr" seems to be the "culprit" - without it, it regression
> > tests run just fine. 
> 
> The pg_regress script sets LD_LIBRARY_PATH to try to cause libpq and
> the other shlibs to be picked up from the temp installation tree.
> Perhaps this is wrong or insufficient on your platform.  It certainly
> sounds like the dynamic linker is choosing the installed libpq over
> the one that we want it to use.  Any thoughts on fixing that?

Since it works when prefix isn't /usr, I'd guess that the build
process sets rpath. This takes precedence over LD_LIBRARY_PATH.

Fix? Don't use rpath - it's evil, and should be avoided anyway.

ld(1) contains some info on how libraries are resolved.
-- 
Trond Eivind Glomsrï¿½d
Red Hat, Inc.

16 Nov 2001 16:50:49 -0500
teg@redhat.com (Trond Eivind =?iso-8859-1?q?Glomsr=F8d?=)
Re: 7.2b2 "make check" failure on Red Hat Linux 7.2
teg@redhat.com (Trond Eivind =?iso-8859-1?q?Glomsr=F8d?=) writes:
> Since it works when prefix isn't /usr, I'd guess that the build
> process sets rpath. This takes precedence over LD_LIBRARY_PATH.

> Fix? Don't use rpath - it's evil, and should be avoided anyway.

Peter, any thoughts on that?  It's not clear to me that removing
rpath settings would be an improvement overall, even if it makes
"make check" more reliable.

			regards, tom lane

Fri, 16 Nov 2001 17:39:16 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: 7.2b2 "make check" failure on Red Hat Linux 7.2 
* Tom Lane <tgl@sss.pgh.pa.us> [011116 16:56]:
> teg@redhat.com (Trond Eivind =?iso-8859-1?q?Glomsr=F8d?=) writes:
> > Since it works when prefix isn't /usr, I'd guess that the build
> > process sets rpath. This takes precedence over LD_LIBRARY_PATH.
> 
> > Fix? Don't use rpath - it's evil, and should be avoided anyway.
> 
> Peter, any thoughts on that?  It's not clear to me that removing
> rpath settings would be an improvement overall, even if it makes
> "make check" more reliable.
My general take is rpath is a *GOOD* thing, as it prevents USERS from
not being able to find REQUIRED shared libs. 

I vote for it staying.

LER

> 
> 			regards, tom lane
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 6: Have you searched our list archives?
> 
> http://archives.postgresql.org

-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 972-414-9812                 E-Mail: ler@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749

Fri, 16 Nov 2001 16:58:19 -0600
Larry Rosenman <ler@lerctr.org>
Re: 7.2b2 "make check" failure on Red Hat Linux 7.2
Trond Eivind Glomsrï¿½d writes:

> Fix? Don't use rpath - it's evil, and should be avoided anyway.

configure --disable-rpath is your friend.

Overall, rpath gives us much more benefit than the occasional trouble it
creates.

-- 
Peter Eisentraut   peter_e@gmx.net


Sun, 18 Nov 2001 17:42:37 +0100 (CET)
Peter Eisentraut <peter_e@gmx.net>
Re: 7.2b2 "make check" failure on Red Hat Linux 7.2
On Sun, 18 Nov 2001, Peter Eisentraut wrote:

> Trond Eivind Glomsrï¿½d writes:
> 
> > Fix? Don't use rpath - it's evil, and should be avoided anyway.
> 
> configure --disable-rpath is your friend.

Didn't know about that one - it will certainly be put to use :)

> Overall, rpath gives us much more benefit than the occasional trouble it
> creates.

When you install (not configure for, just install) into a separate tree 
(for easier packaging), it's a hole which can be exploited - some packages 
will rpath into /var/tmp/<foo>, for  instance. Hackers can then put their 
own library there. One big offender here is perl's automatic module 
creation script which will change the rpaths from what you told it when 
you built it to what you do when you install it.

-- 
Trond Eivind Glomsrï¿½d
Red Hat, Inc.


Sun, 18 Nov 2001 11:45:24 -0500 (EST)
=?ISO-8859-1?Q?Trond_Eivind_Glomsr=F8d?= <teg@redhat.com>
Re: 7.2b2 "make check" failure on Red Hat Linux 7.2
Trond Eivind Glomsrï¿½d writes:

> When you install (not configure for, just install) into a separate tree
> (for easier packaging), it's a hole which can be exploited - some packages
> will rpath into /var/tmp/<foo>, for  instance. Hackers can then put their
> own library there.

"Some packages"... ;-)

> One big offender here is perl's automatic module
> creation script which will change the rpaths from what you told it when
> you built it to what you do when you install it.

This should be fixed now, although the perl module will actually not obey
the --disable-rpath switch.  Can't have everything, I guess...

-- 
Peter Eisentraut   peter_e@gmx.net


Sun, 18 Nov 2001 21:40:12 +0100 (CET)
Peter Eisentraut <peter_e@gmx.net>
Re: 7.2b2 "make check" failure on Red Hat Linux 7.2
Larry Rosenman <ler@lerctr.org> writes:

> * Tom Lane <tgl@sss.pgh.pa.us> [011116 16:56]:
> > teg@redhat.com (Trond Eivind =?iso-8859-1?q?Glomsr=F8d?=) writes:
> > > Since it works when prefix isn't /usr, I'd guess that the build
> > > process sets rpath. This takes precedence over LD_LIBRARY_PATH.
> > 
> > > Fix? Don't use rpath - it's evil, and should be avoided anyway.
> > 
> > Peter, any thoughts on that?  It's not clear to me that removing
> > rpath settings would be an improvement overall, even if it makes
> > "make check" more reliable.
> My general take is rpath is a *GOOD* thing, as it prevents USERS from
> not being able to find REQUIRED shared libs. 

Lib-directories should be listed in /etc/ld.so.conf, and no rpath used
- at least on linux.

-- 
Trond Eivind Glomsrï¿½d
Red Hat, Inc.

19 Nov 2001 11:47:45 -0500
teg@redhat.com (Trond Eivind =?iso-8859-1?q?Glomsr=F8d?=)
Re: 7.2b2 "make check" failure on Red Hat Linux 7.2
* Trond Eivind Glomsr?d <teg@redhat.com> [011119 10:47]:
> Larry Rosenman <ler@lerctr.org> writes:
> 
> > * Tom Lane <tgl@sss.pgh.pa.us> [011116 16:56]:
> > > teg@redhat.com (Trond Eivind =?iso-8859-1?q?Glomsr=F8d?=) writes:
> > > > Since it works when prefix isn't /usr, I'd guess that the build
> > > > process sets rpath. This takes precedence over LD_LIBRARY_PATH.
> > > 
> > > > Fix? Don't use rpath - it's evil, and should be avoided anyway.
> > > 
> > > Peter, any thoughts on that?  It's not clear to me that removing
> > > rpath settings would be an improvement overall, even if it makes
> > > "make check" more reliable.
> > My general take is rpath is a *GOOD* thing, as it prevents USERS from
> > not being able to find REQUIRED shared libs. 
> 
> Lib-directories should be listed in /etc/ld.so.conf, and no rpath used
> - at least on linux.
no such animal on OpenUNIX 8 (In native "unix" mode).


> 
> -- 
> Trond Eivind Glomsrï¿½d
> Red Hat, Inc.

-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 972-414-9812                 E-Mail: ler@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749

Mon, 19 Nov 2001 10:53:37 -0600
Larry Rosenman <ler@lerctr.org>
Re: 7.2b2 "make check" failure on Red Hat Linux 7.2