
As I was documenting my current project the other day, a couple of things
occurred to me:

1. It would be nice to be able to dump my schema in some XML format that I
could then process using XSLT into whatever format I desired. If there is
not already some easy way to do this, and there is interest, I am prepared
to put some work into such a project, (based, I guess, on pg_dump, but I am
open to suggestions on how to approach this). I am slightly unclear about
the current roadmap for releases - is 7.5 targeted at the end of the year,
roughly?

2. Although I can comment on a constraint, none of the psql metacommands
appear to show me the relevant comment, which is a pity.

cheers

andrew



Tue, 22 Jul 2003 06:16:22 -0500 (EST)
"Andrew Dunstan" <andrew@dunslane.net>
a couple of suggestions
I've been thinking some about how exactly to provide the new option of
thread-safe clients (libpq, ecpg).  Let me state the following goals:

a. Thread-safeness, where it makes a difference, is generally thought to
be a performance hit, so the user needs to have a choice to use
thread-safe libraries or not.

b. The user needs to be able to make the choice at the time he builds his
application, *not* at the time the PostgreSQL distribution is built or
installed.

Clearly, a thread-safe ecpg library is always going to be significantly
different from the "normal" one, with all the mutex things that get pulled
in, so it seems reasonable to always offer a libecpg_r alongside the
libecpg.

The question is whether a libpq_r should be provided if libpq is
thread-safe by default (no *_r functions, libc_r, or special flags).  I
think yes.  It could be a symlink, so it doesn't really waste space. But
it would convenience users: Those who want to be sure to always link
against a thread-safe version can point to libpq_r and don't have to
create complicated detection mechanisms. Those who know that their system
is thread-safe by default can simply use libpq to follow that convention.
And of course it creates consistency with libecpg_r and does not bother
the user with complicated internal artifacts.

A final note on the name of the configure option, --with-threads. First,
it does not control an external package but an internal feature, so it
should be --enable-.  Secondly, it does not use threads, only enable
thread-safeness.  So --enable-thread-safe might be a better name.  Or if
you want to be more precise, --enable-thread-safe-client.  The latter is
what MySQL uses, in case anyone cares about that.

-- 
Peter Eisentraut   peter_e@gmx.net

Tue, 22 Jul 2003 16:55:55 +0200 (CEST)
Peter Eisentraut <peter_e@gmx.net>
None
Peter Eisentraut <peter_e@gmx.net> writes:
> I've been thinking some about how exactly to provide the new option of
> thread-safe clients (libpq, ecpg).  Let me state the following goals:

Sounds good to me.  Do you have time to make these things happen?

			regards, tom lane

Tue, 22 Jul 2003 11:11:33 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: 


--On Tuesday, July 22, 2003 11:11:33 -0400 Tom Lane <tgl@sss.pgh.pa.us> 
wrote:

> Peter Eisentraut <peter_e@gmx.net> writes:
>> I've been thinking some about how exactly to provide the new option of
>> thread-safe clients (libpq, ecpg).  Let me state the following goals:
>
> Sounds good to me.  Do you have time to make these things happen?
Please see my post to -hackers last week about -K[p]thread on UnixWare. 
There
are some issues there for us.

Also, Peter, can you look at the forwards I've made from Kean Johnson of 
SCO on
-Patches about the OSR5 patch, and DT_SONAME?

Thanks,
LER

>
> 			regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
>



-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 972-414-9812                 E-Mail: ler@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749


Tue, 22 Jul 2003 10:19:00 -0500
Larry Rosenman <ler@lerctr.org>
Re: 
On 22 Jul 2003 at 16:55, Peter Eisentraut wrote:

> I've been thinking some about how exactly to provide the new option of
> thread-safe clients (libpq, ecpg).  Let me state the following goals:
> 
> a. Thread-safeness, where it makes a difference, is generally thought to
> be a performance hit, so the user needs to have a choice to use
> thread-safe libraries or not.

On linux and freeBSD, that is not an issue. I can attest that from my 
experiments with a webserver. Thread function calls often time less than or of 
the order of 1us even with CPU capped.

I would really like to know for what platforms, locking mutex while selecting 
from connection list or some such object, is going to be such a great 
performance hit.

Performance degradation in threaded programs comes from contentions. It goes 
for frequency of contention and number of threads fighting over it.

I doubt any threaded ecpg program would reach that level of contention anytime. 
If a lock canbe obtained/released in less than 10us and subsequent database 
query is going to take at least a ms, IMO that performance degradation is not 
worth that much trouble.

But that is for linux and freeBSD. What other platforms have serious thread 
issues?

> Clearly, a thread-safe ecpg library is always going to be significantly
> different from the "normal" one, with all the mutex things that get pulled
> in, so it seems reasonable to always offer a libecpg_r alongside the
> libecpg.

I would say, it should be thread-safe by default. No point polluting possible 
linkages.

I repeat what I have said earlier. If there are two libraries A using libecpg_r 
and B, using libecpg, then program linking against both of them is going to 
have tough time living with symbol conflicts. 

I suppose problem will be reproducible even under freeBSD if you try to create  
a postgresql function in C which uses threads. Link the library against libc_r 
and link postgresql against libc. It would run into problems.

I am just stating my experiences.I might have missed solution to this problem. 

But overall I like GNU libc approach of everything thread safe by default. If 
thread performance is an issue, then it should be improved. Not worked around 
with two libraries.

Just a thought..

Bye
 Shridhar

--
2180, U.S. History question:	What 20th Century U.S. President was almost impeached and what	office did he later hold?


Tue, 22 Jul 2003 20:57:45 +0530
"Shridhar Daithankar" <shridhar_daithankar@persistent.co.in>
Re: 
Peter Eisentraut wrote:
> I've been thinking some about how exactly to provide the new option of
> thread-safe clients (libpq, ecpg).  Let me state the following goals:
> 
> a. Thread-safeness, where it makes a difference, is generally thought to
> be a performance hit, so the user needs to have a choice to use
> thread-safe libraries or not.

OK.

> b. The user needs to be able to make the choice at the time he builds his
> application, *not* at the time the PostgreSQL distribution is built or
> installed.

Agreed.

> Clearly, a thread-safe ecpg library is always going to be significantly
> different from the "normal" one, with all the mutex things that get pulled
> in, so it seems reasonable to always offer a libecpg_r alongside the
> libecpg.

True.  And we have lots of platforms that we don't know how to enable
thread-safeness.

> The question is whether a libpq_r should be provided if libpq is
> thread-safe by default (no *_r functions, libc_r, or special flags).  I
> think yes.  It could be a symlink, so it doesn't really waste space. But
> it would convenience users: Those who want to be sure to always link
> against a thread-safe version can point to libpq_r and don't have to
> create complicated detection mechanisms. Those who know that their system
> is thread-safe by default can simply use libpq to follow that convention.
> And of course it creates consistency with libecpg_r and does not bother
> the user with complicated internal artifacts.

I think adding a libpq_r on a platform that doesn't use *_r libraries is
just too confusing.

> A final note on the name of the configure option, --with-threads. First,
> it does not control an external package but an internal feature, so it
> should be --enable-.  Secondly, it does not use threads, only enable
> thread-safeness.  So --enable-thread-safe might be a better name.  Or if
> you want to be more precise, --enable-thread-safe-client.  The latter is
> what MySQL uses, in case anyone cares about that.

Peter, you are good with these subtle distinctions.  How about
--enable-thread-safeness?

I have an idea.  Seems we have two issues, libpq and ecpg.

For ecpg, we need a command-line flag to specify if we want threading in
the application.

For libpq and libpecpg, we will just produce whatever therading library
should be created on that platform, meaning either just one library, or
one with _r and one without.

So we tell them, if you want the ability to do threaded libpq and ecpg,
use the --enable-thread-safeness configure flag.  If it succeeds, it
means we know how to do it for that operating system, and you get
threading capability.  If it fails, you don't have threading.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Tue, 22 Jul 2003 14:31:58 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: 
Larry Rosenman writes:

> Please see my post to -hackers last week about -K[p]thread on UnixWare.
> There are some issues there for us.

The take-home message from those posts is that if you use non-thread-safe
code in threaded applications, bad things will happen.  We knew that. ;-)
As far as mixing code compiled one way with code compiled the other way,
there is no problem, at least not more than on any other platform.

-- 
Peter Eisentraut   peter_e@gmx.net

Wed, 23 Jul 2003 12:12:45 +0200 (CEST)
Peter Eisentraut <peter_e@gmx.net>
Re: 


--On Wednesday, July 23, 2003 12:12:45 +0200 Peter Eisentraut 
<peter_e@gmx.net> wrote:

> Larry Rosenman writes:
>
>> Please see my post to -hackers last week about -K[p]thread on UnixWare.
>> There are some issues there for us.
>
> The take-home message from those posts is that if you use non-thread-safe
> code in threaded applications, bad things will happen.  We knew that. ;-)
> As far as mixing code compiled one way with code compiled the other way,
> there is no problem, at least not more than on any other platform.
if we are heading for thread-safe libpq, I'd suggest, at least on UnixWare, 
compiling
libpq with -D_REENTRANT, because of the macro vs. function issue, at least 
for errno,
and possibly others.

LER




-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 972-414-9812                 E-Mail: ler@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749


Wed, 23 Jul 2003 07:16:58 -0500
Larry Rosenman <ler@lerctr.org>
Re: 
Shridhar Daithankar writes:

> On linux and freeBSD, that is not an issue.

FreeBSD has a separate libc_r.  I don't think they would bother with that
if there was no significant difference.

> I would really like to know for what platforms, locking mutex while selecting
> from connection list or some such object, is going to be such a great
> performance hit.

The performance hit isn't in the mutexes or specifically here or there.
But when you go into thread-safe/reentrant land, wild and wonderful things
happen, different on each platform.  We could work out a specific solution
for each platform that fits well with the local custom and is most
convenient to users.  But that is not maintainable.  So we need to work
out some kind of generalization that works similarly on most platforms.
I'm still unsure what that should really be, but everything thread-safe by
default probably won't suffice.

-- 
Peter Eisentraut   peter_e@gmx.net

Wed, 23 Jul 2003 16:23:08 +0200 (CEST)
Peter Eisentraut <peter_e@gmx.net>
Re: 
Peter Eisentraut wrote:
> A final note on the name of the configure option, --with-threads. First,
> it does not control an external package but an internal feature, so it
> should be --enable-.  Secondly, it does not use threads, only enable
> thread-safeness.  So --enable-thread-safe might be a better name.  Or if
> you want to be more precise, --enable-thread-safe-client.  The latter is
> what MySQL uses, in case anyone cares about that.

Here is an applied patch to implement the rename you suggested, to
--enable-thread-safeness.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Index: configure
===================================================================
RCS file: /cvsroot/pgsql-server/configure,v
retrieving revision 1.277
diff -c -c -r1.277 configure
*** configure	22 Jul 2003 16:39:54 -0000	1.277
--- configure	23 Jul 2003 17:08:27 -0000
***************
*** 855,861 ****
    --with-libraries=DIRS   look for additional libraries in DIRS
    --with-libs=DIRS        alternative spelling of --with-libraries
    --with-pgport=PORTNUM   change default port number 5432
!   --with-threads          allow libpq and ecpg to be thread-safe
    --with-tcl              build Tcl and Tk interfaces
    --without-tk            do not build Tk interfaces if Tcl is enabled
    --with-tclconfig=DIR    tclConfig.sh and tkConfig.sh are in DIR
--- 855,861 ----
    --with-libraries=DIRS   look for additional libraries in DIRS
    --with-libs=DIRS        alternative spelling of --with-libraries
    --with-pgport=PORTNUM   change default port number 5432
!   --enable-thread-safeness allow libpq and ecpg to be thread-safe
    --with-tcl              build Tcl and Tk interfaces
    --without-tk            do not build Tk interfaces if Tcl is enabled
    --with-tclconfig=DIR    tclConfig.sh and tkConfig.sh are in DIR
***************
*** 2811,2820 ****
  IFS=$ac_save_IFS
  
  #
! # Enable libpq to be thread-safe
  #
! echo "$as_me:$LINENO: checking allow threaded libpq" >&5
! echo $ECHO_N "checking allow threaded libpq... $ECHO_C" >&6
  
  
  
--- 2811,2820 ----
  IFS=$ac_save_IFS
  
  #
! # Enable libpq to be thread-safeness
  #
! echo "$as_me:$LINENO: checking allow thread-safe libpq and ecpg" >&5
! echo $ECHO_N "checking allow thread-safe libpq and ecpg... $ECHO_C" >&6
  
  
  
***************
*** 2846,2853 ****
  fi;
  
  
! echo "$as_me:$LINENO: result: $with_threads" >&5
! echo "${ECHO_T}$with_threads" >&6
  
  
  #
--- 2846,2853 ----
  fi;
  
  
! echo "$as_me:$LINENO: result: $enable_thread_safeness" >&5
! echo "${ECHO_T}$enable_thread_safeness" >&6
  
  
  #
***************
*** 12807,12813 ****
  # For each platform, we need to know about any special compile and link
  # libraries, and whether the normal C function names are thread-safe.
  #
! if test "$with_threads" = yes; then
  if test "${ac_cv_header_pthread_h+set}" = set; then
    echo "$as_me:$LINENO: checking for pthread.h" >&5
  echo $ECHO_N "checking for pthread.h... $ECHO_C" >&6
--- 12807,12813 ----
  # For each platform, we need to know about any special compile and link
  # libraries, and whether the normal C function names are thread-safe.
  #
! if test "$enable_thread_safeness" = yes; then
  if test "${ac_cv_header_pthread_h+set}" = set; then
    echo "$as_me:$LINENO: checking for pthread.h" >&5
  echo $ECHO_N "checking for pthread.h... $ECHO_C" >&6
***************
*** 12949,12955 ****
  # One trick here is that if we don't call AC_CHECK_FUNCS, the
  # functions are marked "not found", which is perfect.
  #
! if test "$with_threads" = yes -a "$NEED_REENTRANT_FUNC_NAMES" = yes ; then
  _CFLAGS="$CFLAGS"
  _LIB="$LIBS"
  CFLAGS="$CFLAGS $TREAD_CFLAGS"
--- 12949,12955 ----
  # One trick here is that if we don't call AC_CHECK_FUNCS, the
  # functions are marked "not found", which is perfect.
  #
! if test "$enable_thread_safeness" = yes -a "$NEED_REENTRANT_FUNC_NAMES" = yes ; then
  _CFLAGS="$CFLAGS"
  _LIB="$LIBS"
  CFLAGS="$CFLAGS $TREAD_CFLAGS"
***************
*** 17807,17813 ****
  s,@GCC@,$GCC,;t t
  s,@autodepend@,$autodepend,;t t
  s,@INCLUDES@,$INCLUDES,;t t
! s,@with_threads@,$with_threads,;t t
  s,@with_tcl@,$with_tcl,;t t
  s,@with_tk@,$with_tk,;t t
  s,@with_perl@,$with_perl,;t t
--- 17807,17813 ----
  s,@GCC@,$GCC,;t t
  s,@autodepend@,$autodepend,;t t
  s,@INCLUDES@,$INCLUDES,;t t
! s,@enable_thread_safeness@,$enable_thread_safeness,;t t
  s,@with_tcl@,$with_tcl,;t t
  s,@with_tk@,$with_tk,;t t
  s,@with_perl@,$with_perl,;t t
Index: configure.in
===================================================================
RCS file: /cvsroot/pgsql-server/configure.in,v
retrieving revision 1.268
diff -c -c -r1.268 configure.in
*** configure.in	22 Jul 2003 16:39:55 -0000	1.268
--- configure.in	23 Jul 2003 17:08:31 -0000
***************
*** 320,333 ****
  IFS=$ac_save_IFS
  
  #
! # Enable libpq to be thread-safe
  #
! AC_MSG_CHECKING([allow threaded libpq])
! PGAC_ARG_BOOL(with, threads, no, [  --with-threads          allow libpq and ecpg to be thread-safe],
!               [AC_DEFINE([USE_THREADS], 1, [Define to 1 to build libpq and ecpg to be thread-safe. (--with-threads)])])
  
! AC_MSG_RESULT([$with_threads])
! AC_SUBST(with_threads)
  
  #
  # Tcl/Tk
--- 320,333 ----
  IFS=$ac_save_IFS
  
  #
! # Enable libpq to be thread-safeness
  #
! AC_MSG_CHECKING([allow thread-safe libpq and ecpg])
! PGAC_ARG_BOOL(with, threads, no, [  --enable-thread-safeness allow libpq and ecpg to be thread-safe],
!               [AC_DEFINE([USE_THREADS], 1, [Define to 1 to build libpq and ecpg to be thread-safe. (--enable-thread-safeness)])])
  
! AC_MSG_RESULT([$enable_thread_safeness])
! AC_SUBST(enable_thread_safeness)
  
  #
  # Tcl/Tk
***************
*** 965,971 ****
  # For each platform, we need to know about any special compile and link
  # libraries, and whether the normal C function names are thread-safe.
  #
! if test "$with_threads" = yes; then
  AC_CHECK_HEADER(pthread.h, [], [AC_MSG_ERROR([pthread.h not found, required for --with-threads])])
  
  if test "$SUPPORTS_THREADS" != yes; then
--- 965,971 ----
  # For each platform, we need to know about any special compile and link
  # libraries, and whether the normal C function names are thread-safe.
  #
! if test "$enable_thread_safeness" = yes; then
  AC_CHECK_HEADER(pthread.h, [], [AC_MSG_ERROR([pthread.h not found, required for --with-threads])])
  
  if test "$SUPPORTS_THREADS" != yes; then
***************
*** 991,997 ****
  # One trick here is that if we don't call AC_CHECK_FUNCS, the
  # functions are marked "not found", which is perfect.
  #
! if test "$with_threads" = yes -a "$NEED_REENTRANT_FUNC_NAMES" = yes ; then
  _CFLAGS="$CFLAGS"
  _LIB="$LIBS"
  CFLAGS="$CFLAGS $TREAD_CFLAGS"
--- 991,997 ----
  # One trick here is that if we don't call AC_CHECK_FUNCS, the
  # functions are marked "not found", which is perfect.
  #
! if test "$enable_thread_safeness" = yes -a "$NEED_REENTRANT_FUNC_NAMES" = yes ; then
  _CFLAGS="$CFLAGS"
  _LIB="$LIBS"
  CFLAGS="$CFLAGS $TREAD_CFLAGS"
Index: doc/src/sgml/ecpg.sgml
===================================================================
RCS file: /cvsroot/pgsql-server/doc/src/sgml/ecpg.sgml,v
retrieving revision 1.44
diff -c -c -r1.44 ecpg.sgml
*** doc/src/sgml/ecpg.sgml	15 Jun 2003 04:07:58 -0000	1.44
--- doc/src/sgml/ecpg.sgml	23 Jul 2003 17:08:34 -0000
***************
*** 751,757 ****
  
    <para>
     <application>ecpg</application> is thread-safe if it is compiled using
!    the <literal>--with-threads</> <filename>configure</filename>
     command-line option.  (You might need to use other threading
     command-line options to compile your client code.)
    </para>
--- 751,757 ----
  
    <para>
     <application>ecpg</application> is thread-safe if it is compiled using
!    the <literal>--enable-thread-safeness</> <filename>configure</filename>
     command-line option.  (You might need to use other threading
     command-line options to compile your client code.)
    </para>
Index: doc/src/sgml/installation.sgml
===================================================================
RCS file: /cvsroot/pgsql-server/doc/src/sgml/installation.sgml,v
retrieving revision 1.135
diff -c -c -r1.135 installation.sgml
*** doc/src/sgml/installation.sgml	15 Jun 2003 04:07:58 -0000	1.135
--- doc/src/sgml/installation.sgml	23 Jul 2003 17:08:37 -0000
***************
*** 915,921 ****
        </varlistentry>
  
        <varlistentry>
!        <term><option>--with-threads</option></term>
         <listitem>
          <para>
  	 Allow separate libpq and ecpg threads to safely control their
--- 915,921 ----
        </varlistentry>
  
        <varlistentry>
!        <term><option>--enable-thread-safeness</option></term>
         <listitem>
          <para>
  	 Allow separate libpq and ecpg threads to safely control their
Index: doc/src/sgml/libpq.sgml
===================================================================
RCS file: /cvsroot/pgsql-server/doc/src/sgml/libpq.sgml,v
retrieving revision 1.127
diff -c -c -r1.127 libpq.sgml
*** doc/src/sgml/libpq.sgml	27 Jun 2003 19:08:37 -0000	1.127
--- doc/src/sgml/libpq.sgml	23 Jul 2003 17:08:44 -0000
***************
*** 3268,3276 ****
  
  <para>
  <application>libpq</application> is thread-safe if the library is
! compiled using <filename>configure</filename>'s <literal>--with-threads</>
! command-line option.  (In addition, you might need to
! use other threading command-line options to compile your client code.)
  </para>
  
  <para>
--- 3268,3277 ----
  
  <para>
  <application>libpq</application> is thread-safe if the library is
! compiled using <filename>configure</filename>'s 
! <literal>--enable-thread-safeness</> command-line option.  
! (In addition, you might need to use other threading command-line 
! options to compile your client code.)
  </para>
  
  <para>
Wed, 23 Jul 2003 13:26:42 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: 
Shridhar Daithankar wrote:
> I repeat what I have said earlier. If there are two libraries A using libecpg_r 
> and B, using libecpg, then program linking against both of them is going to 
> have tough time living with symbol conflicts. 
> 
> I suppose problem will be reproducible even under freeBSD if you try to create  
> a postgresql function in C which uses threads. Link the library against libc_r 
> and link postgresql against libc. It would run into problems.
> 
> I am just stating my experiences.I might have missed solution to this problem. 
> 
> But overall I like GNU libc approach of everything thread safe by default. If 
> thread performance is an issue, then it should be improved. Not worked around 
> with two libraries.

I thought glibc was the one to introduce libc_r in the first place ---
are they making libc thread-safe now?

What OS's are still using libc_r for threaded-ness?  I never liked that
approach myself, and I resist adding it to our setup unless it is
required.

One problem now is that we don't have a way to create a separate libpq_r
for operating systems that use libc_r.  We just create libpq and it is
thread-safe.

As for the configure flag, we still need it because we don't know the
flags required by all our supported OS's.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Wed, 23 Jul 2003 13:35:25 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: 

Larry, please see configure.in and the template file and report the
changes needed for unixware and sco.  See linux and freebsd for
examples.

---------------------------------------------------------------------------

Larry Rosenman wrote:
> 
> 
> --On Wednesday, July 23, 2003 12:12:45 +0200 Peter Eisentraut 
> <peter_e@gmx.net> wrote:
> 
> > Larry Rosenman writes:
> >
> >> Please see my post to -hackers last week about -K[p]thread on UnixWare.
> >> There are some issues there for us.
> >
> > The take-home message from those posts is that if you use non-thread-safe
> > code in threaded applications, bad things will happen.  We knew that. ;-)
> > As far as mixing code compiled one way with code compiled the other way,
> > there is no problem, at least not more than on any other platform.
> if we are heading for thread-safe libpq, I'd suggest, at least on UnixWare, 
> compiling
> libpq with -D_REENTRANT, because of the macro vs. function issue, at least 
> for errno,
> and possibly others.
> 
> LER
> 
> 
> 
> 
> -- 
> Larry Rosenman                     http://www.lerctr.org/~ler
> Phone: +1 972-414-9812                 E-Mail: ler@lerctr.org
> US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
> 
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 6: Have you searched our list archives?
> 
>                http://archives.postgresql.org
> 

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Wed, 23 Jul 2003 13:38:25 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: 


--On Wednesday, July 23, 2003 13:38:25 -0400 Bruce Momjian 
<pgman@candle.pha.pa.us> wrote:

>
> Larry, please see configure.in and the template file and report the
> changes needed for unixware and sco.  See linux and freebsd for
> examples.
Can I deal with this after the beta tarball is up?

LER


-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 972-414-9812                 E-Mail: ler@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749


Wed, 23 Jul 2003 21:19:21 -0500
Larry Rosenman <ler@lerctr.org>
Re: 
Larry Rosenman wrote:
> 
> 
> --On Wednesday, July 23, 2003 13:38:25 -0400 Bruce Momjian 
> <pgman@candle.pha.pa.us> wrote:
> 
> >
> > Larry, please see configure.in and the template file and report the
> > changes needed for unixware and sco.  See linux and freebsd for
> > examples.
> Can I deal with this after the beta tarball is up?

Sure.  It can be fiddled with up until final release.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Wed, 23 Jul 2003 22:20:58 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: 
On 23 Jul 2003 at 13:35, Bruce Momjian wrote:
> I thought glibc was the one to introduce libc_r in the first place ---
> are they making libc thread-safe now?

AFAIK, glibc plays all threadsafe in glibc2. Don't know prior to that. Of 
course they do record if some of the routines are not thread safe but there is 
no separate library as such.

by now, it is over all threadsafe. I don't have list of exceptions, but surely 
they are pretty small, if any.

> 
> What OS's are still using libc_r for threaded-ness?  I never liked that
> approach myself, and I resist adding it to our setup unless it is
> required.

FreeBSD and windows..:-) Two I know of..

> As for the configure flag, we still need it because we don't know the
> flags required by all our supported OS's.

Agreed.

Bye
 Shridhar

--
"Waving away a cloud of smoke, I look up, and am blinded by a bright, whitelight. It's God. No, not Richard Stallman, or Linus Torvalds, but God. Ina booming voice, He says: "THIS IS A SIGN. USE LINUX, THE FREE UNIX SYSTEMFOR THE 386."(Matt Welsh)


Thu, 24 Jul 2003 12:15:52 +0530
"Shridhar Daithankar" <shridhar_daithankar@persistent.co.in>
Re: 
Guys, take a look at what was done in libpq to make it
thread-safe... No locks! No overheaded - just using "proper" reentrant
functions...

If we have libpq_r then we're making a complete hash of it all - being
reentrant is good, even if you're not using threads!

Now, ecpg is another issue...

L.

Bruce Momjian writes:
 > Shridhar Daithankar wrote:
 > > I repeat what I have said earlier. If there are two libraries A using libecpg_r 
 > > and B, using libecpg, then program linking against both of them is going to 
 > > have tough time living with symbol conflicts. 
 > > 
 > > I suppose problem will be reproducible even under freeBSD if you try to create  
 > > a postgresql function in C which uses threads. Link the library against libc_r 
 > > and link postgresql against libc. It would run into problems.
 > > 
 > > I am just stating my experiences.I might have missed solution to this problem. 
 > > 
 > > But overall I like GNU libc approach of everything thread safe by default. If 
 > > thread performance is an issue, then it should be improved. Not worked around 
 > > with two libraries.
 > 
 > I thought glibc was the one to introduce libc_r in the first place ---
 > are they making libc thread-safe now?
 > 
 > What OS's are still using libc_r for threaded-ness?  I never liked that
 > approach myself, and I resist adding it to our setup unless it is
 > required.
 > 
 > One problem now is that we don't have a way to create a separate libpq_r
 > for operating systems that use libc_r.  We just create libpq and it is
 > thread-safe.
 > 
 > As for the configure flag, we still need it because we don't know the
 > flags required by all our supported OS's.

Thu, 24 Jul 2003 09:22:19 +0100
Lee Kindness <lkindness@csl.co.uk>
libpq_r
Lee Kindness wrote:
> Guys, take a look at what was done in libpq to make it
> thread-safe... No locks! No overheaded - just using "proper" reentrant
> functions...
> 
> If we have libpq_r then we're making a complete hash of it all - being
> reentrant is good, even if you're not using threads!
> 
> Now, ecpg is another issue...
> 

I think the issue is that using a threaded library to link into libpq
could have locking stuff.

My guess is that if the OS has separate threaded libs, we have to mimic
that stuff.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Thu, 24 Jul 2003 08:52:37 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: libpq_r
Shridhar Daithankar wrote:
> On 23 Jul 2003 at 13:35, Bruce Momjian wrote:
> > I thought glibc was the one to introduce libc_r in the first place ---
> > are they making libc thread-safe now?
> 
> AFAIK, glibc plays all threadsafe in glibc2. Don't know prior to that. Of 
> course they do record if some of the routines are not thread safe but there is 
> no separate library as such.
> 
> by now, it is over all threadsafe. I don't have list of exceptions, but surely 
> they are pretty small, if any.
> 
> > 
> > What OS's are still using libc_r for threaded-ness?  I never liked that
> > approach myself, and I resist adding it to our setup unless it is
> > required.
> 
> FreeBSD and windows..:-) Two I know of..

The strange thing is that accoring to template/freebsd, libc_r was
_added_ in FreeBSD 5.0, while most OS are moving away from separate
threaded libs.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Thu, 24 Jul 2003 08:59:38 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: 


--On Thursday, July 24, 2003 08:52:37 -0400 Bruce Momjian 
<pgman@candle.pha.pa.us> wrote:

> Lee Kindness wrote:
>> Guys, take a look at what was done in libpq to make it
>> thread-safe... No locks! No overheaded - just using "proper" reentrant
>> functions...
>>
>> If we have libpq_r then we're making a complete hash of it all - being
>> reentrant is good, even if you're not using threads!
>>
>> Now, ecpg is another issue...
>>
>
> I think the issue is that using a threaded library to link into libpq
> could have locking stuff.
>
> My guess is that if the OS has separate threaded libs, we have to mimic
> that stuff.
possibly.  On UnixWare, the original executable in the process needs to 
have -K[p]thread
specified to get libthread included.  I'll be adding -D_REENTRANT to the 
libpq build for
UnixWare to make sure that libpq is threadsafe, but doesn't include the 
thread library as
a pre-requisite per input from SCO.

BTW, Bruce, SCO OpenServer does NOT have threads.

LER




-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 972-414-9812                 E-Mail: ler@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749


Thu, 24 Jul 2003 08:09:52 -0500
Larry Rosenman <ler@lerctr.org>
Re: libpq_r
On 24 Jul 2003 at 8:59, Bruce Momjian wrote:

> Shridhar Daithankar wrote:
> > FreeBSD and windows..:-) Two I know of..
> 
> The strange thing is that accoring to template/freebsd, libc_r was
> _added_ in FreeBSD 5.0, while most OS are moving away from separate
> threaded libs.

Well, I have used libc_r for native freeBSD threads in 4.7 so that information 
is certainly not correct..

Bye
 Shridhar

--
Beam me up, Scotty!  It ate my phaser!


Thu, 24 Jul 2003 18:41:40 +0530
"Shridhar Daithankar" <shridhar_daithankar@persistent.co.in>
Re: 
Bruce Momjian writes:
 > Lee Kindness wrote:
 > > Guys, take a look at what was done in libpq to make it
 > > thread-safe... No locks! No overheaded - just using "proper" reentrant
 > > functions...
 > > If we have libpq_r then we're making a complete hash of it all - being
 > > reentrant is good, even if you're not using threads!
 > > Now, ecpg is another issue...
 > I think the issue is that using a threaded library to link into libpq
 > could have locking stuff.
 > 
 > My guess is that if the OS has separate threaded libs, we have to mimic
 > that stuff.

But there are NO thread primitives/calls in libpq - it's just a normal
shared library, it has has no thread voodoo! A threaded app/library
using libpq doesn't have to do anything special... libpq doesn't need
link the threads library (and it explicitly should not) - only define
_REENTRANT.

The reentrant calls libpq may be using are in the standard system
libraries (not the system thread library), the same libraries that the
app/libs will be linked against.

Now, libecpg on the otherhand does need mutex locks compiled in. Thus
if this is enabled then anyone who links against it also needs to link
in threads (unless the system is "intelligent" like Solaris and
includes empty stubs of the thread functions in libc).

However in this day and age the overhead of the locks are negligible
and it is more desirable to have one library...

L.

Thu, 24 Jul 2003 14:12:04 +0100
Lee Kindness <lkindness@csl.co.uk>
Re: libpq_r
Bruce Momjian writes:

> The strange thing is that accoring to template/freebsd, libc_r was
> _added_ in FreeBSD 5.0, while most OS are moving away from separate
> threaded libs.

The template is wrong.  libc_r has existed in FreeBSD for quite a while,
but your're not supposed to use it directly.  Instead, you use the flag
-pthread, which takes care of libc_r and possibly other things (possibly
more things than we like?).  In fact, if you specify -lc_r directly, then
the compiler driver will also link in the normal libc, resulting in
complete desaster.

-- 
Peter Eisentraut   peter_e@gmx.net

Thu, 24 Jul 2003 16:00:05 +0200 (CEST)
Peter Eisentraut <peter_e@gmx.net>
Re: 
Lee Kindness writes:

> Guys, take a look at what was done in libpq to make it
> thread-safe... No locks! No overheaded - just using "proper" reentrant
> functions...

There is a difference between being reentrant and being thread-safe.
Making libpq reentrant is relatively easy if you use the right functions.
But we have no guarantee that it will be thread-safe by default, because
some platforms require special flags to make standard library functions
thread-safe.

-- 
Peter Eisentraut   peter_e@gmx.net

Thu, 24 Jul 2003 16:04:28 +0200 (CEST)
Peter Eisentraut <peter_e@gmx.net>
Re: libpq_r
Lee Kindness <lkindness@csl.co.uk> writes:
> Bruce Momjian writes:
>>> My guess is that if the OS has separate threaded libs, we have to mimic
>>> that stuff.

> But there are NO thread primitives/calls in libpq 

That's not the point.  The point is stuff that isn't necessarily visible
in the source code --- such as what method it uses to get at "errno",
whether it's linked to thread-safe versions of malloc and other libc
routines, etc.

If the OS supplies both libc and libc_r, it is unlikely to be a good
idea to link a threaded libpq with libc, or a non-threaded libpq with
libc_r.

			regards, tom lane

Thu, 24 Jul 2003 10:09:53 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: libpq_r 
Tom Lane writes:
 > Lee Kindness <lkindness@csl.co.uk> writes:
 > > Bruce Momjian writes:
 > >>> My guess is that if the OS has separate threaded libs, we have to mimic
 > >>> that stuff.
 > > But there are NO thread primitives/calls in libpq 
 > That's not the point.  The point is stuff that isn't necessarily visible
 > in the source code --- such as what method it uses to get at "errno",
 > whether it's linked to thread-safe versions of malloc and other libc
 > routines, etc.
 > If the OS supplies both libc and libc_r, it is unlikely to be a good
 > idea to link a threaded libpq with libc, or a non-threaded libpq with
 > libc_r.

No, that is the point exactly... I know fine well the visibility of
errno and the different ways it may be implemented. You'll find it
hard to come across an OS which doesn't "do the right thing" WRT to
using reentrant functions and "wrappers" if _REENTRANT is defined
during the compile...

That is ALL that is needed, lets not complicate the issue...

And what do you mean by "threaded libpq" and "non-threaded libpq" - as
I say above there are NO thread primitives being used in libpq. It's
just clean reentrant code... This change/patch should be done
irregardless of the move to thread safety!

L.

Thu, 24 Jul 2003 15:32:57 +0100
Lee Kindness <lkindness@csl.co.uk>
Re: libpq_r 
Peter Eisentraut wrote:
> Bruce Momjian writes:
> 
> > The strange thing is that accoring to template/freebsd, libc_r was
> > _added_ in FreeBSD 5.0, while most OS are moving away from separate
> > threaded libs.
> 
> The template is wrong.  libc_r has existed in FreeBSD for quite a while,
> but your're not supposed to use it directly.  Instead, you use the flag
> -pthread, which takes care of libc_r and possibly other things (possibly
> more things than we like?).  In fact, if you specify -lc_r directly, then
> the compiler driver will also link in the normal libc, resulting in
> complete desaster.

How is the template wrong?  The following code was gotten from a FreeBSD
user.  Should the top case be used with all FreeBSD's?
	
	case $host_os in
	                freebsd2*|freebsd3*|freebsd4*)
	                        THREAD_CFLAGS="-pthread"
	                        NEED_REENTRANT_FUNC_NAMES=yes
	                        ;;
	                *)
	                        THREAD_LIBS="-lc_r"
	                        NEED_REENTRANT_FUNC_NAMES=yes
	                        ;;
	esac


-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Thu, 24 Jul 2003 10:37:53 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: 
Lee Kindness wrote:
> Tom Lane writes:
>  > Lee Kindness <lkindness@csl.co.uk> writes:
>  > > Bruce Momjian writes:
>  > >>> My guess is that if the OS has separate threaded libs, we have to mimic
>  > >>> that stuff.
>  > > But there are NO thread primitives/calls in libpq 
>  > That's not the point.  The point is stuff that isn't necessarily visible
>  > in the source code --- such as what method it uses to get at "errno",
>  > whether it's linked to thread-safe versions of malloc and other libc
>  > routines, etc.
>  > If the OS supplies both libc and libc_r, it is unlikely to be a good
>  > idea to link a threaded libpq with libc, or a non-threaded libpq with
>  > libc_r.
> 
> No, that is the point exactly... I know fine well the visibility of
> errno and the different ways it may be implemented. You'll find it
> hard to come across an OS which doesn't "do the right thing" WRT to
> using reentrant functions and "wrappers" if _REENTRANT is defined
> during the compile...
> 
> That is ALL that is needed, lets not complicate the issue...
> 
> And what do you mean by "threaded libpq" and "non-threaded libpq" - as
> I say above there are NO thread primitives being used in libpq. It's
> just clean reentrant code... This change/patch should be done
> irregardless of the move to thread safety!

Now I see what you are saying, that _REENTRANT just makes it reentrant,
and doesn't have a downside in terms of performance.

However, notice the flags needed under Linux:

	THREAD_CFLAGS="-D_REENTRANT -D_THREAD_SAFE -D_POSIX_PTHREAD_SEMANTICS"

or are you saying libpq needs only the first one because libpq, itself,
doesn't use threads --- interesting distinction, and perhaps a way we
can get by with one libpq even on platforms that require *_r libraries.

As you mentioned, ecpg is another manner, but I think perhaps an ecpg
flag could allow both threaded and nonthreaded.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Thu, 24 Jul 2003 10:44:27 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: libpq_r
Tom Lane writes:
 > If the OS supplies both libc and libc_r, it is unlikely to be a good
 > idea to link a threaded libpq with libc, or a non-threaded libpq with
 > libc_r.

What modern OS has libc_r? The majors (Linux, Soalris, HPUX, AIX)
certainly do not.

L.

Thu, 24 Jul 2003 15:45:13 +0100
Lee Kindness <lkindness@csl.co.uk>
Re: libpq_r 
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Now I see what you are saying, that _REENTRANT just makes it reentrant,
> and doesn't have a downside in terms of performance.

That's at best an unsupported assertion.  Why would the platform bother
with supplying two copies of libc if they didn't think there was a
performance hit?

			regards, tom lane

Thu, 24 Jul 2003 10:48:06 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: libpq_r 
Tom Lane writes:
 > Bruce Momjian <pgman@candle.pha.pa.us> writes:
 > > Now I see what you are saying, that _REENTRANT just makes it reentrant,
 > > and doesn't have a downside in terms of performance.
 > That's at best an unsupported assertion.  Why would the platform bother
 > with supplying two copies of libc if they didn't think there was a
 > performance hit?

Better remove transactions then, yeah? Performace hit! Profile it and
see how minor (or likely non-existent) it is...

L.

Thu, 24 Jul 2003 15:51:44 +0100
Lee Kindness <lkindness@csl.co.uk>
Re: libpq_r 
Bruce Momjian writes:
 > However, notice the flags needed under Linux:
 > 
 > 	THREAD_CFLAGS="-D_REENTRANT -D_THREAD_SAFE -D_POSIX_PTHREAD_SEMANTICS"
 > 
 > or are you saying libpq needs only the first one because libpq, itself,
 > doesn't use threads --- interesting distinction, and perhaps a way we
 > can get by with one libpq even on platforms that require *_r
 > libraries.

Urgh, I think I was a bit over zealous witht he first patch and just
bundled the later two in... After looking around the includes they
don't do anything under Linux. I think they're used on AIX.

however the best way to do this is to look at another package which
has a wide distribution and wide OS base - OpenLDAP being a good
example (or even MySQL!)- and see how they do things... No need to
reinvent the wheel!

L.

Thu, 24 Jul 2003 15:55:03 +0100
Lee Kindness <lkindness@csl.co.uk>
Re: libpq_r
Peter Eisentraut wrote:
> Bruce Momjian writes:
> 
> > Now I see what you are saying, that _REENTRANT just makes it reentrant,
> 
> _REENTRANT only makes additional functions visible in the header files, it
> doesn't change any functions to behave differently.  (This is not hard to
> imagine, because the lack of reentrancy of most functions lies in the
> public interface, and you can't change that transparently.)
> 
> > and doesn't have a downside in terms of performance.
> 
> It follows from the above that this is irrelevant.
> 
> > However, notice the flags needed under Linux:
> >
> > 	THREAD_CFLAGS="-D_REENTRANT -D_THREAD_SAFE -D_POSIX_PTHREAD_SEMANTICS"
> 
> Those flags are bogus.  You don't need any flags under Linux.

Is that true for all versions of Linux?  Don't we need the first one so
we compile cleanly.

Also, the remaining to linux settings:

	THREAD_LIBS="-lpthread"
	NEED_REENTRANT_FUNC_NAMES=yes

I assume -lpthread does have an effect on the library, or does it?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Thu, 24 Jul 2003 11:18:25 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: libpq_r
Bruce Momjian writes:

> Now I see what you are saying, that _REENTRANT just makes it reentrant,

_REENTRANT only makes additional functions visible in the header files, it
doesn't change any functions to behave differently.  (This is not hard to
imagine, because the lack of reentrancy of most functions lies in the
public interface, and you can't change that transparently.)

> and doesn't have a downside in terms of performance.

It follows from the above that this is irrelevant.

> However, notice the flags needed under Linux:
>
> 	THREAD_CFLAGS="-D_REENTRANT -D_THREAD_SAFE -D_POSIX_PTHREAD_SEMANTICS"

Those flags are bogus.  You don't need any flags under Linux.

-- 
Peter Eisentraut   peter_e@gmx.net

Thu, 24 Jul 2003 17:21:17 +0200 (CEST)
Peter Eisentraut <peter_e@gmx.net>
Re: libpq_r
Bruce Momjian writes:
 > Peter Eisentraut wrote:
 > > Bruce Momjian writes:
 > > > 	THREAD_CFLAGS="-D_REENTRANT -D_THREAD_SAFE -D_POSIX_PTHREAD_SEMANTICS"
 > > Those flags are bogus.  You don't need any flags under Linux.
 > Is that true for all versions of Linux?  Don't we need the first one so
 > we compile cleanly.

Yeah, you want to define _REENTRANT...

 > Also, the remaining to linux settings:
 > 	THREAD_LIBS="-lpthread"
 > 	NEED_REENTRANT_FUNC_NAMES=yes
 > I assume -lpthread does have an effect on the library, or does it?

You DO NOT want -lpthread for libpq, but you do want it for ecpg.

L.

Thu, 24 Jul 2003 16:29:58 +0100
Lee Kindness <lkindness@csl.co.uk>
Re: libpq_r


--On Thursday, July 24, 2003 17:21:17 +0200 Peter Eisentraut 
<peter_e@gmx.net> wrote:

> Bruce Momjian writes:
>
>> Now I see what you are saying, that _REENTRANT just makes it reentrant,
>
> _REENTRANT only makes additional functions visible in the header files, it
> doesn't change any functions to behave differently.  (This is not hard to
> imagine, because the lack of reentrancy of most functions lies in the
> public interface, and you can't change that transparently.)
>
I beg to differ.  Explicitly, on UnixWare, the <errno.h> header, reproduced 
below, under
fair use, show an EXPLICIT difference in what happens with _REENTRANT:
$ cat /usr/include/errno.h
/*
 * Copyright (c) 2002 Caldera International, Inc. All Rights Reserved.
 *
 *        THIS IS UNPUBLISHED PROPRIETARY SOURCE CODE OF
 *                  Caldera International, Inc.
 *
 *   The copyright notice above does not evidence any actual or intended
 *   publication of such source code.
 */

#ifndef _ERRNO_H
#define _ERRNO_H
#ident  "@(#)unixsrc:usr/src/common/head/errno.h /main/uw7_nj/2"

#include <sys/errno.h>

#ifdef _REENTRANT

#ifdef __cplusplus
extern "C" {
#endif

extern int      *__thr_errno(void);

#ifdef __cplusplus
}
#endif

#define errno   (*__thr_errno())

#else /*!_REENTRANT*/

extern int      errno;

#ifdef __cplusplus /* C++ requires that this be a macro! */
#define errno   errno
#endif

#endif /*_REENTRANT*/

#endif /*_ERRNO_H*/
$


-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 972-414-9812                 E-Mail: ler@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749


Thu, 24 Jul 2003 10:30:10 -0500
Larry Rosenman <ler@lerctr.org>
Re: libpq_r
Larry Rosenman writes:

> I beg to differ.  Explicitly, on UnixWare, the <errno.h> header,
> reproduced below, under fair use, show an EXPLICIT difference in what
> happens with _REENTRANT:

Hmm, I was too optimistic.  I guess we'll just have to handcraft a
different solution for each platform.  But clearly on some platforms we'll
need a libpq_r, so for the reasons I outlined in my initial post, it'd be
good to provide one on all platforms.

-- 
Peter Eisentraut   peter_e@gmx.net

Thu, 24 Jul 2003 18:00:01 +0200 (CEST)
Peter Eisentraut <peter_e@gmx.net>
Re: libpq_r
Peter Eisentraut writes:
 > Larry Rosenman writes:
 > > I beg to differ.  Explicitly, on UnixWare, the <errno.h> header,
 > > reproduced below, under fair use, show an EXPLICIT difference in what
 > > happens with _REENTRANT:
 > Hmm, I was too optimistic.  I guess we'll just have to handcraft a
 > different solution for each platform.  But clearly on some platforms we'll
 > need a libpq_r, so for the reasons I outlined in my initial post, it'd be
 > good to provide one on all platforms.

Sorry, but can you elaborate? I can't see any need with the above
reasoning. I managed to find your original post (amazingly hard when
you're in digest mode and the post has an empty subject!) and I think
the responses in this thread have demonstrated that there is no need
for a libpq_r.so, but there is need for a libecpg_r.so

I guess we're going to have get down to the code level to demonstrate
this! If you compile a example library with _REENTRANT and use _r
functions and the function version of errno then it can be used no
problem by a library compiled without _REENTRANT and expecting errno
to be an int...

Unfortunately i'm leaving on holiday tonight until Monday so will not
be able to get the example done...

Regards, Lee.

Thu, 24 Jul 2003 17:08:31 +0100
Lee Kindness <lkindness@csl.co.uk>
Re: libpq_r
> Better remove transactions then, yeah? Performace hit! Profile it and
> see how minor (or likely non-existent) it is...

You can do that.  Turn fsync to off and you remove a large majority of
the transaction hit, but I understand what you're saying.
24 Jul 2003 16:46:42 +0000
Rod Taylor <rbt@rbt.ca>
Re: libpq_r
Peter Eisentraut wrote:
> Larry Rosenman writes:
> 
> > I beg to differ.  Explicitly, on UnixWare, the <errno.h> header,
> > reproduced below, under fair use, show an EXPLICIT difference in what
> > happens with _REENTRANT:
> 
> Hmm, I was too optimistic.  I guess we'll just have to handcraft a
> different solution for each platform.  But clearly on some platforms we'll
> need a libpq_r, so for the reasons I outlined in my initial post, it'd be
> good to provide one on all platforms.

I still think it is confusing to create a libpq_r on platforms that have
no _r libraries.  I am on BSD/OS and I can find only _r library on my
entire system, and guess who make that one:

	/usr/contrib/lib/mysql/libmysqlclient_r.a

and they actually have two different libraries:

-rwxr-xr-x  1 root  wheel  127413 Jun  5  2001 /usr/contrib/lib/mysql/libmysqlclient.so.10.0.0*
-rwxr-xr-x  1 root  wheel  144471 Jun  5  2001 /usr/contrib/lib/mysql/libmysqlclient_r.so.10.0.0*

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Thu, 24 Jul 2003 16:30:14 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: libpq_r


--On Thursday, July 24, 2003 16:30:14 -0400 Bruce Momjian 
<pgman@candle.pha.pa.us> wrote:

> Peter Eisentraut wrote:
>> Larry Rosenman writes:
>>
>> > I beg to differ.  Explicitly, on UnixWare, the <errno.h> header,
>> > reproduced below, under fair use, show an EXPLICIT difference in what
>> > happens with _REENTRANT:
>>
>> Hmm, I was too optimistic.  I guess we'll just have to handcraft a
>> different solution for each platform.  But clearly on some platforms
>> we'll need a libpq_r, so for the reasons I outlined in my initial post,
>> it'd be good to provide one on all platforms.
>
> I still think it is confusing to create a libpq_r on platforms that have
> no _r libraries.  I am on BSD/OS and I can find only _r library on my
> entire system, and guess who make that one:
>
> 	/usr/contrib/lib/mysql/libmysqlclient_r.a
>
> and they actually have two different libraries:
>
> -rwxr-xr-x  1 root  wheel  127413 Jun  5  2001
> /usr/contrib/lib/mysql/libmysqlclient.so.10.0.0* -rwxr-xr-x  1 root
> wheel  144471 Jun  5  2001
> /usr/contrib/lib/mysql/libmysqlclient_r.so.10.0.0*
I think it needs to be a port/platform by port/platform decision.

I DO think we need to -D_REENTRANT on those platforms that we can to allow
libpq's use in threaded programs.

LER




-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 972-414-9812                 E-Mail: ler@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749


Thu, 24 Jul 2003 15:35:02 -0500
Larry Rosenman <ler@lerctr.org>
Re: libpq_r


--On Friday, July 25, 2003 11:23:05 +0200 Peter Eisentraut 
<peter_e@gmx.net> wrote:

> Bruce Momjian writes:
>
>> I still think it is confusing to create a libpq_r on platforms that have
>> no _r libraries.  I am on BSD/OS and I can find only _r library on my
>> entire system,
>
> The criterion is not whether the platform has _r libraries, it's whether
> special actions are required in order to make a library fit for threaded
> programs, at the time that library is built.  Such special actions may
> include:
>
> 1. need to link in special libc_r (FreeBSD)
> 2. need to use magic flags for compiler and/or linker (FreeBSD, UnixWare)
> 3. need to compile with special preprocessor symbols (UnixWare, AIX)
> 4. need to use different compiler altogether (AIX)
>
> (If you read between the lines of the manufacturers' documentation, all
> these methods end up accomplishing similar things, they are only different
> interfaces for it.)
The -D_REENTRANT causes NO pain, and makes it possible to have ONE library
for UnixWare.

the -K[p]thread issue is for EXECUTABLES to cause the libthread library to 
be invoked
at PROCESS startup.

I feel that libpq should be compiled with -D_REENTRANT, and we have ONE copy
of it on UnixWare.


>
> If any of these cases apply, we need to provide a separate libpq_r, or
> else either threaded land or nonthreaded land will suffer pain, from what
> I read.  (The simplest example is errno being defined in different ways
> and referring to different symbols.)
>
> If none of these cases apply, then a separate libpq_r will be redundant,
> perhaps "confusing" to some, but it can be ignored by the user.

see my above comments.


-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 972-414-9812                 E-Mail: ler@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749


Fri, 25 Jul 2003 04:19:42 -0500
Larry Rosenman <ler@lerctr.org>
Re: libpq_r
Bruce Momjian writes:

> I still think it is confusing to create a libpq_r on platforms that have
> no _r libraries.  I am on BSD/OS and I can find only _r library on my
> entire system,

The criterion is not whether the platform has _r libraries, it's whether
special actions are required in order to make a library fit for threaded
programs, at the time that library is built.  Such special actions may
include:

1. need to link in special libc_r (FreeBSD)
2. need to use magic flags for compiler and/or linker (FreeBSD, UnixWare)
3. need to compile with special preprocessor symbols (UnixWare, AIX)
4. need to use different compiler altogether (AIX)

(If you read between the lines of the manufacturers' documentation, all
these methods end up accomplishing similar things, they are only different
interfaces for it.)

If any of these cases apply, we need to provide a separate libpq_r, or
else either threaded land or nonthreaded land will suffer pain, from what
I read.  (The simplest example is errno being defined in different ways
and referring to different symbols.)

If none of these cases apply, then a separate libpq_r will be redundant,
perhaps "confusing" to some, but it can be ignored by the user.

-- 
Peter Eisentraut   peter_e@gmx.net

Fri, 25 Jul 2003 11:23:05 +0200 (CEST)
Peter Eisentraut <peter_e@gmx.net>
Re: libpq_r


--On Friday, July 25, 2003 11:51:50 +0200 Peter Eisentraut 
<peter_e@gmx.net> wrote:

> Larry Rosenman writes:
>
>> The -D_REENTRANT causes NO pain, and makes it possible to have ONE
>> library for UnixWare.
>
> Didn't you just say that -D_REENTRANT changes the definition of errno on
> your system?  Surely it would not be a good idea to link a libpq that uses
> variant 1 with a program that uses variant 2.  Maybe I'm wrong and you can
> do that, but it sounds problematic to me.
Yes, it DOES change errno, but it is **NOT** problematic.  the static int 
is there
ONLY FOR THE FIRST LWP (thread) of a process.  The reentrant version 
accesses the same
information for ANY LWP.  the first LWP can also use it for getting access.

It is SAFER to use the reentrant version, so that if a THREADED program 
uses libpq,
we are now thread-safe.

LER




-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 972-414-9812                 E-Mail: ler@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749


Fri, 25 Jul 2003 04:47:56 -0500
Larry Rosenman <ler@lerctr.org>
Re: libpq_r
Larry Rosenman writes:

> The -D_REENTRANT causes NO pain, and makes it possible to have ONE library
> for UnixWare.

Didn't you just say that -D_REENTRANT changes the definition of errno on
your system?  Surely it would not be a good idea to link a libpq that uses
variant 1 with a program that uses variant 2.  Maybe I'm wrong and you can
do that, but it sounds problematic to me.

-- 
Peter Eisentraut   peter_e@gmx.net

Fri, 25 Jul 2003 11:51:50 +0200 (CEST)
Peter Eisentraut <peter_e@gmx.net>
Re: libpq_r
Peter Eisentraut wrote:
> Bruce Momjian writes:
> 1. need to link in special libc_r (FreeBSD)
> 2. need to use magic flags for compiler and/or linker (FreeBSD, UnixWare)
> 3. need to compile with special preprocessor symbols (UnixWare, AIX)
> 4. need to use different compiler altogether (AIX)
> 
> (If you read between the lines of the manufacturers' documentation, all
> these methods end up accomplishing similar things, they are only different
> interfaces for it.)
> 
> If any of these cases apply, we need to provide a separate libpq_r, or
> else either threaded land or nonthreaded land will suffer pain, from what
> I read.  (The simplest example is errno being defined in different ways
> and referring to different symbols.)
> 
> If none of these cases apply, then a separate libpq_r will be redundant,
> perhaps "confusing" to some, but it can be ignored by the user.

I see no reason to symlink to libpq_r if it is identical to libpq. 
Operating systems are moving away from _r libraries, and I see no reason
to create one just for consistency with platforms that still use _r.
Basically, I don't like the confusion.

I agree with the goal of having a libpq_r if it is different from libpq.

This is going to require us to compile libpq twice and install libpq and
libpq_r.  Can that be done cleanly?  I don't see how we can get this
done before beta.  Will we fiddle with this during beta?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Fri, 25 Jul 2003 11:35:54 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: libpq_r
Peter Eisentraut <peter_e@gmx.net> writes:

> Bruce Momjian writes:
> 
> > I still think it is confusing to create a libpq_r on platforms that have
> > no _r libraries.  I am on BSD/OS and I can find only _r library on my
> > entire system,
> 
> The criterion is not whether the platform has _r libraries, it's whether
> special actions are required in order to make a library fit for threaded
> programs, at the time that library is built.  Such special actions may
> include:

No, there's one more criterion: And those special actions make the library
unsuitable for use in programs not compiled to use threads.

For example, on Debian *all* libraries are compiled with -D_REENTANT -fPIC.
Anything else would be a major headache for the distribution.

It causes a 3-5% performance drop on i386 and much less on other
architectures. However then all libraries are suitable for use in both
threaded and non-threaded applications.

There may still be some architectures out there where two versions are needed.
But I'm not aware of any. 

-- 
greg


25 Jul 2003 18:20:10 -0400
Greg Stark <gsstark@mit.edu>
Re: libpq_r


--On Friday, July 25, 2003 18:20:10 -0400 Greg Stark <gsstark@mit.edu> 
wrote:

> Peter Eisentraut <peter_e@gmx.net> writes:
>
>> Bruce Momjian writes:
>>
>> > I still think it is confusing to create a libpq_r on platforms that
>> > have no _r libraries.  I am on BSD/OS and I can find only _r library
>> > on my entire system,
>>
>> The criterion is not whether the platform has _r libraries, it's whether
>> special actions are required in order to make a library fit for threaded
>> programs, at the time that library is built.  Such special actions may
>> include:
>
> No, there's one more criterion: And those special actions make the library
> unsuitable for use in programs not compiled to use threads.
>
> For example, on Debian *all* libraries are compiled with -D_REENTANT
> -fPIC. Anything else would be a major headache for the distribution.
This, I believe, is the same situation for UnixWare.

>
> It causes a 3-5% performance drop on i386 and much less on other
> architectures. However then all libraries are suitable for use in both
> threaded and non-threaded applications.
>
> There may still be some architectures out there where two versions are
> needed. But I'm not aware of any.
I believe that FreeBSD still needs 2, but there are 3 different threading 
libs at
the moment on 5.x, and 4.x still has _r libs.

LER



-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 972-414-9812                 E-Mail: ler@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749


Fri, 25 Jul 2003 17:26:18 -0500
Larry Rosenman <ler@lerctr.org>
Re: libpq_r