
Hi all,

I'm planning an upgrade from 7.2.3 to 7.3.1 on some HP-UX systems.
With the major number bump in 7.3.1 I was expecting to be able to do
this upgrade without a flag day to re-compile clients, however due to
misnaming of the PostgreSQL shared libraries on HP-UX I can't.

A summary of the problem is that libraries are installed in such a way
that clients link to 'libpq.sl' et al and not to the versioned library
names such as 'libpq.2'.

I doubt that this is solely a problem with my installation, but in
case it is here is the configure command line I use:

    CC=cc CFLAGS="-O" ./configure  --prefix=/opt/pgsql-7.3.1 \
         --with-includes=$HOME/include --with-libraries=$HOME/lib

The extra includes and libraries are to pick up libz and libreadline.
I am building on HP-UX 11.11 with the HP ANSI C compiler.  The problem
is similar with both 7.2.3 and 7.3.1.

template1=> select version();
                            version                            
---------------------------------------------------------------
 PostgreSQL 7.2.3 on hppa2.0w-hp-hpux11.11, compiled by cc -Ae
(1 row)

Below are the details as I understand them currently.

There appear to be two problems.  I've not yet investigated to find
out if only one of them is the root cause.

1. the libraries are installed with their base names such as
   'libpq.sl' being regular files, with versioned names being
   symbolic links to the base names:

   $ ls -l libpq*
   -rw-r--r--   1 root  users   103544 Oct  6 14:54 libpq.a
   -r-xr-xr-x   1 root  users   102400 Oct  6 14:54 libpq.sl
   lrwxr-xr-x   1 root  users        8 Oct  6 14:54 libpq.sl.2 -> libpq.sl

   This should be the other way around:

   $ ls -l libc.2 libc.sl
   -r-xr-xr-x   1 bin   bin    1843200 Jul 30  2001 libc.2
   lrwxr-xr-x   1 root  root        15 Apr 18  2002 libc.sl -> /usr/lib/libc.2

2. the versioned names are possibly incorrect: 'libpq.sl.2' should be
   'libpq.2' by analogy to the libc example above.

The effect of these problems is that client programs linked to these
libraries link to the non-versioned names:

    $ ldd psql
        /usr/lib/libm.2 =>      /usr/lib/libm.2
        /usr/lib/libnsl.1 =>    /usr/lib/libnsl.1
        /usr/lib/libxti.2 =>    /usr/lib/libxti.2
        /usr/lib/libdld.2 =>    /usr/lib/libdld.2
        /usr/lib/libc.2 =>      /usr/lib/libc.2
        /usr/lib/libdld.2 =>    /usr/lib/libdld.2
        /usr/lib/libgen.2 =>    /usr/lib/libgen.2
        /usr/lib/libc.2 =>      /usr/lib/libc.2
        ../../../src/interfaces/libpq/libpq.sl =>       /opt/pgsql-7.2.3/lib/libpq.sl
        /usr/lib/libnsl.1 =>    /usr/lib/libnsl.1


(Aside: the ../../../src/... stuff is ugly, but harmless as far as I
know.)

Looking at the build process I see that Makefile.shlib can install
either the versioned names or the base names: I don't yet understand
why it chooses to install the base names on HP-UX.  On NetBSD it does
it the other way around.  Someone familiar with the build process
might know what the cause is here?

Dropping the ".sl" from the name might be trickier, since it is
expected to be present as a suffix on the base name but not included
in the versioned names.

Wrong:

$ gmake -n install
/bin/sh ../../../config/mkinstalldirs /opt/pgsql-7.3.1/lib/pa20_64 /opt/pgsql-7.3.1/include /opt/pgsql-7.3.1/include/internal
/bin/sh ../../../config/install-sh -c -m 644  libpq.a /opt/pgsql-7.3.1/lib/pa20_64/libpq.a
/bin/sh ../../../config/install-sh -c -m 555  libpq.sl /opt/pgsql-7.3.1/lib/pa20_64/libpq.sl
cd /opt/pgsql-7.3.1/lib/pa20_64 && \
rm -f libpq.sl.3 && \
ln -s libpq.sl libpq.sl.3
/bin/sh ../../../config/install-sh -c -m 644 ./libpq-fe.h /opt/pgsql-7.3.1/include
/bin/sh ../../../config/install-sh -c -m 644 ./libpq-int.h /opt/pgsql-7.3.1/include/internal
/bin/sh ../../../config/install-sh -c -m 644 ./pqexpbuffer.h /opt/pgsql-7.3.1/include/internal

Using NetBSD as an example, it's seen that the symbolic links are
generated the other way around:

$ gmake -n install
/bin/sh ../../../config/mkinstalldirs /usr/local/pgsql-7.3/lib /usr/local/pgsql-7.3/include /usr/local/pgsql-7.3/include/internal
/bin/sh ../../../config/install-sh -c -m 644  libpq.a /usr/local/pgsql-7.3/lib/libpq.a
/bin/sh ../../../config/install-sh -c -m 755  libpq.so.2.2 /usr/local/pgsql-7.3/lib/libpq.so.2.2
cd /usr/local/pgsql-7.3/lib && \
rm -f libpq.so.2 && \
ln -s libpq.so.2.2 libpq.so.2
cd /usr/local/pgsql-7.3/lib && \
rm -f libpq.so && \
ln -s libpq.so.2.2 libpq.so
/bin/sh ../../../config/install-sh -c -m 644 ./libpq-fe.h /usr/local/pgsql-7.3/include
/bin/sh ../../../config/install-sh -c -m 644 ./libpq-int.h /usr/local/pgsql-7.3/include/internal
/bin/sh ../../../config/install-sh -c -m 644 ./pqexpbuffer.h /usr/local/pgsql-7.3/include/internal

If fixing this isn't obvious to someone, I'll investigate in detail in
the new year.  I can see no workaround for existing installations. New
installations will want to adjust the libraries before linking
clients to them.

Regards,

Giles


Mon, 23 Dec 2002 13:24:44 +1100
Giles Lean <giles@nemeton.com.au>
HP-UX shared library installation is incorrect
Giles Lean <giles@nemeton.com.au> writes:
> 1. the libraries are installed with their base names such as
>    'libpq.sl' being regular files, with versioned names being
>    symbolic links to the base names:
>    This should be the other way around:

Probably so.  I had not realized that HP's linker is affected by which
way the symlinks run, but it appears that it is.

> 2. the versioned names are possibly incorrect: 'libpq.sl.2' should be
>    'libpq.2' by analogy to the libc example above.

This I disagree with: I consider HP's naming convention ugly and
misleading.  ".sl" should be in the name *somewhere*.

The open-source libraries I have on my machine seem to mostly put the
version number after .sl, eg

lrwxrwxrwx   1 gnu        users           14 Apr 27  2000 libgdbm.sl@ -> libgdbm.sl.2.0
lrwxrwxrwx   1 gnu        users           14 Apr 27  2000 libgdbm.sl.2@ -> libgdbm.sl.2.0
-r-xr-xr-x   1 gnu        users        45203 Nov 18  1999 libgdbm.sl.2.0*

I'd be in favor of following this convention.

			regards, tom lane

Sun, 22 Dec 2002 21:55:28 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: HP-UX shared library installation is incorrect 

Tom Lane wrote:

> Probably so.  I had not realized that HP's linker is affected by which
> way the symlinks run, but it appears that it is.

I've just spoken to someone who knows more about the HP-UX toolchain
than I do.

The situation is that the library can have an internal name if the +h
option was provided to ld.  Since no name is specified in the link
command in Makefile.shlib for HP-UX, the following (from the ld(1)
manual page) applies instead:

   +h internal_name
        ....	      
        That is, if +h is not used, the shared library does not have
        an internal name and when an executable is built with the
        shared library, the linker records the library name that it
        looks at.
	...

That doesn't specifically say what happens when the library that it
finds without an internal name is a symbolic link, but doing some
testing shows me that the name used is the name that ld found the
library as, i.e. libpq.sl rather than the desired libpq.sl.3. ld(1)
doesn't read the link when it finds a symbolic link.

My recommendation (with a pinch of salt, since I'm still not a HP-UX
toolchain guru) is to add

    +h lib$(NAME)$(DLSUFFIX).$SO_MAJOR_VERSION)

to the HP-UX LINK.shared line in src/Makefile.shlib, and to change the
way the symlinks run as well, so that libpq.sl is a link to the latest
version of libpq that is installed.

I'm not sure where/how the symlink change can be made in the build
environment, but clearly it can be.  If I can figure out the answer,
I'll try a test build and see how it goes.

> > 2. the versioned names are possibly incorrect: 'libpq.sl.2' should be
> >    'libpq.2' by analogy to the libc example above.
> 
> This I disagree with: I consider HP's naming convention ugly and
> misleading.  ".sl" should be in the name *somewhere*.

I'll leave this to your taste and good judgement.  The examples above
follow your preference. :-)

Regards,

Giles







Mon, 23 Dec 2002 15:47:33 +1100
Giles Lean <giles@nemeton.com.au>
Re: HP-UX shared library installation is incorrect 
I wrote:

> My recommendation (with a pinch of salt, since I'm still not a HP-UX
> toolchain guru) is to add
> 
>     +h lib$(NAME)$(DLSUFFIX).$SO_MAJOR_VERSION)
> 
> to the HP-UX LINK.shared line in src/Makefile.shlib, and to change the
> way the symlinks run as well, so that libpq.sl is a link to the latest
> version of libpq that is installed.

OK, I've done this.  A patch is below, against 7.3.1 source.  I've
sent it here instead of to the -patches list, since:

o here is where the discussion was

o so far I've only tested with the HP ANSI C compiler on 11.11, not
  with gcc or on other HP-UX releases.

  I can test other releases and probably with gcc, but if someone gets
  to it before me that would be just fine.  I probably won't until the
  new year.

Checking the 32 bit install:

$ ls -l libpq*
-rw-r--r--   1 root  users  158556 Dec 22 21:22 libpq.a
lrwxr-xr-x   1 root  users      10 Dec 22 21:22 libpq.sl@ -> libpq.sl.3
-r-xr-xr-x   1 root  users  143360 Dec 22 21:22 libpq.sl.3*

$ /usr/ccs/bin/ldd psql
        /usr/lib/libm.2 =>      /usr/lib/libm.2
        /usr/lib/libnsl.1 =>    /usr/lib/libnsl.1
        /usr/lib/libxti.2 =>    /usr/lib/libxti.2
        /usr/lib/libdld.2 =>    /usr/lib/libdld.2
        /usr/lib/libc.2 =>      /usr/lib/libc.2
        /usr/lib/libdld.2 =>    /usr/lib/libdld.2
        /usr/lib/libgen.2 =>    /usr/lib/libgen.2
        /usr/lib/libc.2 =>      /usr/lib/libc.2
        ../../../src/interfaces/libpq/libpq.sl.3 => /opt/pgsql-7.3.1/lib/libpq.sl.3
        /usr/lib/libnsl.1 =>    /usr/lib/libnsl.1

There's still that "../../../src/.." in the output, but the library
has been found correctly with the version number, which is what is
most important.

I ran configure with a --prefix=/opt/pgsql-7.3.1 argument, but most
clients will link against /opt/pgsql/lib so their paths won't have the
version number.

The 64 bit libraries are similar:

$ ls -l libpq*
-rw-r--r--   1 root  users  219202 Dec 22 21:38 libpq.a
lrwxr-xr-x   1 root  users      10 Dec 22 21:38 libpq.sl@ -> libpq.sl.3
-r-xr-xr-x   1 root  users  131024 Dec 22 21:38 libpq.sl.3*

$ /usr/ccs/bin/ldd psql
        libpq.sl.3 =>   /opt/pgsql-7.3.1/lib/pa20_64/libpq.sl.3
        libc.2 =>       /lib/pa20_64/libc.2
        libxcurses.1 => /lib/pa20_64/libxcurses.1
        libgen.2 =>     /lib/pa20_64/libgen.2
        libdl.1 =>      /lib/pa20_64/libdl.1
        libnsl.1 =>     /lib/pa20_64/libnsl.1
        libm.2 =>       /lib/pa20_64/libm.2
        libnsl.1 =>     /lib/pa20_64/libnsl.1
        libdl.1 =>      /usr/lib/pa20_64/libdl.1
        libdl.1 =>      /usr/lib/pa20_64/libdl.1
        libxti.2 =>     /usr/lib/pa20_64/libxti.2

Again, the 64 bit binaries have linked to the versioned library. As
a bonus the ldd output looks better too. :-)

Regards,

Giles

*** src/Makefile.shlib-orig	Wed Oct  9 09:21:54 2002
--- src/Makefile.shlib	Sun Dec 22 21:07:43 2002
***************
*** 126,134 ****
  endif
  
  ifeq ($(PORTNAME), hpux)
! # HPUX doesn't believe in version numbers for shlibs
!   shlib			:= lib$(NAME)$(DLSUFFIX)
!   LINK.shared		= $(LD) -b +b $(libdir)
  endif
  
  ifeq ($(PORTNAME), irix5)
--- 126,133 ----
  endif
  
  ifeq ($(PORTNAME), hpux)
!   shlib			:= lib$(NAME)$(DLSUFFIX).$(SO_MAJOR_VERSION)
!   LINK.shared		= $(LD) +h $(shlib) -b +b $(libdir)
  endif
  
  ifeq ($(PORTNAME), irix5)



Mon, 23 Dec 2002 17:02:53 +1100
Giles Lean <giles@nemeton.com.au>
Re: HP-UX shared library installation is incorrect 

I wrote:

> o so far I've only tested with the HP ANSI C compiler on 11.11, not
>   with gcc or on other HP-UX releases.
> 
>   I can test other releases and probably with gcc, but if someone gets
>   to it before me that would be just fine.  I probably won't until the
>   new year.

Today is quiet so I did some checking on other HP-UX releases
including a build with gcc and the fix seems OK, and I sent it
to -patches.

Giles 


Tue, 24 Dec 2002 09:24:36 +1100
Giles Lean <giles@nemeton.com.au>
Re: HP-UX shared library installation is incorrect 