Subject: Developer's guide to buildlink3
To: None <>
From: Johnny C. Lam <>
List: tech-pkg
Date: 01/22/2004 23:16:51
Content-Disposition: inline

I've attached a start of a developers's guide to using buildlink3.
This contents of this file will eventually be added to Package.txt.
Comments on how to improve this documentation are appreciated.


	-- Johnny Lam <>

Content-Disposition: attachment; filename=buildlink3_dg

 0 Developer's guide to buildlink3

This is a tutorial for pkgsrc developers to understand and to use the
buildlink3 framework in pkgsrc.

 1 Changes between buildlink2 and buildlink3

 1.1 Better behavior with libtool

One of the biggest problems in buildlink2 is handling packages that
install libtool archive files for libraries that are also present in
the base system.  buildlink3 is significantly better at this as it
more tightly controls where libtool can find libtool archives.  One
side effect of this is that we no longer need to create fake libtool
archives to work around cases where the pkgsrc libraries were being
used instead of the system libraries if they shared the same name.

 1.3 Support for native compilers

The buildlink3 wrapper scripts have better support for using SunPro
and MIPSpro compilers to build pkgsrc software.  For the most part,
packages can use any compiler, but some third-party software is
written assuming that it will be compiled using GCC.  The buildlink3
wrapper scripts can capture some common GCC options and convert them
into native toolchain equivalents.

 1.4 New file structure
files.  The first, most noticeable difference is that
generally don't have contain a BUILDLINK_FILES definition.  This is
because buildlink3 automatically determines which files to symlink
into ${BUILDLINK_DIR} by examining the PLIST of the installed package.
The second difference is that files keep track of how
"deep" we are in including files, and only creates
dependencies on packages encountered at depth 1.  This means that
packages that want to add a dependency must directly include the file for that dependency.

  2 Writing files

The following example is taken from graphics/tiff:

# $NetBSD:,v 1.3 2004/01/05 11:05:46 jlam Exp $


.if !empty(BUILDLINK_DEPTH:M+)

.if !empty(TIFF_BUILDLINK3_MK:M+)
BUILDLINK_DEPENDS.tiff?=	tiff>=3.5.4
BUILDLINK_PKGSRCDIR.tiff?=	../../graphics/tiff



The header and footer manipulate BUILDLINK_DEPTH, which is common
across all files and is used to track at what depth we
are in including files.

The first section controls if the dependency on tiff is added.
BUILDLINK_DEPENDS is the global list of packages for which
dependencies are added by buildlink3.  The tiff package is only
appended to this list if the is included directly by a
package Makefile.

The second section is protected from multiple inclusion and control
how the dependency on tiff is added.  Several important variables
are set in the section:

 (1) BUILDLINK_PACKAGES is the global list of packages for which files have been included.  It must _always_ be
     appended to within a;

 (2) BUILDLINK_DEPENDS.tiff is the actual dependency recorded in the
     installed package;

 (3) BUILDLINK_PKGSRCDIR.tiff is the location of the tiff pkgsrc

 (4) BUILDLINK_DEPMETHOD.tiff (not shown above) controls whether we
     use BUILD_DEPENDS or DEPENDS to add the dependency on tiff.  The
     build dependency is selected by setting BUILDLINK_DEPMETHOD.tiff
     to "build".  By default, the full dependency is used.

 (5) BUILDLINK_INCDIRS.tiff and BUILDLINK_LIBDIRS.tiff (not shown
     above) are lists of subdirectories of ${BUILDLINK_PREFIX.tiff} to
     add the header and library search paths.  These default to
     "include" and "lib" respectively.

 (6) BUILDLINK_CPPFLAGS.tiff is the list of preprocessor flags to add
     CPPFLAGS, which are passed on to the configure and build phases.
     -I should be avoided and instead be added using
     BUILDLINK_INCDIRS.tiff as above.

Any for tiff's dependencies are also included at this
point.  Including these files means that the headers
and libraries for these dependencies are also symlinked into
${BULIDLINK_DIR} whenever the tiff file is included.

I highly recommend using pkgtools/createbuildlink, written by Rene
Hexel, to generate an initial file.

There are several considerations that arise when figuring out how
to set BUILDLINK_DEPENDS.<pkg> correctly:

 (1) If the package has a pre-existing file, then
     match the BUILDLINK_DEPENDS.<pkg> lines between the file and the newly-created file.

 (2) If there is no pre-existing file, then set to the first version of the package that
     had the last change in the major number of a shared library or
     that had a major API change.

 3 bl3ifying a package

The process of "bl3ifying" a package, or converting a package to use
the buildlink3 framework, is surprisingly easy.  The things to keep in
mind are:

  (1) Set USE_BUILDLINK3=yes.

  (2) Change references to files into

  (3) Ensure that the build always calls the wrapper scripts instead of
      the actual toolchain.  Some are tricky, e.g. openssl, cdrecord,
      ocaml, and the only way to know for sure is the check .work.log
      to see if the wrappers are being invoked.

  (4) Don't override PREFIX from within the package Makefile, e.g.
      Java VMs, standalone shells, etc., because the code to symlink
      files into ${BUILDLINK_DIR} looks for files relative to
      "pkg_info -qp <pkgname>".

  4 Troubleshooting

Q1: I'm trying to bl3ify a package but I get an error that looks like:

	make: don't know how to make _BUILDLINK_USE. Stop

A1: You forgot to change a reference to a file into a file.

Q2: Where can I see the actual command executed by the wrapper

A2: You should examine the contents of the ${WRKDIR}/.work.log file.
    The lines preceded with [*] are the commands that are intercepted
    by the wrapper scripts, and the lines preceded with <.> are the
    commands that are executed by the wrapper scripts.