title: Cross-compiling and linking to a library in a sysroot - what's wrong?
body: <p>I am not inexperienced but it's become clear to me that I lack some fundamental understanding in the area of cross-compilation and sysroots. I'm hoping someone can provide me that gem of information I need to continue.</p>
  
  <p>I have downloaded a pre-built armhf gcc cross-compiler/toolchain and it came with a directory called <code>arm-linux-gnueabihf</code> that contains the following directories:</p>
  
  <pre><code>bin/
  include/
  lib/  - contains libstdc++.a, libgcc_s.so, but does NOT contain `libcrypt.so` or `libcrypt.so.1`, etc.
  libc/
  </code></pre>
  
  <p>Curiously <code>libc</code> contains yet another set of directories, that looks a bit like a separate sysroot, but I'm not sure why it's here. I've looked in other toolchains, including one I built myself with crosstool-ng, and I've not seen anything similar:</p>
  
  <pre><code>libc/
       etc/
       lib/  - contains files like libcrypt.so.1 / libcrypt-2.24.so
       sbin/
       usr/
           bin/
           include/
           lib/  - contains files like libc.a, libc.so, libcrypt.a, libcrypt.so, 
           libexec/
           sbin/
           share/
       var/
  </code></pre>
  
  <p>Anyway, I'm not sure if that's a problem, or if I have to merge those two sysroots into one</p>
  
  <p>I have installed it within a Docker container at the path <code>/cross-pi-gcc-9.1.0-1</code>. I am using cmake to cross-compile, and my toolchain.cmake file refers to this toolchain:</p>
  
  <pre><code>SET(CMAKE_SYSTEM_NAME Linux)
  SET(CMAKE_SYSTEM_VERSION 1)
  SET(CMAKE_C_COMPILER /cross-pi-gcc-9.1.0-1/bin/arm-linux-gnueabihf-gcc)
  SET(CMAKE_CXX_COMPILER /cross-pi-gcc-9.1.0-1/bin/arm-linux-gnueabihf-g++)
  SET(CMAKE_FIND_ROOT_PATH /cross-pi-gcc-9.1.0-1/arm-linux-gnueabihf/)
  SET(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
  SET(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
  SET(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
  </code></pre>
  
  <p>This seems to be sufficient to build most code with the cross-compiler (including Boost).</p>
  
  <p>As part of this container, I wish to have WiringPi installed in the sysroot, so I can build and link against it.</p>
  
  <p>To do this, I have created a custom <code>CMakeLists.txt</code> file that successfully builds and installs WiringPi:</p>
  
  <pre><code>cmake_minimum_required(VERSION 3.0)
  project(WiringPi)
  set(CMAKE_THREAD_PREFER_PTHREAD TRUE)
  find_package(Threads REQUIRED)
  add_library(wiringPi SHARED ads1115.c &lt;snip a bunch of .c files&gt;)
  target_include_directories(wiringPi PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})
  target_link_libraries(wiringPi PUBLIC ${CMAKE_THREAD_LIBS_INIT} crypt m rt)
  install(TARGETS wiringPi DESTINATION lib)
  install(FILES ads1115.h &lt;snip a bunch of .h files&gt;
          DESTINATION include)
  </code></pre>
  
  <p>This indicates that the resultant <code>libwiringpi.so</code> should be linked with <code>libcrypt</code>, <code>libpthread</code>, <code>libm</code> and <code>librt</code>. And if I run the target's <code>ldd</code> tool on <code>libwiringpi.so</code> it does indeed show these libraries as dependencies:</p>
  
  <pre><code>$ ldd libwiringPi.so 
      linux-vdso.so.1 (0x7eee6000)
      /usr/lib/arm-linux-gnueabihf/libarmmem-${PLATFORM}.so =&gt; /usr/lib/arm-linux-gnueabihf/libarmmem-v7l.so (0x76f0a000)
      libpthread.so.0 =&gt; /lib/arm-linux-gnueabihf/libpthread.so.0 (0x76ee0000)
      libcrypt.so.1 =&gt; /lib/arm-linux-gnueabihf/libcrypt.so.1 (0x76ea0000)
      libm.so.6 =&gt; /lib/arm-linux-gnueabihf/libm.so.6 (0x76e1e000)
      librt.so.1 =&gt; /lib/arm-linux-gnueabihf/librt.so.1 (0x76e07000)
      libc.so.6 =&gt; /lib/arm-linux-gnueabihf/libc.so.6 (0x76cb9000)
      /lib/ld-linux-armhf.so.3 (0x76f43000)
  </code></pre>
  
  <p>The problem I'm having is that I have an application that links against wiringpi using a cmake line like this:</p>
  
  <pre><code>target_link_libraries(myapp wiringPi)
  </code></pre>
  
  <p>When I build this on a raspberry pi using the native toolchain, I don't need to explicitly link against <code>libcrypt</code>. But in my Docker/cross-compiler environment, I get this error at link time:</p>
  
  <pre><code>/cross-pi-gcc-9.1.0-1/bin/arm-linux-gnueabihf-g++     CMakeFiles/app.dir/main.cpp.o   -o myapp -lpthread -lwiringPi -lrt 
  /cross-pi-gcc-9.1.0-1/bin/../lib/gcc/arm-linux-gnueabihf/9.1.0/../../../../arm-linux-gnueabihf/bin/ld: warning: libcrypt.so.1, needed by /cross-pi-gcc-9.1.0-1/bin/../lib/gcc/arm-linux-gnueabihf/9.1.0/../../../../arm-linux-gnueabihf/lib/libwiringPi.so, not found (try using -rpath or -rpath-link)
  /cross-pi-gcc-9.1.0-1/bin/../lib/gcc/arm-linux-gnueabihf/9.1.0/../../../../arm-linux-gnueabihf/lib/libwiringPi.so: undefined reference to `crypt@GLIBC_2.4'
  collect2: error: ld returned 1 exit status
  </code></pre>
  
  <p>Note that <code>-lrt</code> and <code>-lpthread</code> seem to have been automatically included in the link library list. But <code>-lcrypt</code> is not present.</p>
  
  <p>If I copy/paste the <code>make VERBOSE=1</code> output that corresponds to this error, and manually add <code>-lcrypt</code> to the end of the command, it links successfully and the application compilation is complete.</p>
  
  <p>I realise this is a long description but what I'm ultimately trying to do is find the hole in my knowledge that is preventing me from understanding why I would need to explicitly link <code>libcrypt</code> into this application in this environment.</p>
  
  <p>I would have thought, perhaps wrongly, that since <code>libwiringpi.so</code> is already linked against <code>libcrypt</code>, that it wouldn't need to be linked at the top level. If that's not how it works, is there anyone that can help me repair my mental model, please?</p>
  
  <p>Note: I could just add <code>target_link_libraries(myapp wiringPi crypt)</code> however I don't think it's necessary (not needed when building natively) and I'd like to learn a bit more about the process rather than just finding a workaround.</p>