{"action":"created","issue":{"url":"https://api.github.com/repos/postmarketOS/pmbootstrap/issues/1333","repository_url":"https://api.github.com/repos/postmarketOS/pmbootstrap","labels_url":"https://api.github.com/repos/postmarketOS/pmbootstrap/issues/1333/labels{/name}","comments_url":"https://api.github.com/repos/postmarketOS/pmbootstrap/issues/1333/comments","events_url":"https://api.github.com/repos/postmarketOS/pmbootstrap/issues/1333/events","html_url":"https://github.com/postmarketOS/pmbootstrap/issues/1333","id":304997996,"number":1333,"title":"busybox: tar applet fails to extract (some) symlinks","user":{"login":"alive4ever","id":7654911,"avatar_url":"https://avatars0.githubusercontent.com/u/7654911?v=4","gravatar_id":"","url":"https://api.github.com/users/alive4ever","html_url":"https://github.com/alive4ever","followers_url":"https://api.github.com/users/alive4ever/followers","following_url":"https://api.github.com/users/alive4ever/following{/other_user}","gists_url":"https://api.github.com/users/alive4ever/gists{/gist_id}","starred_url":"https://api.github.com/users/alive4ever/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alive4ever/subscriptions","organizations_url":"https://api.github.com/users/alive4ever/orgs","repos_url":"https://api.github.com/users/alive4ever/repos","events_url":"https://api.github.com/users/alive4ever/events{/privacy}","received_events_url":"https://api.github.com/users/alive4ever/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-03-14T02:07:23Z","updated_at":"2018-03-14T19:17:54Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"After upgrading busybox to version 1.28, busybox tar applet fails to extract\r\n(some) symlinks.\r\n\r\nWith my recycled device ([armani](https://wiki.postmarketos.org/wiki/Xiaomi_Redmi_1S_(xiaomi-armani))/armhf), the failed symlinks extraction occurs\r\nwith busybox applets inside /bin directory. There are no symlinks to busybox\r\napplets at all here. Even important applet such as /bin/ash and /bin/sh are not\r\navailable, causing `chroot` inside the debug-shell to fail unless a specific\r\ncommand is specified, such as `/bin/bash` or `/bin/busybox sh`.\r\n\r\n```\r\n/tmp/postmarketos/chroot/mnt/pmOS # ls -F bin/\r\nbash*           findmnt*        lsmod@          mountpoint*     umount*\r\nbbsuid*         groups*         mksh*           rc-status*      uniso*\r\nbusybox*        kmod*           modinfo@        su*             wdctl*\r\nbusybox-extras* login*          more*           tar*\r\ndmesg*          lsblk*          mount*          udevadm*\r\n```\r\nAs comparison, here are files present inside device chroot (`pmbootstrap chroot -r`)\r\n```\r\n/ # ls -F bin/\r\narch@            date@           fsync@     linux32@   more*           rc-status*  sync@\r\nash@             dd@             getopt@    linux64@   mount*          reformime@  tar*\r\nbase64@          df@             grep@      ln@        mountpoint*     rev@        touch@\r\nbash*            dmesg*          groups*    login*     mpstat@         rm@         true@\r\nbbconfig@        dnsdomainname@  gunzip@    ls@        mv@             rmdir@      udevadm*\r\nbbsuid*          dumpkmap@       gzip@      lsblk*     netstat@        run-parts@  umount*\r\nbusybox*         echo@           hostname@  lsmod@     nice@           sed@        uname@\r\nbusybox-extras*  ed@             ionice@    lzop@      pidof@          setpriv@    uniso*\r\ncat@             egrep@          iostat@    makemime@  ping@           setserial@  usleep@\r\nchgrp@           false@          ipcalc@    mkdir@     ping6@          sh@         watch@\r\nchmod@           fatattr@        kbd_mode@  mknod@     pipe_progress@  sleep@      wdctl*\r\nchown@           fdflush@        kill@      mksh*      printenv@       stat@       zcat@\r\nconspy@          fgrep@          kmod*      mktemp@    ps@             stty@\r\ncp@              findmnt*        link@      modinfo@   pwd@            su*\r\n```\r\n\r\nAnother noticeable failure is missing `/sbin/init` symlink, which causes\r\n`switch_root` to fail miserably.\r\n\r\n```\r\n/tmp/postmarketos/chroot/mnt/pmOS # ls -F sbin/  \r\nagetty*                     mpathpersist* \r\napk*                        multipath*\r\narptables-compat@           multipathd*\r\nblkdiscard*                 nologin*\r\nblockdev*                   openrc*\r\nchcpu*                      openrc-run*\r\ncryptsetup*                 pam_tally*\r\ncryptsetup-reencrypt*       pam_tally2*\r\nctrlaltdel*                 pam_timestamp_check*\r\ndhcpcd*                     pivot_root*\r\ndmsetup*                    raw*\r\ndmstats@                    rc*\r\ne2fsck*                     rc-service*\r\neapol_test*                 rc-update*\r\nebtables-compat@            runscript*\r\nfdisk*                      service*\r\nfindfs*                     setup-acf*\r\nfsck*                       setup-alpine*\r\nfsck.cramfs*                setup-apkcache*\r\nfsck.ext2@                  setup-apkrepos*\r\nfsck.ext3@                  setup-bootable*\r\nfsck.ext4@                  setup-disk*\r\nfsck.minix*                 setup-dns*\r\nfsfreeze*                   setup-gparted-desktop*\r\nfstrim*                     setup-hostname*\r\nhwclock*                    setup-interfaces*\r\nintegritysetup*             setup-keymap*\r\niptables@                   setup-lbu*\r\niptables-compat@            setup-mta*\r\niptables-compat-restore@    setup-ntp*\r\niptables-compat-save@       setup-proxy*\r\niptables-restore@           setup-sshd*\r\niptables-restore-translate@ setup-timezone*\r\niptables-save@              setup-udev*\r\niptables-translate@         setup-xen-dom0*\r\nkpartx*                     setup-xorg-base*\r\nlbu*                        start-stop-daemon*\r\nlbu_commit@                 supervise-daemon*\r\nlbu_exclude@                swapfile*\r\nlbu_include@                swaplabel*\r\nlbu_status@                 swapoff*\r\nlbu_update@                 swapon*\r\nldconfig*                   switch_root*\r\nlosetup*                    udevd*\r\nmke2fs*                     unix_chkpwd*\r\nmkfs*                       unix_update*\r\nmkfs.bfs*                   update-conf*\r\nmkfs.cramfs*                update-kernel*\r\nmkfs.ext2@                  veritysetup*\r\nmkfs.ext3@                  wipefs*\r\nmkfs.ext4@                  wpa_cli*\r\nmkfs.minix*                 wpa_passphrase*\r\nmkhomedir_helper*           wpa_supplicant*\r\nmkinitfs*                   xtables-compat-multi*\r\nmkmntdirs*                  xtables-multi*\r\nmkswap*                     zramctl*\r\n```\r\nHere are files inside sbin on device chroot\r\n```\r\n/ # ls -F sbin/\r\nacpid@                    iptables-restore-translate@  route@\r\nadjtimex@                 iptables-save@               runscript*\r\nagetty*                   iptables-translate@          service*\r\napk*                      iptunnel@                    setconsole@\r\narp@                      klogd@                       setup-acf*\r\narptables-compat@         kpartx*                      setup-alpine*\r\nblkdiscard*               lbu*                         setup-apkcache*\r\nblkid@                    lbu_commit@                  setup-apkrepos*\r\nblockdev*                 lbu_exclude@                 setup-bootable*\r\nchcpu*                    lbu_include@                 setup-disk*\r\ncryptsetup*               lbu_status@                  setup-dns*\r\ncryptsetup-reencrypt*     lbu_update@                  setup-gparted-desktop*\r\nctrlaltdel*               ldconfig*                    setup-hostname*\r\ndepmod@                   loadkmap@                    setup-interfaces*\r\ndhcpcd*                   logread@                     setup-keymap*\r\ndmsetup*                  losetup*                     setup-lbu*\r\ndmstats@                  lsmod@                       setup-mta*\r\ne2fsck*                   mdev@                        setup-ntp*\r\neapol_test*               mkdosfs@                     setup-proxy*\r\nebtables-compat@          mke2fs*                      setup-sshd*\r\nfbsplash@                 mkfs*                        setup-timezone*\r\nfdisk*                    mkfs.bfs*                    setup-udev*\r\nfindfs*                   mkfs.cramfs*                 setup-xen-dom0*\r\nfsck*                     mkfs.ext2@                   setup-xorg-base*\r\nfsck.cramfs*              mkfs.ext3@                   slattach@\r\nfsck.ext2@                mkfs.ext4@                   start-stop-daemon*\r\nfsck.ext3@                mkfs.minix*                  supervise-daemon*\r\nfsck.ext4@                mkfs.vfat@                   swapfile*\r\nfsck.minix*               mkhomedir_helper*            swaplabel*\r\nfsfreeze*                 mkinitfs*                    swapoff*\r\nfstrim*                   mkmntdirs*                   swapon*    \r\ngetty@                    mkswap*                      switch_root*\r\nhalt@                     modinfo@                     sysctl@      \r\nhdparm@                   modprobe@                    syslogd@       \r\nhwclock*                  mpathpersist*                tunctl@        \r\nifconfig@                 multipath*                   udevadm@       \r\nifdown@                   multipathd*                  udevd*     \r\nifenslave@                nameif@                      udhcpc@   \r\nifup@                     nologin*                     unix_chkpwd*          \r\ninit@                     openrc*                      unix_update*   \r\ninotifyd@                 openrc-run*                  update-conf*     \r\ninsmod@                   pam_tally*                   update-kernel*\r\nintegritysetup*           pam_tally2*                  vconfig@  \r\nip@                       pam_timestamp_check*         veritysetup*\r\nipaddr@                   pivot_root*                  watchdog@ \r\niplink@                   poweroff@                    wipefs*     \r\nipneigh@                  raidautorun@                 wpa_cli*   \r\niproute@                  raw*                         wpa_passphrase*\r\niprule@                   rc*                          wpa_supplicant*\r\niptables@                 rc-service*                  xtables-compat-multi*\r\niptables-compat@          rc-sstat@                    xtables-multi*  \r\niptables-compat-restore@  rc-update*                   zramctl* \r\niptables-compat-save@     reboot@                                        \r\niptables-restore@         rmmod@\r\n```\r\n\r\nAnother failed symlink extraction occurs inside `/etc/runlevels`, with no\r\nsymlinks inside `boot`, `default`, `shutdown`, and `sysinit`. The symlinks here\r\nare important, as they indicate whether a service is enabled for the specific\r\nrunlevel.\r\n\r\n```\r\n/tmp/postmarketos/chroot/mnt/pmOS # ls -F etc/runlevels/*\r\netc/runlevels/boot:\r\n\r\netc/runlevels/default:\r\n\r\netc/runlevels/nonetwork:\r\n\r\netc/runlevels/shutdown:\r\n\r\netc/runlevels/sysinit:\r\n```\r\nFor comparison, here are files inside /etc/runlevels inside device rootfs chroot.\r\n\r\n```\r\n/ # ls -F etc/runlevels/*\r\netc/runlevels/boot:\r\nbootmisc@  hostname@  hwclock@  modules@  msm-fb-refresher@  swclock@  sysctl@  syslog@\r\n\r\netc/runlevels/default:\r\nchronyd@  haveged@  networkmanager@  swapfile@        wcnss-wlan@\r\ndbus@     local@    sshd@            udev-postmount@  wpa_supplicant@\r\n\r\netc/runlevels/nonetwork:\r\n\r\netc/runlevels/shutdown:\r\nkillprocs@  mount-ro@  savecache@\r\n\r\netc/runlevels/sysinit:\r\ndevfs@  dmesg@  udev@  udev-trigger@  wlanarmani@\r\n```\r\nThis issue breaks installation using `--android-bootable-recovery` method,\r\nsince it uses `busybox tar` applet to extract rootfs tarball into root\r\nsubpartition.\r\n\r\nWhen extracting the rootfs tarball into root subpartition, failure is not an\r\noption.\r\n\r\nWith this fatal error, I suggest including better `tar` for use with\r\npostmarketos installation, such as GNU tar or `bsdtar` from `libarchive-tools`\r\npackage.\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/postmarketOS/pmbootstrap/issues/comments/373143247","html_url":"https://github.com/postmarketOS/pmbootstrap/issues/1333#issuecomment-373143247","issue_url":"https://api.github.com/repos/postmarketOS/pmbootstrap/issues/1333","id":373143247,"user":{"login":"ollieparanoid","id":6033024,"avatar_url":"https://avatars1.githubusercontent.com/u/6033024?v=4","gravatar_id":"","url":"https://api.github.com/users/ollieparanoid","html_url":"https://github.com/ollieparanoid","followers_url":"https://api.github.com/users/ollieparanoid/followers","following_url":"https://api.github.com/users/ollieparanoid/following{/other_user}","gists_url":"https://api.github.com/users/ollieparanoid/gists{/gist_id}","starred_url":"https://api.github.com/users/ollieparanoid/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ollieparanoid/subscriptions","organizations_url":"https://api.github.com/users/ollieparanoid/orgs","repos_url":"https://api.github.com/users/ollieparanoid/repos","events_url":"https://api.github.com/users/ollieparanoid/events{/privacy}","received_events_url":"https://api.github.com/users/ollieparanoid/received_events","type":"User","site_admin":false},"created_at":"2018-03-14T19:17:54Z","updated_at":"2018-03-14T19:17:54Z","author_association":"OWNER","body":"@alive4ever: Thank you very much for this excellent bug report! The details you've provided allowed to write a short script that reproduces the problem quickly.\r\n\r\nI've made a patch in #1335, could you please test that?\r\n\r\n### Short script for reproducing the error\r\n\r\n```shell\r\n#!/bin/sh -e\r\n\r\n# Reset, install zip\r\npmbootstrap -q -y zap\r\npmbootstrap -q chroot --add=\"zip\" -- true\r\n\r\n# Create \"rootfs.tar.gz\" inside \"pmos.zip\" like in recovery\r\npmbootstrap -q chroot -- tar -pcf rootfs.tar -C / bin/sh\r\npmbootstrap -q chroot -- gzip -f1 /rootfs.tar\r\npmbootstrap -q chroot -- zip -0 -r \"/pmos.zip\" /rootfs.tar.gz\r\n\r\n# Extract\r\npmbootstrap -q chroot -- mkdir -p /tmp/extract\r\npmbootstrap -q chroot -- sh -c \\\r\n    \"unzip -p /pmos.zip rootfs.tar.gz | tar -xz -C /tmp/extract\"\r\npmbootstrap -q chroot -- ls -lahR /tmp/extract\r\n```\r\n\r\n### Output\r\n\r\n```\r\n  adding: rootfs.tar.gz (stored 0%)\r\ntar: skipping unsafe symlink to '/bin/busybox' in archive, set EXTRACT_UNSAFE_SYMLINKS=1 to extract\r\n/tmp/extract:\r\ntotal 12\r\ndrwxr-xr-x    3 root     root        4.0K Mar 14 19:35 .\r\ndrwxrwxrwt    3 root     root        4.0K Mar 14 19:35 ..\r\ndrwxr-xr-x    2 root     root        4.0K Mar 14 19:35 bin\r\n\r\n/tmp/extract/bin:\r\ntotal 8\r\ndrwxr-xr-x    2 root     root        4.0K Mar 14 19:35 .\r\ndrwxr-xr-x    3 root     root        4.0K Mar 14 19:35 ..\r\n```\r\n\r\n### Output with EXTRACT_UNSAFE_SYMLINKS\r\n\r\nNow if we export `EXTRACT_UNSAFE_SYMLINKS=1` as the error suggests:\r\n```shell\r\npmbootstrap -q chroot -- sh -c \\\r\n    \"export EXTRACT_UNSAFE_SYMLINKS=1; \\\r\n    unzip -p /pmos.zip rootfs.tar.gz | tar -xz -C /tmp/extract\"\r\n```\r\n\r\n```\r\n  adding: rootfs.tar.gz (stored 0%)\r\n/tmp/extract:\r\ntotal 12\r\ndrwxr-xr-x    3 root     root        4.0K Mar 14 19:38 .\r\ndrwxrwxrwt    3 root     root        4.0K Mar 14 19:38 ..\r\ndrwxr-xr-x    2 root     root        4.0K Mar 14 19:38 bin\r\n\r\n/tmp/extract/bin:\r\ntotal 8\r\ndrwxr-xr-x    2 root     root        4.0K Mar 14 19:38 .\r\ndrwxr-xr-x    3 root     root        4.0K Mar 14 19:38 ..\r\nlrwxrwxrwx    1 root     root          12 Mar 14 19:38 sh -> /bin/busybox\r\n```\r\n\r\nThe check was introduced in https://github.com/mirror/busybox/commit/bc9bbeb2b81001e8731cd2ae501c8fccc8d87cc7."}}
{"actor":{"display_login":"ollieparanoid"}}