{"action":"created","issue":{"url":"https://api.github.com/repos/freebsd/poudriere/issues/259","repository_url":"https://api.github.com/repos/freebsd/poudriere","labels_url":"https://api.github.com/repos/freebsd/poudriere/issues/259/labels{/name}","comments_url":"https://api.github.com/repos/freebsd/poudriere/issues/259/comments","events_url":"https://api.github.com/repos/freebsd/poudriere/issues/259/events","html_url":"https://github.com/freebsd/poudriere/issues/259","id":51969003,"number":259,"title":"Provide support for passing build flags down the dependency chain","user":{"login":"marcusva","id":5445744,"avatar_url":"https://avatars.githubusercontent.com/u/5445744?v=3","gravatar_id":"","url":"https://api.github.com/users/marcusva","html_url":"https://github.com/marcusva","followers_url":"https://api.github.com/users/marcusva/followers","following_url":"https://api.github.com/users/marcusva/following{/other_user}","gists_url":"https://api.github.com/users/marcusva/gists{/gist_id}","starred_url":"https://api.github.com/users/marcusva/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/marcusva/subscriptions","organizations_url":"https://api.github.com/users/marcusva/orgs","repos_url":"https://api.github.com/users/marcusva/repos","events_url":"https://api.github.com/users/marcusva/events{/privacy}","received_events_url":"https://api.github.com/users/marcusva/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/freebsd/poudriere/labels/Feature_Request","name":"Feature_Request","color":"ededed"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2014-12-15T10:15:44Z","updated_at":"2016-02-19T20:52:53Z","closed_at":null,"body":"# Abstract\r\npoudriere creates packages from ports in isolated jails, which feature their own set of environment variables that are usually set on starting the jail. A dependency X of a port Y thus is unaware of any environment settings the ports framework does on Y's behalf.\r\nIn some situations, it is necessary that environment settings are passed down the build chain to allow dependencies to build correctly based on whatever the framework decides \"to be the best\" for port Y.\r\n\r\n# Problem Space\r\nAs of writing, the FreeBSD ports tree allows users and ports to build, install and use different versions of the Python language (`lang/python`) at the same time. Ports using the Python bits of the ports framework (namely `Mk/Uses/python.mk`) are enabled to choose a specific Python version or can be marked to support a series of Python versions, giving users (and other ports) the opportunity and flexibility to pick their specific preferred and supported version of the Python language.\r\n\r\nThe pre-pkg era did not allow ports to install packages for different Python versions, which used the same origin information, which was resolved with the availability of pkg 1.3. The ports and pkg framework are capable of installing e.g. py33-foo _and_ py27-foo originating from `devel/py-foo` at the same time.\r\n\r\nA single poudriere instance however is unable to create such a set of packages. Within a single run, it can only create a package py33-foo _or_ py27-foo, but not both. This in turn limits the possibilities of creating different version-specific packages based on the same origin and won't allow different ports to refer to a shared dependency, which _would_ support those versions.\r\n\r\n## Previous Workarounds\r\n\r\nThe limiting situation was resolved in the pre-pkg era by duplicating ports, renaming them and referring to either one of them within the specific consumers.\r\n\r\nA port `py-onlypy33` may be limited to Python 3.3 and may depend on `py-foo`, which supports not only Python 3.3, but also Python 2.7. Another port `py-onlypy27`may also depend on `py-foo`, but only works with Python 2.7. This lead to the typical following workaround:\r\n\r\n* Create port for py-foo, using Python 2.7\r\n* Create port for py33-foo, using Python 3.3\r\n* py-onlypy33 refers to py33-foo in the `BUILD_DEPENDS` or `RUN_DEPENDS`\r\n* py-onlypy27 refers to py-foo in the `BUILD_DEPENDS` or `RUN_DEPENDS`\r\n\r\n## Current Situation within the Ports\r\nWith pkg 1.3 and newer, a workaround as shown above is not necessary anymore. The Python bits of the ports framework allow ports to pass certain flags (specifically `DEPENDS_ARGS=PYTHON_VERSION=<versionnumber>`) to their dependencies, which can (re)act according to those. If a user wants to install `py-onlypy33`, the following will happen:\r\n\r\n1. User executes `make install` for `py-onlypy33`\r\n2. `python.mk` will append `PYTHON_VERSION=python3.3` to the `DEPENDS_ARGS`\r\n3. ports framework traverses into the `py-foo`dependency\r\n4. `python.mk` for `py-foo` notices `PYTHON_VERSION` and will instruct it to be built for Python 3.3\r\n5. `py33-foo` is installed\r\n6. `py-onlypy33` detects the proper `py33-foo` package and will continue to build and install\r\n\r\nIf the user is going to install `py-onlypy27`, the following will happen:\r\n\r\n1. User executes `make install` for `py-onlypy27`\r\n2. `python.mk` will append `PYTHON_VERSION=python2.7` to the `DEPENDS_ARGS`\r\n3. ports framework traverses into the `py-foo`dependency\r\n4. `python.mk` for `py-foo` notices `PYTHON_VERSION` and will instruct it to be built for Python 2.7\r\n5. `py27-foo` is installed\r\n6. `py-onlypy27` detects the proper `py27-foo` package and will continue to build and install\r\n\r\n## Current Situation with Poudriere\r\n\r\nPoudriere builds ports and their dependencies in (mostly) isolated jail environments, with the ability to parallelize builds. Information, such as the `DEPENDS_ARGS` from above are not shared or passed down the build chain. This results in the following behaviour for `py-onlypy33`:\r\n\r\n1. poudriere gathers dependencies for `py-onlypy33`\r\n2. poudriere runs the build for `py-foo` using the configured default python version (e.g. 2.7)\r\n3. poudriere registers a package `py27-foo`\r\n4. poudriere raises an error for `py-onlypy33` due to the unsatisfied dependency for `py33-foo`\r\n\r\n# Solution\r\nPoudriere needs to be able to pass around specific flags or settings, which can be set by a port or the ports framework. The ports framework may define a specific set of global variables, which should be respected by package builders (poudriere, tinderbox, etc.).\r\n\r\nTo prevent misbehaviour and to avoid limiting the parallelization features of package builders, those global variables should be available prior to any specific task for a port (`make fetch`, `make patch`, etc.), so that the internal package builder workflow may look like:\r\n\r\n1. detect port\r\n2. calculate global flags\r\n3. gather dependencies to be built (with the global flags)\r\n4. build dependencies (with the global flags)\r\n5. build port\r\n\r\nA single variable `PACKAGE_BUILD_FLAGS` may be sufficient and should be able to be passed down to dependencies of dependencies of dependencies of ..."},"comment":{"url":"https://api.github.com/repos/freebsd/poudriere/issues/comments/186401091","html_url":"https://github.com/freebsd/poudriere/issues/259#issuecomment-186401091","issue_url":"https://api.github.com/repos/freebsd/poudriere/issues/259","id":186401091,"user":{"login":"bdrewery","id":33693,"avatar_url":"https://avatars.githubusercontent.com/u/33693?v=3","gravatar_id":"","url":"https://api.github.com/users/bdrewery","html_url":"https://github.com/bdrewery","followers_url":"https://api.github.com/users/bdrewery/followers","following_url":"https://api.github.com/users/bdrewery/following{/other_user}","gists_url":"https://api.github.com/users/bdrewery/gists{/gist_id}","starred_url":"https://api.github.com/users/bdrewery/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bdrewery/subscriptions","organizations_url":"https://api.github.com/users/bdrewery/orgs","repos_url":"https://api.github.com/users/bdrewery/repos","events_url":"https://api.github.com/users/bdrewery/events{/privacy}","received_events_url":"https://api.github.com/users/bdrewery/received_events","type":"User","site_admin":false},"created_at":"2016-02-19T20:52:53Z","updated_at":"2016-02-19T20:52:53Z","body":"<pre>\r\ndns/unbound/Makefile:DEPENDS_ARGS+=     WITH_GOST=yes\r\ndns/unbound/Makefile:DEPENDS_ARGS+=     WITH_ECDSA=yes\r\n</pre>\r\n\r\nThese are wrong since we don't generally support multiple packages installed at once but the plan with Poudriere will allow supporting something like this in the future perhaps."}}