{"action":"created","issue":{"url":"https://api.github.com/repos/LLNL/spack/issues/2590","repository_url":"https://api.github.com/repos/LLNL/spack","labels_url":"https://api.github.com/repos/LLNL/spack/issues/2590/labels{/name}","comments_url":"https://api.github.com/repos/LLNL/spack/issues/2590/comments","events_url":"https://api.github.com/repos/LLNL/spack/issues/2590/events","html_url":"https://github.com/LLNL/spack/pull/2590","id":195600423,"number":2590,"title":"[HACK] Make concretization great again!","user":{"login":"adamjstewart","id":12021217,"avatar_url":"https://avatars.githubusercontent.com/u/12021217?v=3","gravatar_id":"","url":"https://api.github.com/users/adamjstewart","html_url":"https://github.com/adamjstewart","followers_url":"https://api.github.com/users/adamjstewart/followers","following_url":"https://api.github.com/users/adamjstewart/following{/other_user}","gists_url":"https://api.github.com/users/adamjstewart/gists{/gist_id}","starred_url":"https://api.github.com/users/adamjstewart/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/adamjstewart/subscriptions","organizations_url":"https://api.github.com/users/adamjstewart/orgs","repos_url":"https://api.github.com/users/adamjstewart/repos","events_url":"https://api.github.com/users/adamjstewart/events{/privacy}","received_events_url":"https://api.github.com/users/adamjstewart/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":13,"created_at":"2016-12-14T17:52:56Z","updated_at":"2016-12-14T19:41:02Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/LLNL/spack/pulls/2590","html_url":"https://github.com/LLNL/spack/pull/2590","diff_url":"https://github.com/LLNL/spack/pull/2590.diff","patch_url":"https://github.com/LLNL/spack/pull/2590.patch"},"body":"I _really_ didn't want to have to do this, but I've been left with no other choice. Spack's concretization algorithm is so crippled at the moment that simple commands no longer work more often than I would like. This PR does not fix the concretizer, as I wouldn't even no where to begin.\r\n\r\nSo what does this PR do?\r\n\r\n- [x] Always default to `+mpi` and `~X` for _every_ package\r\n- [x] Never even try to forward variants, as it doesn't work anyway\r\n\r\nWhy in the world would you want to do something so drastic?\r\n\r\nTry checking out a fresh copy of Spack, and with no `packages.yaml`, run the following command:\r\n```\r\n$ spack spec gtkplus\r\n...\r\n==> Error: Invalid spec: 'pango@1.40.1%gcc@6.2.1~X arch=linux-fedora25-x86_64'. Package pango requires variant +X, but spec asked for ~X\r\n```\r\nThe problem here is that `gtkplus` and `cairo` default to `+X` while `pango` defaults to `~X`. Spack's concretization algorithm is incapable of resolving this conflict because it settles on the default value for all dependencies. This bug has been reported over and over and over again (see #2546, #2574, #1552, etc.).\r\n\r\nThe solution that I decided on in this PR is to make sure common variants like `+X` and `+mpi` are at least consistent. Of course, as soon as you run `spack spec gtkplus+X`, everything breaks again, but at least `spack spec gtkplus` works.\r\n\r\nWhat solutions have we used in the past?\r\n\r\nWhen going through the packages, I noticed a lot of packages that looked like:\r\n```python\r\ndepends_on('hdf5+mpi', when='+mpi')\r\ndepends_on('hdf5~mpi', when='~mpi')\r\n```\r\nWhat's wrong with this? Well, first of all, it doesn't forward the variants properly, as seen with `spack spec gtkplus`. Second, try running the following:\r\n```\r\nspack spec packageA ^hdf5@1.8.12\r\n```\r\nWith a packageA containing those two lines, it will inform you that packageA does not depend on hdf5. This is because Spack doesn't concretize default variants before it concretizes dependency versions. Either of the following commands will work:\r\n```\r\nspack spec packageA +mpi ^hdf5@1.8.12\r\nspack spec packageA -mpi ^hdf5@1.8.12\r\n```\r\nbut the original won't. So what if we always do:\r\n```python\r\ndepends_on('hdf5')\r\ndepends_on('hdf5+mpi', when='+mpi')\r\ndepends_on('hdf5~mpi', when='~mpi')\r\n```\r\nOk, first of all, that's a pain in the ass. Second of all, it doesn't work, as evidenced by `spack spec gtkplus`. But what if we did:\r\n```python\r\ndepends_on('hdf5')\r\ndepends_on('hdf5+mpi', when='+mpi')\r\n```\r\nDoesn't work either. I just tried it with the `gtkplus`, `cairo`, `pango` chain and you still see the problem with a conflict between `+X` and `~X`.\r\n\r\nBut won't everything be happy if we keep variant forwarding and make sure all of the variants have the same default? Nope. Things like `spack spec gtkplus` will work, but `spack spec gtkplus +X` won't. And if you decide to change the default for any package in `packages.yaml`, we're right back to square one, except it's harder to debug now unless you tell us what is in your `packages.yaml`.\r\n\r\nThere is another standing issue that this PR does not address. Try running the following:\r\n```\r\nspack spec python+tk\r\n...\r\n==> Error: +tk does not satisfy ~tk\r\n```\r\nThis may look like the problem we just solved, but it's not that simple. Here is the dependency chain:\r\n\r\npython+tk -> tk -> libx11 -> libxcb -> python~tk\r\n\r\nSince libxcb has a build dependency on python~tk, it says it doesn't satisfy python+tk. Of course, if you try to fix this by setting the default to `python+tk`, you end up in infinite recursion hell, as seen in #2495 and #2565. Even more fun, if you try to build py-matplotlib for Python 3, it requires python+tk, and you end up with the problem that xcb-proto, another dependency of libxcb, depends on Python 2, which is most certainly not Python 3. \r\n\r\nThis will hopefully be fixed when we start resolving build dependencies separately, but for now it is an absolute nightmare.\r\n\r\nI _really_ hope this PR won't be merged, but it does provide immediate relief for a lot of open issues. In my opinion, these issues should block the release of `0.10`. If users can't build `python+tk` or `gtkplus` without hacking up Spack, we have a real problem."},"comment":{"url":"https://api.github.com/repos/LLNL/spack/issues/comments/267134755","html_url":"https://github.com/LLNL/spack/pull/2590#issuecomment-267134755","issue_url":"https://api.github.com/repos/LLNL/spack/issues/2590","id":267134755,"user":{"login":"citibeth","id":1924215,"avatar_url":"https://avatars.githubusercontent.com/u/1924215?v=3","gravatar_id":"","url":"https://api.github.com/users/citibeth","html_url":"https://github.com/citibeth","followers_url":"https://api.github.com/users/citibeth/followers","following_url":"https://api.github.com/users/citibeth/following{/other_user}","gists_url":"https://api.github.com/users/citibeth/gists{/gist_id}","starred_url":"https://api.github.com/users/citibeth/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/citibeth/subscriptions","organizations_url":"https://api.github.com/users/citibeth/orgs","repos_url":"https://api.github.com/users/citibeth/repos","events_url":"https://api.github.com/users/citibeth/events{/privacy}","received_events_url":"https://api.github.com/users/citibeth/received_events","type":"User","site_admin":false},"created_at":"2016-12-14T19:41:02Z","updated_at":"2016-12-14T19:41:02Z","body":"Thank you for the detailed example.  I question one thing in it:\n\n>  In an ideal world, Spack would say that since pango needs +X, so does\n> gtkplus,\n>\nLet's think in terms of PATTERNS and SPECS.  They both have the same\nsyntax, so we tend to get them confused.\n\nI thought the way things were SUPPOSED to work is that the PATTERN\n`packageA` will match the SPEC `packageA+x` or `packageA~x`.  The rationale\nis that if `packageB` -> `packageA` without anything else, then `packageB`\nshould not really care whether or not `packageA` does or does not have the\n`+x` feature turned on.  Boolean variants really do mean that \"this is an\nextra feature.\"  If not, then maybe multi-valued variants should be used\ninstead (once they're merged).\n\nSo... I believe that THIS little corner works as it should for now: a non-x\ngtk should work find with pango+x\n"}}
{"actor":{"display_login":"citibeth"}}