{"action":"created","issue":{"url":"https://api.github.com/repos/catkin/catkin_tools/issues/276","labels_url":"https://api.github.com/repos/catkin/catkin_tools/issues/276/labels{/name}","comments_url":"https://api.github.com/repos/catkin/catkin_tools/issues/276/comments","events_url":"https://api.github.com/repos/catkin/catkin_tools/issues/276/events","html_url":"https://github.com/catkin/catkin_tools/pull/276","id":127919749,"number":276,"title":"[PR196 2/3] Adding linked develspace support","user":{"login":"jbohren","id":1253450,"avatar_url":"https://avatars.githubusercontent.com/u/1253450?v=3","gravatar_id":"","url":"https://api.github.com/users/jbohren","html_url":"https://github.com/jbohren","followers_url":"https://api.github.com/users/jbohren/followers","following_url":"https://api.github.com/users/jbohren/following{/other_user}","gists_url":"https://api.github.com/users/jbohren/gists{/gist_id}","starred_url":"https://api.github.com/users/jbohren/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jbohren/subscriptions","organizations_url":"https://api.github.com/users/jbohren/orgs","repos_url":"https://api.github.com/users/jbohren/repos","events_url":"https://api.github.com/users/jbohren/events{/privacy}","received_events_url":"https://api.github.com/users/jbohren/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/catkin/catkin_tools/labels/enhancement","name":"enhancement","color":"84b6eb"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/catkin/catkin_tools/milestones/3","html_url":"https://github.com/catkin/catkin_tools/milestones/0.4.0%20-%20Second%20Beta%20Announcement","labels_url":"https://api.github.com/repos/catkin/catkin_tools/milestones/3/labels","id":1161028,"number":3,"title":"0.4.0 - Second Beta Announcement","description":"This release will be included the second `catkin_tools` beta announcement. It will include numerous bugfixes, a new execution engine, better I/O capture with automatic warning display, per-package cleaning via linked develspace support, and other minor features.","creator":{"login":"jbohren","id":1253450,"avatar_url":"https://avatars.githubusercontent.com/u/1253450?v=3","gravatar_id":"","url":"https://api.github.com/users/jbohren","html_url":"https://github.com/jbohren","followers_url":"https://api.github.com/users/jbohren/followers","following_url":"https://api.github.com/users/jbohren/following{/other_user}","gists_url":"https://api.github.com/users/jbohren/gists{/gist_id}","starred_url":"https://api.github.com/users/jbohren/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jbohren/subscriptions","organizations_url":"https://api.github.com/users/jbohren/orgs","repos_url":"https://api.github.com/users/jbohren/repos","events_url":"https://api.github.com/users/jbohren/events{/privacy}","received_events_url":"https://api.github.com/users/jbohren/received_events","type":"User","site_admin":false},"open_issues":18,"closed_issues":31,"state":"open","created_at":"2015-06-11T21:08:49Z","updated_at":"2016-01-27T05:10:49Z","due_on":"2016-02-15T05:00:00Z","closed_at":null},"comments":19,"created_at":"2016-01-21T13:03:48Z","updated_at":"2016-01-27T22:27:10Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/catkin/catkin_tools/pulls/276","html_url":"https://github.com/catkin/catkin_tools/pull/276","diff_url":"https://github.com/catkin/catkin_tools/pull/276.diff","patch_url":"https://github.com/catkin/catkin_tools/pull/276.patch"},"body":"This adds the prototype linked develspace support from #196. \r\n\r\n## Why?\r\n\r\nCatkin packages build their products into a usable FHS tree called a \"develspace.\" We'd like to be able to clean the products from a specific package. Unfortunately, it's impossible to determine which files were generated by which package without some OS-specific hacks or wrapper programs such as `checkinstall`. See #196 and #194 for more details on the development and history of this implementation.\r\n\r\n## Basic Design\r\n\r\nThis PR implements the same linked develspace support  and the same workarounds as in #196. It adds the additional functions required for the forthcoming enhancements to the `clean` verb, but does change it.\r\n\r\nThis adds a new `linked` develspace layout, which appears to the user as a merged develspace. This first builds Catkin packages into a *hidden* isolated develspace, and then symbolically links all of the created files into the *visible* merged develspace. All of these files are listed in a `devel_manifest.txt` file stored in the develspace. If two packages generate files with the same path, the linking code checks if the two files are identical. If so, it increments a counter in a `devel_collisions.txt` file. If they are different, it raises a warning. Pure CMake and other build types which support install logs can simply use their normal install targets with the visible develspace.\r\n\r\nMaking this work requires a major workaround due to the way Catkin generates the develspace setup files. Catkin's CMake macros generate setup files which encode the actual *hidden* isolated develspace. This means that they can't be used in the visible merged develspace. This means that these files need to be generated by some other means. The solution used by this PR is to run a \"prebuild\" step which builds either a minimal catkin package called `catkin_tools_prebuild` or if it exists in the source space, the `catkin` CMake package, itself.\r\n\r\nThis does have another benefit, which is that workspaces containing only non-Catkin packages will still have the normal Catkin setup files.  See #87 for an example use case.\r\n\r\n## Interface\r\n\r\nSince this PR includes three options for the develspace layout, the `catkin_tools` `Context` reflects the develspace layout with a string identifier which is either `merged`, `isolated`, or `linked`. This is reflected in the `catkin config` options and the context summary:\r\n\r\n```\r\nDevel Space:\r\n  Options for configuring the structure of the devel space.\r\n\r\n  --merge-devel         Build products from each catkin package into a single\r\n                        merged devel spaces.\r\n  --link-devel          Build products from each catkin package into isolated\r\n                        spaces, then symbolically link them into a merged\r\n                        devel space.\r\n  --isolate-devel       Build products from each catkin package into isolated\r\n                        devel spaces.\r\n```\r\n\r\n```\r\n--------------------------------------------------\r\nProfile:                     default\r\nExtending:                   None\r\nWorkspace:                   /home/jbohren\r\nSource Space:      [missing] /home/jbohren/src\r\nBuild Space:       [missing] /home/jbohren/build\r\nDevel Space:       [missing] /home/jbohren/devel\r\nInstall Space:     [missing] /home/jbohren/install\r\nDESTDIR:                     None\r\n--------------------------------------------------\r\nDevel Space Layout:          merged\r\nInstall Packages:            False\r\nIsolate Installs:            False\r\n--------------------------------------------------\r\nAdditional CMake Args:       None\r\nAdditional Make Args:        None\r\nAdditional catkin Make Args: None\r\nInternal Make Job Server:    True\r\nCache Job Environments:      False\r\n--------------------------------------------------\r\nWhitelisted Packages:        None\r\nBlacklisted Packages:        None\r\n--------------------------------------------------\r\n```\r\n\r\n## Additional Changes\r\n\r\n* Moves common functionality out of poorly named `catkin_tools.jobs.job` module into `catkin_tools.common` and `catkin_tools.jobs.utils`.\r\n* Clarifies and extends `context` path functions\r\n* Gets rid of no-longer-used `build_env.sh` generation source\r\n* Fixes some line-break handling and path formatting issues with the CMake io handler\r\n* Fixes the dependency gathering functions so they preserve topological order\r\n* Fixes #45 / #65 by using the unbuilt list for the isolated devel space list\r\n* Fixes #148, #224 by including exec and run dependencies of top-level packages\r\n* Fixes #227 by automatically calling `--force-cmake` if `cmake_args` changes\r\n* Fixes regression #225\r\n* Adds version, fixes #272 \r\n\r\n## Tasks\r\n\r\n- [x] Incorporate `exec_depend` dependencies into build deps #278, #148\r\n- [x] Review first draft by @wjwwood \r\n- [x] Use `dateutil` for copyright dates\r\n- [x] Fix trailing commas / etc in https://github.com/catkin/catkin_tools/commit/d1ef3e1506c5e0ca29d463b14ef20c076a278289\r\n- [ ] Update autocomplete args"},"comment":{"url":"https://api.github.com/repos/catkin/catkin_tools/issues/comments/175892415","html_url":"https://github.com/catkin/catkin_tools/pull/276#issuecomment-175892415","issue_url":"https://api.github.com/repos/catkin/catkin_tools/issues/276","id":175892415,"user":{"login":"wjwwood","id":100427,"avatar_url":"https://avatars.githubusercontent.com/u/100427?v=3","gravatar_id":"","url":"https://api.github.com/users/wjwwood","html_url":"https://github.com/wjwwood","followers_url":"https://api.github.com/users/wjwwood/followers","following_url":"https://api.github.com/users/wjwwood/following{/other_user}","gists_url":"https://api.github.com/users/wjwwood/gists{/gist_id}","starred_url":"https://api.github.com/users/wjwwood/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wjwwood/subscriptions","organizations_url":"https://api.github.com/users/wjwwood/orgs","repos_url":"https://api.github.com/users/wjwwood/repos","events_url":"https://api.github.com/users/wjwwood/events{/privacy}","received_events_url":"https://api.github.com/users/wjwwood/received_events","type":"User","site_admin":false},"created_at":"2016-01-27T22:27:09Z","updated_at":"2016-01-27T22:27:09Z","body":"Ok, that cleared it up for me. My acceptance testing is going well, I'm just finishing up testing `--link-devel` with `--install` and I think we'll be good to go."}}