{"action":"created","issue":{"url":"https://api.github.com/repos/machinekit/machinekit/issues/874","repository_url":"https://api.github.com/repos/machinekit/machinekit","labels_url":"https://api.github.com/repos/machinekit/machinekit/issues/874/labels{/name}","comments_url":"https://api.github.com/repos/machinekit/machinekit/issues/874/comments","events_url":"https://api.github.com/repos/machinekit/machinekit/issues/874/events","html_url":"https://github.com/machinekit/machinekit/issues/874","id":135107554,"number":874,"title":"RFC: getting rid of multiple rtap_app binaries, enable alternatives when loading components","user":{"login":"mhaberler","id":901479,"avatar_url":"https://avatars.githubusercontent.com/u/901479?v=3","gravatar_id":"","url":"https://api.github.com/users/mhaberler","html_url":"https://github.com/mhaberler","followers_url":"https://api.github.com/users/mhaberler/followers","following_url":"https://api.github.com/users/mhaberler/following{/other_user}","gists_url":"https://api.github.com/users/mhaberler/gists{/gist_id}","starred_url":"https://api.github.com/users/mhaberler/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mhaberler/subscriptions","organizations_url":"https://api.github.com/users/mhaberler/orgs","repos_url":"https://api.github.com/users/mhaberler/repos","events_url":"https://api.github.com/users/mhaberler/events{/privacy}","received_events_url":"https://api.github.com/users/mhaberler/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-02-20T18:18:21Z","updated_at":"2016-02-21T11:21:06Z","closed_at":null,"body":"[a bit longish but fundamental change - I'd appreciate a look over the shoulder while at it]\r\n\r\nthis lays out a proposed change to solve two issues at once:\r\n\r\n1. getting rid of the per-flavor `rtapi_app_<flavor>` binaries (helps simplifiying the build and distribution, including the cmake build)\r\n2. enable loading of components from different directories specified by a path preference (needed for the multicore branch)\r\n\r\n(1) is an annoyance due to the way shared libraries are resolved by dlopen() in setuid binaries (rtapi_app needs to be setuid so it can talk to hardware); see man dlopen for the gory details.\r\n\r\nThe current solution @zultron came up with is:\r\n\r\nThe component shared libs (.so files) live under `rtlib/<flavor>/<compname>.so`, and to force loading from a particular `rtlib/<flavor>/` the per-thread rtapi_app binary gets an [rpath](https://en.wikipedia.org/wiki/Rpath) set during [build](https://github.com/machinekit/machinekit/blob/master/src/rtapi/Submakefile#L173) - this means that once you start `rtapi_app_<flavor>`, this binary will load shared libs with dlopen only from its rpath (`rtlib/<flavor>/`). \r\n\r\nWhile this works great, the downside is: we need multiple rtapi_app binaries - one per flavor - and they really only differ in the `rpath` attribute. Also, the scripts/realtime code needs to figure the flavor and start the right binary. Not much of an issue though.\r\n\r\n(2) The harder issue is the introduction of a versioned HAL API in the multicore branch. To make a component multicore-safe, it must employ the new HAL pin API, and accessor functions; manipulating raw memory locations through pointers as done in the old API is impossible to make thread-safe in a sane manner. However, at the pin/signal level the API's are compatible. So a v1 pin and a v2 pin can be linked to a signal, and things will work as before.\r\n\r\n@ArcEye has adapted comp to icomp to employ the v2 HAL API, and rewritten most of the components to use it, and I have converted some of the C comps to v2 (stepgen, pwmgen, encoder, sim_encoder).\r\n\r\nThis means we now have two sets of components: the existing set of legacy components, and all the rewritten v2 components. This bears the question how these are distinguished and loaded. As per above rpath scheme, all components need to reside in the same trusted directory. What @ArcEye and me have done so far is to give  these comps different basenames; so for example there exists a `rtlib/posix/xor.so` and `rtlib/posix/xorv2.so` component.\r\n\r\nThis works, but means that ALL halcmd/Python HAL configs which should be converted to v2 need to be rewritten as the comp name changes. It also makes regression testing v2 against v1 comps harder as this is a somewhat ad-doc name change.\r\n\r\nA better solution would be to provide a preference path of directories to load components from, like for example `rtlib/posix/v2:rtlib/posix`, and have [the comp loading code in rtapi_app](https://github.com/machinekit/machinekit/blob/master/src/rtapi/rtapi_app.cc#L528-L530) do the right thing depending on this path preference. This avoids wholesale rewrite of HAL configs as the preference path could be passed by some other means (halcmd primitive, API parameter, environment variable etc).\r\n\r\nHowever, this is at odds with the security intentions of the rpath feature: directories of an rpath on a setuid binary are trusted. And the rpath resolution ONLY works if there is no slash in the first argument to dlopen().\r\n\r\nTo cite the manpage:\r\n\r\n>  If filename contains a slash (\"/\"), then it is interpreted as a (relative or absolute) pathname. Otherwise, the dynamic linker searches for the library as follows (see ld.so(8) for further details): ... details of loading, including rpath feature..\r\n\r\nSo as soon as a slash is found in the dlopen() argument the rpath feature is essentially disabled. This is a problem if we want to distinguish shared libaries aka versioned components by pathnames like `v2/comp.so` etc.\r\n\r\nwhat does work - and that could be the start towards a solution to both issues - is:\r\n\r\n- provide a way for rtapi_app to learn the path to the `rtlib/<flavor>` (eg from config values, like EMC2_RTLIB_BASE_DIR which points to `rtlib/` - since rtapi_app knows the current flavor, it can construct the path to `rtlib/<flavor>` from EMC2_RTLIB_BASE_DIR \r\n- before loading a shared library, do a chdir(`rtlib/<flavor>`)\r\n- ALWAYS use a relative path in the dlopen argument, for instance dlopen(`./xor.so`) or dlopen(`./v2/xor.so`)\r\n- post loading, pop back to original working directory (so among others, an accidential core file is dropped in the right directory)\r\n\r\nthe gist of the idea is - the preference path is just a subdir under a trusted directory, and hence trusted as well.\r\n\r\nNow if we also set the rpath to EMC2_RTLIB_BASE_DIR rather than to `rtlib/<flavor>`, then we also do not need separate binaries because there will be only one rtapi_app with an rpath which applies to all flavors.\r\n\r\nFrom what I've tried this works. I would appreciate feedback on the idea before I do this in full.\r\n\r\n[edit: the way dlopen() works, it seems this scheme avoids the rpath functionality as it does not apply altogether to dlopen() loading from relative paths. This suggests the rpath to  EMC2_RTLIB_BASE_DIR  can be dropped as long as  EMC2_RTLIB_BASE_DIR  is available some way to chdir to. ]\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/machinekit/machinekit/issues/comments/186801259","html_url":"https://github.com/machinekit/machinekit/issues/874#issuecomment-186801259","issue_url":"https://api.github.com/repos/machinekit/machinekit/issues/874","id":186801259,"user":{"login":"ArcEye","id":770392,"avatar_url":"https://avatars.githubusercontent.com/u/770392?v=3","gravatar_id":"","url":"https://api.github.com/users/ArcEye","html_url":"https://github.com/ArcEye","followers_url":"https://api.github.com/users/ArcEye/followers","following_url":"https://api.github.com/users/ArcEye/following{/other_user}","gists_url":"https://api.github.com/users/ArcEye/gists{/gist_id}","starred_url":"https://api.github.com/users/ArcEye/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ArcEye/subscriptions","organizations_url":"https://api.github.com/users/ArcEye/orgs","repos_url":"https://api.github.com/users/ArcEye/repos","events_url":"https://api.github.com/users/ArcEye/events{/privacy}","received_events_url":"https://api.github.com/users/ArcEye/received_events","type":"User","site_admin":false},"created_at":"2016-02-21T11:21:06Z","updated_at":"2016-02-21T11:21:06Z","body":"I think you are quite right, but I would actually go further.\r\n\r\nThe introduction of the multiple kernel builds, build flavour etc was an innovation to Linuxcnc ( one it ultimately was not ready for :disappointed:  ) as part of the support for other rt kernels and platforms.\r\nIt then defined the difference between Linuxcnc and Machinekit in terms of platform support.\r\n\r\nNow that we are splitting the project into discreet, albeit interconnected blocks, a lot of other things have changed and it might be time to consider what we actually need.\r\n\r\nrtai has gone and next to no-one actually uses posix as a platform, there is simply no need, a simulation will run on a rt kernel too plus there is no latency issue with a simulation and rt-preempt will run on just about everything.\r\n\r\nThis leaves rt-preempt and xenomai, in the future possibly just rt-preempt.\r\n\r\nI would greatly favour a build system where either the running kernel is detected and built for, or it is specified at configuration.\r\nNo posix builds because they are irrelevant and if you want it to run on a kernel, you build whilst running that kernel.\r\nThe build is thereafter marked as that flavour and out of tree builds etc can be managed simply by reference to that tag.\r\n\r\nThat immediately simplifies a lot of the rpath stuff because there are no flavours, just what you built for.  \r\nIf a default choice of component, with a legacy fallback, can be managed as you describe, that would be ideal.  \r\n\r\nThe alternative of having different configurations etc. is a non starter for users, it needs to be backwardly compatible and unseen, in the same way that we made `loadrt` function in apparently the same way with instantiated components as it had previously with rt ones.\r\n\r\nI look forward to getting the multicore work into HAL, and this split is probably the best time to do it.\r\nWe have a clean sheet to some extent and don't need to be shackled by previous norms\r\n\r\n\r\n\r\n\r\n"}}