{"action":"created","issue":{"url":"https://api.github.com/repos/ocaml/dune/issues/427","repository_url":"https://api.github.com/repos/ocaml/dune","labels_url":"https://api.github.com/repos/ocaml/dune/issues/427/labels{/name}","comments_url":"https://api.github.com/repos/ocaml/dune/issues/427/comments","events_url":"https://api.github.com/repos/ocaml/dune/issues/427/events","html_url":"https://github.com/ocaml/dune/issues/427","id":290141709,"number":427,"title":"Better isolation of .cm* files and support of private modules","user":{"login":"diml","id":1251584,"avatar_url":"https://avatars1.githubusercontent.com/u/1251584?v=4","gravatar_id":"","url":"https://api.github.com/users/diml","html_url":"https://github.com/diml","followers_url":"https://api.github.com/users/diml/followers","following_url":"https://api.github.com/users/diml/following{/other_user}","gists_url":"https://api.github.com/users/diml/gists{/gist_id}","starred_url":"https://api.github.com/users/diml/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/diml/subscriptions","organizations_url":"https://api.github.com/users/diml/orgs","repos_url":"https://api.github.com/users/diml/repos","events_url":"https://api.github.com/users/diml/events{/privacy}","received_events_url":"https://api.github.com/users/diml/received_events","type":"User","site_admin":false},"labels":[{"id":471845593,"url":"https://api.github.com/repos/ocaml/dune/labels/enhancement","name":"enhancement","color":"84b6eb","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":13,"created_at":"2018-01-19T23:54:57Z","updated_at":"2018-04-26T09:26:25Z","closed_at":null,"author_association":"MEMBER","body":"Currently we always wait for all the rules of a directory to be loaded before executing any rule from this directory. Technically, we could start executing rules that don't depend on globs from this directory before all the rules are known. That might be useful when we add support for dynamic rules for instance. Indeed, it seems quite natural to write something like this for instance:\r\n\r\n```scheme\r\n(executable ((name foo) (module (foo))))\r\n\r\n(rule (with-stdout-to jbuild.inc (run ./foo.exe)))\r\n\r\n(include jbuild.inc)\r\n```\r\n\r\nThere is however one issue with such approach: because the compiler uses include directories, the compilation of any .ml/.mli file implicitly depend on `*.cmi`. In fact, there is already an issue currently when there are several independent libraries in the same directories, since they can see each others modules. Though, the problem gets worse if we start building before we have loaded all the rules: because we can only delete stale artefacts when all the rules are known, we might end up leaving stale .cmi files and the compiler could end up reading them.\r\n\r\n## Solution\r\n\r\nThe idea is to put the .cm* files in sub-directories. For instance we could put the .cm* files for a library `foo` in a sub-directory `.foo.objs`. All the rules for this sub-directory would always be all generated at once, so we wouldn't have the issue describe before.\r\n\r\n## Private modules\r\n\r\nThis should provide a simpler way to support private modules: the .cm* files for private modules of libraries could be stored in `.foo.objs/private`. This way we could easily control who can see them by passing or not the relevant `-I .../private` option.\r\n\r\n## Work\r\n\r\nThere should be only three things to change:\r\n1. the path to the various .cm* files\r\n2. the include paths passed to the various tools\r\n3. merlin files\r\n\r\nWe could proceed as follow:\r\n- change the functions `Module.cm*_file` to take an `obj_dir` argument rather than a `dir` one, as well as various functions in `gen_rules.ml` returning other OCaml object files\r\n- define `obj_dir` in `library_rules` and `executables_rules` appropriately and fold these values everywhere needed\r\n- adapt `Lib.include_flags`, `Lib.link_flags`, ...\r\n- adapt `merlin.ml`"},"comment":{"url":"https://api.github.com/repos/ocaml/dune/issues/comments/384574501","html_url":"https://github.com/ocaml/dune/issues/427#issuecomment-384574501","issue_url":"https://api.github.com/repos/ocaml/dune/issues/427","id":384574501,"user":{"login":"diml","id":1251584,"avatar_url":"https://avatars1.githubusercontent.com/u/1251584?v=4","gravatar_id":"","url":"https://api.github.com/users/diml","html_url":"https://github.com/diml","followers_url":"https://api.github.com/users/diml/followers","following_url":"https://api.github.com/users/diml/following{/other_user}","gists_url":"https://api.github.com/users/diml/gists{/gist_id}","starred_url":"https://api.github.com/users/diml/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/diml/subscriptions","organizations_url":"https://api.github.com/users/diml/orgs","repos_url":"https://api.github.com/users/diml/repos","events_url":"https://api.github.com/users/diml/events{/privacy}","received_events_url":"https://api.github.com/users/diml/received_events","type":"User","site_admin":false},"created_at":"2018-04-26T09:26:25Z","updated_at":"2018-04-26T09:26:25Z","author_association":"MEMBER","body":"@rgrinberg I imagine that supporting only the simplest version, i.e. where private modules are visible only to the library itself would be easier, since we only need to pass the `-I .../private` when compiling the library itself. Maybe we could start like this.\r\n\r\nAdditionally, one way to make the private view easier to understand and implement would be to always install the private cmi files in a `.private` sub-directory, and allow to give the private view to anyone who wants it, for instance by writing something special in the jbuild file:\r\n\r\n```scheme\r\n(library\r\n  ((name ...)\r\n   (libraries (foo (private bar) ...))))\r\n```"}}
{"actor":{"display_login":"diml"}}