{"action":"created","issue":{"url":"https://api.github.com/repos/ganga-devs/ganga/issues/412","repository_url":"https://api.github.com/repos/ganga-devs/ganga","labels_url":"https://api.github.com/repos/ganga-devs/ganga/issues/412/labels{/name}","comments_url":"https://api.github.com/repos/ganga-devs/ganga/issues/412/comments","events_url":"https://api.github.com/repos/ganga-devs/ganga/issues/412/events","html_url":"https://github.com/ganga-devs/ganga/issues/412","id":151154430,"number":412,"title":"Get rid of File","user":{"login":"milliams","id":61316,"avatar_url":"https://avatars.githubusercontent.com/u/61316?v=3","gravatar_id":"","url":"https://api.github.com/users/milliams","html_url":"https://github.com/milliams","followers_url":"https://api.github.com/users/milliams/followers","following_url":"https://api.github.com/users/milliams/following{/other_user}","gists_url":"https://api.github.com/users/milliams/gists{/gist_id}","starred_url":"https://api.github.com/users/milliams/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/milliams/subscriptions","organizations_url":"https://api.github.com/users/milliams/orgs","repos_url":"https://api.github.com/users/milliams/repos","events_url":"https://api.github.com/users/milliams/events{/privacy}","received_events_url":"https://api.github.com/users/milliams/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/ganga-devs/ganga/labels/cleanup","name":"cleanup","color":"0052cc"},{"url":"https://api.github.com/repos/ganga-devs/ganga/labels/enhancement","name":"enhancement","color":"009800"},{"url":"https://api.github.com/repos/ganga-devs/ganga/labels/question","name":"question","color":"fbca04"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2016-04-26T14:45:23Z","updated_at":"2016-04-27T08:38:17Z","closed_at":null,"body":"Currently in a number of Ganga APIs we have the option of using the `File` type. This is completely separate and unrelated to `IGangaFile` (and therefore `LocalFile`, `DiracFile` etc.). I believe that historically (before we had an inputfiles/outputfiles system) it was used to signal that, for example, the file should be copied into the user's sandbox.\r\n\r\nIt is currently usable in a number of places, for example in `Executable.exe` and `CustomChecker.module`. The problem is that is used very inconsistently. The API they seem to *want* to provide is the ability to seamlessly do either of the following:\r\n```python\r\nj.application.exe = 'myscript.sh'\r\nj.application.exe = File('myscript.sh')\r\n```\r\nBut there is a fundamental problem that `File` is a `GangaObject` and therefore needs to be recorded as a `ComponentItem` in the schema but a string is not and must be recorded as a `SimpleItem`. This means that we've ended up in a bunch of different 'solutions' in the code, the main two that I've seen are:\r\n```python\r\nComponentItem(category='files')  # e.g. CustomChecker.module\r\nSimpleItem(defvalue='echo', typelist=[str, File])  # e.g. Executable.exe\r\n```\r\nDue to Ganga's currently lax and inconsistent type checking they are both able to accept both `str` and `File` objects via assignment but what happens afterwards only seems to work (where it *does* work) by chance.\r\n\r\n# Serialisation\r\n\r\nConsider the following 4 variants:\r\n```python\r\nbox.add(Executable(exe='~/mychecker.py'), 'e')  # 1. SimpleItem str\r\nbox.add(Executable(exe=File('~/mychecker.py')), 'e2')  # 2. SimpleItem File\r\nbox.add(CustomChecker(module='~/mychecker.py'), 'cc')  # 3. ComponentItem str\r\nbox.add(CustomChecker(module=File('~/mychecker.py')), 'cc2')  # 4. ComponentItem File\r\n```\r\nWhen they are serialised they end up with very different XML outputs (snipped):\r\n```xml\r\n1.\r\n<attribute name=\"exe\"> \r\n  <value>'~/mychecker.py'</value>\r\n</attribute>\r\n\r\n2.\r\n<attribute name=\"exe\"> \r\n  <value>File(name='/afs/cern.ch/user/m/mwilliam/mychecker.py',subdir='.')</value>\r\n</attribute>\r\n\r\n3.\r\n<attribute name=\"module\">\r\n  <value>'~/mychecker.py'</value>\r\n</attribute>\r\n\r\n4.\r\n<attribute name=\"module\">\r\n  <class name=\"File\" version=\"1.1\" category=\"files\">\r\n    <attribute name=\"name\"> \r\n      <value>'/afs/cern.ch/user/m/mwilliam/mychecker.py'</value>\r\n    </attribute>\r\n    <attribute name=\"subdir\"> \r\n      <value>'.'</value>\r\n    </attribute>\r\n  </class>\r\n</attribute>\r\n```\r\n\r\n`SimpleItem`/`str` (1) and `ComponentItem`/`File` (4) both work as expected and round-trip correctly but 2 and 3 do not. 2 streams out 'successfully' as `SimpleItem`s use the object's `repr`. 3 streams out 'successfully' due to an unassuming (and unexplained) `if isType(s, str)` in `acceptOptional` (added by @rob-c in 717775211e84ea). The latter is not present in `VPrinter` so that a plain `CustomChecker(module='~/mychecker.py')` will raise an exception.\r\n\r\nAs for reading (after a Ganga restart), `box['e']` works and `exe` is set to a string, `box['e2']` works with `exe` set to `File` (which works due to the stored XML data being passed through `eval`). `box['cc']` fails due to the inconsistent types and `box['cc2']` works with `module` being `File`.\r\n\r\n# Usage\r\n\r\n`File` has two public attributes, `name` and `subdir`. On the whole, only `name` is ever used (and people will just put the full path in `name`). For example, `Executable`, `CustomChecker`, `CustomMerger`, `Gaudi` and co., `Bender`, and `Athena` and co. just use `name` and never `subdir`. Only `Root` and `GaudiPython` use `subdir`.\r\n\r\n# Partial proposal\r\n\r\nI suggest that we find a way to get rid of `File` as it does not add any *functionality* over a simple string. There is an open question about a safe transition mechanism (both from the point-of-view of the XML repository and also and user scripts) about which I am open to discussion."},"comment":{"url":"https://api.github.com/repos/ganga-devs/ganga/issues/comments/215011912","html_url":"https://github.com/ganga-devs/ganga/issues/412#issuecomment-215011912","issue_url":"https://api.github.com/repos/ganga-devs/ganga/issues/412","id":215011912,"user":{"login":"drmarkwslater","id":13677368,"avatar_url":"https://avatars.githubusercontent.com/u/13677368?v=3","gravatar_id":"","url":"https://api.github.com/users/drmarkwslater","html_url":"https://github.com/drmarkwslater","followers_url":"https://api.github.com/users/drmarkwslater/followers","following_url":"https://api.github.com/users/drmarkwslater/following{/other_user}","gists_url":"https://api.github.com/users/drmarkwslater/gists{/gist_id}","starred_url":"https://api.github.com/users/drmarkwslater/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/drmarkwslater/subscriptions","organizations_url":"https://api.github.com/users/drmarkwslater/orgs","repos_url":"https://api.github.com/users/drmarkwslater/repos","events_url":"https://api.github.com/users/drmarkwslater/events{/privacy}","received_events_url":"https://api.github.com/users/drmarkwslater/received_events","type":"User","site_admin":false},"created_at":"2016-04-27T08:38:16Z","updated_at":"2016-04-27T08:38:16Z","body":"@rob-c So if I'm understanding correctly, there are two outstanding issues:\r\n\r\n1. Some users are requesting a system of making sure files go in a particular sub folder on the WN\r\n2. We may want to support a user running an Executable from a sub folder.\r\n\r\nFor 1) I'd say that this is something that needs to be added to `IGangaFile`  and if it's holding some users back from using 6.1 then we should probably get that added ASAP. For 2), `File` doesn't provide that anyway and certainly for `Executable` jobs I can't see any way of really doing that short of explicitly adding it into the application (which I wouldn't do anyway unless it was specifically requested 😄  ). So aren't both of these orthogonal to removing `File` as it stands in 6.1? It's not used in Sandboxes anymore and the only other place would be in applications that just use it as a string anyway. Or am I missing something?\r\n\r\nNow having said all that, what the best thing to do as regards phasing it out and not annoying lots of people is a much harder question..."}}