{"action":"created","issue":{"url":"https://api.github.com/repos/archivematica/Issues/issues/187","repository_url":"https://api.github.com/repos/archivematica/Issues","labels_url":"https://api.github.com/repos/archivematica/Issues/issues/187/labels{/name}","comments_url":"https://api.github.com/repos/archivematica/Issues/issues/187/comments","events_url":"https://api.github.com/repos/archivematica/Issues/issues/187/events","html_url":"https://github.com/archivematica/Issues/issues/187","id":362284997,"node_id":"MDU6SXNzdWUzNjIyODQ5OTc=","number":187,"title":"Problem: xlink fields in metsrw are not url encoded","user":{"login":"ross-spencer","id":1880412,"node_id":"MDQ6VXNlcjE4ODA0MTI=","avatar_url":"https://avatars1.githubusercontent.com/u/1880412?v=4","gravatar_id":"","url":"https://api.github.com/users/ross-spencer","html_url":"https://github.com/ross-spencer","followers_url":"https://api.github.com/users/ross-spencer/followers","following_url":"https://api.github.com/users/ross-spencer/following{/other_user}","gists_url":"https://api.github.com/users/ross-spencer/gists{/gist_id}","starred_url":"https://api.github.com/users/ross-spencer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ross-spencer/subscriptions","organizations_url":"https://api.github.com/users/ross-spencer/orgs","repos_url":"https://api.github.com/users/ross-spencer/repos","events_url":"https://api.github.com/users/ross-spencer/events{/privacy}","received_events_url":"https://api.github.com/users/ross-spencer/received_events","type":"User","site_admin":false},"labels":[{"id":1003414127,"node_id":"MDU6TGFiZWwxMDAzNDE0MTI3","url":"https://api.github.com/repos/archivematica/Issues/labels/OCUL:%20AM-Dataverse","name":"OCUL: AM-Dataverse","color":"2fef93","default":false},{"id":964107530,"node_id":"MDU6TGFiZWw5NjQxMDc1MzA=","url":"https://api.github.com/repos/archivematica/Issues/labels/Status:%20review","name":"Status: review","color":"25a18e","default":false},{"id":964107548,"node_id":"MDU6TGFiZWw5NjQxMDc1NDg=","url":"https://api.github.com/repos/archivematica/Issues/labels/Type:%20bug","name":"Type: bug","color":"ff9f70","default":false}],"state":"open","locked":false,"assignee":{"login":"ablwr","id":3260492,"node_id":"MDQ6VXNlcjMyNjA0OTI=","avatar_url":"https://avatars2.githubusercontent.com/u/3260492?v=4","gravatar_id":"","url":"https://api.github.com/users/ablwr","html_url":"https://github.com/ablwr","followers_url":"https://api.github.com/users/ablwr/followers","following_url":"https://api.github.com/users/ablwr/following{/other_user}","gists_url":"https://api.github.com/users/ablwr/gists{/gist_id}","starred_url":"https://api.github.com/users/ablwr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ablwr/subscriptions","organizations_url":"https://api.github.com/users/ablwr/orgs","repos_url":"https://api.github.com/users/ablwr/repos","events_url":"https://api.github.com/users/ablwr/events{/privacy}","received_events_url":"https://api.github.com/users/ablwr/received_events","type":"User","site_admin":false},"assignees":[{"login":"ablwr","id":3260492,"node_id":"MDQ6VXNlcjMyNjA0OTI=","avatar_url":"https://avatars2.githubusercontent.com/u/3260492?v=4","gravatar_id":"","url":"https://api.github.com/users/ablwr","html_url":"https://github.com/ablwr","followers_url":"https://api.github.com/users/ablwr/followers","following_url":"https://api.github.com/users/ablwr/following{/other_user}","gists_url":"https://api.github.com/users/ablwr/gists{/gist_id}","starred_url":"https://api.github.com/users/ablwr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ablwr/subscriptions","organizations_url":"https://api.github.com/users/ablwr/orgs","repos_url":"https://api.github.com/users/ablwr/repos","events_url":"https://api.github.com/users/ablwr/events{/privacy}","received_events_url":"https://api.github.com/users/ablwr/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/archivematica/Issues/milestones/2","html_url":"https://github.com/archivematica/Issues/milestone/2","labels_url":"https://api.github.com/repos/archivematica/Issues/milestones/2/labels","id":3454023,"node_id":"MDk6TWlsZXN0b25lMzQ1NDAyMw==","number":2,"title":"1.8.0","description":"","creator":{"login":"sallain","id":4612276,"node_id":"MDQ6VXNlcjQ2MTIyNzY=","avatar_url":"https://avatars1.githubusercontent.com/u/4612276?v=4","gravatar_id":"","url":"https://api.github.com/users/sallain","html_url":"https://github.com/sallain","followers_url":"https://api.github.com/users/sallain/followers","following_url":"https://api.github.com/users/sallain/following{/other_user}","gists_url":"https://api.github.com/users/sallain/gists{/gist_id}","starred_url":"https://api.github.com/users/sallain/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sallain/subscriptions","organizations_url":"https://api.github.com/users/sallain/orgs","repos_url":"https://api.github.com/users/sallain/repos","events_url":"https://api.github.com/users/sallain/events{/privacy}","received_events_url":"https://api.github.com/users/sallain/received_events","type":"User","site_admin":false},"open_issues":77,"closed_issues":4,"state":"open","created_at":"2018-06-26T18:39:54Z","updated_at":"2018-10-06T14:58:35Z","due_on":null,"closed_at":null},"comments":4,"created_at":"2018-09-20T17:19:37Z","updated_at":"2018-10-09T03:08:39Z","closed_at":null,"author_association":"MEMBER","body":"**Expected behaviour**\r\n\r\nRelated to archivematica/issues#131 the filename `30_CFLQ_271_13-3-13_1524[1].pdf` needs to be urlencoded to pass validation.\r\n\r\n**Current behaviour**\r\n\r\nThis portion of code is responsible for writing that particular filename out at present: https://github.com/artefactual/archivematica/blob/16e0f04ff3510c0abc1425c33da58da91fda80a1/src/MCPClient/lib/clientScripts/convert_dataverse_structure.py#L378-L385 but when written to the METS three entries are created, including an xlink (this one is urlencoded): `<mets:FLocat xlink:href=\"30_CFLQ_271_13-3-13_1524[1].pdf\" LOCTYPE=\"OTHER\" OTHERLOCTYPE=\"SYSTEM\"/>`\r\n\r\nWhen urlencoded, the rest of the path entries are also impacted which results in those paths being incorrect unless later on in the workflow our tools know how to decode those values.\r\n\r\nJoel D notes a possible addition to `metsrw`:\r\n\r\n```\r\nimport urllib\r\nimport urlparse\r\n\r\n\r\ndef urlencode(url):\r\n    parsed = urlparse.urlparse(url)\r\n    for attr in ('path', 'params', 'query', 'fragment'):\r\n        parsed = parsed._replace(**{attr: urllib.quote_plus(getattr(parsed, attr), safe='/')})\r\n    return urlparse.urlunparse(parsed)\r\n```\r\nFurther, he notes that there are only a handful of places in metsrw where xlink:href appears to be touched, so running adding a function like the above shouldn't be much problem:\r\n```\r\n$ ack --type=python href ./metsrw\r\nmetsrw/metadata.py\r\n250:        target = root.get(utils.lxmlns('xlink') + 'href')\r\n252:            raise exceptions.ParseError('mdRef must have an xlink:href.')\r\n279:            el.attrib[utils.lxmlns('xlink') + 'href'] = self.target\r\n\r\nmetsrw/fsentry.py\r\n52:        populate FLocat @xlink:href\r\n339:            flocat.set(utils.lxmlns('xlink') + 'href', self.path)\r\n\r\nmetsrw/mets.py\r\n388:                    utils.lxmlns('xlink') + 'href')```\r\n```\r\n**Steps to reproduce**\r\n\r\nTransfer the dataset `Canadian Relocation Cases: Heading Towards Guidelines, 2011 [test] ` using the Dataverse transfer type at time of writing.\r\n\r\n**Additional context**\r\n\r\nDOI of dataset is: https://doi.org/10.5072/FK2/QYH45Z\r\n\r\n---\r\n**For Artefactual use:**\r\nPlease make sure these steps are taken before moving this issue from Review to Verified in Waffle:\r\n\r\n- All PRs related to this issue are properly linked 👍\r\n- All PRs related to this issue have been merged 👍\r\n- Test plan for this issue has been implemented and passed 👍\r\n- Release documentation (e.g. release notes, wiki documentation, etc) regarding this issue has been written 👍\r\n"},"comment":{"url":"https://api.github.com/repos/archivematica/Issues/issues/comments/428044685","html_url":"https://github.com/archivematica/Issues/issues/187#issuecomment-428044685","issue_url":"https://api.github.com/repos/archivematica/Issues/issues/187","id":428044685,"node_id":"MDEyOklzc3VlQ29tbWVudDQyODA0NDY4NQ==","user":{"login":"ross-spencer","id":1880412,"node_id":"MDQ6VXNlcjE4ODA0MTI=","avatar_url":"https://avatars1.githubusercontent.com/u/1880412?v=4","gravatar_id":"","url":"https://api.github.com/users/ross-spencer","html_url":"https://github.com/ross-spencer","followers_url":"https://api.github.com/users/ross-spencer/followers","following_url":"https://api.github.com/users/ross-spencer/following{/other_user}","gists_url":"https://api.github.com/users/ross-spencer/gists{/gist_id}","starred_url":"https://api.github.com/users/ross-spencer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ross-spencer/subscriptions","organizations_url":"https://api.github.com/users/ross-spencer/orgs","repos_url":"https://api.github.com/users/ross-spencer/repos","events_url":"https://api.github.com/users/ross-spencer/events{/privacy}","received_events_url":"https://api.github.com/users/ross-spencer/received_events","type":"User","site_admin":false},"created_at":"2018-10-09T03:08:39Z","updated_at":"2018-10-09T03:08:39Z","author_association":"MEMBER","body":"Hi @ablwr, to help others recreate this, providing the name of the dataset will be useful. Also, if it is failing in Archivematica and you look up the `failed` directory for the transfer's metadata folder, and access `dataset.json` you'll find a doi field under a persistent identifiers field: `    \"persistentUrl\": \"https://doi.org/10.5072/FK2/WZTJWN\",`. We can just link this back to specific failing datasets. \r\n\r\n"}}
{"actor":{"display_login":"ross-spencer"}}