{"action":"created","issue":{"url":"https://api.github.com/repos/NCEAS/datamgmt/issues/175","repository_url":"https://api.github.com/repos/NCEAS/datamgmt","labels_url":"https://api.github.com/repos/NCEAS/datamgmt/issues/175/labels{/name}","comments_url":"https://api.github.com/repos/NCEAS/datamgmt/issues/175/comments","events_url":"https://api.github.com/repos/NCEAS/datamgmt/issues/175/events","html_url":"https://github.com/NCEAS/datamgmt/pull/175","id":372146298,"node_id":"MDExOlB1bGxSZXF1ZXN0MjI0NDMzNzA4","number":175,"title":"Modularize and expand qa_package()","user":{"login":"drkrynstrng","id":36723171,"node_id":"MDQ6VXNlcjM2NzIzMTcx","avatar_url":"https://avatars3.githubusercontent.com/u/36723171?v=4","gravatar_id":"","url":"https://api.github.com/users/drkrynstrng","html_url":"https://github.com/drkrynstrng","followers_url":"https://api.github.com/users/drkrynstrng/followers","following_url":"https://api.github.com/users/drkrynstrng/following{/other_user}","gists_url":"https://api.github.com/users/drkrynstrng/gists{/gist_id}","starred_url":"https://api.github.com/users/drkrynstrng/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/drkrynstrng/subscriptions","organizations_url":"https://api.github.com/users/drkrynstrng/orgs","repos_url":"https://api.github.com/users/drkrynstrng/repos","events_url":"https://api.github.com/users/drkrynstrng/events{/privacy}","received_events_url":"https://api.github.com/users/drkrynstrng/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2018-10-19T23:27:32Z","updated_at":"2018-11-29T19:00:31Z","closed_at":null,"author_association":"MEMBER","pull_request":{"url":"https://api.github.com/repos/NCEAS/datamgmt/pulls/175","html_url":"https://github.com/NCEAS/datamgmt/pull/175","diff_url":"https://github.com/NCEAS/datamgmt/pull/175.diff","patch_url":"https://github.com/NCEAS/datamgmt/pull/175.patch"},"body":"This update for the QA functions has multiple dimensions.\r\n\r\nFirst, I modularized the existing `qa_package()` function by creating separate helper functions to download and read data objects. This closes #159. For checking the congruence of data and metadata attributes, the primary purpose of the existing `qa_package()`, I renamed the function to `qa_congruence()`. I also changed the output messages to be returned in a list.\r\n\r\nSecond, I redefined `qa_package()` to include all types of QA checks, including checks for EML metadata based on the MetaDIG checks as well as checks for system metadata (effectively a wrapper around `qa_eml()` and `qa_sysmeta()`). The default is set to return only results with a FAILURE or ERROR status. As part of this, `qa_congruence()` is included as an option, with the default set to not run for load/speed reasons.\r\n\r\nThe general intended use when processing data packages is `qa_package(mn, rm_pid)` or with congruence checks `qa_package(mn, rm_pid, congruence = TRUE)`. When checking an EML object in R before or after making edits, then the intended use is `qa_eml(eml)`. For prettier output I also included `qa_view()`, a convenience function that uses the listviewer package to output results. Using `qa_view(qa_package(mn, rm_pid))` is a better way to see the results in the viewer in RStudio.\r\n\r\nThird, these updates are meant to feedback into the relevant MetaDIG quality suites. As part of this, we would need to identify which checks we want to add and then classify these new checks into Identification, Discovery, Interpretation, or Informational (which do not impact quality scores). The potential additions are QA checks for the physical, congruence of data and metadata attributes, and data object duplication. All of these additions seem to be Informational checks. I think the QA checks for system metadata would be excluded from MetaDIG, but correct me if I'm wrong. We could potentially expand these checks too.\r\n\r\nI did a lot of debugging with these functions already and there are probably more bugs, but we can fix them as we find them. I also need to add more unit tests. Let me know if you have suggestions for improvements in the code, in the organization of the code or output, or in output messages."},"comment":{"url":"https://api.github.com/repos/NCEAS/datamgmt/issues/comments/442952760","html_url":"https://github.com/NCEAS/datamgmt/pull/175#issuecomment-442952760","issue_url":"https://api.github.com/repos/NCEAS/datamgmt/issues/175","id":442952760,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0Mjk1Mjc2MA==","user":{"login":"drkrynstrng","id":36723171,"node_id":"MDQ6VXNlcjM2NzIzMTcx","avatar_url":"https://avatars3.githubusercontent.com/u/36723171?v=4","gravatar_id":"","url":"https://api.github.com/users/drkrynstrng","html_url":"https://github.com/drkrynstrng","followers_url":"https://api.github.com/users/drkrynstrng/followers","following_url":"https://api.github.com/users/drkrynstrng/following{/other_user}","gists_url":"https://api.github.com/users/drkrynstrng/gists{/gist_id}","starred_url":"https://api.github.com/users/drkrynstrng/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/drkrynstrng/subscriptions","organizations_url":"https://api.github.com/users/drkrynstrng/orgs","repos_url":"https://api.github.com/users/drkrynstrng/repos","events_url":"https://api.github.com/users/drkrynstrng/events{/privacy}","received_events_url":"https://api.github.com/users/drkrynstrng/received_events","type":"User","site_admin":false},"created_at":"2018-11-29T19:00:31Z","updated_at":"2018-11-29T19:00:31Z","author_association":"MEMBER","body":"What do you mean by wrapper in this case? I redefined `qa_package()` here to be the main wrapper around the newer functions `qa_eml()`, `qa_congruence()`, and `qa_sysmeta()`, which could all be used separately. "}}
{"actor":{"display_login":"drkrynstrng"}}