{"action":"created","issue":{"url":"https://api.github.com/repos/tdwg/bdq/issues/175","repository_url":"https://api.github.com/repos/tdwg/bdq","labels_url":"https://api.github.com/repos/tdwg/bdq/issues/175/labels{/name}","comments_url":"https://api.github.com/repos/tdwg/bdq/issues/175/comments","events_url":"https://api.github.com/repos/tdwg/bdq/issues/175/events","html_url":"https://github.com/tdwg/bdq/issues/175","id":365679795,"node_id":"MDU6SXNzdWUzNjU2Nzk3OTU=","number":175,"title":"Expected Response versus Pass/Fail","user":{"login":"tucotuco","id":462827,"node_id":"MDQ6VXNlcjQ2MjgyNw==","avatar_url":"https://avatars2.githubusercontent.com/u/462827?v=4","gravatar_id":"","url":"https://api.github.com/users/tucotuco","html_url":"https://github.com/tucotuco","followers_url":"https://api.github.com/users/tucotuco/followers","following_url":"https://api.github.com/users/tucotuco/following{/other_user}","gists_url":"https://api.github.com/users/tucotuco/gists{/gist_id}","starred_url":"https://api.github.com/users/tucotuco/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tucotuco/subscriptions","organizations_url":"https://api.github.com/users/tucotuco/orgs","repos_url":"https://api.github.com/users/tucotuco/repos","events_url":"https://api.github.com/users/tucotuco/events{/privacy}","received_events_url":"https://api.github.com/users/tucotuco/received_events","type":"User","site_admin":false},"labels":[{"id":806717757,"node_id":"MDU6TGFiZWw4MDY3MTc3NTc=","url":"https://api.github.com/repos/tdwg/bdq/labels/NEEDS%20WORK","name":"NEEDS WORK","color":"e80293","default":false},{"id":796044414,"node_id":"MDU6TGFiZWw3OTYwNDQ0MTQ=","url":"https://api.github.com/repos/tdwg/bdq/labels/TG2","name":"TG2","color":"ededed","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":5,"created_at":"2018-10-01T23:18:02Z","updated_at":"2018-10-01T23:58:15Z","closed_at":null,"author_association":"MEMBER","body":"Our tests and assertions are missing consistent specifications of the expected responses. Sometimes we include that information in a Description, sometimes in the Examples, sometimes we include a Fail description and Pass description (for Notifications and Validations). Those descriptions strike me as odd. Wouldn't it be better, and \"closer to code\" to simply have an \"Expected response\" column for all our tests and assertions? As a coder I would find that easier to grasp. I trip on the logic that a notification should have a pass or fail condition, for example. The tests don't pass or fail, they simply run and return results. For all Amendments, Notifications, they return either True or False, for measures, they return a number. Here are a few examples of what I propose.  \r\n\r\nFor TG2-NOTIFICATION_ESTABLISHMENTMEANS_NOTEMPTY:\r\n\r\nExpected response: INTERNAL_PREREQUISITES_NOT_MET if there is no field dwc:establishmentMeans in the input, False if field dwc:establishmentMeans is EMPTY, otherwise True.\r\n\r\nrather than:\r\n\r\nFail Description: The field dwc:establishmentMeans is not EMPTY\r\nPass Description: The field dwc:establishmentMeans is EMPTY\r\nTest Prerequisites: The field dwc:establishmentMeans is present in the record.\r\n\r\nFor TG2-VALIDATION_DAY_NOTSTANDARD:\r\n\r\nExpected response: INTERNAL_PREREQUISITES_NOT_MET if there is no field dwc:day, False if the value of the field dwc:day is an integer between 1 and 31 inclusive, otherwise True. \r\n\r\nrather than:\r\n\r\nFail Description: The value of dwc:day is not an integer between 1 and 31 inclusive\r\nPass Description: The value of dwc:day is an integer between 1 and 31 inclusive\r\nTest Prerequisites: The field dwc:day is not EMPTY.\r\n\r\nFor TG2-VALIDATION_GENUS_NOTFOUND:\r\n\r\nExpected response: INTERNAL_PREREQUISITES_NOT_MET if there is no field dwc:genus in the input or if the field dwc:genus is EMPTY, EXTERNAL_PREREQUISITES_NOT_MET if the supplied source authority does not provide a response, True if the value of dwc:genus was not found as a value at the rank of genus in the specified source authority, otherwise False.\r\n\r\nrather than:\r\n\r\nFail Description: The value of dwc:genus was not found as a value at the rank of genus in the specified source authority\r\nPass Description: The value of dwc:genus was found as a value at the rank of genus in the specified source authority\r\nNotes: The purpose of this test is to check whether the value is a name that is a result of a nomenclatural act at this rank. This excludes unpublished names, misspellings and vernacular names. It is expected that the test will designate the source authority against to check. The same test might return distinct results when using distinct source authorities.\r\nTest Prerequisites: A specified target source authority. The field dwc:genus must not be EMPTY.\r\n\r\nLooking at the tests this way makes me nervous about many of those of output type Measure, for example TG2-MEASURE_VALIDATIONTESTS_COMPLIANT (https://github.com/tdwg/bdq/issues/135). That one doesn't have a well-defined expected response. If we want to go in this direction in general, I propose we revisit the measure tests, make their expected results explicit."},"comment":{"url":"https://api.github.com/repos/tdwg/bdq/issues/comments/426101999","html_url":"https://github.com/tdwg/bdq/issues/175#issuecomment-426101999","issue_url":"https://api.github.com/repos/tdwg/bdq/issues/175","id":426101999,"node_id":"MDEyOklzc3VlQ29tbWVudDQyNjEwMTk5OQ==","user":{"login":"chicoreus","id":3864169,"node_id":"MDQ6VXNlcjM4NjQxNjk=","avatar_url":"https://avatars2.githubusercontent.com/u/3864169?v=4","gravatar_id":"","url":"https://api.github.com/users/chicoreus","html_url":"https://github.com/chicoreus","followers_url":"https://api.github.com/users/chicoreus/followers","following_url":"https://api.github.com/users/chicoreus/following{/other_user}","gists_url":"https://api.github.com/users/chicoreus/gists{/gist_id}","starred_url":"https://api.github.com/users/chicoreus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chicoreus/subscriptions","organizations_url":"https://api.github.com/users/chicoreus/orgs","repos_url":"https://api.github.com/users/chicoreus/repos","events_url":"https://api.github.com/users/chicoreus/events{/privacy}","received_events_url":"https://api.github.com/users/chicoreus/received_events","type":"User","site_admin":false},"created_at":"2018-10-01T23:58:15Z","updated_at":"2018-10-01T23:58:15Z","author_association":"COLLABORATOR","body":"@ArthurChapman yes, for each test which is a Validation at the singlerecord level, there is a corresponding Measure of the proportion of records which are COMPLIANT at the multirecord level."}}
{"actor":{"display_login":"chicoreus"}}