{"action":"created","issue":{"url":"https://api.github.com/repos/asgorobets/SoapExtension/issues/14","repository_url":"https://api.github.com/repos/asgorobets/SoapExtension","labels_url":"https://api.github.com/repos/asgorobets/SoapExtension/issues/14/labels{/name}","comments_url":"https://api.github.com/repos/asgorobets/SoapExtension/issues/14/comments","events_url":"https://api.github.com/repos/asgorobets/SoapExtension/issues/14/events","html_url":"https://github.com/asgorobets/SoapExtension/issues/14","id":147207241,"number":14,"title":"Step that tests Exceptions does not provide reason of failure for all formatters","user":{"login":"asgorobets","id":869079,"avatar_url":"https://avatars.githubusercontent.com/u/869079?v=3","gravatar_id":"","url":"https://api.github.com/users/asgorobets","html_url":"https://github.com/asgorobets","followers_url":"https://api.github.com/users/asgorobets/followers","following_url":"https://api.github.com/users/asgorobets/following{/other_user}","gists_url":"https://api.github.com/users/asgorobets/gists{/gist_id}","starred_url":"https://api.github.com/users/asgorobets/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/asgorobets/subscriptions","organizations_url":"https://api.github.com/users/asgorobets/orgs","repos_url":"https://api.github.com/users/asgorobets/repos","events_url":"https://api.github.com/users/asgorobets/events{/privacy}","received_events_url":"https://api.github.com/users/asgorobets/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/asgorobets/SoapExtension/labels/bug","name":"bug","color":"fc2929"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-04-10T08:52:38Z","updated_at":"2016-04-15T05:04:08Z","closed_at":null,"body":"When implementing the step for catching exceptions and throwing back during next step if the exception was not expected we use @BeforeStep hook to report the exception back. While this thrown exception is shown in the Console output when using Behat's own Console Pretty formatter, this is not always the case for other formatters, who prefer not to output before and after events output, or don't have it enabled by default. So a step after the Exception step is marked as skipped, but no visible reason if BeforeScope output is not shown.\r\n\r\nExample is HTML formatter, while it is possible to override the twig template and extend some classes to implement this feature for HTML formatter, it is not out-of-the-box support and makes things more complex then expected.\r\n![image](https://cloud.githubusercontent.com/assets/869079/14408920/ca30f5a6-ff31-11e5-925f-a27651c8e78f.png)\r\n\r\nWe should solve it either by:\r\n1. Implementing error throwing inside the step itself (is there a way to hook into step during execution?)\r\n2. Document workarounds to allow this visibility for other formatters. (HTML at least)\r\n3. Rethink what we're doing, maybe it's a bad thing to do the request and the exception checking in different steps, maybe a single step will be better, we will end up duplicating soap execute steps, but we may actually get some benefit here. I'm thinking about something similar to what behat/tqextension does with RedirectContext:\r\nhttps://github.com/BR0kEN-/TqExtension/blob/3abe4445b90c17df2e5e9fb52bf1603bf706bb36/src/Context/Redirect/RedirectContext.php#L98\r\n```\r\nI send SOAP request (?:| and expect exception with code \"([^\"]*)\")\r\n```"},"comment":{"url":"https://api.github.com/repos/asgorobets/SoapExtension/issues/comments/210288494","html_url":"https://github.com/asgorobets/SoapExtension/issues/14#issuecomment-210288494","issue_url":"https://api.github.com/repos/asgorobets/SoapExtension/issues/14","id":210288494,"user":{"login":"jboffel","id":6637391,"avatar_url":"https://avatars.githubusercontent.com/u/6637391?v=3","gravatar_id":"","url":"https://api.github.com/users/jboffel","html_url":"https://github.com/jboffel","followers_url":"https://api.github.com/users/jboffel/followers","following_url":"https://api.github.com/users/jboffel/following{/other_user}","gists_url":"https://api.github.com/users/jboffel/gists{/gist_id}","starred_url":"https://api.github.com/users/jboffel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jboffel/subscriptions","organizations_url":"https://api.github.com/users/jboffel/orgs","repos_url":"https://api.github.com/users/jboffel/repos","events_url":"https://api.github.com/users/jboffel/events{/privacy}","received_events_url":"https://api.github.com/users/jboffel/received_events","type":"User","site_admin":false},"created_at":"2016-04-15T05:04:08Z","updated_at":"2016-04-15T05:04:08Z","body":"I confirm the problem and it also true with the junit formatter.\r\n\r\nIt does not detect any failure and report all tests as success.\r\n\r\nPlease check my pull request as a quick and dirty counter measure until a new design is found.\r\n\r\nBasically you can not change the status of the result of a step whatever in afterStep or beforeStep hook.\r\n\r\nIt would have do the job if afterStep would let you do it but it seems not to be possible. So normally to fit with behat design the step function itself should determine in some way if it pass or not."}}