{"action":"created","issue":{"url":"https://api.github.com/repos/rackerlabs/api-checker/issues/304","repository_url":"https://api.github.com/repos/rackerlabs/api-checker","labels_url":"https://api.github.com/repos/rackerlabs/api-checker/issues/304/labels{/name}","comments_url":"https://api.github.com/repos/rackerlabs/api-checker/issues/304/comments","events_url":"https://api.github.com/repos/rackerlabs/api-checker/issues/304/events","html_url":"https://github.com/rackerlabs/api-checker/pull/304","id":152575742,"number":304,"title":"URI Step Wildcard Does Not Match Encoded Newline Characters","user":{"login":"dmnjohns","id":1923408,"avatar_url":"https://avatars.githubusercontent.com/u/1923408?v=3","gravatar_id":"","url":"https://api.github.com/users/dmnjohns","html_url":"https://github.com/dmnjohns","followers_url":"https://api.github.com/users/dmnjohns/followers","following_url":"https://api.github.com/users/dmnjohns/following{/other_user}","gists_url":"https://api.github.com/users/dmnjohns/gists{/gist_id}","starred_url":"https://api.github.com/users/dmnjohns/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dmnjohns/subscriptions","organizations_url":"https://api.github.com/users/dmnjohns/orgs","repos_url":"https://api.github.com/users/dmnjohns/repos","events_url":"https://api.github.com/users/dmnjohns/events{/privacy}","received_events_url":"https://api.github.com/users/dmnjohns/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/rackerlabs/api-checker/labels/BUG","name":"BUG","color":"02e10c"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/rackerlabs/api-checker/milestones/34","html_url":"https://github.com/rackerlabs/api-checker/milestones/2.0.1","labels_url":"https://api.github.com/repos/rackerlabs/api-checker/milestones/34/labels","id":1781394,"number":34,"title":"2.0.1","description":"","creator":{"login":"RackerWilliams","id":348314,"avatar_url":"https://avatars.githubusercontent.com/u/348314?v=3","gravatar_id":"","url":"https://api.github.com/users/RackerWilliams","html_url":"https://github.com/RackerWilliams","followers_url":"https://api.github.com/users/RackerWilliams/followers","following_url":"https://api.github.com/users/RackerWilliams/following{/other_user}","gists_url":"https://api.github.com/users/RackerWilliams/gists{/gist_id}","starred_url":"https://api.github.com/users/RackerWilliams/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/RackerWilliams/subscriptions","organizations_url":"https://api.github.com/users/RackerWilliams/orgs","repos_url":"https://api.github.com/users/RackerWilliams/repos","events_url":"https://api.github.com/users/RackerWilliams/events{/privacy}","received_events_url":"https://api.github.com/users/RackerWilliams/received_events","type":"User","site_admin":false},"open_issues":5,"closed_issues":2,"state":"open","created_at":"2016-05-22T21:20:35Z","updated_at":"2016-06-06T20:15:43Z","due_on":"2016-06-30T05:00:00Z","closed_at":null},"comments":6,"created_at":"2016-05-02T15:20:30Z","updated_at":"2016-06-08T14:04:57Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/rackerlabs/api-checker/pulls/304","html_url":"https://github.com/rackerlabs/api-checker/pull/304","diff_url":"https://github.com/rackerlabs/api-checker/pull/304.diff","patch_url":"https://github.com/rackerlabs/api-checker/pull/304.patch"},"body":"When a request contains an encoded newline character in the URI, api-checker will throw a `ValidatorException` caused by a Scala `MatchError` resulting from trying to match a `MistmatchResult` against an `ErrorResult`.\r\n\r\nI believe that percent-encoded newline characters are legal in the URI of a request. The rest of this description will be based on that assumption. For convenience, I've linked a relevant RFC:\r\nhttps://tools.ietf.org/html/rfc3986#section-3.3\r\n\r\nI've only tested this issue in Repose as I was initially unaware of the cause. I used the following WADL with the api-validator filter for testing:\r\n```\r\n<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\r\n\r\n<application xmlns=\"http://wadl.dev.java.net/2009/02\"\r\n             xmlns:xs=\"http://www.w3.org/2001/XMLSchema\">\r\n    <resources base=\"http://localhost:${targetPort}/\">\r\n        <resource path=\"{wildcard}\">\r\n            <param name=\"wildcard\" style=\"template\" type=\"xs:string\"/>\r\n\r\n            <method name=\"GET\" id=\"get\"/>\r\n        </resource>\r\n    </resources>\r\n</application>\r\n```\r\n\r\nThe Repose log shows:\r\n> 2016-05-02 10:02:59,838 570330 [qtp2045895214-69] ERROR org.openrepose.filters.apivalidator.ApiValidatorHandler - Error processing validation\r\n> com.rackspace.com.papi.components.checker.ValidatorException: Error while validating request\r\n>         at com.rackspace.com.papi.components.checker.Validator.validate(Validator.scala:187) ~[checker-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]\r\n>         at org.openrepose.filters.apivalidator.ApiValidatorHandler.doFilter(ApiValidatorHandler.java:145) [api-validator-filter-8.0.0.0-SNAPSHOT.jar:8.0.0.0-SNAPSHOT]\r\n>         at org.openrepose.filters.apivalidator.ApiValidatorFilter.doFilter(ApiValidatorFilter.java:83) [api-validator-filter-8.0.0.0-SNAPSHOT.jar:8.0.0.0-SNAPSHOT]\r\n>         at org.openrepose.powerfilter.PowerFilterChain.doReposeFilter(PowerFilterChain.java:212) [repose-valve.jar:8.0.0.0-SNAPSHOT]\r\n>         at org.openrepose.powerfilter.PowerFilterChain.doFilter(PowerFilterChain.java:330) [repose-valve.jar:8.0.0.0-SNAPSHOT]\r\n>         at org.openrepose.powerfilter.PowerFilterChain.startFilterChain(PowerFilterChain.java:116) [repose-valve.jar:8.0.0.0-SNAPSHOT]\r\n>         at org.openrepose.powerfilter.PowerFilter.doFilter(PowerFilter.java:390) [repose-valve.jar:8.0.0.0-SNAPSHOT]\r\n>         at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:344) [repose-valve.jar:8.0.0.0-SNAPSHOT]\r\n>         at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:261) [repose-valve.jar:8.0.0.0-SNAPSHOT]\r\n>         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1636) [repose-valve.jar:8.0.0.0-SNAPSHOT]\r\n>         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:564) [repose-valve.jar:8.0.0.0-SNAPSHOT]\r\n>         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1111) [repose-valve.jar:8.0.0.0-SNAPSHOT]\r\n>         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:498) [repose-valve.jar:8.0.0.0-SNAPSHOT]\r\n>         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1045) [repose-valve.jar:8.0.0.0-SNAPSHOT]\r\n>         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [repose-valve.jar:8.0.0.0-SNAPSHOT]\r\n>         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:98) [repose-valve.jar:8.0.0.0-SNAPSHOT]\r\n>         at org.eclipse.jetty.server.Server.handle(Server.java:461) [repose-valve.jar:8.0.0.0-SNAPSHOT]\r\n>         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:284) [repose-valve.jar:8.0.0.0-SNAPSHOT]\r\n>         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244) [repose-valve.jar:8.0.0.0-SNAPSHOT]\r\n>         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:534) [repose-valve.jar:8.0.0.0-SNAPSHOT]\r\n>         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607) [repose-valve.jar:8.0.0.0-SNAPSHOT]\r\n>         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536) [repose-valve.jar:8.0.0.0-SNAPSHOT]\r\n>         at java.lang.Thread.run(Thread.java:745) [?:1.8.0_74]\r\n> Caused by: scala.MatchError: (S0 d61e4) (of class com.rackspace.com.papi.components.checker.step.results.MismatchResult)\r\n>         at com.rackspace.com.papi.components.checker.handler.ServletResultHandler.handle(ServletResultHandler.scala:32) ~[?:?]\r\n>         at com.rackspace.com.papi.components.checker.handler.DispatchResultHandler$$anonfun$handle$1.apply(DispatchResultHandler.scala:38) ~[?:?]\r\n>         at com.rackspace.com.papi.components.checker.handler.DispatchResultHandler$$anonfun$handle$1.apply(DispatchResultHandler.scala:38) ~[?:?]\r\n>         at scala.collection.immutable.List.foreach(List.scala:381) ~[repose-valve.jar:8.0.0.0-SNAPSHOT]\r\n>         at com.rackspace.com.papi.components.checker.handler.DispatchResultHandler.handle(DispatchResultHandler.scala:38) ~[?:?]\r\n>         at com.rackspace.com.papi.components.checker.Validator.validate(Validator.scala:191) ~[?:?]\r\n>         ... 22 more\r\n\r\nThe root of the issue appears to be the api-checker URI wildcard regex not matching newline characters. Remember that the newline character was presented as an encoded character, but was decoded when wrapped in a `CheckerServletRequest` and added to the `URISegment` array. Note that the Scala `Regex` class delegates to the Java `java.util.regex` package. From https://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.html :\r\n\r\n> The regular expression . matches any character except a line terminator unless the DOTALL flag is specified.\r\n\r\nIf we intend to only compare decoded strings in the URI step, it seems like the simplest fix would be to enable the `DOTALL` flag for the wildcard regex. I don't believe there will be an issue if a WADL declares a newline as part of the path explicitly, but that should probably be tested.\r\n\r\nAnother facet to this issue is the fact that a `MismatchResult` is **not** an `ErrorResult` even though calling `isValid()` on a `MismatchResult` will always return false. This is particularly relevant in the `ServletResultHandler` which seems to assume that if a result is not valid, it must be an `ErrorResult`. That is, the match being used is not exhaustive."},"comment":{"url":"https://api.github.com/repos/rackerlabs/api-checker/issues/comments/224599338","html_url":"https://github.com/rackerlabs/api-checker/pull/304#issuecomment-224599338","issue_url":"https://api.github.com/repos/rackerlabs/api-checker/issues/304","id":224599338,"user":{"login":"dmnjohns","id":1923408,"avatar_url":"https://avatars.githubusercontent.com/u/1923408?v=3","gravatar_id":"","url":"https://api.github.com/users/dmnjohns","html_url":"https://github.com/dmnjohns","followers_url":"https://api.github.com/users/dmnjohns/followers","following_url":"https://api.github.com/users/dmnjohns/following{/other_user}","gists_url":"https://api.github.com/users/dmnjohns/gists{/gist_id}","starred_url":"https://api.github.com/users/dmnjohns/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dmnjohns/subscriptions","organizations_url":"https://api.github.com/users/dmnjohns/orgs","repos_url":"https://api.github.com/users/dmnjohns/repos","events_url":"https://api.github.com/users/dmnjohns/events{/privacy}","received_events_url":"https://api.github.com/users/dmnjohns/received_events","type":"User","site_admin":false},"created_at":"2016-06-08T14:04:57Z","updated_at":"2016-06-08T14:04:57Z","body":"Regarding the `MismatchResult` and `ErrorResult` stuff, I thought that might be the case.\r\n\r\nGlad to see that the fix for now was pretty simple."}}
{"actor":{"display_login":"dmnjohns"}}