{"action":"closed","issue":{"url":"https://api.github.com/repos/sjdirect/nrobots/issues/4","labels_url":"https://api.github.com/repos/sjdirect/nrobots/issues/4/labels{/name}","comments_url":"https://api.github.com/repos/sjdirect/nrobots/issues/4/comments","events_url":"https://api.github.com/repos/sjdirect/nrobots/issues/4/events","html_url":"https://github.com/sjdirect/nrobots/issues/4","id":127515760,"number":4,"title":"Issue with prefixed disallowed sections","user":{"login":"xavivars","id":134083,"avatar_url":"https://avatars.githubusercontent.com/u/134083?v=3","gravatar_id":"","url":"https://api.github.com/users/xavivars","html_url":"https://github.com/xavivars","followers_url":"https://api.github.com/users/xavivars/followers","following_url":"https://api.github.com/users/xavivars/following{/other_user}","gists_url":"https://api.github.com/users/xavivars/gists{/gist_id}","starred_url":"https://api.github.com/users/xavivars/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/xavivars/subscriptions","organizations_url":"https://api.github.com/users/xavivars/orgs","repos_url":"https://api.github.com/users/xavivars/repos","events_url":"https://api.github.com/users/xavivars/events{/privacy}","received_events_url":"https://api.github.com/users/xavivars/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2016-01-19T19:08:16Z","updated_at":"2016-01-19T19:51:47Z","closed_at":"2016-01-19T19:51:47Z","body":"I'm finding some issues with the following configuration\r\n\r\nRobots.txt\r\n\r\n    User-agent: *\r\n    Disallow: /w/\r\n\r\nWhen trying to validate any URL that starts with /wiki/*, it goes blocked [by this line](https://github.com/sjdirect/nrobots/blob/master/Robots/Robots.cs#L401)\r\n\r\n    if (testedPart.StartsWith(regsiteredPart, StringComparison.InvariantCultureIgnoreCase))\r\n                return false;\r\n\r\nI don't think any validation \"startswith\" should be done here, as the entry had a `/` at the end.\r\n\r\nSo maybe a new \"flag\" has to be added to the entries to determine if they represent \"complete\" directory names or just directory prefixes.\r\n\r\nI'll try to come up with a solution, but couldn't come up of any working implementation yet.\r\n"}}