{"action":"created","issue":{"url":"https://api.github.com/repos/guyskk/validr/issues/17","repository_url":"https://api.github.com/repos/guyskk/validr","labels_url":"https://api.github.com/repos/guyskk/validr/issues/17/labels{/name}","comments_url":"https://api.github.com/repos/guyskk/validr/issues/17/comments","events_url":"https://api.github.com/repos/guyskk/validr/issues/17/events","html_url":"https://github.com/guyskk/validr/issues/17","id":363035524,"node_id":"MDU6SXNzdWUzNjMwMzU1MjQ=","number":17,"title":"list.unique validator is SLOW","user":{"login":"survivorm","id":2362776,"node_id":"MDQ6VXNlcjIzNjI3NzY=","avatar_url":"https://avatars0.githubusercontent.com/u/2362776?v=4","gravatar_id":"","url":"https://api.github.com/users/survivorm","html_url":"https://github.com/survivorm","followers_url":"https://api.github.com/users/survivorm/followers","following_url":"https://api.github.com/users/survivorm/following{/other_user}","gists_url":"https://api.github.com/users/survivorm/gists{/gist_id}","starred_url":"https://api.github.com/users/survivorm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/survivorm/subscriptions","organizations_url":"https://api.github.com/users/survivorm/orgs","repos_url":"https://api.github.com/users/survivorm/repos","events_url":"https://api.github.com/users/survivorm/events{/privacy}","received_events_url":"https://api.github.com/users/survivorm/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":12,"created_at":"2018-09-24T07:19:02Z","updated_at":"2018-09-24T14:11:02Z","closed_at":null,"author_association":"NONE","body":"I have a schema like \r\n```python\r\nplain_rule_schema_in = T.dict(\r\n    enabled=T.bool.optional.default,\r\n    ttl=T.int.optional,\r\n    settings=T.dict(\r\n        addr=T.netaddr\r\n    )\r\n)\r\n\r\nplain_rule_list_in = T.dict(\r\n    name=T.str,\r\n    enabled=T.bool,\r\n    description=T.str.optional.default(''),\r\n    category=T.enum(\r\n        ','.join([category.name for category in RuleListCategoriesEnum])\r\n    ),\r\n    rules=T.list(\r\n        plain_rule_schema_in,\r\n    ).maxlen(100000)\r\n)\r\n```\r\nof a relatively big list of rule entities inside of the list container. It processes fast enough (0.23 sec for 20000 entities), but that's before i add a unique checker to the list. Time instantly jumps to 15-16 s!\r\nI can imagine that unique check is a little slow, yes, but not SUCH slow. There may be a catch here, of course, for example, we can try to drop list/dict objects to json, and many others to str to do substantially faster checks. You may think of something even faster, but for now, this great check is literally unusable\r\nI use validr==1.0.4, netaddr check is performed via \r\n```python\r\n@validator(string=True)\r\ndef netaddr_validator(compiler):\r\n    \"\"\"Custom validator\r\n\r\n    Params:\r\n        compiler: can be used for compile inner schema\r\n        items: optional, and can only be scalar type, passed by schema in `T.validator(items)` form\r\n        some_param: other params\r\n    Returns:\r\n        validate function\r\n    \"\"\"\r\n    def validate(value):\r\n        \"\"\"Validate function\r\n\r\n        Params:\r\n            value: data to be validate\r\n        Returns:\r\n            valid value or converted value\r\n        Raises:\r\n            Invalid: value invalid\r\n        \"\"\"\r\n        try:\r\n            value = str(netaddr.IPNetwork(value))\r\n        except netaddr.core.AddrFormatError as ex:\r\n            raise Invalid('{} is invalid addr'.format(value))\r\n\r\n        return value\r\n    return validate\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/guyskk/validr/issues/comments/423988452","html_url":"https://github.com/guyskk/validr/issues/17#issuecomment-423988452","issue_url":"https://api.github.com/repos/guyskk/validr/issues/17","id":423988452,"node_id":"MDEyOklzc3VlQ29tbWVudDQyMzk4ODQ1Mg==","user":{"login":"survivorm","id":2362776,"node_id":"MDQ6VXNlcjIzNjI3NzY=","avatar_url":"https://avatars0.githubusercontent.com/u/2362776?v=4","gravatar_id":"","url":"https://api.github.com/users/survivorm","html_url":"https://github.com/survivorm","followers_url":"https://api.github.com/users/survivorm/followers","following_url":"https://api.github.com/users/survivorm/following{/other_user}","gists_url":"https://api.github.com/users/survivorm/gists{/gist_id}","starred_url":"https://api.github.com/users/survivorm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/survivorm/subscriptions","organizations_url":"https://api.github.com/users/survivorm/orgs","repos_url":"https://api.github.com/users/survivorm/repos","events_url":"https://api.github.com/users/survivorm/events{/privacy}","received_events_url":"https://api.github.com/users/survivorm/received_events","type":"User","site_admin":false},"created_at":"2018-09-24T14:11:02Z","updated_at":"2018-09-24T14:11:02Z","author_association":"NONE","body":"simple enough and fast enough, i think"}}
{"actor":{"display_login":"survivorm"}}