{"action":"opened","issue":{"url":"https://api.github.com/repos/scrapy/scrapy/issues/1918","repository_url":"https://api.github.com/repos/scrapy/scrapy","labels_url":"https://api.github.com/repos/scrapy/scrapy/issues/1918/labels{/name}","comments_url":"https://api.github.com/repos/scrapy/scrapy/issues/1918/comments","events_url":"https://api.github.com/repos/scrapy/scrapy/issues/1918/events","html_url":"https://github.com/scrapy/scrapy/issues/1918","id":147426450,"number":1918,"title":"scrapy contracts issues","user":{"login":"pawelmhm","id":2700942,"avatar_url":"https://avatars.githubusercontent.com/u/2700942?v=3","gravatar_id":"","url":"https://api.github.com/users/pawelmhm","html_url":"https://github.com/pawelmhm","followers_url":"https://api.github.com/users/pawelmhm/followers","following_url":"https://api.github.com/users/pawelmhm/following{/other_user}","gists_url":"https://api.github.com/users/pawelmhm/gists{/gist_id}","starred_url":"https://api.github.com/users/pawelmhm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pawelmhm/subscriptions","organizations_url":"https://api.github.com/users/pawelmhm/orgs","repos_url":"https://api.github.com/users/pawelmhm/repos","events_url":"https://api.github.com/users/pawelmhm/events{/privacy}","received_events_url":"https://api.github.com/users/pawelmhm/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2016-04-11T13:24:29Z","updated_at":"2016-04-11T13:24:29Z","closed_at":null,"body":"I tried using Scrapy contracts today and found following problems, I;m creating issue for future reference in case others find it helpful (perhaps it could also serve as guide for future improvements). \r\n\r\nIt's very difficult to customize test case to suit your needs. In my project some class of spiders always expect response with some meta keys. This seems like common use case, but there is no easy way to pass meta to spider contract. I found out the only way is to create custom contract that updates args for request, but this custom contract has to be added to contract docstring as well which is not documented (I thought adding custom contract to settings is enough, but no you also have to add @custom_contract_name to docstring).\r\n\r\nThere is no easy way to customize request being made from contract. I need to test callback that is response to POST with some formdata, headers, cookies and some meta keys. Passing all those values to Request init should do the trick, but there is no simple api to do that. \r\n\r\nThere is no timeouts for some tests, I noticed that one of the spider was hanging for long time, it would be better to close it down rather than wait for it to end. \r\n\r\nThere is no way to pass command line arguments to spider test. It would be very useful but currently the way contracts are designed you cannot do this in any way (you can create custom contract that will pass arguments to Request(), but there's no contract for spider init). \r\n\r\nI think the problems are mostly result of docstring test format. Ideally you should be able to specify meta in dosctring, e.g.\r\n\r\n```\r\n@url http://example.com\r\n@meta {\"a\": \"b\"}\r\n@return Item\r\n```\r\nbut with current implementation args for meta will be parsed as strings and they are split on whitespace, so to parse this you'd have to actually write some JSON and this JSON must be without whitespace. Same for specyfing request init args. Ideally I would do something like this\r\n\r\n```\r\n@request {\"method\": \"POST\", \"headers\": {\"foo\":\"bar\"}}\r\n```\r\n\r\nbut this is not possible with current implementation.\r\n\r\nHow about switching to yaml? This will allow us to have test description like this\r\n\r\n```yaml\r\n\r\nrequest: \r\n    url: http://example.com\r\n    method: POST,\r\n    meta : \r\n         variant_request: True\r\n         item: \r\n               name: foobar\r\n    headers:\r\n          header-one: header-value\r\n          header-two: another-value\r\nspider_init_args:\r\n    zipcode: 14001\r\nreturns:\r\n     item: \r\n         name: \"bar\"\r\n```\r\n\r\nthis should allow detailed specification of test case. Above would translate to following description: \"create POST Request with following meta and following headers; initialize spider with following argument; return item with following specs\". \r\n\r\nAside from switching to yaml we could move contracts out of spider docstrings. Maybe this yaml file with test specs could be stored outside spider code, e.g. you could file folder spider_tests full of yaml files describing each test case. "}}