{"action":"created","issue":{"url":"https://api.github.com/repos/haccer/twint/issues/141","repository_url":"https://api.github.com/repos/haccer/twint","labels_url":"https://api.github.com/repos/haccer/twint/issues/141/labels{/name}","comments_url":"https://api.github.com/repos/haccer/twint/issues/141/comments","events_url":"https://api.github.com/repos/haccer/twint/issues/141/events","html_url":"https://github.com/haccer/twint/issues/141","id":329490573,"node_id":"MDU6SXNzdWUzMjk0OTA1NzM=","number":141,"title":"not getting all the tweets from all the days when using \"since\" and \"until\" when the query result is big ","user":{"login":"Nestor75","id":12037308,"node_id":"MDQ6VXNlcjEyMDM3MzA4","avatar_url":"https://avatars2.githubusercontent.com/u/12037308?v=4","gravatar_id":"","url":"https://api.github.com/users/Nestor75","html_url":"https://github.com/Nestor75","followers_url":"https://api.github.com/users/Nestor75/followers","following_url":"https://api.github.com/users/Nestor75/following{/other_user}","gists_url":"https://api.github.com/users/Nestor75/gists{/gist_id}","starred_url":"https://api.github.com/users/Nestor75/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Nestor75/subscriptions","organizations_url":"https://api.github.com/users/Nestor75/orgs","repos_url":"https://api.github.com/users/Nestor75/repos","events_url":"https://api.github.com/users/Nestor75/events{/privacy}","received_events_url":"https://api.github.com/users/Nestor75/received_events","type":"User","site_admin":false},"labels":[{"id":955168203,"node_id":"MDU6TGFiZWw5NTUxNjgyMDM=","url":"https://api.github.com/repos/haccer/twint/labels/Triaged","name":"Triaged","color":"f79b6c","default":false}],"state":"open","locked":false,"assignee":{"login":"haccer","id":22013186,"node_id":"MDQ6VXNlcjIyMDEzMTg2","avatar_url":"https://avatars3.githubusercontent.com/u/22013186?v=4","gravatar_id":"","url":"https://api.github.com/users/haccer","html_url":"https://github.com/haccer","followers_url":"https://api.github.com/users/haccer/followers","following_url":"https://api.github.com/users/haccer/following{/other_user}","gists_url":"https://api.github.com/users/haccer/gists{/gist_id}","starred_url":"https://api.github.com/users/haccer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/haccer/subscriptions","organizations_url":"https://api.github.com/users/haccer/orgs","repos_url":"https://api.github.com/users/haccer/repos","events_url":"https://api.github.com/users/haccer/events{/privacy}","received_events_url":"https://api.github.com/users/haccer/received_events","type":"User","site_admin":false},"assignees":[{"login":"haccer","id":22013186,"node_id":"MDQ6VXNlcjIyMDEzMTg2","avatar_url":"https://avatars3.githubusercontent.com/u/22013186?v=4","gravatar_id":"","url":"https://api.github.com/users/haccer","html_url":"https://github.com/haccer","followers_url":"https://api.github.com/users/haccer/followers","following_url":"https://api.github.com/users/haccer/following{/other_user}","gists_url":"https://api.github.com/users/haccer/gists{/gist_id}","starred_url":"https://api.github.com/users/haccer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/haccer/subscriptions","organizations_url":"https://api.github.com/users/haccer/orgs","repos_url":"https://api.github.com/users/haccer/repos","events_url":"https://api.github.com/users/haccer/events{/privacy}","received_events_url":"https://api.github.com/users/haccer/received_events","type":"User","site_admin":false}],"milestone":null,"comments":3,"created_at":"2018-06-05T14:45:37Z","updated_at":"2018-06-05T21:10:28Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"# Issue Template\r\nPlease use this template!\r\n\r\n### Initial Check \r\n> If the issue is a request please specify that it is a request in the title (Example: [REQUEST] more features). If this is a question regarding 'twint' please specify that it's a question in the title (Example: [QUESTION] What is x?). Please **only** submit issues related to 'twint'. Thanks.\r\n\r\n>Make sure you've checked the following:\r\n\r\n- [X] Python version is 3.5 or higher.\r\n- [X] Using the latest version of Twint.\r\n- [X] I have searched the issues and there are no duplicates of this issue/question/request.\r\n\r\n### Command Ran\r\n```python\r\nimport  twint\r\nc = twint.Config()\r\nc.Count = True\r\noutput csv\r\nc.Output = \"file.csv\"\r\nc.Store_csv = True\r\nc.search_name =\"testmysql1_en\"\r\n\r\nc.Search = \"elonmusk%20OR%20testla%20OR%20electric%20OR%20car%20OR%20apple%20OR%20volvo%20OR%20uber\"\r\n\r\nc.Since = \"2017-09-01\"\r\nc.Until = \"2017-10-27\"\r\nc.Lang = \"en\"\r\ntwint.run.Search(c) \r\n```\r\n\r\n**it gives only  returns 25831 tweets from only these not consecutive days 2017-10-27, 2017-10-26 and 2017-09-27**\r\n\r\nit makes me think there is a problem with how the system manages the dates. \r\n\r\nif I use the same parameters but changing the since and until parameters by:\r\nc.Since = \"2017-10-23\"\r\nc.Until = \"2017-10-24\"\r\n\r\n**then I get 81557 tweets from  2017-10-24 and 2017-10-23**  Why did not appear in the previous search? this time frame is included in the previous one\r\n\r\nand for doing it more weird... if I use the first time frame but searching for a not common stuff\r\n```\r\nimport  twint\r\nc = twint.Config()\r\nc.Count = True\r\noutput csv\r\nc.Output = \"file.csv\"\r\nc.Store_csv = True\r\nc.search_name =\"testmysql3_en\"\r\nc.Search = \"gimeno\"\r\nc.Since = \"2017-09-01\"\r\nc.Until = \"2017-10-27\"\r\nc.Lang = \"en\"\r\ntwint.run.Search(c) \r\n```\r\n\r\nI get 670 tweets from all these days:\r\n'2017-08-28'\r\n'2017-08-29'\r\n'2017-08-30'\r\n'2017-08-31'\r\n'2017-09-01'\r\n'2017-09-02'\r\n'2017-09-03'\r\n'2017-09-04'\r\n'2017-09-05'\r\n'2017-09-06'\r\n'2017-09-07'\r\n'2017-09-08'\r\n'2017-09-09'\r\n'2017-09-10'\r\n'2017-09-11'\r\n'2017-09-12'\r\n'2017-09-13'\r\n'2017-09-14'\r\n'2017-09-15'\r\n'2017-09-16'\r\n'2017-09-17'\r\n'2017-09-18'\r\n'2017-09-19'\r\n'2017-09-20'\r\n'2017-09-21'\r\n'2017-09-22'\r\n'2017-09-23'\r\n'2017-09-24'\r\n'2017-09-25'\r\n'2017-09-26'\r\n'2017-09-27'\r\n'2017-09-28'\r\n'2017-09-29'\r\n'2017-09-30'\r\n'2017-10-01'\r\n'2017-10-02'\r\n'2017-10-03'\r\n'2017-10-04'\r\n'2017-10-05'\r\n'2017-10-06'\r\n'2017-10-07'\r\n'2017-10-08'\r\n'2017-10-09'\r\n'2017-10-10'\r\n'2017-10-11'\r\n'2017-10-12'\r\n'2017-10-13'\r\n'2017-10-14'\r\n'2017-10-15'\r\n'2017-10-16'\r\n'2017-10-17'\r\n'2017-10-18'\r\n'2017-10-19'\r\n'2017-10-20'\r\n'2017-10-21'\r\n'2017-10-22'\r\n'2017-10-23'\r\n'2017-10-24'\r\n'2017-10-25'\r\n'2017-10-26'\r\n'2017-10-27'\r\n\r\n\r\n### Description of Issue\r\n>Please use **as much detail as possible.**\r\nit only gets tweets from some not consecutive days when you search for more than a month and searching for common words with tones of results\r\n\r\n### Environment Details\r\n>Using Windows, Linux? Linux What OS version? ubuntu Running this in Anaconda? NO Jupyter Notebook? No Terminal? Yes\r\n"},"comment":{"url":"https://api.github.com/repos/haccer/twint/issues/comments/394861498","html_url":"https://github.com/haccer/twint/issues/141#issuecomment-394861498","issue_url":"https://api.github.com/repos/haccer/twint/issues/141","id":394861498,"node_id":"MDEyOklzc3VlQ29tbWVudDM5NDg2MTQ5OA==","user":{"login":"Nestor75","id":12037308,"node_id":"MDQ6VXNlcjEyMDM3MzA4","avatar_url":"https://avatars2.githubusercontent.com/u/12037308?v=4","gravatar_id":"","url":"https://api.github.com/users/Nestor75","html_url":"https://github.com/Nestor75","followers_url":"https://api.github.com/users/Nestor75/followers","following_url":"https://api.github.com/users/Nestor75/following{/other_user}","gists_url":"https://api.github.com/users/Nestor75/gists{/gist_id}","starred_url":"https://api.github.com/users/Nestor75/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Nestor75/subscriptions","organizations_url":"https://api.github.com/users/Nestor75/orgs","repos_url":"https://api.github.com/users/Nestor75/repos","events_url":"https://api.github.com/users/Nestor75/events{/privacy}","received_events_url":"https://api.github.com/users/Nestor75/received_events","type":"User","site_admin":false},"created_at":"2018-06-05T21:10:28Z","updated_at":"2018-06-05T21:10:28Z","author_association":"CONTRIBUTOR","body":"after applying the last commit you did, I launched again the first query and  the result was not really much better.\r\nthis is the code:\r\n\r\nc = twint.Config()\r\nc.Count = True\r\noutput csv\r\nc.Output = \"file.csv\"\r\nc.Store_csv = True\r\nc.search_name =\"testmysql1_en\"\r\n\r\nc.Search = \"elonmusk%20OR%20testla%20OR%20electric%20OR%20car%20OR%20apple%20OR%20volvo%20OR%20uber\"\r\n\r\nc.Since = \"2017-09-01\"\r\nc.Until = \"2017-10-27\"\r\nc.Lang = \"en\"\r\ntwint.run.Search(c) \r\n\r\nbefore the change this was the result:\r\nit only returned **25831** tweets from only these not consecutive days \r\n**2017-09-27\r\n2017-10-26 \r\n2017-10-27**\r\n\r\nafter the change:\r\nit returned 181.687 tweets which is a really bigger number than before but only for four days:\r\n**'2017-09-26'\r\n'2017-09-27'\r\n'2017-10-26'\r\n'2017-10-27'**\r\n\r\nand like I checked previously, if y use \r\nc.Since = \"2017-10-23\"\r\nc.Until = \"2017-10-24\"\r\nwith the same query I got 81557  tweets so..... it seems there is something else :(\r\n"}}
{"actor":{"display_login":"Nestor75"}}