{"action":"created","issue":{"url":"https://api.github.com/repos/Flexget/Flexget/issues/1589","repository_url":"https://api.github.com/repos/Flexget/Flexget","labels_url":"https://api.github.com/repos/Flexget/Flexget/issues/1589/labels{/name}","comments_url":"https://api.github.com/repos/Flexget/Flexget/issues/1589/comments","events_url":"https://api.github.com/repos/Flexget/Flexget/issues/1589/events","html_url":"https://github.com/Flexget/Flexget/pull/1589","id":197519137,"number":1589,"title":"Log data fetched from URLs to make debugging easier (at trace log level).","user":{"login":"tobinjt","id":218222,"avatar_url":"https://avatars.githubusercontent.com/u/218222?v=3","gravatar_id":"","url":"https://api.github.com/users/tobinjt","html_url":"https://github.com/tobinjt","followers_url":"https://api.github.com/users/tobinjt/followers","following_url":"https://api.github.com/users/tobinjt/following{/other_user}","gists_url":"https://api.github.com/users/tobinjt/gists{/gist_id}","starred_url":"https://api.github.com/users/tobinjt/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tobinjt/subscriptions","organizations_url":"https://api.github.com/users/tobinjt/orgs","repos_url":"https://api.github.com/users/tobinjt/repos","events_url":"https://api.github.com/users/tobinjt/events{/privacy}","received_events_url":"https://api.github.com/users/tobinjt/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2016-12-25T23:41:33Z","updated_at":"2016-12-26T13:40:48Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/Flexget/Flexget/pulls/1589","html_url":"https://github.com/Flexget/Flexget/pull/1589","diff_url":"https://github.com/Flexget/Flexget/pull/1589.diff","patch_url":"https://github.com/Flexget/Flexget/pull/1589.patch"},"body":"### Motivation for changes:\r\n\r\nMake it easier to debug Flexget not getting useful results from a URL.\r\n\r\n### Detailed changes:\r\n\r\nTo make debugging easier, log contents of fetched URLs at trace level.\r\nBrowsers process Javascript, handle iframes, and more that flexget\r\ndoesn't; seeing the actual data processed by flexget rather than looking\r\nat the URL in your browser makes it easier to figure out why you're not\r\ngetting any results.\r\n\r\n### Addressed issues:\r\n\r\n- Fixes # .\r\n\r\n### Config usage if relevant (new plugin or updated schema):\r\n```\r\npaste_config_here\r\n```\r\n### Log and/or tests output (preferably both):\r\nThe test failures appear to be completely unrelated to my change.\r\n\r\n```\r\n================================================================================================= FAILURES =================================================================================================\r\n______________________________________________________________________________________ TestFilesystem.test_non_ascii _______________________________________________________________________________________\r\n\r\nself = <flexget.tests.test_filesystem.TestFilesystem object at 0x1067a4290>, execute_task = <function execute at 0x1072da0c8>\r\n\r\n    def test_non_ascii(self, execute_task):\r\n        task_name = 'non_ascii'\r\n        should_exist = ['\\u0161 dir', '\\u0152 file.txt']\r\n        task = execute_task(task_name)\r\n\r\n>       self.assert_check(task, task_name, 'positive', should_exist)\r\n\r\ntest_filesystem.py:210:\r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _\r\n\r\nself = <flexget.tests.test_filesystem.TestFilesystem object at 0x1067a4290>, task = <flexget.task.Task object at 0x107ed7210>, task_name = 'non_ascii', test_type = 'positive'\r\nfilenames = ['š dir', 'Œ file.txt']\r\n\r\n    def assert_check(self, task, task_name, test_type, filenames):\r\n        for file in filenames:\r\n            file = Path(file)\r\n            if test_type == 'positive':\r\n                assertion_error = 'Failed %s %s test, did not find %s' % (test_type, task_name, file)\r\n>               assert task.find_entry(title=file.namebase), assertion_error\r\nE               AssertionError: Failed positive non_ascii test, did not find š dir\r\nE               assert None\r\nE                +  where None = <bound method Task.find_entry of <flexget.task.Task object at 0x107ed7210>>(title=Path(u'\\u0161 dir'))\r\nE                +    where <bound method Task.find_entry of <flexget.task.Task object at 0x107ed7210>> = <flexget.task.Task object at 0x107ed7210>.find_entry\r\nE                +    and   Path(u'\\u0161 dir') = Path(u'\\u0161 dir').namebase\r\n\r\ntest_filesystem.py:91: AssertionError\r\n----------------------------------------------------------------------------------------------- Captured log -----------------------------------------------------------------------------------------------\r\nmanager.py                 115 INFO     last manager was not torn down correctly\r\nmanager.py                 172 DEBUG    sys.defaultencoding: ascii\r\nmanager.py                 173 DEBUG    sys.getfilesystemencoding: utf-8\r\nmanager.py                 174 DEBUG    flexget detected io encoding: utf-8\r\nmanager.py                 175 DEBUG    os.path.supports_unicode_filenames: True\r\nconftest.py                309 DEBUG    database_uri: sqlite:///:memory:\r\nplugin.py                  383 DEBUG    Trying to load plugins from: [u'/Users/johntobin/src/upstream-flexget/flexget/tests/plugins', '/Users/johntobin/src/upstream-flexget/flexget/plugins']\r\nplugin.py                  404 DEBUG    (u'Plugin `%s` requires `%s` to load.', u'memusage', u'ext lib `guppy`')\r\nplugin.py                  468 DEBUG    Plugins took 0.07 seconds to load. 271 plugins in registry.\r\nmanager.py                 713 DEBUG    Connecting to: sqlite:///:memory:\r\ndb_schema.py                92 DEBUG    Initializing plugin feed schema version to 0\r\ndb_schema.py                92 DEBUG    Initializing plugin api_trakt schema version to 7\r\ndb_schema.py                92 DEBUG    Initializing plugin trakt_auth schema version to 0\r\ndb_schema.py                92 DEBUG    Initializing plugin make_rss schema version to 0\r\ndb_schema.py                92 DEBUG    Initializing plugin series schema version to 13\r\ndb_schema.py                92 DEBUG    Initializing plugin pending_approval schema version to 0\r\ndb_schema.py                92 DEBUG    Initializing plugin alpharatio schema version to 0\r\ndb_schema.py                92 DEBUG    Initializing plugin seen schema version to 4\r\ndb_schema.py                92 DEBUG    Initializing plugin pogcal_acquired schema version to 0\r\ndb_schema.py                92 DEBUG    Initializing plugin archive schema version to 0\r\ndb_schema.py                92 DEBUG    Initializing plugin digest schema version to 1\r\ndb_schema.py                92 DEBUG    Initializing plugin remember_rejected schema version to 3\r\ndb_schema.py                92 DEBUG    Initializing plugin simple_persistence schema version to 4\r\ndb_schema.py                92 DEBUG    Initializing plugin myepisodes schema version to 0\r\ndb_schema.py                92 DEBUG    Initializing plugin delay schema version to 2\r\ndb_schema.py                92 DEBUG    Initializing plugin failed schema version to 3\r\ndb_schema.py                92 DEBUG    Initializing plugin tail schema version to 0\r\ndb_schema.py                92 DEBUG    Initializing plugin api_tvdb schema version to 7\r\ndb_schema.py                92 DEBUG    Initializing plugin imdb_lookup schema version to 8\r\ndb_schema.py                92 DEBUG    Initializing plugin import_series schema version to 0\r\ndb_schema.py                92 DEBUG    Initializing plugin template_hash schema version to 0\r\ndb_schema.py                92 DEBUG    Initializing plugin status schema version to 2\r\ndb_schema.py                92 DEBUG    Initializing plugin movie_list schema version to 0\r\ndb_schema.py                92 DEBUG    Initializing plugin torrent411 schema version to 0\r\ndb_schema.py                92 DEBUG    Initializing plugin api_tmdb schema version to 5\r\ndb_schema.py                92 DEBUG    Initializing plugin log_once schema version to 0\r\ndb_schema.py                92 DEBUG    Initializing plugin discover schema version to 0\r\ndb_schema.py                92 DEBUG    Initializing plugin entry_list schema version to 1\r\ndb_schema.py                92 DEBUG    Initializing plugin regexp_list schema version to 1\r\ndb_schema.py                92 DEBUG    Initializing plugin api_bluray schema version to 0\r\ndb_schema.py                92 DEBUG    Initializing plugin subtitle_list schema version to 1\r\ndb_schema.py                92 DEBUG    Initializing plugin api_rottentomatoes schema version to 2\r\ndb_schema.py                92 DEBUG    Initializing plugin version_checker schema version to 0\r\ndb_schema.py                92 DEBUG    Initializing plugin secrets schema version to 0\r\ndb_schema.py                92 DEBUG    Initializing plugin morethantv schema version to 0\r\ndb_schema.py                92 DEBUG    Initializing plugin t411 schema version to 0\r\ndb_schema.py                92 DEBUG    Initializing plugin input_cache schema version to 1\r\ndb_schema.py                92 DEBUG    Initializing plugin telegram_chat_ids schema version to 0\r\ndb_schema.py                92 DEBUG    Initializing plugin tvmaze schema version to 6\r\ndb_schema.py                92 DEBUG    Initializing plugin imdb_list schema version to 0\r\ndb_schema.py                92 DEBUG    Initializing plugin rutracker_auth schema version to 0\r\ndb_schema.py                92 DEBUG    Initializing plugin backlog schema version to 2\r\nmanager.py                 639 DEBUG    New config data loaded.\r\ndb_schema.py                39 DEBUG    entering flexget version 2.8.12.dev to db\r\nplugin_parsing.py           32 DEBUG    setting default movie parser to internal. (options: {u'internal': <flexget.plugins.parsers.parser_internal.ParserInternal object at 0x1042f5fd0>, u'guessit': <flexg\r\net.plugins.parsers.parser_guessit.ParserGuessit object at 0x1043e4090>})\r\nplugin_parsing.py           32 DEBUG    setting default series parser to internal. (options: {u'internal': <flexget.plugins.parsers.parser_internal.ParserInternal object at 0x1042f5fd0>, u'guessit': <flex\r\nget.plugins.parsers.parser_guessit.ParserGuessit object at 0x1043e4090>})\r\nconftest.py                 89 INFO     ********** Running task: non_ascii **********\r\ntask.py                    576 DEBUG    executing non_ascii\r\nstatus.py                  110 DEBUG    Adding new task non_ascii\r\nlogger.py                  129 VERBOSE  Starting to scan folders.\r\nlogger.py                  129 VERBOSE  Scanning folder filesystem_test_dir//Test3. Recursion is set to True.\r\nfilesystem.py              165 DEBUG    Scanning filesystem_test_dir//Test3\r\nfilesystem.py              170 DEBUG    Checking if filesystem_test_dir//Test3/š dir qualifies to be added as an entry.\r\nfilesystem.py              170 DEBUG    Checking if filesystem_test_dir//Test3/s dir/Œ file.txt qualifies to be added as an entry.\r\nbacklog.py                 182 DEBUG    0 entries purged from backlog\r\nlogger.py                  129 VERBOSE  Produced 2 entries.\r\nurlrewriting.py             30 DEBUG    Checking 0 entries\r\nlogger.py                  129 VERBOSE  Summary - Accepted: 0 (Rejected: 0 Undecided: 2 Failed: 0)\r\nsimple_persistence.py      150 DEBUG    Flushing simple persistence for task non_ascii to db.\r\n========================================================================================== pytest-warning summary ==========================================================================================\r\nWC1 None [pytest] section in setup.cfg files is deprecated, use [tool:pytest] instead.\r\nWI1 /Users/johntobin/src/upstream-flexget/lib/python2.7/site-packages/pytest_capturelog.py:171 'pytest_runtest_makereport' hook uses deprecated __multicall__ argument\r\nWC1 None pytest_funcarg__cov: declaring fixtures using \"pytest_funcarg__\" prefix is deprecated and scheduled to be removed in pytest 4.0.  Please remove the prefix and use the @pytest.fixture decorator in\r\nstead.\r\nWC1 None pytest_funcarg__caplog: declaring fixtures using \"pytest_funcarg__\" prefix is deprecated and scheduled to be removed in pytest 4.0.  Please remove the prefix and use the @pytest.fixture decorator\r\n instead.\r\nWC1 None pytest_funcarg__capturelog: declaring fixtures using \"pytest_funcarg__\" prefix is deprecated and scheduled to be removed in pytest 4.0.  Please remove the prefix and use the @pytest.fixture decor\r\nator instead.\r\n======================================================= 1 failed, 1065 passed, 26 skipped, 3 xfailed, 1 xpassed, 5 pytest-warnings in 421.41 seconds =======================================================```\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/Flexget/Flexget/issues/comments/269212386","html_url":"https://github.com/Flexget/Flexget/pull/1589#issuecomment-269212386","issue_url":"https://api.github.com/repos/Flexget/Flexget/issues/1589","id":269212386,"user":{"login":"liiight","id":4374581,"avatar_url":"https://avatars.githubusercontent.com/u/4374581?v=3","gravatar_id":"","url":"https://api.github.com/users/liiight","html_url":"https://github.com/liiight","followers_url":"https://api.github.com/users/liiight/followers","following_url":"https://api.github.com/users/liiight/following{/other_user}","gists_url":"https://api.github.com/users/liiight/gists{/gist_id}","starred_url":"https://api.github.com/users/liiight/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/liiight/subscriptions","organizations_url":"https://api.github.com/users/liiight/orgs","repos_url":"https://api.github.com/users/liiight/repos","events_url":"https://api.github.com/users/liiight/events{/privacy}","received_events_url":"https://api.github.com/users/liiight/received_events","type":"User","site_admin":false},"created_at":"2016-12-26T13:40:48Z","updated_at":"2016-12-26T13:40:48Z","body":"OK and what does that data provide you with? What problem is this solving? "}}
{"actor":{"display_login":"liiight"}}