{"action":"created","issue":{"url":"https://api.github.com/repos/orgtechservice/roskombox/issues/16","repository_url":"https://api.github.com/repos/orgtechservice/roskombox","labels_url":"https://api.github.com/repos/orgtechservice/roskombox/issues/16/labels{/name}","comments_url":"https://api.github.com/repos/orgtechservice/roskombox/issues/16/comments","events_url":"https://api.github.com/repos/orgtechservice/roskombox/issues/16/events","html_url":"https://github.com/orgtechservice/roskombox/issues/16","id":173620542,"number":16,"title":"При запросе с локального хоста, имя пира пустое, а ресурс отображается как незаблокированный.","user":{"login":"dserov","id":13929278,"avatar_url":"https://avatars.githubusercontent.com/u/13929278?v=3","gravatar_id":"","url":"https://api.github.com/users/dserov","html_url":"https://github.com/dserov","followers_url":"https://api.github.com/users/dserov/followers","following_url":"https://api.github.com/users/dserov/following{/other_user}","gists_url":"https://api.github.com/users/dserov/gists{/gist_id}","starred_url":"https://api.github.com/users/dserov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dserov/subscriptions","organizations_url":"https://api.github.com/users/dserov/orgs","repos_url":"https://api.github.com/users/dserov/repos","events_url":"https://api.github.com/users/dserov/events{/privacy}","received_events_url":"https://api.github.com/users/dserov/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/orgtechservice/roskombox/labels/%D0%9E%D1%88%D0%B8%D0%B1%D0%BA%D0%B0","name":"Ошибка","color":"fc2929"}],"state":"open","locked":false,"assignee":{"login":"WST","id":190773,"avatar_url":"https://avatars.githubusercontent.com/u/190773?v=3","gravatar_id":"","url":"https://api.github.com/users/WST","html_url":"https://github.com/WST","followers_url":"https://api.github.com/users/WST/followers","following_url":"https://api.github.com/users/WST/following{/other_user}","gists_url":"https://api.github.com/users/WST/gists{/gist_id}","starred_url":"https://api.github.com/users/WST/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/WST/subscriptions","organizations_url":"https://api.github.com/users/WST/orgs","repos_url":"https://api.github.com/users/WST/repos","events_url":"https://api.github.com/users/WST/events{/privacy}","received_events_url":"https://api.github.com/users/WST/received_events","type":"User","site_admin":false},"assignees":[{"login":"WST","id":190773,"avatar_url":"https://avatars.githubusercontent.com/u/190773?v=3","gravatar_id":"","url":"https://api.github.com/users/WST","html_url":"https://github.com/WST","followers_url":"https://api.github.com/users/WST/followers","following_url":"https://api.github.com/users/WST/following{/other_user}","gists_url":"https://api.github.com/users/WST/gists{/gist_id}","starred_url":"https://api.github.com/users/WST/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/WST/subscriptions","organizations_url":"https://api.github.com/users/WST/orgs","repos_url":"https://api.github.com/users/WST/repos","events_url":"https://api.github.com/users/WST/events{/privacy}","received_events_url":"https://api.github.com/users/WST/received_events","type":"User","site_admin":false}],"milestone":null,"comments":7,"created_at":"2016-08-27T23:27:18Z","updated_at":"2016-08-30T12:45:57Z","closed_at":null,"body":"Доброго времени суток!\r\n\r\nЯ слегка добавил диагностики, чтоб при запуске их командной строки было удобнее видеть ип пира и код статуса ответа сервера. Видно, что на доменах, которые ресолвятся в 127.0.0.1 отсутствует ип пира, и они, соответственно не помечаются, как local-ip. \r\nПлюс отключил проверку при открытии https ресурсов. У нас там тоже заглушка. Сертификат невалидный, поэтому отключил. \r\n```\r\n(roskombox) admin@xbtt:~/www/roskombox/portal/management/commands$ diff -p ./roskom_check.py.origin ./roskom_check.py\r\n*** ./roskom_check.py.origin    2016-08-25 21:00:06.000000000 +0300\r\n--- ./roskom_check.py   2016-08-28 01:38:03.108796516 +0300\r\n*************** import resource, time, sys, threading, l\r\n*** 21,26 ****\r\n--- 21,30 ----\r\n  import requests\r\n  from lxml import etree\r\n\r\n+ # Отключим ругань на невалидный сертификат\r\n+ from requests.packages.urllib3.exceptions import InsecureRequestWarning\r\n+ requests.packages.urllib3.disable_warnings(InsecureRequestWarning)\r\n+\r\n  # Время начала работы скрипта\r\n  execution_start = time.time()\r\n\r\n*************** class Worker(threading.Thread):\r\n*** 88,100 ****\r\n                        percents = int((float(counter) / float(self.total_count)) * 100.0)\r\n                        self.scan.set_progress(percents)\r\n\r\n!               print(u\"(%d of %d) [%s] %s\" % (counter, self.total_count, item['status'], item['url']))\r\n\r\n        def process_item(self, item):\r\n                global search_substring, request_headers\r\n                try:\r\n!                       response = requests.get(item['url'], timeout = self.timeout, stream = True, headers = request_headers)\r\n!\r\n                        maxsize = 100000\r\n                        content = ''\r\n                        for chunk in response.iter_content(2048):\r\n--- 92,107 ----\r\n                        percents = int((float(counter) / float(self.total_count)) * 100.0)\r\n                        self.scan.set_progress(percents)\r\n\r\n!               print(u\"(%d of %d) [%s] %s(%s) -  %s\" % (counter, self.total_count, item['status'], item['peer'], item['status_code'], item['url']))\r\n\r\n        def process_item(self, item):\r\n                global search_substring, request_headers\r\n                try:\r\n!                       # Умолчательное значение, если выскочит исключение\r\n!                       item['peer'] = 'exception'\r\n!                       item['status_code'] = 'Null'\r\n!                       response = requests.get(item['url'], timeout = self.timeout, stream = True, headers = request_headers, verify = False)\r\n!\r\n                        maxsize = 100000\r\n                        content = ''\r\n                        for chunk in response.iter_content(2048):\r\n*************** class Worker(threading.Thread):\r\n*** 105,125 ****\r\n                                item['status'] = 'available'\r\n                                item['reply'] = 'data too large'\r\n\r\n                        if search_substring in content:\r\n                                item['status'] = 'blocked'\r\n                        else:\r\n-                               peer = response.raw._original_response.peer\r\n                                if peer is not None:\r\n                                        address = peer[0]\r\n                                        if address.startswith('127') or address.startswith('192.168') or address.startswith('10.10'):\r\n                                                item['status'] = 'local-ip'\r\n                                        else:\r\n                                                item['status'] = 'available'\r\n-                                               item['reply'] = content\r\n                                else:\r\n                                        item['status'] = 'available'\r\n-                                       item['reply'] = content\r\n- #                                     print('status: available, reply: %s' % content[:100])\r\n                except Exception as e:\r\n                        item['status'] = 'failure'\r\n\r\n--- 112,131 ----\r\n                                item['status'] = 'available'\r\n                                item['reply'] = 'data too large'\r\n\r\n+                       peer = response.raw._original_response.peer\r\n+                       item['peer'] = (peer[0] if peer is not None else 'None')\r\n+                       item['status_code'] = response.status_code\r\n                        if search_substring in content:\r\n                                item['status'] = 'blocked'\r\n                        else:\r\n                                if peer is not None:\r\n                                        address = peer[0]\r\n                                        if address.startswith('127') or address.startswith('192.168') or address.startswith('10.10'):\r\n                                                item['status'] = 'local-ip'\r\n                                        else:\r\n                                                item['status'] = 'available'\r\n                                else:\r\n                                        item['status'] = 'available'\r\n                except Exception as e:\r\n                        item['status'] = 'failure'\r\n\r\n```"},"comment":{"url":"https://api.github.com/repos/orgtechservice/roskombox/issues/comments/243426662","html_url":"https://github.com/orgtechservice/roskombox/issues/16#issuecomment-243426662","issue_url":"https://api.github.com/repos/orgtechservice/roskombox/issues/16","id":243426662,"user":{"login":"dserov","id":13929278,"avatar_url":"https://avatars.githubusercontent.com/u/13929278?v=3","gravatar_id":"","url":"https://api.github.com/users/dserov","html_url":"https://github.com/dserov","followers_url":"https://api.github.com/users/dserov/followers","following_url":"https://api.github.com/users/dserov/following{/other_user}","gists_url":"https://api.github.com/users/dserov/gists{/gist_id}","starred_url":"https://api.github.com/users/dserov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dserov/subscriptions","organizations_url":"https://api.github.com/users/dserov/orgs","repos_url":"https://api.github.com/users/dserov/repos","events_url":"https://api.github.com/users/dserov/events{/privacy}","received_events_url":"https://api.github.com/users/dserov/received_events","type":"User","site_admin":false},"created_at":"2016-08-30T12:45:57Z","updated_at":"2016-08-30T12:45:57Z","body":"urllib3-1.9.1-py2.py3-none-any.whl"}}
{"actor":{"display_login":"dserov"}}