{"action":"created","issue":{"url":"https://api.github.com/repos/z3APA3A/3proxy/issues/263","repository_url":"https://api.github.com/repos/z3APA3A/3proxy","labels_url":"https://api.github.com/repos/z3APA3A/3proxy/issues/263/labels{/name}","comments_url":"https://api.github.com/repos/z3APA3A/3proxy/issues/263/comments","events_url":"https://api.github.com/repos/z3APA3A/3proxy/issues/263/events","html_url":"https://github.com/z3APA3A/3proxy/issues/263","id":318413862,"number":263,"title":"Проблема производительности связки connect back","user":{"login":"flyrider","id":23336296,"avatar_url":"https://avatars2.githubusercontent.com/u/23336296?v=4","gravatar_id":"","url":"https://api.github.com/users/flyrider","html_url":"https://github.com/flyrider","followers_url":"https://api.github.com/users/flyrider/followers","following_url":"https://api.github.com/users/flyrider/following{/other_user}","gists_url":"https://api.github.com/users/flyrider/gists{/gist_id}","starred_url":"https://api.github.com/users/flyrider/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/flyrider/subscriptions","organizations_url":"https://api.github.com/users/flyrider/orgs","repos_url":"https://api.github.com/users/flyrider/repos","events_url":"https://api.github.com/users/flyrider/events{/privacy}","received_events_url":"https://api.github.com/users/flyrider/received_events","type":"User","site_admin":false},"labels":[{"id":90906471,"url":"https://api.github.com/repos/z3APA3A/3proxy/labels/question","name":"question","color":"cc317c","default":true}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":9,"created_at":"2018-04-27T13:40:48Z","updated_at":"2018-04-28T12:16:20Z","closed_at":"2018-04-27T14:08:09Z","author_association":"NONE","body":"Добрый день!\r\n\r\nOчень давно используем 3proxy в своих продуктивах, но столкнулись со странным эффектом в производительности на связке connect-back и никак не можем разобраться.\r\n\r\nПри превышении ESTABLISHED соединений на прокси connect-back свыше 100 отмечается существенное увеличение времени ответа от прокси:\r\n```\r\nsrv4007 ~$ netstat -an | grep EST | grep 4007 | wc -l\r\n542\r\nsrv4007 ~$ netstat -an | grep EST | grep 4008 | wc -l\r\n2\r\n```\r\n```\r\nsrv4007 ~$ time curl -x xxx:123@srv4007:4008 http://ya.ru/ -s\r\n\r\nreal\t0m0.230s\r\nuser\t0m0.008s\r\nsys\t0m0.000s\r\n```\r\n```\r\nsrv4007 ~$ time curl -x xxx:123@srv4007:4007 http://ya.ru/ -s\r\n\r\nreal\t0m14.469s\r\nuser\t0m0.008s\r\nsys\t0m0.004s\r\n```\r\nКонфигурационные файлы:\r\n\r\n4007\r\n\r\n```\r\ndaemon\r\nlog /log/proxy.log\r\ntimeouts 8 40 240 480 720 1800 60 60\r\nmaxconn 20000\r\nauth iponly\r\nallow * * *\r\ntcppm -R0.0.0.0:3007 4007 0.0.0.0 4007\r\n```\r\n4008\r\n\r\n```\r\ndaemon\r\nlog /log/proxy.log\r\ntimeouts 8 40 240 480 720 1800 60 60\r\nmaxconn 20000\r\nauth iponly\r\nallow * * *\r\ntcppm -R0.0.0.0:3008 4008 0.0.0.0 4008\r\n```\r\n3007\r\n\r\n```\r\ndaemon\r\ntimeouts 8 40 240 480 720 1800 60 60\r\nlog /log/proxy.log\r\nmaxconn 20000\r\nusers xxx:CL:123\r\nauth strong\r\nallow xxx *\r\nproxy -rsrv1.prxdomain.com:3007\r\n```\r\n3008\r\n\r\n```\r\ndaemon\r\ntimeouts 8 40 240 480 720 1800 60 60\r\nlog /log/proxy.log\r\nmaxconn 20000\r\nusers xxx:CL:123\r\nauth strong\r\nallow xxx *\r\nproxy -rsrv1.prxdomain.com:3008\r\n```\r\n\r\nВ логах на инициаторе петли \"3007\" записи типа:\r\n```\r\n180427202631.692 3128 00000 xxx 31.201.143.105:3007 77.240.29.19:443 679 5336 0 CONNECT 7.240.29.19:443 HTTP/1.0\r\n```\r\nВ логах на srv4007 все аналогично. Видно что запрос уходит, доходит, но как уменьшить время ответа?\r\n\r\nМежду srv4007 и 3007 пинг не превышает 20ms и не растет при нагрузке на прокси.\r\n\r\nsysctl на srv4007\r\n```\r\nsrv4007 ~$ sysctl -p\r\nnet.ipv4.tcp_keepalive_time = 320\r\nnet.ipv4.tcp_keepalive_intvl = 300\r\nnet.ipv4.tcp_fin_timeout = 300\r\nnet.netfilter.nf_conntrack_generic_timeout = 320\r\nnet.netfilter.nf_conntrack_tcp_timeout_established = 1200\r\nnet.netfilter.nf_conntrack_max = 1000048576\r\nnet.netfilter.nf_conntrack_max = 1000048576\r\nnet.core.rmem_max = 1677721600\r\nnet.core.rmem_default = 167772160\r\nnet.core.wmem_max = 1677721600\r\nnet.core.wmem_default = 167772160\r\nnet.ipv4.tcp_rmem = 1024000 8738000 1677721600\r\nnet.ipv4.tcp_wmem = 1024000 8738000 1677721600\r\nnet.ipv4.tcp_mem = 1024000 8738000 1677721600\r\nnet.ipv4.udp_mem = 1024000 8738000 1677721600\r\n```\r\nlimits на srv4007 (для root, 3proxy запускается под root) \r\n```\r\ncore file size          (blocks, -c) 0\r\ndata seg size           (kbytes, -d) unlimited\r\nscheduling priority             (-e) 0\r\nfile size               (blocks, -f) unlimited\r\npending signals                 (-i) 9860\r\nmax locked memory       (kbytes, -l) 64\r\nmax memory size         (kbytes, -m) unlimited\r\nopen files                      (-n) 204708\r\npipe size            (512 bytes, -p) 8\r\nPOSIX message queues     (bytes, -q) 819200\r\nreal-time priority              (-r) 0\r\nstack size              (kbytes, -s) unlimited\r\ncpu time               (seconds, -t) unlimited\r\nmax user processes              (-u) unlimited\r\nvirtual memory          (kbytes, -v) unlimited\r\nfile locks                      (-x) unlimited\r\n```"},"comment":{"url":"https://api.github.com/repos/z3APA3A/3proxy/issues/comments/385170674","html_url":"https://github.com/z3APA3A/3proxy/issues/263#issuecomment-385170674","issue_url":"https://api.github.com/repos/z3APA3A/3proxy/issues/263","id":385170674,"user":{"login":"flyrider","id":23336296,"avatar_url":"https://avatars2.githubusercontent.com/u/23336296?v=4","gravatar_id":"","url":"https://api.github.com/users/flyrider","html_url":"https://github.com/flyrider","followers_url":"https://api.github.com/users/flyrider/followers","following_url":"https://api.github.com/users/flyrider/following{/other_user}","gists_url":"https://api.github.com/users/flyrider/gists{/gist_id}","starred_url":"https://api.github.com/users/flyrider/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/flyrider/subscriptions","organizations_url":"https://api.github.com/users/flyrider/orgs","repos_url":"https://api.github.com/users/flyrider/repos","events_url":"https://api.github.com/users/flyrider/events{/privacy}","received_events_url":"https://api.github.com/users/flyrider/received_events","type":"User","site_admin":false},"created_at":"2018-04-28T12:16:20Z","updated_at":"2018-04-28T12:16:20Z","author_association":"NONE","body":"Спасибо за информацию, проведем эксперименты с данными опциями. Нагрузка реальна такая - до тысячи соединений на петлю. А каким-то образом можно \"увидеть\" тормоза NAT таблицы?\r\nВообще раньше транспортом между клиентом и сервером вместо \"петлевой схемы 3proxy\" служил старый добрый p2p туннель (3прокси заворачивался в него на сервере) и такой проблемы не было, но скорость соединения была процентов на 25 хуже чем сейчас  с петлей.   "}}
{"actor":{"display_login":"flyrider"}}