{"action":"opened","issue":{"url":"https://api.github.com/repos/curl/curl/issues/2369","repository_url":"https://api.github.com/repos/curl/curl","labels_url":"https://api.github.com/repos/curl/curl/issues/2369/labels{/name}","comments_url":"https://api.github.com/repos/curl/curl/issues/2369/comments","events_url":"https://api.github.com/repos/curl/curl/issues/2369/events","html_url":"https://github.com/curl/curl/issues/2369","id":303143838,"number":2369,"title":"Pathological performance when setting MAX_TOTAL_CONNECTIONS","user":{"login":"cbay","id":725049,"avatar_url":"https://avatars3.githubusercontent.com/u/725049?v=4","gravatar_id":"","url":"https://api.github.com/users/cbay","html_url":"https://github.com/cbay","followers_url":"https://api.github.com/users/cbay/followers","following_url":"https://api.github.com/users/cbay/following{/other_user}","gists_url":"https://api.github.com/users/cbay/gists{/gist_id}","starred_url":"https://api.github.com/users/cbay/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cbay/subscriptions","organizations_url":"https://api.github.com/users/cbay/orgs","repos_url":"https://api.github.com/users/cbay/repos","events_url":"https://api.github.com/users/cbay/events{/privacy}","received_events_url":"https://api.github.com/users/cbay/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-03-07T15:30:38Z","updated_at":"2018-03-07T15:30:38Z","closed_at":null,"author_association":"NONE","body":"libcurl version: 7.58.0\r\nOperating system: Archlinux (Linux 4.15.5).\r\n\r\nI'm trying to use libcurl to do as many requests as possible to a local server (similarly to load testing tools, although that's not my purpose). I've used the [multi-uv.c](https://raw.githubusercontent.com/curl/curl/master/docs/examples/multi-uv.c) example and slightly simplified it to not save files to disk and accept the number of requests to do as argument ([full source](https://gist.github.com/cbay/c97334c6b202d27558ee70ccf2902614)).\r\n\r\nThe local server is nginx serving a tiny static page. It's not the bottleneck.\r\n\r\nDoing 1000 requests takes about ~160ms:\r\n\r\n```\r\n$ time ./multi-uv 1000\r\n./multi-uv 1000  0.08s user 0.06s system 89% cpu 0.162 total\r\n```\r\n\r\nFor comparison, [h2load](https://nghttp2.org/documentation/h2load.1.html) takes a similar amount of time, although using less CPU:\r\n\r\n```\r\n$ time h2load -c 1000 -n 1000 --h1 http://127.0.0.1:8080/hello.html >/dev/null\r\nh2load -c 1000 -n 1000 --h1 http://127.0.0.1:8080/hello.html  0.02s user 0.07s system 51% cpu 0.166 total\r\n```\r\n\r\nObviously, reusing connections should improve performance. Confirmed with h2load:\r\n\r\n```\r\n$ time h2load -c 10 -n 1000 --h1 http://127.0.0.1:8080/hello.html >/dev/null\r\nh2load -c 10 -n 1000 --h1 http://127.0.0.1:8080/hello.html > /dev/null  0.01s user 0.02s system 70% cpu 0.037 total\r\n```\r\n\r\nHowever, adding:\r\n\r\n```\r\ncurl_multi_setopt(curl_handle, CURLMOPT_MAX_TOTAL_CONNECTIONS, 10L);\r\n```\r\n\r\nto `multi-uv.c` actually degrades performance a lot:\r\n\r\n```\r\n$ time ./multi-uv 1000\r\n./multi-uv 1000  0.35s user 0.09s system 90% cpu 0.485 total\r\n```\r\n\r\nEven worse, the performance degradation is not linear at all. Let's do more requests in total (still limiting to 10 connections):\r\n\r\n```\r\n$ time ./multi-uv 2000\r\n./multi-uv 2000  1.38s user 0.18s system 95% cpu 1.629 total\r\n$ time ./multi-uv 5000\r\n./multi-uv 5000  8.28s user 0.99s system 98% cpu 9.367 total\r\n```\r\n\r\nAm I doing something wrong here? missing an obvious optimization?\r\n\r\nHere's what profiling with `perf` returns (truncated):\r\n\r\n```\r\n    81.56%     0.00%  multi-uv     multi-uv                [.] curl_multi_socket_action\r\n            |\r\n            ---curl_multi_socket_action\r\n               |          \r\n                --81.51%--multi_socket\r\n                          |          \r\n                          |--73.87%--multi_runsingle\r\n                          |          |          \r\n                          |          |--65.54%--Curl_connect\r\n                          |          |          |          \r\n                          |          |           --65.10%--create_conn\r\n                          |          |                     |          \r\n                          |          |                     |--11.41%--ConnectionExists\r\n                          |          |                     |          |          \r\n                          |          |                     |          |--6.37%--Curl_conncache_find_bundle\r\n                          |          |                     |          |          |          \r\n                          |          |                     |          |          |--4.86%--hashkey\r\n                          |          |                     |          |          |          |          \r\n                          |          |                     |          |          |           --4.78%--curl_msnprintf\r\n                          |          |                     |          |          |                     |          \r\n                          |          |                     |          |          |                      --4.70%--curl_mvsnprintf\r\n                          |          |                     |          |          |                                |          \r\n                          |          |                     |          |          |                                 --4.44%--dprintf_formatf\r\n                          |          |                     |          |          |                                           |          \r\n                          |          |                     |          |          |                                           |--1.30%--dprintf_Pass1\r\n                          |          |                     |          |          |                                           |          \r\n                          |          |                     |          |          |                                            --1.10%--addbyter\r\n                          |          |                     |          |          |          \r\n                          |          |                     |          |           --1.43%--Curl_hash_pick\r\n                          |          |                     |          |                     |          \r\n                          |          |                     |          |                      --0.89%--Curl_hash_str\r\n                          |          |                     |          |          \r\n                          |          |                     |           --1.97%--extract_if_dead\r\n                          |          |                     |          \r\n                          |          |                     |--7.11%--conn_free\r\n                          |          |                     |          |          \r\n                          |          |                     |          |--2.31%--curl_dofree\r\n                          |          |                     |          |          |          \r\n                          |          |                     |          |           --1.07%--curl_memlog\r\n                          |          |                     |          |          \r\n                          |          |                     |          |--1.58%--Curl_free_primary_ssl_config\r\n                          |          |                     |          |          |          \r\n                          |          |                     |          |           --0.84%--curl_dofree\r\n                          |          |                     |          |          \r\n                          |          |                     |           --0.52%--Curl_ssl_close\r\n                          |          |                     |          \r\n                          |          |                     |--6.40%--Curl_conncache_extract_oldest\r\n                          |          |                     |          |          \r\n                          |          |                     |           --4.44%--Curl_hash_next_element\r\n                          |          |                     |          \r\n                          |          |                     |--5.98%--Curl_conncache_find_bundle\r\n                          |          |                     |          |          \r\n                          |          |                     |          |--4.51%--hashkey\r\n                          |          |                     |          |          |          \r\n                          |          |                     |          |           --4.41%--curl_msnprintf\r\n                          |          |                     |          |                     |          \r\n                          |          |                     |          |                      --4.33%--curl_mvsnprintf\r\n                          |          |                     |          |                                |          \r\n                          |          |                     |          |                                 --4.07%--dprintf_formatf\r\n                          |          |                     |          |                                           |          \r\n                          |          |                     |          |                                           |--1.19%--dprintf_Pass1\r\n                          |          |                     |          |                                           |          \r\n                          |          |                     |          |                                            --0.96%--addbyter\r\n                          |          |                     |          |          \r\n                          |          |                     |           --1.30%--Curl_hash_pick\r\n                          |          |                     |                     |          \r\n                          |          |                     |                      --0.92%--Curl_hash_str\r\n                          |          |                     |          \r\n                          |          |                     |--5.96%--parseurlandfillconn\r\n                          |          |                     |          |          \r\n                          |          |                     |          |--1.99%--findprotocol\r\n                          |          |                     |          |          \r\n                          |          |                     |           --1.08%--Curl_dedotdotify\r\n                          |          |                     |          \r\n                          |          |                     |--4.32%--allocate_conn\r\n                          |          |                     |          \r\n                          |          |                     |--4.01%--create_conn_helper_init_proxy\r\n                          |          |                     |          |          \r\n                          |          |                     |           --2.53%--detect_proxy\r\n                          |          |                     |                     |          \r\n                          |          |                     |                     |--1.06%--Curl_strcasecompare\r\n                          |          |                     |                     |          |          \r\n                          |          |                     |                     |           --0.51%--Curl_raw_toupper\r\n                          |          |                     |                     |          \r\n                          |          |                     |                      --0.77%--curl_getenv\r\n                          |          |                     |          \r\n                          |          |                     |--3.88%--setup_connection_internals\r\n                          |          |                     |          |          \r\n                          |          |                     |          |--2.26%--Curl_free_request_state\r\n                          |          |                     |          |          |          \r\n                          |          |                     |          |           --1.45%--cfree@GLIBC_2.2.5\r\n                          |          |                     |          |          \r\n                          |          |                     |           --1.40%--Curl_http_setup_conn\r\n                          |          |                     |          \r\n                          |          |                     |--3.03%--Curl_clone_primary_ssl_config\r\n                          |          |                     |          |          \r\n                          |          |                     |          |--2.05%--__strlen_avx2\r\n                          |          |                     |          |          \r\n                          |          |                     |           --0.53%--curl_dostrdup\r\n                          |          |                     |          \r\n                          |          |                     |--1.20%--parse_remote_port\r\n                          |          |                     |          \r\n                          |          |                     |--0.88%--set_login\r\n                          |          |                     |          |          \r\n                          |          |                     |           --0.54%--curl_dostrdup\r\n                          |          |                     |          \r\n                          |          |                     |--0.80%--curl_dostrdup\r\n                          |          |                     |          \r\n                          |          |                     |--0.64%--curl_dofree\r\n                          |          |                     |          \r\n                          |          |                     |--0.61%--fix_hostname\r\n                          |          |                     |          \r\n                          |          |                     |--0.60%--cfree@GLIBC_2.2.5\r\n                          |          |                     |          \r\n                          |          |                      --0.55%--__strlen_avx2\r\n                          |          |          \r\n                          |           --4.74%--Curl_multi_process_pending_handles\r\n                          |                     |          \r\n                          |                     |--2.32%--Curl_expire\r\n                          |                     |          |          \r\n                          |                     |           --1.37%--multi_deltimeout\r\n                          |                     |          \r\n                          |                      --0.99%--mstate\r\n                          |                                |          \r\n                          |                                 --0.61%--Curl_infof\r\n                          |          \r\n                          |--2.80%--Curl_splaygetbest\r\n                          |          |          \r\n                          |           --2.24%--Curl_splay\r\n                          |          \r\n                          |--1.85%--add_next_timeout\r\n                          |          \r\n                          |--0.74%--sigpipe_ignore\r\n                          |          \r\n                          |--0.74%--__libc_sigaction\r\n                          |          \r\n                           --0.62%--0xffffffffab800158\r\n```"}}
{"actor":{"display_login":"cbay"}}