{"action":"created","issue":{"url":"https://api.github.com/repos/openresty/lua-nginx-module/issues/872","repository_url":"https://api.github.com/repos/openresty/lua-nginx-module","labels_url":"https://api.github.com/repos/openresty/lua-nginx-module/issues/872/labels{/name}","comments_url":"https://api.github.com/repos/openresty/lua-nginx-module/issues/872/comments","events_url":"https://api.github.com/repos/openresty/lua-nginx-module/issues/872/events","html_url":"https://github.com/openresty/lua-nginx-module/issues/872","id":178849941,"number":872,"title":"Memory overhead when using lua-nginx-module","user":{"login":"Zaynullin","id":1665900,"avatar_url":"https://avatars.githubusercontent.com/u/1665900?v=3","gravatar_id":"","url":"https://api.github.com/users/Zaynullin","html_url":"https://github.com/Zaynullin","followers_url":"https://api.github.com/users/Zaynullin/followers","following_url":"https://api.github.com/users/Zaynullin/following{/other_user}","gists_url":"https://api.github.com/users/Zaynullin/gists{/gist_id}","starred_url":"https://api.github.com/users/Zaynullin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Zaynullin/subscriptions","organizations_url":"https://api.github.com/users/Zaynullin/orgs","repos_url":"https://api.github.com/users/Zaynullin/repos","events_url":"https://api.github.com/users/Zaynullin/events{/privacy}","received_events_url":"https://api.github.com/users/Zaynullin/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":11,"created_at":"2016-09-23T11:11:57Z","updated_at":"2016-11-29T17:03:45Z","closed_at":null,"body":"I have nginx with lua-nginx-module.\nWhen I send range requests to nginx with lua-nginx-module and enabled http_slice_module, nginx takes all free memory and does not free even if all connections closed.\nWithout lua-nginx-module nginx does not use so much memory, one of worker processes can use some memory and free it during work.\n\n$ nginx -V\nnginx version: nginx/1.11.2\nbuilt by gcc 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04.3)\nbuilt with OpenSSL 1.0.1f 6 Jan 2014\nTLS SNI support enabled\nconfigure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-http_ssl_module --with-http_realip_module --with-http_addition_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_stub_status_module --with-http_auth_request_module --with-http_xslt_module=dynamic --with-http_image_filter_module=dynamic --with-http_geoip_module=dynamic --with-http_perl_module=dynamic --add-dynamic-module=debian/extra/njs-ef2b708510b1/nginx --with-threads --with-stream --with-stream_ssl_module --with-http_slice_module --with-mail --with-mail_ssl_module --with-file-aio --with-ipv6 --with-http_v2_module --with-cc-opt='-g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security' --with-ld-opt='-Wl,-Bsymbolic-functions -Wl,-z,relro' --add-module=ngx_devel_kit-0.3.0 --add-module=lua-nginx-module-0.10.6\n\nlibluajit-5.1-2 version:\n\n```\n$ ldd /usr/sbin/nginx | grep lua\n    libluajit-5.1.so.2 => /usr/lib/x86_64-linux-gnu/libluajit-5.1.so.2 \n$ ls -al /usr/lib/x86_64-linux-gnu/libluajit-5.1.so.2\nlrwxrwxrwx 1 root root 22 Jun  9  2013 /usr/lib/x86_64-linux-gnu/libluajit-5.1.so.2 -> libluajit-5.1.so.2.0.2\n```\n\nWith libluajit-5.1.so.2.0.4 memory usage same as with libluajit-5.1.so.2.0.2.\n\nFor example nginx with lua-nginx-module:\n\nmemory usage before test:\n\n```\n$ ps -e -orss=,args= | grep nginx: | awk 'BEGIN{sum=0;}{sum+=$1; print $0; }END{print \"Total memory = \", sum;}'\n 2712 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf\n13340 nginx: worker process\n13340 nginx: worker process\n13340 nginx: worker process\n13340 nginx: worker process\n13340 nginx: worker process\n13340 nginx: worker process\n13340 nginx: worker process\n13340 nginx: worker process\n13340 nginx: worker process\n13340 nginx: worker process\n13340 nginx: worker process\n13340 nginx: worker process\n13340 nginx: worker process\n13340 nginx: worker process\n13340 nginx: worker process\n13340 nginx: worker process\n 3360 nginx: cache manager process\n 2948 nginx: cache loader process\n  944 grep --color=auto nginx:\nTotal memory =  223404\n```\n\nmemory usage after test (all requests finished and all connections to nginx closed):\n\n```\n$ ps -e -orss=,args= | grep nginx: | awk 'BEGIN{sum=0;}{sum+=$1; print $0; }END{print \"Total memory = \", sum;}'\n 2372 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf\n2033368 nginx: worker process\n1993736 nginx: worker process\n2010688 nginx: worker process\n2072408 nginx: worker process\n1899844 nginx: worker process\n2036096 nginx: worker process\n2081336 nginx: worker process\n1829648 nginx: worker process\n1992792 nginx: worker process\n1705992 nginx: worker process\n2074204 nginx: worker process\n1805180 nginx: worker process\n2003932 nginx: worker process\n1902636 nginx: worker process\n2081448 nginx: worker process\n2048644 nginx: worker process\n 3008 nginx: cache manager process\n  804 grep --color=auto nginx:\nTotal memory =  31578136\n```\n\nnginx config:\n\n```\nworker_processes  16;\nworker_rlimit_nofile 1000000;\n\nevents {\n}\n\nhttp {\n    output_buffers            1 1m;\n\n    log_format main '$remote_addr [$time_local] \"$request\" $status $bytes_sent'\n                    ' \"$request_time\" \"$upstream_cache_status\" \"$http_range\"';\n\n    proxy_cache_path       /var/cache/nginx/proxy_cache levels=2:2 keys_zone=proxy_slice:500m inactive=3d use_temp_path=off;\n\n    server {\n      listen      127.0.0.1:8080 default_server backlog=65535;\n      access_log  /var/log/nginx/proxy_slice.log main;\n\n      slice             1m;\n      proxy_cache       proxy_slice;\n      proxy_cache_key   \"$request_uri|$slice_range\";\n      proxy_cache_valid 200 206 1d;\n      proxy_cache_valid any 0;\n      proxy_set_header  Range    $slice_range;\n      proxy_ignore_client_abort on;\n      proxy_http_version 1.1;\n\n      location / {\n        proxy_pass http://127.0.0.1:8088;\n      }\n    }\n\n    server {\n      listen 127.0.0.1:8088;\n\n      access_log /var/log/nginx/storage.log main;\n      rewrite ^.*$ /1.data last;\n\n      location / {\n        root /var/data;\n      }\n    }\n}\n```\n\nHow to reproduce:\n\n```\n$ mkdir /var/data; \n$ dd if=/dev/urandom of=/var/data/1.data bs=1M count=4500\n```\n\nCommand for test:\n\n```\n$ while true; do curl -H \"Range: bytes=$(echo $(od -An -N3 -i /dev/urandom)-$(( 100 * $(od -An -N3 -i /dev/urandom) )))\" \"http://127.0.0.1:8080/$(head /dev/urandom | tr -dc A-Za-z0-9 | head -c 13)\" > /dev/null ; done\n```\n"},"comment":{"url":"https://api.github.com/repos/openresty/lua-nginx-module/issues/comments/263631513","html_url":"https://github.com/openresty/lua-nginx-module/issues/872#issuecomment-263631513","issue_url":"https://api.github.com/repos/openresty/lua-nginx-module/issues/872","id":263631513,"user":{"login":"agentzh","id":56241,"avatar_url":"https://avatars.githubusercontent.com/u/56241?v=3","gravatar_id":"","url":"https://api.github.com/users/agentzh","html_url":"https://github.com/agentzh","followers_url":"https://api.github.com/users/agentzh/followers","following_url":"https://api.github.com/users/agentzh/following{/other_user}","gists_url":"https://api.github.com/users/agentzh/gists{/gist_id}","starred_url":"https://api.github.com/users/agentzh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/agentzh/subscriptions","organizations_url":"https://api.github.com/users/agentzh/orgs","repos_url":"https://api.github.com/users/agentzh/repos","events_url":"https://api.github.com/users/agentzh/events{/privacy}","received_events_url":"https://api.github.com/users/agentzh/received_events","type":"User","site_admin":false},"created_at":"2016-11-29T17:03:45Z","updated_at":"2016-11-29T17:03:45Z","body":"@Zaynullin How do you configure `lua_malloc_trim` in your `nginx.conf`? You can try smaller values there and also `0`."}}
{"actor":{"display_login":"agentzh"}}