title: How to speed up file uploading using Nginx + uWSGI
body: <p>I have used nginx built-in functionality <strong>clientbodyinfileonly</strong> to upload my files. And in order to know its performance. I used Apache ab to test it. </p>
  
  <p>when I set <strong>concurrency level</strong> to <strong>100</strong>,</p>
  
  <pre><code>ab -n 10000 -c 100 -p xxxx.rar -T application/x-www-form-urlencoded http://??????
  </code></pre>
  
  <p>the result was</p>
  
  <pre><code>Requests per second:    1477.95 [#/sec] (mean)
  </code></pre>
  
  <p>When the <strong>concurrency level</strong> is set to 120, I got 502 errors.</p>
  
  <pre><code>connect() to unix:/???/uwsgi.sock failed (11: Resource temporarily unavailable) while connecting to upstream
  </code></pre>
  
  <p>The nginx config as following shown [ visit <a href="https://coderwall.com/p/swgfvw/nginx-direct-file-upload-without-passing-them-through-backend" rel="nofollow">Nginx direct file upload without passing them through backend</a> for more details ]</p>
  
  <pre><code>location /upload {
          auth_request               /upload/authenticate;
  
          client_body_temp_path      /tmp/;
          client_body_in_file_only   on;
          client_body_buffer_size    16M;
          client_max_body_size       80M;
  
          uwsgi_pass_request_headers on;
  
          uwsgi_param                X-FILE $request_body_file;
          uwsgi_pass_request_body    off;
          uwsgi_pass                 unix://tmp/uwsgi.sock;
          include  uwsgi_params;
  }
  
  location = /upload/authenticate {
          uwsgi_pass              unix://tmp/uwsgi.sock;
          uwsgi_pass_request_body off;
          uwsgi_param             CONTENT-LENGTH "";
          uwsgi_param             X-Original-URI $request_uri;
          include  uwsgi_params;
  }
  </code></pre>
  
  <p>The way I took to around it was providing much more back-end servers together with <strong>upstream</strong> directive. And now the 502 went away. However, the TPS is still less than 1600 no matter how many uWSGi instances I added [ in practice, I used at most 8 instance, each with 2 processes ].</p>
  
  <p>I also looked at the cpu, memory usage [ top, free, vmstat] during loading test. They are not too high. </p>
  
  <p>Why the performance does not become better even if I used more uWSGI instances?? How could I know where to optimize?</p>
  
  <p>The machine used is AWS EC2 with 2 CPUs ( 2.5GHz), 4G memory. And the file size used to upload is 30K.</p>