title: Processes are running slow in Docker container on Ubuntu 18.04
body: <p><strong>Processes are taking time when running them in a Docker container on an Ubuntu 18 machine. But the same process with the same Docker version is running fine on an Ubuntu 16 machine.</strong></p>
  <p>I have a node application listening on some port. Accepting get requests on the path &quot;/&quot; and &quot;/docker&quot; which simply runs a command &quot;whoami&quot; in the host machine and in a Docker container respectively and returns the result. The same node application with the same Docker container is running on both the machines (Ubuntu 16 and Ubuntu 18).</p>
  <p>Firstly, I tried sending 20 concurrent get requests with path &quot;/&quot; to both the machines. And both the machines executed the command in an average of<br />
  35-40ms.</p>
  <p>Secondly, I tried sending 20 concurrent get requests with path &quot;/docker&quot; to both the machines. Here, the Ubuntu 16 machine took a maximum of 4.3 seconds and an average of 3 seconds. But the Ubuntu 18 machine takes a maximum of 10 seconds and an average of 9 seconds.</p>
  <p>I tried the above test multiple times and concluded that when running the process inside Docker, the time taken to execute is almost double in the Ubuntu 18 machine compared to Ubuntu 16.</p>
  <p><strong>I checked the following:</strong></p>
  <ol>
  <li><p>I tried monitoring through <code>top</code> and <code>htop</code> while hitting 20 requests. But everything seems the same there.</p>
  </li>
  <li><p>Also tried monitoring using <code>perf</code> command. But unable to find any unusual difference there. But I am not very used to perf command and so unable to understand clearly.</p>
  </li>
  <li><p>While these 20 requests were in processing. I run the same Docker command manually with <code>strace</code>. And found random results i.e. sometimes time taken in <code>clock_gettime</code> or <code>futex (FUTEX_WAIT)</code> or sometimes in <code>+++ exited with 0 +++</code> message on Ubuntu 18, but it took less time on Ubuntu 16.</p>
  </li>
  </ol>
  <p>Below are the different configurations and code snippets I am using and running:</p>
  <pre>
  <b>Machine1: Giving better performance.</b>
  node    v10.16.0
  npm     6.9.0
  docker  18.09.8
  ubuntu  16.04.3 LTS, xenial
  
  <b>Machine2: Giving poor performance.</b>
  node    v10.16.0
  npm     6.9.0
  docker  18.09.8
  ubuntu  18.04.2 LTS, bionic
  </pre>
  <p>Node application code snippet:</p>
  <pre class="lang-js prettyprint-override"><code>// for path &quot;/docker&quot;
  var excuteInDocker = function() {
      var cmd = &quot;docker&quot;;
      var args = [&quot;exec&quot;, &quot;ubuntu&quot;, &quot;whoami&quot;];
      return executeCmd(cmd, args);
  }
  
  // for path &quot;/&quot;
  var execute = function(){
      var cmd = 'whoami';
      var args = [];
      return executeCmd(cmd, args);
  }
  
  </code></pre>
  <p><strong>Output of <code>docker info</code> which are common to both ubuntu 16 and 18:</strong></p>
  <pre><code>Containers: 1
   Running: 1
   Paused: 0
   Stopped: 0
  Images: 2
  Server Version: 18.09.8
  Storage Driver: overlay2
   Backing Filesystem: extfs
   Supports d_type: true
   Native Overlay Diff: true
  Logging Driver: json-file
  Cgroup Driver: cgroupfs
  Plugins:
   Volume: local
   Network: bridge host macvlan null overlay
   Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
  Swarm: inactive
  Runtimes: runc
  Default Runtime: runc
  Init Binary: docker-init
  containerd version: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
  runc version: 425e105d5a03fabd737a126ad93d62a9eeede87f
  init version: fec3683
  Security Options:
   apparmor
   seccomp
    Profile: default
  OSType: linux
  Architecture: x86_64
  CPUs: 4
  Total Memory: 7.296GiB
  Name: myhostname
  ID: LLLO:OMTS:PNNM:T3MP:AD2F:UMDG:IIZK:OGBO:3ZLL:YDBX:ONAO:AY5G
  Docker Root Dir: /var/lib/docker
  Debug Mode (client): false
  Debug Mode (server): true
   File Descriptors: 27
   Goroutines: 42
   System Time: 2019-07-25T15:25:54.991694211+05:30
   EventsListeners: 0
  Registry: https://index.docker.io/v1/
  Labels:
  Experimental: false
  Insecure Registries:
   127.0.0.0/8
  Live Restore Enabled: false
  Product License: Community Engine
  
  WARNING: No swap limit support
  
  </code></pre>
  <p><strong><code>docker info</code> specific to Ubuntu 16:</strong></p>
  <pre><code>Kernel Version: 4.4.0-112-generic
  Operating System: Ubuntu 16.04.3 LTS
  Total Memory: 7.303GiB
  ID: FOFI:RW7N:RZSP:HHKH:BKS3:LMWL:TC2J:W7V2:222Y:Q2AU:XMU3:KLU7
  </code></pre>
  <p><strong><code>docker info</code> specific to Ubuntu 18:</strong></p>
  <pre><code>Kernel Version: 4.15.0-1040-aws
  Operating System: Ubuntu 18.04.2 LTS
  Total Memory: 7.296GiB
  ID: LLLO:OMTS:PNNM:T3MP:AD2F:UMDG:IIZK:OGBO:3ZLL:YDBX:ONAO:AY5G
  </code></pre>
  <hr />
  <p><strong>Ubuntu 16 machine Data:</strong></p>
  <p><strong>1. Data of time taken in execution</strong></p>
  <pre><code>2019-07-25 14:06:42.851 INFO  uid: 540ae880-aeb7-11e9-919d-dd32b3cf84d5 time: 475 result: {&quot;success&quot;:true,&quot;data&quot;:&quot;root&quot;}
  2019-07-25 14:06:43.183 INFO  uid: 54145e60-aeb7-11e9-919d-dd32b3cf84d5 time: 745 result: {&quot;success&quot;:true,&quot;data&quot;:&quot;root&quot;}
  2019-07-25 14:06:45.711 INFO  uid: 540c4810-aeb7-11e9-919d-dd32b3cf84d5 time: 3326 result: {&quot;success&quot;:true,&quot;data&quot;:&quot;root&quot;}
  .
  .
  .
  2019-07-25 14:06:46.835 INFO  uid: 541d5f10-aeb7-11e9-919d-dd32b3cf84d5 time: 4338 result: {&quot;success&quot;:true,&quot;data&quot;:&quot;root&quot;}
  </code></pre>
  <ol start="2">
  <li>Logs of command <code>strace -t docker exec ubuntu whoami</code>:</li>
  </ol>
  <p><img src="https://i.stack.imgur.com/PUoyh.png" alt="Logs of command strace Ubuntu 16" /></p>
  <ol start="3">
  <li>Result of <code>perf top --sort comm,dso</code>:</li>
  </ol>
  <p><img src="https://i.stack.imgur.com/Yja5k.png" alt="Result of perf command Ubuntu 16" /></p>
  <hr />
  <p><strong>Ubuntu 18 machine Data:</strong></p>
  <p><strong>1. Data of time taken in execution:</strong></p>
  <pre><code>2019-07-25 14:07:32.559 INFO  uid: 715a6af0-aeb7-11e9-a5a9-2fffd4e800d1 time: 1008 result: {&quot;success&quot;:true,&quot;data&quot;:&quot;root&quot;}
  2019-07-25 14:07:32.941 INFO  uid: 7178c860-aeb7-11e9-a5a9-2fffd4e800d1 time: 1191 result: {&quot;success&quot;:true,&quot;data&quot;:&quot;root&quot;}
  2019-07-25 14:07:40.363 INFO  uid: 71767e70-aeb7-11e9-a5a9-2fffd4e800d1 time: 8628 result: {&quot;success&quot;:true,&quot;data&quot;:&quot;root&quot;}
  .
  .
  .
  2019-07-25 14:07:41.970 INFO  uid: 718af0d0-aeb7-11e9-a5a9-2fffd4e800d1 time: 10101 result: {&quot;success&quot;:true,&quot;data&quot;:&quot;root&quot;}
  
  </code></pre>
  <ol start="2">
  <li>Logs of command <code>strace -t docker exec ubuntu whoami</code>:</li>
  </ol>
  <p><img src="https://i.stack.imgur.com/W0cz4.png" alt="Logs of command strace Ubuntu 18" /></p>
  <ol start="3">
  <li>result of <code>perf top --sort comm,dso</code>:</li>
  </ol>
  <p><img src="https://i.stack.imgur.com/Kyi5f.png" alt="Result of perf command Ubuntu 18" /></p>
  <p>So, I need help in debugging what is wrong with Docker on the Ubuntu 18 machine. Or if there is any limitation with Docker on Ubuntu 18 or maybe some machine or Docker configuration issue.</p>