title: How to keep executable code in memory even under memory pressure ? in Linux
body: <p>The goal here is to keep every running process' executable code in memory during memory pressure, in Linux.<br>
  In Linux, I am able to instantly (1 sec) cause high memory pressure and trigger the OOM-killer by
  <code>stress --vm-bytes $(awk '/MemAvailable/{printf "%d\n", $2 + 4000;}' &lt; /proc/meminfo)k --vm-keep -m 4 --timeout 10s</code> (code from <a href="https://unix.stackexchange.com/q/423261/306023">here</a>)
  with 24000MB max RAM inside a Qubes OS R4.0 Fedora 28 AppVM. <strong>EDIT4:</strong> Perhaps relevant, and yet I forgot to mention, is the fact that I've no swap enabled (ie. <code>CONFIG_SWAP</code> is not set)  </p>
  
  <p>dmesg reports:  </p>
  
  <pre><code>[  867.746593] Mem-Info:
  [  867.746607] active_anon:1390927 inactive_anon:4670 isolated_anon:0
                  active_file:94 inactive_file:72 isolated_file:0
                  unevictable:13868 dirty:0 writeback:0 unstable:0
                  slab_reclaimable:5906 slab_unreclaimable:12919
                  mapped:1335 shmem:4805 pagetables:5126 bounce:0
                  free:40680 free_pcp:978 free_cma:0
  </code></pre>
  
  <p>The interesting parts are <code>active_file:94 inactive_file:72</code> they are in kilobytes and are very low.  </p>
  
  <p>The problem here is that, during that period of memory pressure, executable code is being re-read from disk causing disk thrashing which leads to <a href="https://unix.stackexchange.com/a/458855/306023">frozen OS</a>. (but in the above case it only happens for less than 1 second)  </p>
  
  <p>I see an interesting code in kernel <a href="https://elixir.bootlin.com/linux/v4.18.5/source/mm/vmscan.c#L1960" rel="noreferrer"><code>mm/vmscan.c</code></a><a href="https://elixir.bootlin.com/linux/v4.18.5/source/mm/vmscan.c#L881" rel="noreferrer">:</a></p>
  
  <pre><code>        if (page_referenced(page, 0, sc-&gt;target_mem_cgroup,
                              &amp;vm_flags)) {
                  nr_rotated += hpage_nr_pages(page);
                  /*
                   * Identify referenced, file-backed active pages and
                   * give them one more trip around the active list. So
                   * that executable code get better chances to stay in
                   * memory under moderate memory pressure.  Anon pages
                   * are not likely to be evicted by use-once streaming
                   * IO, plus JVM can create lots of anon VM_EXEC pages,
                   * so we ignore them here.
                   */
                  if ((vm_flags &amp; VM_EXEC) &amp;&amp; page_is_file_cache(page)) {
                          list_add(&amp;page-&gt;lru, &amp;l_active);
                          continue;
                  }
          }
  </code></pre>
  
  <p>I'm think that if someone could point out how to change this so that instead of <code>give them one more trip around the active list</code> we get it to <code>give them infinite trips around the active list</code>, then job should be done. Or maybe there's some other way?  </p>
  
  <p>I can patch and test custom kernel. I just don't have the know-how as to what to change in the code in order to always keep active executable code in memory(which in effect, I believe, would avoid disk thrashing).</p>
  
  <p><strong>EDIT:</strong> Here's what I got working so far (applied on top of kernel 4.18.5):  </p>
  
  <pre><code>diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
  index 32699b2..7636498 100644
  --- a/include/linux/mmzone.h
  +++ b/include/linux/mmzone.h
  @@ -208,7 +208,7 @@ enum lru_list {
  
   #define for_each_lru(lru) for (lru = 0; lru &lt; NR_LRU_LISTS; lru++)
  
  -#define for_each_evictable_lru(lru) for (lru = 0; lru &lt;= LRU_ACTIVE_FILE; lru++)
  +#define for_each_evictable_lru(lru) for (lru = 0; lru &lt;= LRU_INACTIVE_FILE; lru++)
  
   static inline int is_file_lru(enum lru_list lru)
   {
  diff --git a/mm/vmscan.c b/mm/vmscan.c
  index 03822f8..1f3ffb5 100644
  --- a/mm/vmscan.c
  +++ b/mm/vmscan.c
  @@ -2234,7 +2234,7 @@ static void get_scan_count(struct lruvec *lruvec, struct mem_cgroup *memcg,
  
      anon  = lruvec_lru_size(lruvec, LRU_ACTIVE_ANON, MAX_NR_ZONES) +
          lruvec_lru_size(lruvec, LRU_INACTIVE_ANON, MAX_NR_ZONES);
  -   file  = lruvec_lru_size(lruvec, LRU_ACTIVE_FILE, MAX_NR_ZONES) +
  +   file  = //lruvec_lru_size(lruvec, LRU_ACTIVE_FILE, MAX_NR_ZONES) +
          lruvec_lru_size(lruvec, LRU_INACTIVE_FILE, MAX_NR_ZONES);
  
      spin_lock_irq(&amp;pgdat-&gt;lru_lock);
  @@ -2345,7 +2345,7 @@ static void shrink_node_memcg(struct pglist_data *pgdat, struct mem_cgroup *memc
               sc-&gt;priority == DEF_PRIORITY);
  
      blk_start_plug(&amp;plug);
  -   while (nr[LRU_INACTIVE_ANON] || nr[LRU_ACTIVE_FILE] ||
  +   while (nr[LRU_INACTIVE_ANON] || //nr[LRU_ACTIVE_FILE] ||
                      nr[LRU_INACTIVE_FILE]) {
          unsigned long nr_anon, nr_file, percentage;
          unsigned long nr_scanned;
  @@ -2372,7 +2372,8 @@ static void shrink_node_memcg(struct pglist_data *pgdat, struct mem_cgroup *memc
           * stop reclaiming one LRU and reduce the amount scanning
           * proportional to the original scan target.
           */
  -       nr_file = nr[LRU_INACTIVE_FILE] + nr[LRU_ACTIVE_FILE];
  +       nr_file = nr[LRU_INACTIVE_FILE] //+ nr[LRU_ACTIVE_FILE]
  +           ;
          nr_anon = nr[LRU_INACTIVE_ANON] + nr[LRU_ACTIVE_ANON];
  
          /*
  @@ -2391,7 +2392,8 @@ static void shrink_node_memcg(struct pglist_data *pgdat, struct mem_cgroup *memc
              percentage = nr_anon * 100 / scan_target;
          } else {
              unsigned long scan_target = targets[LRU_INACTIVE_FILE] +
  -                       targets[LRU_ACTIVE_FILE] + 1;
  +                       //targets[LRU_ACTIVE_FILE] + 
  +                       1;
              lru = LRU_FILE;
              percentage = nr_file * 100 / scan_target;
          }
  </code></pre>
  
  <p>Also seen <a href="https://github.com/constantoverride/qubes-linux-kernel/blob/acd686a5019c7ab6ec10dc457bdee4830e2d741f/patches.addon/le9b.patch" rel="noreferrer">here</a> on github because in the above code, tabs got transformed into spaces! (<a href="https://launchpadlibrarian.net/386196358/le9b.patch" rel="noreferrer">mirror1</a>, <a href="https://groups.google.com/group/qubes-users/attach/3373eab85a4fe/le9b.patch?part=0.1&amp;authuser=0" rel="noreferrer">mirror2</a>)<br>
  I've tested the above patch(on 4000MB max RAM now, yes 20G less than before!) even with a Firefox compilation that was known to disk thrash the OS into a permanent freeze, and it does not happen anymore (oom-killer is almost instantly killing the offending process(es)), also with the above <code>stress</code> command which now yields:  </p>
  
  <pre><code>[  745.830511] Mem-Info:
  [  745.830521] active_anon:855546 inactive_anon:20453 isolated_anon:0
                  active_file:26925 inactive_file:76 isolated_file:0
                  unevictable:10652 dirty:0 writeback:0 unstable:0
                  slab_reclaimable:26975 slab_unreclaimable:13525
                  mapped:24238 shmem:20456 pagetables:4028 bounce:0
                  free:14935 free_pcp:177 free_cma:0
  </code></pre>
  
  <p>That's <code>active_file:26925 inactive_file:76</code>, almost 27 megs of active file...<br>
  So, I don't know how good this is. Am I keeping all active files instead of just executable files in memory ? During firefox compilation I've had like 500meg of <code>Active(file)</code>(<strong>EDIT2:</strong> but that's according to: <code>cat /proc/meminfo|grep -F -- 'Active(file)'</code> which shows different value than the above <code>active_file:</code> from dmesg!!!) which makes me doubt it was only exes/libs...<br>
   Maybe someone can suggest how to keep ONLY executable code ?(if that's not what's already happening)<br>
  Thoughts?  </p>
  
  <p><strong>EDIT3:</strong> with the above patch, it seems perhaps necessary to (periodically?) run <code>sudo sysctl vm.drop_caches=1</code> to free some stale memory(?), so that if I call <code>stress</code> after a firefox compilation I get: <code>active_file:142281 inactive_file:0 isolated_file:0</code> (142megs) then drop file caches (another way: <code>echo 1|sudo tee /proc/sys/vm/drop_caches</code>) then run <code>stress</code> again, I get: <code>active_file:22233 inactive_file:160 isolated_file:0</code> (22megs) - I am unsure...  </p>
  
  <p>Results without the above patch: <a href="https://gist.github.com/constantoverride/84eba764f487049ed642eb2111a20830#gistcomment-2693256" rel="noreferrer">here</a><br>
  Results with the above patch: <a href="https://gist.github.com/constantoverride/84eba764f487049ed642eb2111a20830#gistcomment-2693349" rel="noreferrer">here</a>  </p>