title: Bash: Add empty column(s) to CSV with unknown positions
body: <p>currently we receive multiple large CSV files, which we need to insert/update into our database.
  The schema of our database does <strong>not</strong> change.
  We only need specific columns in a specfic order which are stated in a header-database. These could change at any given point. The CSV files we receive can also change in order at any time as well.</p>
  <p>So what I did is piping the required columns from the header-DB into this script (<em>$TEMP_FILE</em>) and extract my required columns from the received CSV (<em>$REC_CSV</em>).</p>
  <p>This is working fine thus far:</p>
  <pre class="lang-sh prettyprint-override"><code>awk 'NR==FNR{
          Clm=Clm (Clm?&quot;|&quot;:&quot;&quot;)$1
          next
      }
  
      FNR==1{
          for (i=1;i&lt;=NF;i++) 
          {
              if (match($i,Clm)) 
              {
                  Ar[++n]=i
              }
          }
      }
  
      FNR&gt;=2{
          for (i=1; i&lt;=n; i++)
          {
              printf (i&lt;n)? $(Ar[i]) FS : $(Ar[i])
          }
          printf &quot;\n&quot;
      }' FS=&quot;|&quot; ${TEMP_FILE} $REC_CSV &gt;&gt; $MY_NEW_CSV_WITH_HEADER
  </code></pre>
  <p><strong>TEMP_FILE</strong> (From Header DB):</p>
  <pre><code>id|anotherId|hahahaIdontExist|timestamp
  </code></pre>
  <p><strong>Input-CSV:</strong></p>
  <pre><code>id|timestamp|anotherId|thisId|andThisId|andAnotherId
  1|2:00|34|44|44|41
  2|2:00|34|45|44|41
  3|3:00|35|46|44|41
  </code></pre>
  <p><strong>Output:</strong></p>
  <pre><code>id|anotherId|timestamp
  1|34|2:00
  2|34|2:00
  3|35|3:00
  </code></pre>
  <p>But here is the problem: <strong>hahahaIdontExist</strong> is not in the output.</p>
  <p>As it is vaguely implied this varaible was not supposed to exist in the first place, but needs to be in the output as well with empty FS.</p>
  <p><strong>Desired Output:</strong></p>
  <pre><code>id|anotherId|hahahaIdontExist|timestamp
  1|34||2:00
  2|34||2:00
  3|35||3:00
  </code></pre>
  <p>As I believe it is easier (and safer) to keep the first script (And I tried 1000st of drafts) do you have a suggestion on how to fill the non-existant columns into the output?</p>
  <p>Best regards</p>