<p>I've update my Airbyte image from <code>0.35.2-alpha</code> to <code>0.35.37-alpha</code>.
[running in kubernetes]</p>
<p>When the system rolled out the db pod wouldn't terminate and I [a terrible mistake] deleted the pod.
When it came back up, I get an error -</p>
<pre><code>PostgreSQL Database directory appears to contain a database; Skipping initialization

2022-02-24 20:19:44.065 UTC [1] LOG:  starting PostgreSQL 13.6 on x86_64-pc-linux-musl, compiled by gcc (Alpine 10.3.1_git20211027) 10.3.1 20211027, 64-bit
2022-02-24 20:19:44.065 UTC [1] LOG:  listening on IPv4 address &quot;0.0.0.0&quot;, port 5432
2022-02-24 20:19:44.065 UTC [1] LOG:  listening on IPv6 address &quot;::&quot;, port 5432
2022-02-24 20:19:44.071 UTC [1] LOG:  listening on Unix socket &quot;/var/run/postgresql/.s.PGSQL.5432&quot;
2022-02-24 20:19:44.079 UTC [21] LOG:  database system was shut down at 2022-02-24 20:12:55 UTC
2022-02-24 20:19:44.079 UTC [21] LOG:  invalid resource manager ID in primary checkpoint record
2022-02-24 20:19:44.079 UTC [21] PANIC:  could not locate a valid checkpoint record
2022-02-24 20:19:44.530 UTC [1] LOG:  startup process (PID 21) was terminated by signal 6: Aborted
2022-02-24 20:19:44.530 UTC [1] LOG:  aborting startup due to startup process failure
2022-02-24 20:19:44.566 UTC [1] LOG:  database system is shut down
</code></pre>
<p>Pretty sure the WAL file is corrupted, but I'm not sure how to fix this.</p>

<p><em>Warning</em> - there is a potential for data loss</p>
<p>This is a test system, so I wasn't concerned with keeping the latest transactions, and had no backup.</p>
<p>First I overrode the container command to keep the container running but not try to start postgres.</p>
<pre><code>...
    spec:
      containers:
        - name: airbyte-db-container
          image: airbyte/db
          command: [&quot;sh&quot;]
          args: [&quot;-c&quot;, &quot;while true; do echo $(date -u) &gt;&gt; /tmp/run.log; sleep 5; done&quot;]
...
</code></pre>
<p>And spawned a shell on the pod -</p>
<pre><code>kubectl exec -it -n airbyte airbyte-db-xxxx -- sh
</code></pre>
<p>Run <code>pg_reset_wal</code></p>
<pre><code># dry-run first
pg_resetwal --dry-run /var/lib/postgresql/data/pgdata
</code></pre>
<p>Success!</p>
<pre><code>pg_resetwal /var/lib/postgresql/data/pgdata
Write-ahead log reset
</code></pre>
<p>Then removed the temp command in the container, and postgres started up correctly!</p>
