title: BeautifulSoup - remove children but keep their contents
body: <p>I am creating a web <strong>scraper</strong>, and I have issue fetching the pages whose are most likely generated, like this:</p>
  <pre><code>&lt;html&gt;
  &lt;body&gt;
  &lt;div &gt;
  &lt;code&gt;
      &lt;p class=&quot;nt&quot;&gt;&amp;lt;my-component&lt;/p&gt; &lt;p class=&quot;na&quot;&gt;v-bind:prop1=&lt;/p&gt;&lt;p class=&quot;s&quot;&gt;&quot;parentValue&quot;&lt;/p&gt;&lt;p class=&quot;nt&quot;&gt;&amp;gt;&amp;lt;/my-component&amp;gt;&lt;/p&gt;
      &lt;p class=&quot;c&quot;&gt;&amp;lt;!-- Or more succinctly, --&amp;gt;&lt;/p&gt;
      &lt;p class=&quot;nt&quot;&gt;&amp;lt;my-component&lt;/p&gt; &lt;p class=&quot;na&quot;&gt;:prop1=&lt;/p&gt;&lt;p class=&quot;s&quot;&gt;&quot;parentValue&quot;&lt;/p&gt;&lt;p class=&quot;nt&quot;&gt;&amp;gt;&amp;lt;/my-component&amp;gt;&lt;/p&gt;
  &lt;/code&gt;
  &lt;/div&gt;
  &lt;div&gt;
  &lt;code&gt;
      &lt;p class=&quot;nt&quot;&gt;&amp;lt;my-component&lt;/p&gt; &lt;p class=&quot;na&quot;&gt;v-on:myEvent=&lt;/p&gt;&lt;p class=&quot;s&quot;&gt;&quot;parentHandler&quot;&lt;/p&gt;&lt;p class=&quot;nt&quot;&gt;&amp;gt;&amp;lt;/my-component&amp;gt;&lt;/p&gt;
      &lt;p class=&quot;c&quot;&gt;&amp;lt;!-- Or more succinctly, --&amp;gt;&lt;/p&gt;
      &lt;p class=&quot;nt&quot;&gt;&amp;lt;my-component&lt;/p&gt; &lt;p class=&quot;err&quot;&gt;@&lt;/p&gt;&lt;p class=&quot;na&quot;&gt;myEvent=&lt;/p&gt;&lt;p class=&quot;s&quot;&gt;&quot;parentHandler&quot;&lt;/p&gt;&lt;p class=&quot;nt&quot;&gt;&amp;gt;&amp;lt;/my-component&amp;gt;&lt;/p&gt;
  
  &lt;/code&gt;
  &lt;/div&gt;
  &lt;/body&gt;
  &lt;/html&gt;
  </code></pre>
  <p>Most important is content between <code>code</code>  tags. Plan is to extract text between <p> tags (or , remove those <p> tags and keep the rest of the DOM as it is.</p>
  <p>So I need output like this:</p>
  <pre><code>&lt;html&gt;
  &lt;body&gt;
  &lt;div &gt;
  &lt;code&gt;
    text text and more text
  &lt;/code&gt;
  &lt;/div&gt;
  &lt;/html&gt;
  &lt;/body&gt;
  </code></pre>
  <p>My tries as following..</p>
  <pre><code>from bs4 import BeautifulSoup
  
  
  bs = BeautifulSoup(payload, 'lxml')
  
  with open('/tmp/out.html', 'w+') as f:
      for t in bs.find_all():
          
          for q in t.find_all('code'):
              # print(t.text, t.next_sibling)
              f.write(q.text)
  
  
  </code></pre>
  <p>but this doesn't give good results.. From what I learned, bs main purpose is to extract elements, so that is reason why I tried recreating the dom in another file.</p>
  <p>Thanks!</p>