title: How to break a corpus into paragraphs using custom delimiters
body: <p>I am scraping the New york Times webpages to do some natural language processing on it, I want to split the webpage into paragraphs when using corpus in order to do frequency counts on words that appear in paragraphs which also contain key words or phrases .</p>
  <p>The below works with sentences but the paragraphs are donated by a • in NYT, so I need to replace this into how corpus reads paragraphs - anyone got any idea's? I have tried gsub(&quot;•&quot;,&quot;/n&quot;,...) and gsub(&quot;•&quot;,&quot;/r/n&quot;) but this didn't work.</p>
  <p>If anyone knows how to do this all in tm corpus's rather than having to switch between quanteda and TM that would save some code.</p>
  <pre><code> website&lt;-read_html(&quot;https://www.nytimes.com/2017/01/03/briefing/asia-australia-briefing.html&quot;) #Read URL
       
  
  
    #Obtain any text with the paragraph Html deliminator 
    text&lt;-website%&gt;%
      html_nodes(&quot;p&quot;) %&gt;%
      html_text() %&gt;% as.character()
    
    #Collapse the string as it is currently text[1]=para1 and text[2]= para 2
    text&lt;- str_c(text,collapse=&quot; &quot;)
  
  
  data_corpus_para &lt;- 
    corpus_reshape(corpus((text),to=&quot;paragraphs&quot;))
  
  
  data_corpus_para &lt;-tolower(data_corpus_para )
  
  
  containstarget &lt;- 
    stringr::str_detect(texts(data_corpus_para ), &quot;pull out of peace talks&quot;) #Random string in only one of the paragraphs to proof concept
  
  #Filter for the para's that only contain the sentence above
  data_corpus_para &lt;- 
    corpus_subset(data_corpus_para , containstarget)                 
  
  data_corpus_para &lt;-corpus_reshape(data_corpus_para , to = &quot;documents&quot;)
  
  
  #There are quanteda corpus and TM Corpuses. And so I have to convert to a dataframe and then make back into a vcorupus.. this is very messy
  
  data_corpus_para &lt;-quanteda::convert(data_corpus_para )
  data_corpus_para_VCorpus&lt;-tm::VCorpus(tm::VectorSource(data_corpus_para$text))
  
  dt.dtm = tm::DocumentTermMatrix(data_corpus_para_VCorpus)
  tm::findFreqTerms(dt.dtm, 1)
  </code></pre>