title: CURL missing data from write function
body: <p>I am using the <em>CURLOPT_WRITEFUNCTION</em> CURL option to get website contents, along with the <em>CURLOPT_WRITEDATA</em> option to specify my buffer. According to the documentation when using C++, I must define a static class member function with the following signature for the write callback, otherwise, I will get a segmentation fault:</p>
  <pre><code>size_t write_callback(char *ptr, size_t size, size_t nmemb, void *userdata)
  </code></pre>
  <p>I know that, according to the documentation, that the maximum write size is defined in <em>CURL_MAX_WRITE_SIZE</em>. But when I am simply printing the contents of <em>ptr</em>, I get less content that specified in <em>CURL_MAX_WRITE_SIZE</em>. That is, I get roughly 1/3 of the website contents.</p>
  <p>When I don't add this <em>CURLOPT_WRITEFUNCTION</em> option, CURL will by default use <em>fwrite()</em> to <em>stdout</em> to print the write data, where I can see all the contents of the webpage.</p>
  <p>I am therefore asking; why is it, that I can't get the entire write data from the web page?</p>
  <p>Below is a minimal reproducible example:</p>
  <pre><code>#include &lt;curl/curl.h&gt;
  #include &lt;string&gt;
  #include &lt;iostream&gt;
  
  class curl_write
  {
  public:
      static size_t write_data(void* ptr, size_t size, size_t nmemb, void* buffer)
      {
          ((std::string*) buffer)-&gt;append((char*) ptr, nmemb);
          return nmemb;
      }
  };
  
  int main()
  {
      std::string buffer;
  
      CURL* handle = curl_easy_init();
      curl_easy_setopt(handle, CURLOPT_NOPROGRESS, 1);
      curl_easy_setopt(handle, CURLOPT_NOSIGNAL, 1);
      curl_easy_setopt(handle, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
      curl_easy_setopt(handle, CURLOPT_PROXYAUTH, CURLAUTH_ANY);
      curl_easy_setopt(handle, CURLOPT_URL, &quot;https://ensk.dict.cc/&quot;);
      curl_easy_setopt(handle, CURLOPT_WRITEFUNCTION, 
      curl_write::write_data);
      curl_easy_setopt(handle, CURLOPT_WRITEDATA, &amp;buffer);
      curl_easy_perform(handle);
      curl_easy_cleanup(handle);
  
      std::cout &lt;&lt; buffer &lt;&lt; std::endl;
  
      return 0;
  }
  </code></pre>
  <p>This is the end of the expected output:</p>
  <pre><code>&lt;br&gt;
  &lt;div class=&quot;hl3&quot; style=&quot;margin:25px 0px 3px 4px;&quot;&gt;News &amp;amp; History (&lt;a href=&quot;//www.dict.cc/news_en.rss.php&quot; style=&quot;color:#36c&quot;&gt;RSS&lt;/a&gt;)&lt;/div&gt;
  &lt;/div&gt;&lt;div style=&quot;width:728px; padding:0px 1px&quot;&gt;
  &lt;div class=&quot;news&quot;&gt;&lt;em&gt;2020-08-11:&lt;/em&gt; Keyboard handling improved for the &lt;a href=&quot;https://m.dict.cc/&quot;&gt;mobile dict.cc website&lt;/a&gt; when used on a PC/Mac. Just type your keyword, regardless of where the cursor is. Pressing the Tab key (followed by Enter) selects from the suggestions in most browsers.&lt;/div&gt;
  &lt;div class=&quot;news&quot;&gt;&lt;em&gt;2020-07-07:&lt;/em&gt; The new vocabulary trainer (only web, not yet in the app) now displays a message if there are several possible answers. The message can be clicked to show the translations currently not being asked for. The trainer now also accepts any of the other answers and moves the corresponding card to the next box, but it will keep asking for the missing answer. To get help finding out which term is asked for, it's still possible to press the space bar for the next correct character.&lt;/div&gt;
  &lt;div class=&quot;news&quot;&gt;&lt;em&gt;2020-06-29:&lt;/em&gt; The mobile website now offers audio buttons for both languages, saving the additional click on terms for re-translation.&lt;/div&gt;
  &lt;div class=&quot;news&quot;&gt;&lt;em&gt;2020-03-30:&lt;/em&gt; The new vocabulary trainer is now officially released! The old version stays available for the time being and can be used via the &quot;v1&quot; or &quot;old&quot; links and menu items.&lt;/div&gt;
  &lt;div class=&quot;news&quot;&gt;» &lt;a href=&quot;//www.dict.cc/german-english-dictionary.php&quot;&gt;&lt;b&gt;English: News History&lt;/b&gt;&lt;/a&gt;&lt;br&gt;
  » &lt;a href=&quot;//www.dict.cc/deutsch-englisch-woerterbuch.php&quot;&gt;&lt;b&gt;Deutsch: Verlauf der Mitteilungen&lt;/b&gt;&lt;/a&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;table border=&quot;0&quot; cellpadding=&quot;2&quot; cellspacing=&quot;0&quot; width=&quot;728&quot; style=&quot;margin-left:1px;height:22px&quot;&gt; 
          &lt;tbody&gt;&lt;tr&gt;&lt;td class=&quot;td1&quot;&gt;&lt;a href=&quot;#top&quot; name=&quot;bottom&quot; style=&quot;color:white;&quot;&gt;back to top&lt;/a&gt; | &lt;a href=&quot;https://ensk.dict.cc/&quot; style=&quot;color:white;&quot;&gt;home&lt;/a&gt;&lt;/td&gt;&lt;td class=&quot;td1&quot; align=&quot;right&quot;&gt;© 2002 - 2020 &lt;a href=&quot;https://www.hemetsberger.com&quot; style=&quot;color:white;&quot;&gt;Paul Hemetsberger&lt;/a&gt; | &lt;a href=&quot;//www.dict.cc/?s=about%3A#impressum&quot; style=&quot;color:white;&quot;&gt;contact / privacy&lt;/a&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;&lt;img src=&quot;https://www4.dict.cc/img/hr4.gif&quot; width=&quot;728&quot; height=&quot;4&quot; alt=&quot;&quot; style=&quot;margin:0px 0px 0px 1px&quot;&gt;&lt;div class=&quot;aftertable&quot;&gt;Slovak-English online dictionary (Anglicko-slovenský slovník) developed to help you share your knowledge with others. &lt;a href=&quot;//www.dict.cc/?s=about%3A&quot;&gt;More information&lt;/a&gt;&lt;br&gt;Links to this dictionary or to single translations are very welcome! &lt;a href=&quot;//www.dict.cc/?s=about%3Afaq&quot;&gt;Questions and Answers&lt;/a&gt;&lt;/div&gt;&lt;script type=&quot;text/javascript&quot;&gt;add_inputhasfocus_changer();&lt;/script&gt;&lt;script type=&quot;text/javascript&quot;&gt;var my_vocab_lists = [];
  &lt;/script&gt;&lt;div id=&quot;speechlayer&quot; style=&quot;width:1px;height:1px&quot;&gt;&lt;/div&gt;&lt;div id=&quot;recthomebot&quot; style=&quot;position: absolute; left: 0px; top: 0px;&quot;&gt;&lt;div style=&quot;text-align:center;width:300px&quot;&gt;&lt;span class=&quot;noline&quot; style=&quot;font-size:10px;color:#999&quot;&gt;Advertisement&lt;/span&gt;&lt;/div&gt;&lt;!-- 
  headerbidding part2
  --&gt;&lt;div id=&quot;snhb-detail_rectangle-0&quot;&gt;&lt;/div&gt;&lt;script type=&quot;text/javascript&quot;&gt; snhb.queue.push( function(){ snhb.startAuction(['detail_rectangle']); }); &lt;/script&gt;&lt;/div&gt;&lt;script type=&quot;text/javascript&quot;&gt;
              if (document.getElementById(&quot;recthome&quot;)) {
              document.getElementById(&quot;recthomebot&quot;).style.position=&quot;absolute&quot;;
              document.getElementById(&quot;recthomebot&quot;).style.left=getX(document.getElementById(&quot;recthome&quot;))+&quot;px&quot;;
              document.getElementById(&quot;recthomebot&quot;).style.top=getY(document.getElementById(&quot;recthome&quot;))+&quot;px&quot;;
              }
              &lt;/script&gt;&lt;/div&gt;&lt;iframe name=&quot;__tcfapiLocator&quot; style=&quot;display: none;&quot;&gt;&lt;/iframe&gt;&lt;iframe name=&quot;__uspapiLocator&quot; style=&quot;display: none;&quot;&gt;&lt;/iframe&gt;&lt;iframe name=&quot;__cmpLocator&quot; style=&quot;display: none;&quot;&gt;&lt;/iframe&gt;&lt;/body&gt;&lt;/html&gt;
  </code></pre>
  <p>The difference from the actual output is that the last 6.5 lines are missing.</p>
  <p>SOLUTION</p>
  <p>The solution was to exchange the carriage return characters with the space character.</p>