user3554: I'm using the Adafruit SD shield with a Mega board and wrote a simple program to log data.  I am outputting a constant stream of data at 115200 baudrate from another Arduino to the Mega.  The data output is mostly good, but there are places where some data are skipped.  These skipped data usually occurs around a file flush.  I've tried playing around with the flush frequency and the soft buffer size, but to no avail.  I'm not sure what these hiccups are caused by and would like some help.
void loop(void)
{
  while (Serial1.available())
  {
    softBuffer[i] = Serial1.read();
    i++;
  }

  if (i >= SOFT_BUFFER_SIZE)
  {
    for (x = 0; x < SOFT_BUFFER_SIZE; x++)
    {
      logfile.print(softBuffer[x]);
    }

    logfile.flush();


    i = 0;
  }
}


user1566: The SD card library works by filling up a 512 byte buffer. Once full, the data is written to disk.
The flush command updates the directory listing with the correct size for the file and also updates the file allocation table. It only needs to be done once, just before you pull the card out.
While flush is doing its thing, the internal serial buffer of the Arduino is filling up with new bytes. The internal buffer is 64 bytes long so if flush takes too long, the internal buffer fills up and you lose characters. You could edit HardwareSerial.cpp in the Arduino program directory to make it say 256 bytes.
Because of this softBuffer is not needed. Simply write straight to the card and let the internal buffer handle the buffering. See https://github.com/arduino/Arduino/blob/master/hardware/arduino/cores/arduino/HardwareSerial.cpp for the Serial code.
unsigned long bytesWritten = 0;
void loop(void)
{
  while (Serial1.available())
  {
    logfile.write(Serial1.read());
    bytesWritten++;
    if(bytesWritten > BYTES_PER_FLUSH) logfile.flush();
  }
}


3554: <p>I'm using the Adafruit SD shield with a Mega board and wrote a simple program to log data.  I am outputting a constant stream of data at 115200 baudrate from another Arduino to the Mega.  The data output is mostly good, but there are places where some data are skipped.  These skipped data usually occurs around a file flush.  I've tried playing around with the flush frequency and the soft buffer size, but to no avail.  I'm not sure what these hiccups are caused by and would like some help.</p>

<pre><code>void loop(void)
{
  while (Serial1.available())
  {
    softBuffer[i] = Serial1.read();
    i++;
  }

  if (i &gt;= SOFT_BUFFER_SIZE)
  {
    for (x = 0; x &lt; SOFT_BUFFER_SIZE; x++)
    {
      logfile.print(softBuffer[x]);
    }

    logfile.flush();


    i = 0;
  }
}
</code></pre>
 1566: <p>The SD card library works by filling up a 512 byte buffer. Once full, the data is written to disk.</p>

<p>The <code>flush</code> command updates the directory listing with the correct size for the file and also updates the file allocation table. It only needs to be done once, just before you pull the card out.</p>

<p>While <code>flush</code> is doing its thing, the internal serial buffer of the Arduino is filling up with new bytes. The internal buffer is 64 bytes long so if <code>flush</code> takes too long, the internal buffer fills up and you lose characters. You could edit HardwareSerial.cpp in the Arduino program directory to make it say 256 bytes.</p>

<p>Because of this <code>softBuffer</code> is not needed. Simply write straight to the card and let the internal buffer handle the buffering. See <a href="https://github.com/arduino/Arduino/blob/master/hardware/arduino/cores/arduino/HardwareSerial.cpp" rel="nofollow">https://github.com/arduino/Arduino/blob/master/hardware/arduino/cores/arduino/HardwareSerial.cpp</a> for the Serial code.</p>

<pre><code>unsigned long bytesWritten = 0;
void loop(void)
{
  while (Serial1.available())
  {
    logfile.write(Serial1.read());
    bytesWritten++;
    if(bytesWritten &gt; BYTES_PER_FLUSH) logfile.flush();
  }
}
</code></pre>
