user33548: This is my first post to Stack Exchange so please forgive any formatting errors.
I am conducting an experiment that has 7 total sensors: 5 thermocouples, 2 humidity sensors and 1 pressure sensor. I am reading their signals via Arduino Mega analog input ports. I chose to use Python to save and compile the data in a nice csv file, therefore I am using Pyserial. 
My problem is that I cannot go above 1 sample/second. As you can see in my code, I have a 1 second delay in Arduino and Python. I have ran successful calibration tests with my thermocouples, so I know at that rate Python and Arduino are communicating well. Anything faster than 1 sample/second (i.e. I change my delays to .5 or .25 seconds), a timing error occurs and my data is "cut up" and sent in odd chunks instead of the standard layout I programmed. 
Additionally, I tried to increase the baud rate from 9600 and anything above that will send "garbage" ASCII characters (e.g. little blank blocks and odd characters) to the Serial Comm. I changed the baud rate in Arduino, Python and the COM port in my devices manager settings to no avail. 
I would really like to increase my sampling rate to increase my signal-to-noise ratio and I believe it's good practice as a metrologist to have a higher sampling rate than 1 samp/sec. I am sending ~1072 bits each time I write to the Serial Comm (if I'm counting right??) so I should have enough room to increase the amount of bits to send per second (around 3 times what I am currently sending).
In summary, my two main issues are: if I increase the baud rate, I get garbage in my Serial Comm and if I try to increase my "sampling rate" by decreasing my delays, timing issues appear and my data is split up erroneously. Should I be using a different function, like Serial.flush()?
I greatly appreciate any feedback, thank you for your time!
// Data acquisition
// updated  March 30, 2017


//Note: This version is for TEMPERATURE, HUMIDITY and PRESSURE data         
acquisition ALONE. No motor/conveyor control.

//***GLOBAL VARIABLES***//

float lfactor = 125.0; 
float afactor = 1.15;

/////////////////////////

//Standard setup with analogReference to 5Volts. 

void setup() {

Serial.begin(9600);  
}

void loop() {


//******THERMOCOUPLES*******//

//get voltage reading
// [code]; TC # - Location

float tcv1 = analogRead(A4) * ( 5.0 / 1024.0 ); //TC 1 
float tcv2 = analogRead(A7) * ( 5.0 / 1024.0 ); //TC 2 
float tcv3 = analogRead(A8) * ( 5.0 / 1024.0 ); //TC 3 
float tcv4 = analogRead(A9) * ( 5.0 / 1024.0 ); //TC 4 
float tcv5 = analogRead(A10) * ( 5.0 / 1024.0 ); //TC 5

//convert to temperature using custom equation based on 5V reference
float tc1 = ( 188.7755 * tcv1 ) - 245.3959;  
float tc2 = ( 188.7755 * tcv2 ) - 245.3959;  
float tc3 = ( 188.7755 * tcv3 ) - 245.3959;  
float tc4 = ( 188.7755 * tcv4 ) - 245.3959;  
float tc5 = ( 188.7755 * tcv5 ) - 245.3959;    

//******HUMIDITY SENSORS********//

float hs1 = ((( analogRead(A13) * ( 5.0 / 1024.0) ) / 5.0 ) * 100.0 ); //HS 
1
float hs2 = ((( analogRead(A15) * ( 5.0 / 1024.0) ) / 5.0 ) * 100.0 ); //HS 
2

//******PRESSURE SENSORS********//

//obtain voltage signal from Sensirion sensor
float pvolt = analogRead(A0) * (4.995 / 1024.0); 

//convert to Pascals using equation from datasheet, including altitude 
compensation
//P = lfactor * (voltage[V] - .250 ) / 3.750

float pressure =  lfactor * ( pvolt - .250 ) / 3.750;  
float pascals = pressure * afactor; 

Serial.print("TC1");
Serial.print(":");
Serial.print(tc1);
Serial.print(";");

Serial.print("TC2");
Serial.print(":");
Serial.print(tc2);
Serial.print(";");

Serial.print("TC3");
Serial.print(":");
Serial.print(tc3);
Serial.print(";");

Serial.print("TC4");
Serial.print(":");
Serial.print(tc4);
Serial.print(";");

Serial.print("TC5");
Serial.print(":");
Serial.print(tc5);
Serial.print(";");    

Serial.print("RH 1");
Serial.print(":");
Serial.print(hs1);
Serial.print(";");  

Serial.print("RH 2");
Serial.print(":");
Serial.print(hs2);
Serial.print(";");

Serial.print("Pressure");
Serial.print(":");
Serial.print(pascals);
Serial.print(";");

Serial.println("");
delay(1000); 

  }

Python Code: 
from threading import Thread
import time
import serial
import os 
global datalist
global fileName

global motorspeed

motorspeed = 0 

def serInitialization():
    #activate the serial port, if possible
    try:
        ser = serial.Serial('COM9', 9600) #initialize the serial port
        print "Serial connection successful." 
        return ser
    except:
        print "Error: serial port cannot be initialized"
        while(1):
            voidholder = 1

def getDateTime():
  """Function grabs current time and date, then returns values in a 2-
 element list. """
   timeNow = time.strftime("%H:%M:%S")
   dateToday = time.strftime("%m/%d/%y")
  return [dateToday, timeNow]

def writetocsv(data):

    """ function writes datalist values to a csv file. If daily csv file 
exists already, 
    list values are simply appended to end of file. If it does not, function 
creates the file, 
    then appends values. 
    """

    global csv_success

    header = ["date", "time", " ", "TC1", " ", "TC2", " ", 
"TC3", " ", "TC4", " ", "TC5", " ", "HS1", " ", "HS2"," ", 
"PRESSURE", " ", "PRESSURE(alt)", "\n"]

    fileName = str(time.strftime("%m_%d_%y_")+ "log.csv")

    if os.path.exists(fileName):
        f = open(fileName, "a")

    else:
        f = open(fileName, "a+")

        for element in header:
            f.write(element + ",")
        f.write("\n")

    for element in data:
        if type(element)==str:
            f.write(element + ",")
        if type(element) == list:
            for i in element:
                f.write(i + ",")

    f.write("\n")
    f.close()
    csv_success = True

def mainprogram():

    data = []


    if ser.inWaiting():
        datetimeData = getDateTime()

        for i in datetimeData:
            data.append(i)

        val = ser.readline().strip('\n\r').split(';')
        print "Current readings: "
        for i in range(0,len(val)):
            sensorData = val[i].split(':')
            data.append(sensorData)

            print sensorData

        writetocsv(data)

        time.sleep(1) 


        print "\n" * 50

#Initializes the serial port with the arduino so now we can read what 
Arduino is sending us!
print "*******************************"
print "DATA ACQUISITION PROGRAM       "
print "*******************************"
print "Data files are saved under "
print "C:\\Users\\lpaw\\Downloads\\python workspace"
print "*******************************"


ser = serInitialization()

while True:

   mainprogram()

Screenshots of errors:



user612: Your main problem is in erroneously having a delay in both the sender and the receiver.  While this sometimes works by lucky accident, it essentially never does what is hoped for by those who try it.
In a system such as this, you should have a delay in the sender only, in order to produce quasi-regular sample intervals, but no delay in the receiver.  Rather, the receiver should accept data whenever the transmitter produces it, or more specifically in your case, every time it completes a terminated line of data.
But you also seem to want to write the data in an organized way as sets of measurements.  To do that, you should reject data until you see the first "TC1", then collect lines to fill in a data block until and end of data condition is recognized (in your case a blank line), and write it out to disk in a timestamped file.  You may also want to include code that abandons a corrupted block unwritten if any of the intermediate reading headers aren't as expected, and goes back to looking for the next hopefully clean TC1, or marks it as erroneous, or whatever.
To use a different baud rate, you must apply a matching setting in both the Arduino and the python ends of the link.
Also note that while the Arduino serial monitor is indeed useful for debugging, you must not have it open while trying to collect data with python - this will either cause an error in that one program is disallowed, or the two programs to fight over the data and each get incomplete results.
Another technique which can be simple and useful, especially during development, is to write every received line of serial data out to a text file with a numeric timestamp at the start of each line.  You can then have a second process which reads this file, grabs complete multi-line records, and writes them out using either the timestamp of the first or last element of the record.  In particular this lets you get an intermediate view of the data to understand any oddities.  With various Unix style command line tricks you can even run the first program both saving the intermediate format and feeding the second record extraction program in real time.

33548: <p>This is my first post to Stack Exchange so please forgive any formatting errors.</p>

<p>I am conducting an experiment that has 7 total sensors: 5 thermocouples, 2 humidity sensors and 1 pressure sensor. I am reading their signals via Arduino Mega analog input ports. I chose to use Python to save and compile the data in a nice csv file, therefore I am using Pyserial. </p>

<p>My problem is that I cannot go above 1 sample/second. As you can see in my code, I have a 1 second delay in Arduino and Python. I have ran successful calibration tests with my thermocouples, so I know at that rate Python and Arduino are communicating well. Anything faster than 1 sample/second (i.e. I change my delays to .5 or .25 seconds), a timing error occurs and my data is "cut up" and sent in odd chunks instead of the standard layout I programmed. </p>

<p>Additionally, I tried to increase the baud rate from 9600 and anything above that will send "garbage" ASCII characters (e.g. little blank blocks and odd characters) to the Serial Comm. I changed the baud rate in Arduino, Python <em>and</em> the COM port in my devices manager settings to no avail. </p>

<p>I would really like to increase my sampling rate to increase my signal-to-noise ratio and I believe it's good practice as a metrologist to have a higher sampling rate than 1 samp/sec. I am sending ~1072 bits each time I write to the Serial Comm (if I'm counting right??) so I should have enough room to increase the amount of bits to send per second (around 3 times what I am currently sending).</p>

<p><strong>In summary, my two main issues are: if I increase the baud rate, I get garbage in my Serial Comm and if I try to increase my "sampling rate" by decreasing my delays, timing issues appear and my data is split up erroneously. Should I be using a different function, like Serial.flush()?</strong></p>

<p><strong>I greatly appreciate any feedback, thank you for your time!</strong></p>

<pre><code>// Data acquisition
// updated  March 30, 2017


//Note: This version is for TEMPERATURE, HUMIDITY and PRESSURE data         
acquisition ALONE. No motor/conveyor control.

//***GLOBAL VARIABLES***//

float lfactor = 125.0; 
float afactor = 1.15;

/////////////////////////

//Standard setup with analogReference to 5Volts. 

void setup() {

Serial.begin(9600);  
}

void loop() {


//******THERMOCOUPLES*******//

//get voltage reading
// [code]; TC # - Location

float tcv1 = analogRead(A4) * ( 5.0 / 1024.0 ); //TC 1 
float tcv2 = analogRead(A7) * ( 5.0 / 1024.0 ); //TC 2 
float tcv3 = analogRead(A8) * ( 5.0 / 1024.0 ); //TC 3 
float tcv4 = analogRead(A9) * ( 5.0 / 1024.0 ); //TC 4 
float tcv5 = analogRead(A10) * ( 5.0 / 1024.0 ); //TC 5

//convert to temperature using custom equation based on 5V reference
float tc1 = ( 188.7755 * tcv1 ) - 245.3959;  
float tc2 = ( 188.7755 * tcv2 ) - 245.3959;  
float tc3 = ( 188.7755 * tcv3 ) - 245.3959;  
float tc4 = ( 188.7755 * tcv4 ) - 245.3959;  
float tc5 = ( 188.7755 * tcv5 ) - 245.3959;    

//******HUMIDITY SENSORS********//

float hs1 = ((( analogRead(A13) * ( 5.0 / 1024.0) ) / 5.0 ) * 100.0 ); //HS 
1
float hs2 = ((( analogRead(A15) * ( 5.0 / 1024.0) ) / 5.0 ) * 100.0 ); //HS 
2

//******PRESSURE SENSORS********//

//obtain voltage signal from Sensirion sensor
float pvolt = analogRead(A0) * (4.995 / 1024.0); 

//convert to Pascals using equation from datasheet, including altitude 
compensation
//P = lfactor * (voltage[V] - .250 ) / 3.750

float pressure =  lfactor * ( pvolt - .250 ) / 3.750;  
float pascals = pressure * afactor; 

Serial.print("TC1");
Serial.print(":");
Serial.print(tc1);
Serial.print(";");

Serial.print("TC2");
Serial.print(":");
Serial.print(tc2);
Serial.print(";");

Serial.print("TC3");
Serial.print(":");
Serial.print(tc3);
Serial.print(";");

Serial.print("TC4");
Serial.print(":");
Serial.print(tc4);
Serial.print(";");

Serial.print("TC5");
Serial.print(":");
Serial.print(tc5);
Serial.print(";");    

Serial.print("RH 1");
Serial.print(":");
Serial.print(hs1);
Serial.print(";");  

Serial.print("RH 2");
Serial.print(":");
Serial.print(hs2);
Serial.print(";");

Serial.print("Pressure");
Serial.print(":");
Serial.print(pascals);
Serial.print(";");

Serial.println("");
delay(1000); 

  }
</code></pre>

<p><strong>Python Code:</strong> </p>

<pre><code>from threading import Thread
import time
import serial
import os 
global datalist
global fileName

global motorspeed

motorspeed = 0 

def serInitialization():
    #activate the serial port, if possible
    try:
        ser = serial.Serial('COM9', 9600) #initialize the serial port
        print "Serial connection successful." 
        return ser
    except:
        print "Error: serial port cannot be initialized"
        while(1):
            voidholder = 1

def getDateTime():
  """Function grabs current time and date, then returns values in a 2-
 element list. """
   timeNow = time.strftime("%H:%M:%S")
   dateToday = time.strftime("%m/%d/%y")
  return [dateToday, timeNow]

def writetocsv(data):

    """ function writes datalist values to a csv file. If daily csv file 
exists already, 
    list values are simply appended to end of file. If it does not, function 
creates the file, 
    then appends values. 
    """

    global csv_success

    header = ["date", "time", " ", "TC1", " ", "TC2", " ", 
"TC3", " ", "TC4", " ", "TC5", " ", "HS1", " ", "HS2"," ", 
"PRESSURE", " ", "PRESSURE(alt)", "\n"]

    fileName = str(time.strftime("%m_%d_%y_")+ "log.csv")

    if os.path.exists(fileName):
        f = open(fileName, "a")

    else:
        f = open(fileName, "a+")

        for element in header:
            f.write(element + ",")
        f.write("\n")

    for element in data:
        if type(element)==str:
            f.write(element + ",")
        if type(element) == list:
            for i in element:
                f.write(i + ",")

    f.write("\n")
    f.close()
    csv_success = True

def mainprogram():

    data = []


    if ser.inWaiting():
        datetimeData = getDateTime()

        for i in datetimeData:
            data.append(i)

        val = ser.readline().strip('\n\r').split(';')
        print "Current readings: "
        for i in range(0,len(val)):
            sensorData = val[i].split(':')
            data.append(sensorData)

            print sensorData

        writetocsv(data)

        time.sleep(1) 


        print "\n" * 50

#Initializes the serial port with the arduino so now we can read what 
Arduino is sending us!
print "*******************************"
print "DATA ACQUISITION PROGRAM       "
print "*******************************"
print "Data files are saved under "
print "C:\\Users\\lpaw\\Downloads\\python workspace"
print "*******************************"


ser = serInitialization()

while True:

   mainprogram()
</code></pre>

<p><strong>Screenshots of errors:</strong>
<a href="https://i.stack.imgur.com/agYCD.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/agYCD.png" alt="Arduino Serial Comm and Python IDLE - data becomes messed up"></a></p>

<p><a href="https://i.stack.imgur.com/jozIB.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/jozIB.png" alt="Baud rate error"></a></p>
 612: <p>Your main problem is in erroneously having a delay in both the sender and the receiver.  While this sometimes works by lucky accident, it essentially never does what is hoped for by those who try it.</p>

<p>In a system such as this, you should have a delay in the sender only, in order to produce quasi-regular sample intervals, but no delay in the receiver.  Rather, the receiver should accept data whenever the transmitter produces it, or more specifically in your case, every time it completes a terminated line of data.</p>

<p>But you also seem to want to write the data in an organized way as sets of measurements.  To do that, you should reject data until you see the first "TC1", then collect lines to fill in a data block until and end of data condition is recognized (in your case a blank line), and write it out to disk in a timestamped file.  You may also want to include code that abandons a corrupted block unwritten if any of the intermediate reading headers aren't as expected, and goes back to looking for the next hopefully clean TC1, or marks it as erroneous, or whatever.</p>

<p>To use a different baud rate, you must apply a matching setting in both the Arduino and the python ends of the link.</p>

<p>Also note that while the Arduino serial monitor is indeed useful for debugging, you <em>must not</em> have it open while trying to collect data with python - this will either cause an error in that one program is disallowed, or the two programs to fight over the data and each get incomplete results.</p>

<p>Another technique which can be simple and useful, especially during development, is to write every received line of serial data out to a text file with a numeric timestamp at the start of each line.  You can then have a second process which reads this file, grabs complete multi-line records, and writes them out using either the timestamp of the first or last element of the record.  In particular this lets you get an intermediate view of the data to understand any oddities.  With various Unix style command line tricks you can even run the first program both saving the intermediate format and feeding the second record extraction program in real time.</p>
