Sensor logger for Raspberry Pi in a stratospheric probe

Have you already executed the code to see how it performs and if the battery will last? There is that famous Donald Knuth quote saying premature optimization is the root of all evil (or at least most of it) in programming.

I never had to think about the energy consumption of a program, so I cannot tell you about the power efficieny. But as vnp already did, I can also share my opinion about the code structure to help you to identify bottlenecks more easily. Also, a different structure should help you to still log some data even in case of exceptions.

Here is what struck me on first read:

• most of the code is defined in the main method
• you overwrite the complete data files at the beginning of the program
• very broad exception clause
• repetition of the csv write (violates the zen of python - not dry - dont repeat yourself)

I tried to resolve some of the issues and refactored the structure of the code:

#!/usr/bin/env python3

from sense_hat import SenseHat
import time    
import csv
import datetime

# defined constants on moduel level and capitalized the names (pep8: https://www.python.org/dev/peps/pep-0008/#constants)
RED = (255, 0, 0)
GREEN = (0, 255, 0)
BLACK = (0,0,0)


class DataLogger(object):

    def __init__(self, init_csv_files=False):
        # initalize the commonly ued sensor
        self.sense = SenseHat()
        self.sense.clear()
        self.sense.set_imu_config(True, True, True)
        self.sense.low_light = True

        # only initialize the csv files, if intended
        # I would suggest not to init them in the same program though.
        # If - for some reasons - the python interpreter crashes and the script is restarted,
        # the init of the csv_files will overwrite all the data which was logged so far.
        if init_csv_files:
            self.init_csv_files()

    def write_data_to_file(self, data, file_name, mode='a', delimiter=';', quotechar='"', quoting=csv.QUOTE_MINIMAL):
        """
        Helper method to write the given data to a csv file. Using 'append' as default mode to avoid accidental overwrites.
        """
        with open(file_name, mode=mode) as file:
            writer = csv.writer(file, delimiter=delimiter, quotechar=quotechar, quoting=quoting)
            writer.writerow(data)

    def init_csv_files(self):
        # see comment in init method
        data_headings = ['Zeit','Temperatur1', 'Temperatur2', 'Temperatur3', 'Luftdruck', 'Luftfeuchtigkeit', 'Yaw', 'Pitch', 'Roll', 'Compass X', 'Compass Y', 'Compass Z', 'Gyro X', 'Gyro Y', 'Gyro Z']
        self.write_data_to_file(data_headings, 'data.csv', 'w')

        acc_headings = ['Zeit','Acc_X','Acc_Y','Acc_Z']
        self.write_data_to_file(acc_headings, 'acc.csv', 'w')

        log_headings = ['Zeit','Fehler']
        self.write_data_to_file(log_headings, 'log.csv', 'w')        

    def start_logging(self):
        # actual execution
        sense.set_pixel(0, 0, BLACK)
        counter = 0

        while True:
            # moved the accelleration logging to a different method
            # and catched possible exceptions there, so the counter will still be increase
            # and the rest of the data may still be logged even if the accelleration data
            # always raises exceptions
            self.log_accelleration()
            time.sleep(.250)
            counter += 1

            # using counter % 4 == 0 instead of counter == 4
            # this will evaluate to true for every number divisible by 4
            # If you do the strict comparision, you could find yourself in the scenario
            # where the data logging is never executed, if the counter is larger than 4
            # (in this case this is very unlikely, but in threaded scenarios it would be possible,
            # so doing modulo 4 is more defensive)
            if(counter % 4 == 0):
                self.log_data()
                counter = 0

    def log_accelleration(self):
        acceleration_data = get_accelleration()
        if acceleration_data:
            try:
                self.write_data_to_file(acceleration_data, 'acc.csv')
            except Exception as e:
                self.log_exception(e)
                pass
            else:
                # no exception occurred
                self.sense.set_pixel(0, 0, green)
                time.sleep(.05)
            finally:
                self.sense.set_pixel(0, 0, black)

    def log_data(self):
        # saving datetime first, before reading all the sensor data
        data = [datetime.datetime.now()]

        # moved each of the calls to sense in a separate method
        # exceptions will lead to empty entries being logged but
        # if e.g. get_pressure raises an exceptions, the other data may still get logged 
        data += self.get_temperature()
        data += self.get_pressure()
        data += self.get_humidity()
        data += self.get_orientation()
        data += self.get_mag()
        data += self.get_gyro()

        self.write_data_to_file(data, 'data.csv')

    def log_exception(self, exception):
        sense.set_pixel(1, 0, red)
        self.write_data_to_file([datetime.datetime.now(), str(exception)], 'log.csv')
        sense.set_pixel(0, 0, black)

    def get_accelleration(self):
        try: 
            acceleration = self.sense.get_accelerometer_raw()
        except Exception as e:
            self.log_exception(e)
            return

        acc_x = acceleration['x']
        acc_y = acceleration['y']
        acc_z = acceleration['z']

        return[datetime.datetime.now(), acc_x, acc_y, acc_z]

    def get_temperature(self):
        try:
            temperature1 = sense.get_temperature()
            temperature2 = sense.get_temperature_from_humidity()
            temperature3 = sense.get_temperature_from_pressure()
        except Exception as e:
            return [None, None, None]
        return [temperature1, temperature2, temperature3]

    def get_pressure(self):
        try:
            pressure = sense.get_pressure()
        except Exception as e:
            return [None]
        return [pressure]

    def get_humidity(self):
        try:
            humidty = sense.get_humidity()
        except Exception as e:
            return [None]
        return [humidty]

    def get_orientation(self):       
        try:
            orientation = sense.get_orientation()
        except Exception as e:
            return [None, None, None]
        return [orientation["yaw"], orientation["pitch"], orientation["roll"]]

    def get_mag(self):
        try:
            mag = sense.get_compass_raw()
        except Exception as e:
            return [None, None, None]
        return [mag["x"], mag["y"], mag["z"]]

    def get_gyro(self):
        try:
            gyro = sense.get_gyroscope_raw()
        except Exception as e:
            return [None, None, None]
        return [gyro["x"], gyro["y"], gyro["z"]]


if __name__ == '__main__':
    data_logger = DataLogger(init_csv_files=True)
    try:
        data_logger.start_logging()
    except Exception as e:
        data_logger.log_exception(e)

Further steps for improvements:

• Catch specific exceptions (e.g. IOErrors in the write csv, or SenseHat specific exceptions
• Log exceptions (where needed) and return different defaults in cases of error
• Refactor the write to - as you suggested - log the data in memory and only write every 10th entry to the csv. Attention: If you only log every 10th or even every 100th data entry and the python interpreter crashes, the recently logged data will be lost
• Don't write the csv headers in code, but manually prepare the csv files and put them next to the script
• Use a sqlite database and log the data here instead of in CSVs

In order to figure out where to start with the optimizations, you can now profile the helper methods (write_data_to_file, get_temperature and the other get_... methods) and derive appropriate measurements to take.

PS. Fair warning: I never executed the code in a python shell, so it may not be free from bugs :see_no_evil:.