This little guide is meant to teach you the following concepts:
A software filter is a program or a subset of a program meant to sanitize data input, transforming it into something that is more useful for the end user.
We see software filters all over the place, from image processing to machine learning and down into smoothing out sensor data.
In our application, we will be designing a software filter for the last task: smoothing out data from the voltage and current sensors on our PV Curve Tracer.
That means an ideal software filter will have the following qualities:
In this tutorial I’ll show you a C++ implementation of a SMA filter class. This is a very simple and intuitive filter: essentially we’re taking the average of the last X samples and spitting that out. This filter is decent at removing noise from the dataset, especially cyclic variations in the data, as well as identifying trends.
There are drawbacks with this filter, though, including:
Starting off, we want a couple methods for our filter class:
Simple, right? Here’s the bare skeleton of our class and we’ll begin filling it in.
class SMAFilter {
protected:
// a buffer that will hold the history of samples added to it
double * _dataBuffer;
// the maximum number of samples that our filter can hold
int _maxSamples;
// the number of samples we currently have (saturates at _maxSamples)
int _numSamples;
// the current index of the buffer we want to add a sample at
int _idx;
public:
// our setup method
SMAFilter(int maxSamples);
// method to add a sample of type double
void addSample(double sample);
// method to retrieve a filtered sample
double getResult();
// shutdown method
void shutdown();
}
In our setup method, we’re going to initialize some variables, including setting the size of our buffer.
SMAFilter(int maxSamples) {
// setup the data buffer
_maxSamples = maxSamples + 1;
_dataBuffer = new (std::nothrow) double [_maxSamples];
_idx = 0;
_numSamples = 0;
}
You can see that we set the data buffer to a variable array allocated on the heap; we will need to deallocate that when we shut down the SMAFilter. Notice that we need to include the header #include <new> to not throw an exception if we are unable to allocate memory to the data buffer.
Now in our addSample method, we want to add a value to our buffer:
// method to add a sample of type double
void addSample(double sample) {
// check for exception
if (_dataBuffer == nullptr) { return; }
_dataBuffer[_idx] = sample;
_idx = (_idx + 1) % _maxSamples;
// saturate counter at max samples
if ((_numSamples + 1) < _maxSamples) {
_numSamples ++;
}
}
We see here a couple of things:
_idx = (_idx + 1) % _maxSamples;. This code increments our index until it exceeds _maxSamples and wraps back around to 0. This indicates that the buffer is a circular buffer. This saves us space and doesn’t require us to check if the array is full or to allocate more memory.getResult() function how many elements are in the data buffer to divide our sum by to get the average.// method to retrieve a filtered sample
double getResult() {
// check for exception
if (_dataBuffer == nullptr) { return 0.0; }
double sum = 0.0;
for (int i = (_idx - _numSamples + _maxSamples) % _maxSamples; i != _idx; i = (i + 1) % _maxSamples) {
sum += _dataBuffer[i];
}
return sum / _numSamples;
}
Here as well, we perform the exception checking to make sure we can access valid values in our buffer. If our buffer is unallocated, we get a null pointer and then return a default 0.0 value.
We then take every filled value in the buffer and sum it up, then divide it by the number of samples to get the average. There’s a little bit of pointer math here to follow to get every filled value. First we start at the index of the stalest value (_idx - _numSamples + _maxSamples) % _maxSamples. This code means that we’re starting at the stalest value (_idx - _numSamples), plus the upper bound of the buffer + _maxSamples to keep the index positive, and then modulo by the upper bound % _maxSamples to truncate any overflow.
Finally, here’s a one line for deallocating the data buffer from setup.
void shutdown() { delete[] _dataBuffer; }
In order to use this filter, we can allocate it on the stack like so:
printf("Hello World\n");
// setup
SMAFilter filter(5); // 5 sample buffer
// add 20 samples, increasing linearly by 10, and then some noisy 100s every 5 cycles.
for (int i = 0; i < 20; i++) {
if (i%5 == 0) { filter.addSample(100); }
else { filter.addSample(i*10.0); }
// read the filter output at every point
printf("%i:\t%f\n", i, filter.getResult());
}
// shutdown
filter.shutdown();
The whole code is provided here:
#include <stdio.h>
#include <new>
class SMAFilter {
protected:
// a buffer that will hold the history of samples added to it
double * _dataBuffer;
// the maximum number of samples that our filter can hold
int _maxSamples;
// the number of samples we currently have (saturates at _maxSamples)
int _numSamples;
// the current index of the buffer we want to add a sample at
int _idx;
public:
// our setup method
SMAFilter(int maxSamples) {
// setup the data buffer
_maxSamples = maxSamples + 1;
_dataBuffer = new (std::nothrow) double [_maxSamples];
_idx = 0;
_numSamples = 0;
}
// method to add a sample of type double
void addSample(double sample) {
// check for exception
if (_dataBuffer == nullptr) { return; }
_dataBuffer[_idx] = sample;
_idx = (_idx + 1) % _maxSamples;
// saturate counter at max samples
if ((_numSamples + 1) < _maxSamples) {
_numSamples ++;
}
}
// method to retrieve a filtered sample
double getResult() {
// check for exception
if (_dataBuffer == nullptr) { return 0.0; }
double sum = 0.0;
for (int i = (_idx - _numSamples + _maxSamples) % _maxSamples; i != _idx; i = (i + 1) % _maxSamples
) {
sum += _dataBuffer[i];
}
return sum / _numSamples;
}
// shutdown method
void shutdown() { delete[] _dataBuffer; }
};
int main()
{
printf("Hello World\n");
// setup
SMAFilter filter(5); // 5 sample buffer
// add 20 samples, increasing linearly by 10, and then some noisy 100s every 5 cycles.
for (int i = 0; i < 20; i++) {
if (i%5 == 0) { filter.addSample(100); }
else { filter.addSample(i*10.0); }
// read the filter output at every point
printf("%i:\t%f\n", i, filter.getResult());
}
// shutdown
filter.shutdown();
return 0;
}
I highly recommend popping it into an online compiler and seeing the output and checking it out for yourself.
Alright, some of you readers may have realized that there’s probably a better way to calculate the average of the data buffer. This way we don’t need to loop all of the elements every time to get the sum.
Instead, what we can do is maintain the previous sum and the previous number of elements, and use that to get our new average.
Our new addSample() and getResult() should look like this:
// method to add a sample of type double
void addSample(double sample) {
// check for exception
if (_dataBuffer == nullptr) { return; }
// saturate counter at max samples
if ((_numSamples + 1) <= _maxSamples) {
_numSamples ++;
_sum += sample;
} else {
// add the new value but remove the value at the
// current index we're overwriting
_sum += sample - _dataBuffer[_idx];
}
_dataBuffer[_idx] = sample;
_idx = (_idx + 1) % _maxSamples;
}
// method to retrieve a filtered sample
double getResult() {
// check for exception
if (_dataBuffer == nullptr) { return 0.0; }
return _sum / _numSamples;
}
As you can see, getResult() is reduced to a simple sum / number of valid samples in the window. We’ve added some code to addSample(), which is essentially getting the cumulative sum over the window. We just add the sample to the sum if we haven’t overflowed, and if we do overflow, then we’ll add the sample but remove the sum of the next value that’ll be overwritten.
The full code for this optimization is here:
#include <stdio.h>
#include <new>
class SMAFilter {
protected:
// a buffer that will hold the history of samples added to it
double * _dataBuffer;
// the maximum number of samples that our filter can hold
int _maxSamples;
// the number of samples we currently have (saturates at _maxSamples)
int _numSamples;
// the current index of the buffer we want to add a sample at
int _idx;
// sum over all elements in the window
double _sum;
public:
// our setup method
SMAFilter(int maxSamples) {
// setup the data buffer
_maxSamples = maxSamples;
_dataBuffer = new (std::nothrow) double [_maxSamples];
_idx = 0;
_numSamples = 0;
_sum = 0;
}
// method to add a sample of type double
void addSample(double sample) {
// check for exception
if (_dataBuffer == nullptr) { return; }
// saturate counter at max samples
if ((_numSamples + 1) <= _maxSamples) {
_numSamples ++;
_sum += sample;
} else {
// add the new value but remove the value at the
// current index we're overwriting
_sum += sample - _dataBuffer[_idx];
}
_dataBuffer[_idx] = sample;
_idx = (_idx + 1) % _maxSamples;
}
// method to retrieve a filtered sample
double getResult() {
// check for exception
if (_dataBuffer == nullptr) { return 0.0; }
return _sum / _numSamples;
}
// shutdown method
void shutdown() { delete[] _dataBuffer; }
};
int main()
{
printf("Hello World\n");
// setup
SMAFilter filter(5); // 5 sample buffer
// add 20 samples, increasing linearly by 10, and then some noisy 100s every 5 cycles.
for (int i = 0; i < 20; i++) {
if (i%5 == 0) { filter.addSample(100); }
else { filter.addSample(i*10.0); }
// read the filter output at every point
printf("output:\t%f\n\n", filter.getResult());
}
// shutdown
filter.shutdown();
return 0;
}
There’s one more section here as a postnote: SMA filtering is probably the lowest apple on the tree. There are many more filtering techniques that you can explore, and I highly encourage you to do so. One you’ll probably hear about is the Kalman filter; I’ve linked a couple of different tutorials on how to possibly implement this filter:
This is the end of the sensor filter development tutorial. Hopefully what you should have gotten out of this is how SMA filtering works, how to implement it in code, how to use it, as well as potential ways to optimize it.
This might be the first in a series of technical blog posts, so any feedback for future posts are greatly appreciated!
Author: Matthew Yu