The personal website of Scott W Harden
⚠️ Warning: This article is obsolete.
Articles typically receive this designation when the technology they describe is no longer relevant, code provided is later deemed to be of poor quality, or the topics discussed are better presented in future articles. Articles like this are retained for the sake of preservation, but their content should be critically assessed.

Signal Filtering with Python

⚠️ SEE UPDATED POST: Signal Filtering in Python

I've been spending a lot of time creating a DIY ECGs which produce fairly noisy signals. I have researched the ways to clean-up these signals, and the results are very useful! I document some of these findings here.

This example shows how I take a noisy recording and turn it into a smooth trace. This is achieved by eliminating excess high-frequency components which are in the original recording due to electromagnetic noise. A major source of noise can be from the AC passing through wires traveling through the walls of my apartment. My original ECG circuit was highly susceptible to this kind of interference, but my improved ECG circuit eliminates much of this noise. However, noise is still in the trace and it needs to be removed.

One method of reducing noise uses the FFT (Fast Fourier Transformation) and its inverse (iFFT) algorithm. Let's say you have a trace with repeating sine-wave noise. The output of the FFT is the breakdown of the signal by frequency. Check out this FFT trace of a noisy signal from a few posts ago. High peaks represent frequencies which are common. See the enormous peak around 60 Hz? That's noise from AC power lines. Other peaks (shown in colored bands) are other electromagnetic noise sources, such as wireless networks, TVs, telephones, and maybe my computer. The heart produces changes in electricity that are very slow (a heartbeat is about 1 Hz), so if we can eliminate higher-frequency sine waves we can get a pretty clear trace. This is called a band-stop filter (we block-out certain bands of frequencies). A band-pass filter is the opposite, where we only allow frequencies which are below (low-pass) or above (high-pass) a given frequency. By eliminating each of the peaks in the colored regions (setting each value to 0), then performing an inverse fast Fourier transformation (going backwards from frequency back to time), the result is the signal trace (seen as light gray on the bottom graph) with those high-frequency sine waves removed! (the gray trace on the bottom graph). A little touch-up smoothing makes a great trace (black trace on the bottom graph).

Here's some Python code you may find useful. The image below is the output of the Python code at the bottom of this entry. This python file requires that test.wav (an actual ECG recording of my heartbeat) exist in the same folder.

  • (A) The original signal we want to isolate. (IE: our actual heart signal)

  • (B) Some electrical noise. (3 sine waves of different amplitudes and periods)

  • (C) Electrical noise (what happens when you add those 3 sine waves together)

  • (D) Static (random noise generated by a random number generator)

  • (E) Signal (A) plus static (D)

  • (F) Signal (A) plus static (D) plus electrical noise (C)

  • (G) Total FFT trace of (F). Note the low frequency peak due to the signal and electrical noise (near 0) and the high frequency peak due to static (near 10,000)

  • (H) This is a zoomed-in region of (F) showing 4 peaks (one for the original signal and 3 for high frequency noise). By blocking-out (set it to 0) everything above 10Hz (red), we isolate the peak we want (signal). This is a low-pass filter.

  • (I) Performing an inverse FFT (iFFT) on the low-pass iFFT, we get a nice trace which is our original signal!

  • (J) Comparison of our iFFT with our original signal shows that the amplitude is kinda messed up. If we normalize each of these (set minimum to 0, maximum to 1) they line up. Awesome!

  • (K) How close were we? Graphing the difference of iFFT and the original signal shows that usually we're not far off. The ends are a problem though, but if our data analysis trims off these ends then our center looks great.

  • Note: these ends can be fixed by applying a windowing function to the original data. The FFT works best if the input data starts and ends at zero.

import numpy, scipy, pylab, random

# This script demonstrates how to use band-pass (low-pass)
# filtering to eliminate electrical noise and static
# from signal data!

##################
### PROCESSING ###
##################

xs=numpy.arange(1,100,.01) #generate Xs (0.00,0.01,0.02,0.03,...,100.0)
signal = sin1=numpy.sin(xs*.3) #(A)
sin1=numpy.sin(xs) # (B) sin1
sin2=numpy.sin(xs*2.33)*.333 # (B) sin2
sin3=numpy.sin(xs*2.77)*.777 # (B) sin3
noise=sin1+sin2+sin3 # (C)
static = (numpy.random.random_sample((len(xs)))-.5)*.2 # (D)
sigstat=static+signal # (E)
rawsignal=sigstat+noise # (F)
fft=scipy.fft(rawsignal) # (G) and (H)
bp=fft[:]
for i in range(len(bp)): # (H-red)
    if i>=10:bp[i]=0
ibp=scipy.ifft(bp) # (I), (J), (K) and (L)

################
### GRAPHING ###
################

h,w=6,2
pylab.figure(figsize=(12,9))
pylab.subplots_adjust(hspace=.7)

pylab.subplot(h,w,1);pylab.title("(A) Original Signal")
pylab.plot(xs,signal)

pylab.subplot(h,w,3);pylab.title("(B) Electrical Noise Sources (3 Sine Waves)")
pylab.plot(xs,sin1,label="sin1")
pylab.plot(xs,sin2,label="sin2")
pylab.plot(xs,sin3,label="sin3")
pylab.legend()

pylab.subplot(h,w,5);pylab.title("(C) Electrical Noise (3 sine waves added together)")
pylab.plot(xs,noise)

pylab.subplot(h,w,7);pylab.title("(D) Static (random noise)")
pylab.plot(xs,static)
pylab.axis([None,None,-1,1])

pylab.subplot(h,w,9);pylab.title("(E) Signal + Static")
pylab.plot(xs,sigstat)

pylab.subplot(h,w,11);pylab.title("(F) Recording (Signal + Static + Electrical Noise)")
pylab.plot(xs,rawsignal)

pylab.subplot(h,w,2);pylab.title("(G) FFT of Recording")
fft=scipy.fft(rawsignal)
pylab.plot(abs(fft))
pylab.text(200,3000,"signals",verticalalignment='top')
pylab.text(9500,3000,"static",verticalalignment='top',
        horizontalalignment='right')

pylab.subplot(h,w,4);pylab.title("(H) Low-Pass FFT")
pylab.plot(abs(fft))
pylab.text(17,3000,"sin1",verticalalignment='top',horizontalalignment='left')
pylab.text(37,2000,"sin2",verticalalignment='top',horizontalalignment='center')
pylab.text(45,3000,"sin3",verticalalignment='top',horizontalalignment='left')
pylab.text(6,3000,"signal",verticalalignment='top',horizontalalignment='left')
pylab.axvspan(10,10000,fc='r',alpha='.5')
pylab.axis([0,60,None,None])

pylab.subplot(h,w,6);pylab.title("(I) Inverse FFT")
pylab.plot(ibp)

pylab.subplot(h,w,8);pylab.title("(J) Signal vs. iFFT")
pylab.plot(signal,'k',label="signal",alpha=.5)
pylab.plot(ibp,'b',label="ifft",alpha=.5)

pylab.subplot(h,w,10);pylab.title("(K) Normalized Signal vs. iFFT")
pylab.plot(signal/max(signal),'k',label="signal",alpha=.5)
pylab.plot(ibp/max(ibp),'b',label="ifft",alpha=.5)

pylab.subplot(h,w,12);pylab.title("(L) Difference / Error")
pylab.plot(signal/max(signal)-ibp/max(ibp),'k')

pylab.savefig("SIG.png",dpi=200)
pylab.show()
This article's source was last edited on September 23, 2020.
Have something to say about this article? Let me know!

DIY ECG Improvements

⚠️ Check out my newer ECG designs:

Instead of using a single op-amp circuit like the previous entries which gave me decent but staticky traces, I decided to build a more advanced ECG circuit documented by Jason Nguyen which used 6 op amps! (I'd only been using one). Luckily I got a few couple LM324 quad op-amps from radioshack ($1.40 each), so I had everything I needed.

The results look great! Noise is almost zero, so true details of the trace are visible. I can now clearly see the PQRST features in the wave. I'll detail how I did this in a later entry. For now, here are some photos of the little device.

UPDATE: After analyzing ~20 minutes of heartbeat data I found a peculiarity. Technically this could be some kind of noise (a 'pop' in the microphone signal), but because this peculiarity happened only once in 20 minutes I'm not ruling out the possibility that this is the first irregular heartbeat I captured with my DIY ECG. Note that single-beat irregularities are common in healthy people, and that this does not alarm me so much as fascinate me.

This article's source was last edited on September 29, 2020.
Have something to say about this article? Let me know!
⚠️ Warning: This article is obsolete.
Articles typically receive this designation when the technology they describe is no longer relevant, code provided is later deemed to be of poor quality, or the topics discussed are better presented in future articles. Articles like this are retained for the sake of preservation, but their content should be critically assessed.

DIY ECG Detected an Irregular Heartbeat

⚠️ Check out my newer ECG designs:

Am I going to die? It's unlikely. Upon analyzing ~20 minutes of heartbeat data (some of which is depicted in the previous entry) I found a peculiarity. Technically this could be some kind of noise (a 'pop' in the microphone signal due to the shuffling of wires or a momentary disconnect from the electrodes or perhaps even a static shock to my body from something), but because this peculiarity happened only once in 20 minutes I'm not ruling out the possibility that this is the first irregular heartbeat I captured with my DIY ECG. Note that single-beat irregularities are common, and that this does not alarm me so much as fascinates me. Below is the section of the data which contains this irregular beat.

In the spirit of improvement I wonder how much more interesting this project would be if I were to combine the already-designed ECG machine with a sensor to detect the physical effect of the heart's beating on my vasculature. in other words, can I combine my electrical traces with physical traces? (Blood pressure or blood flow) I found an interesting site that shows how someone built a DIY blood flow meter using a piezo film pulse sensor. Pretty clever I must say... but I think I draw my limit at what I've done. Although blood flow would be interesting to analyze (does the murmur depicted above produce an alteration in normal blood flow?), it's not worth the time, hassle or expense of building.

This article's source was last edited on September 29, 2020.
Have something to say about this article? Let me know!

Vector-Based Racing Game?

While searching for a more efficient band-pass filter for python (to apply to multi-hour WAV files because the WAV->FFT->bandstop->iFFT->smooth process is way too slow) I came across The Stoic Monkey's VROOM! which apparently is a vector-based racing game game (written in Python with PyGame libraries) . It doesn't flaunt the capabilities of PyGame for game design, but it was a cool concept because it's very 3D-ish and is done by mathematically processing anti-aliasing splines at impressively high speed. Check it out! http://www.stoicmonkey.com/vroom/VROOM_setup.exe

This article's source was last edited on September 12, 2020.
Have something to say about this article? Let me know!

DIY ECG Progress

⚠️ Check out my newer ECG designs:

Last night I finished building my DIY ECG as a prototype (I finally got the circuit off the breadboard and onto a plastic sheet). This is a similar circuit to the one used to record data from the last entry (resister values are now identical to the crude circuit described several posts ago). I left-in the crude band-pass filter (made by grounding my primary electrode sensor through a 0.1µF capacitor) because it seemed to help a great deal, and wasn't hard to implement. I picked up all of my parts (including the LM324 quad op-amp microchip) at RadioShack. Of note, the quad-op-amp is overkill because I'm only using one of the 4 op-amps. Theoretically I could add 3 more electrodes to this circuit (which would allow for multi-sensor recording) but this would require multiple microphone jacks, which isn't very common. I guess I could use 2 microphone jacks, and differentiate right/left channels.

I made the prototype by drilling holes in a small rectangular piece of a non-conductive plastic material. I picked up a stack of these rectangular sections for a quarter at a local electrical surplus store and they're perfect for prototyping. The two green wires coming out the left side attach to a power supply (either a plugged in AC->DC transformer, 3 or 4 AA batteries, or even a 9V should work). The blue wires on the right attach to the electrodes I put on my chest. The black wires go to a headphone-jack which I plug into the microphone hole of my PC to record the signal.

This is the back of the device which shows my crummy soldering. I'm a molecular biologist not an electrical engineer. The white/yellow wires correspond to the left/right channels of the microphone connector. I only use the left one (white), but attached the right channel (yellow) to the op-amp just in case I decide to add another sensor later - this is not required.

Here's the full device: You can see the circuit (note its small size - easy to mount inside of a tictac box or something) with the green wires leading to a power supply, black cable being the microphone connector, and the blue wires leading to electrodes made... from... Fanta... cans...? Yes, in the spirit of rigging electronics (my specialty) I found that surprisingly nice chest electrodes can be made from aluminum soda cans! If you go this route, cut them delicately so you don't get metal shards in your skin like I did at first. Also, note that you have to firmly scrape each side of the aluminum to get the insulating waxy-plastic stuff off or it just won't work. I guess it's coated with something to prevent the soda from tasting like metal. Aluminum rapidly transfers heat and it's nearly impossible to solder leads onto these pads, so I wrapped a wire (tipped with a bead of solder) with the edge of the aluminum several times and crushed the heck out of it with pliers and it seems to stay on well and make a good connection. Also, before taping these onto your skin, it helps to put a conductive goo on it to make the connection better. I added skin moisturizer to each electrode and taped the gooey electrode directly onto my chest.

I recorded ~20 minutes of data last night with this rig and it looked pretty good. I went to analyze it with Python and it kept crashing! The python script I gave you previously loads the entire wave file into an array of numbers, but with a 20 minute wave file (at over 40,000 samples a second) it is too big for memory. I wrote an updated wave loader which loads large wave files in parts which is much more efficient. It also performs the condensation method at load time. Basically, it loads 100 data points (or whatever deg is set to), averages them, and adds this value to a list. The result is a smoothed trace with a resolution of 400 Hz instead of 40 kHz. I'd test this on the wave file I recorded last night but that's on my laptop which is in the car and I've got to get back to work. Here's that function:

def loadWav(fname,deg=100):
     global hz
     w=wave.open(fname)
     nchannel, width, rate, length, comptype, compname = w.getparams()
     print "[%s] rate: %d Hz frames: %d length: %.02f sec" %
           (fname, rate, length, float(length)/rate)
     hz=rate/deg
     chunks=int(length/deg)
     data=[]
     for i in range(chunks):
         if i%7777==0:
             print "processing chunk %d of %d (%0.02f%%)" %
                   (i,chunks,100.0*i/chunks)
         vals = struct.unpack("%sh" %deg,w.readframes(deg))
         data.append(sum(vals)/float(deg))
     print "complete!"
     return data
This article's source was last edited on September 29, 2020.
Have something to say about this article? Let me know!
page 1, page 2, page 3, page 4, page 5, page 6, page 7, page 8, page 9, page 10, page 11, page 12, page 13, page 14, page 15, page 16, page 17, page 18, page 19, page 20, page 21, page 22, page 23, page 24, page 25, page 26, page 27, page 28, page 29, page 30, page 31, page 32, page 33, page 34, page 35, page 36, page 37, page 38, page 39, page 40, page 41, page 42, page 43, page 44, page 45, page 46, page 47, page 48, page 49, page 50
All Blog Posts