The personal website of Scott W Harden
⚠️ Warning: This article is obsolete.
Articles typically receive this designation when the technology they describe is no longer relevant, code provided is later deemed to be of poor quality, or the topics discussed are better presented in future articles. Articles like this are retained for the sake of preservation, but their content should be critically assessed.

Smoothing Window Data Averaging in Python - Moving Triangle Tecnique

⚠️ SEE UPDATED POST: Signal Filtering in Python

While I wrote a pervious post on linear data smoothing with python, those scripts were never fully polished. Fred (KJ4LFJ) asked me about this today and I felt bad I had nothing to send him. While I might add that the script below isn't polished, at least it's clean. I've been using this method for all of my smoothing recently. Funny enough, none of my code was clean enough to copy and paste, so I wrote this from scratch tonight. It's a function to take a list in (any size) and smooth it with a triangle window (of any size, given by "degree") and return the smoothed data with or without flanking copies of data to make it the identical length as before. The script also graphs the original data vs. smoothed traces of varying degrees. The output is below. I hope it helps whoever wants it!

import numpy
import pylab


def smoothTriangle(data, degree, dropVals=False):
    """
    performs moving triangle smoothing with a variable degree.
    note that if dropVals is False, output length will be identical
        to input length, but with copies of data at the flanking regions
    """
    triangle = numpy.array(range(degree)+[degree]+range(degree)[::-1])+1
    smoothed = []
    for i in range(degree, len(data)-degree*2):
        point = data[i:i+len(triangle)]*triangle
        smoothed.append(sum(point)/sum(triangle))
    if dropVals:
        return smoothed
    smoothed = [smoothed[0]]*(degree+degree/2)+smoothed
    while len(smoothed) < len(data):
        smoothed.append(smoothed[-1])
    return smoothed


### CREATE SOME DATA ###
data = numpy.random.random(100)  # make 100 random numbers from 0-1
data = numpy.array(data*100, dtype=int)  # make them integers from 1 to 100
for i in range(100):
    data[i] = data[i]+i**((150-i)/80.0)  # give it a funny trend

### GRAPH ORIGINAL/SMOOTHED DATA ###
pylab.plot(data, "k.-", label="original data", alpha=.3)
pylab.plot(smoothTriangle(data, 3), "-", label="smoothed d=3")
pylab.plot(smoothTriangle(data, 5), "-", label="smoothed d=5")
pylab.plot(smoothTriangle(data, 10), "-", label="smoothed d=10")
pylab.title("Moving Triangle Smoothing")
pylab.grid(alpha=.3)
pylab.axis([20, 80, 50, 300])
pylab.legend()
pylab.show()
⚠️ Warning: This article is obsolete.
Articles typically receive this designation when the technology they describe is no longer relevant, code provided is later deemed to be of poor quality, or the topics discussed are better presented in future articles. Articles like this are retained for the sake of preservation, but their content should be critically assessed.

Simple Python Spectrograph with PyGame

While thinking of ways to improve my QRSS VD high-definitions spectrograph software, I often wish I had a better way to display large spectrographs. Currently I'm using PIL (the Python Imaging Library) with TK and it's slow as heck. I looked into the PyGame project, and it seems to be designed with speed in mind. I whipped-up this quick demo, and it's a simple case audio spectrograph which takes in audio from your sound card and graphs it time vs. frequency. This method is far superior to the method I was using previously to display the data, because while QRSS VD can only update the entire GUI (500px by 8,000 px) every 3 seconds, early tests with PyGame suggests it can do it about 20 times a second (wow!). With less time/CPU going into the GUI, the program can be more responsivle and my software can be less of a drain.

import pygame
import numpy
import threading
import pyaudio
import scipy
import scipy.fftpack
import scipy.io.wavfile
import wave
rate = 12000  # try 5000 for HD data, 48000 for realtime
soundcard = 2
windowWidth = 500
fftsize = 512
currentCol = 0
scooter = []
overlap = 5  # 1 for raw, realtime - 8 or 16 for high-definition


def graphFFT(pcm):
    global currentCol, data
    ffty = scipy.fftpack.fft(pcm)  # convert WAV to FFT
    ffty = abs(ffty[0:len(ffty)/2])/500  # FFT is mirror-imaged
    # ffty=(scipy.log(ffty))*30-50 # if you want uniform data
    print "MIN:t%stMAX:t%s" % (min(ffty), max(ffty))
    for i in range(len(ffty)):
        if ffty[i] < 0:
            ffty[i] = 0
        if ffty[i] > 255:
            ffty[i] = 255
    scooter.append(ffty)
    if len(scooter) < 6:
        return
    scooter.pop(0)
    ffty = (scooter[0]+scooter[1]*2+scooter[2]*3+scooter[3]*2+scooter[4])/9
    data = numpy.roll(data, -1, 0)
    data[-1] = ffty[::-1]
    currentCol += 1
    if currentCol == windowWidth:
        currentCol = 0


def record():
    p = pyaudio.PyAudio()
    inStream = p.open(format=pyaudio.paInt16, channels=1, rate=rate,
                      input_device_index=soundcard, input=True)
    linear = [0]*fftsize
    while True:
        linear = linear[fftsize/overlap:]
        pcm = numpy.fromstring(inStream.read(
            fftsize/overlap), dtype=numpy.int16)
        linear = numpy.append(linear, pcm)
        graphFFT(linear)


pal = [(max((x-128)*2, 0), x, min(x*2, 255)) for x in xrange(256)]
print max(pal), min(pal)
data = numpy.array(numpy.zeros((windowWidth, fftsize/2)), dtype=int)
# data=Numeric.array(data) # for older PyGame that requires Numeric
pygame.init()  # crank up PyGame
pygame.display.set_caption("Simple Spectrograph")
screen = pygame.display.set_mode((windowWidth, fftsize/2))
world = pygame.Surface((windowWidth, fftsize/2), depth=8)  # MAIN SURFACE
world.set_palette(pal)
t_rec = threading.Thread(target=record)  # make thread for record()
t_rec.daemon = True  # daemon mode forces thread to quit with program
t_rec.start()  # launch thread
clk = pygame.time.Clock()
while 1:
    for event in pygame.event.get():  # check if we need to exit
        if event.type == pygame.QUIT:
            pygame.quit()
            sys.exit()
    pygame.surfarray.blit_array(world, data)  # place data in window
    screen.blit(world, (0, 0))
    pygame.display.flip()  # RENDER WINDOW
    clk.tick(30)  # limit to 30FPS
⚠️ Warning: This article is obsolete.
Articles typically receive this designation when the technology they describe is no longer relevant, code provided is later deemed to be of poor quality, or the topics discussed are better presented in future articles. Articles like this are retained for the sake of preservation, but their content should be critically assessed.

Simple-Case PyGame Example

I'm starting to investigate PyGame as an alternative to PIL and K for my QRSS VD spectrograph project. This sample code makes a box bounce around a window.

import pygame, sys
pygame.init() #load pygame modules
size = width, height = 320, 240 #size of window
speed = [2, 2] #speed and direction
screen = pygame.display.set_mode(size) #make window
s=pygame.Surface((100,50)) #create surface 100px by 50px
s.fill((33,66,99)) #color the surface blue
r=s.get_rect() #get the rectangle bounds for the surface
clock=pygame.time.Clock() #make a clock
while 1: #infinite loop
        clock.tick(30) #limit framerate to 30 FPS
        for event in pygame.event.get(): #if something clicked
                if event.type == pygame.QUIT: #if EXIT clicked
                        sys.exit() #close cleanly
        r=r.move(speed) #move the box by the "speed" coordinates
        #if we hit a  wall, change direction
        if r.left < 0 or r.right > width: speed[0] = -speed[0]
        if r.top < 0 or r.bottom > height: speed[1] = -speed[1]
        screen.fill((0,0,0)) #make redraw background black
        screen.blit(s,r) #render the surface into the rectangle
        pygame.display.flip() #update the screen

MEPT Insulation Improves Stability

While it may not be perfect, it's a whole lot better. Below is a capture from this morning of my signal (the waves near the bottom). Compare that to how it was before and you should notice a dramatic improvement! The MEPT is inside a metal box inside a 1-inch-thick Styrofoam box. Very cool!

Failing Oscilloscope

My oscilloscope decided to die on my right as I finally was able to view my 10 MHz waveform. I used a piece of coax with a load at the connector to the o-scope, and ran the coax to my test points. It was beautiful! ... and lasted about 30 seconds. The culprit seems to be a failing "focus" knob. My images had been getting blurrier by the day, and now it's completely black unless I twist pretty hard on the focus knob. I'd stick a small pot in there, but I have no idea how much voltage/current is being regulated. I'm sure the schematics are posted somewhere, but for now I'm going to try to clean out the potentiometer manually and see if the situation improves. Here are some photos of the circuitry inside this old scope. They don't make stuff like this anymore!

Update: I never got this scope to work again!

page 1, page 2, page 3, page 4, page 5, page 6, page 7, page 8, page 9, page 10, page 11, page 12, page 13, page 14, page 15, page 16, page 17, page 18, page 19, page 20, page 21, page 22, page 23, page 24, page 25, page 26, page 27, page 28, page 29, page 30, page 31, page 32, page 33, page 34, page 35, page 36, page 37, page 38, page 39, page 40, page 41, page 42, page 43, page 44, page 45, page 46, page 47, page 48, page 49
All Blog Posts