SWHarden.com

The personal website of Scott W Harden

Python-Powered Frequency Activity Logger

__I’m often drawn toward projects involving data analysis with Python. __When I found out a fellow ham in Orlando was using his computer to stream a popular local repeater frequency over the internet I got excited because of the potential for generating data from the setup. Since this guy already has his radio connected to his PC’s microphone jack, I figured I could write a Python app to check the microphone input to determine if anyone is using the frequency. By recording when people start and stop talking, I can create a log of frequency activity. Later I can write software to visualize this data. I’ll talk about that in a later post. For now, here’s how I used Python and a Linux box (Ubuntu, with the python-alsaaudio package installed) to generate such logs.

We can visualize this data using some more simple Python code. Long term it would be useful to visualize frequency activity similarly to how I graphed computer usage at work over the last year but for now since I don’t have any large amount of data to work with. I’ll just write cote to visualize a QSO (conversation) with respect to time. It should be self-explanatory. This data came from data points displayed in the video (provided at the end of this post too).

And, of course, the code I used to generate the log files (seen running in video above): Briefly, this program checks the microphone many times every second to determine if its state has changed (talking/no talking) and records this data in a text file (which it updates every 10 seconds). Matplotlib can EASILY be used to graph data from such a text file.

import alsaaudio, time, audioop, datetime
inp = alsaaudio.PCM(alsaaudio.PCM_CAPTURE,alsaaudio.PCM_NONBLOCK)
inp.setchannels(1)
inp.setrate(4000)
inp.setformat(alsaaudio.PCM_FORMAT_S16_LE)
inp.setperiodsize(1)

squelch = False
lastLog = 0
dataToLog = ""

def logIt(nowSquelch):
 global dataToLog, lastLog
 timeNow = datetime.datetime.now()
 epoch = time.mktime(timeNow.timetuple())
 if nowSquelch==True: nowSquelch=1
 else: nowSquelch=0
 logLine="%s %dn"%(timeNow, nowSquelch)
 print timeNow, nowSquelch
 dataToLog+=logLine
 if epoch-lastLog>10:
 #print "LOGGING..."
 f=open('squelch.txt','a')
 f.write(dataToLog)
 f.close()
 lastLog = epoch
 dataToLog=""

while True:
 l,data = inp.read()
 if l:
 vol = audioop.max(data,2)
 #print vol #USED FOR CALIBRATION
 if vol>800: nowSquelch = True
 else: nowSquelch = False
 if not nowSquelch == squelch:
 logIt(nowSquelch)
 squelch = nowSquelch
 time.sleep(.01)

To use this code make sure that you’ve properly calibrated it. See the “vol>800” line? That means that if the volume in the microphone is at least 800, it’s counted as talking, and less than it’s silence. Hopefully you can find a value that counts as silence when the squelch is active, but as talking when the squelch is broken (even if there’s silence). This is probably best achieved with the radio outputting at maximum volume. You’ll have to run the program live with that line un-commented to view the data values live. Find which values occur for squelch on/off, and pick your threshold accordingly.

After that you can visualize the data with the following code. Note that this is SEVERELY LIMITED and is only useful when graphing a few minutes of data. I don’t have hours/days of data to work with right now, so I won’t bother writing code to graph it. This code produced the graph seen earlier in this page. Make sure matplotlib is installed on your box.

import pylab

def loadData():
 #returns Xs
 import time, datetime, pylab
 f=open('good.txt')
 raw=f.readlines()
 f.close()
 onTimes=[]
 timeStart=None
 lastOn=False
 for line in raw:
 if len(line)<10: continue
 line = line.strip('n').split(" ")
 t=line[0]+" "+line[1]
 t=t.split('.')
 thisDay=time.strptime(t[0], "%Y-%m-%d %H:%M:%S")
 e=time.mktime(thisDay)+float("."+t[1])
 if timeStart==None: timeStart=e
 if line[-1]==1: stat=True
 else: stat=False
 if not lastOn and line[-1]=="1":
 lastOn=e
 else:
 onTimes.append([(lastOn-timeStart)/60.0,
 (e-timeStart)/60.0])
 lastOn=False
 return onTimes

times = loadData()
pylab.figure(figsize=(8,3))
for t in times:
 pylab.fill([t[0],t[0],t[1],t[1]],[0,1,1,0],'k',lw=0,alpha=.5)
pylab.axis([None,None,-3,4])
pylab.title("A little QSO")
pylab.xlabel("Time (minutes)")
pylab.show()

Prime Prototype Construction

Now that I’ve worked-out the software side of the microcontroller-powered prime number generator, it’s time to start working on the hardware. I want to make a prototype which is far smaller and simpler than the final version but lets me practice driving lots of LEDs (30). I expect the final version to have around 80. Also, the heart of this project is an ATTiny2313 microcontroller, and for the full version I’d like to use an ATMEega8. I picked up an unfinished wooden box with a magnetic latch from Michaels. It’s delicate and tends to chip when you drill it, but moving slowly I’m able to make nice evenly-spaced holes.

__This is the circuit concept. __The chip is an ATTiny2313, sourced with 5V, where the left pins control the columns (by providing current) and the right pins control the rows (by providing ground). The “holes” at the top of the circuit represent where I hook up my PC and external power for testing purposes.

Thoughts from Future Scott (10 years later, August, 2019)

A+ for enthusiasm and construction but your design is... just no!

Why are you using an external crystal?

The schematic for the crystal is wrong: those capacitors should be to ground not in series!

You made the circuit diagram in InkScape!

You shouldn't drive current directly out of the microcontroller pins.

The majority of the microcontroller CPU cycles will go into managing multiplexing of the display (not calculating primes).

After a little more work I have a functional device and it looks better than I expected. There are a few more features I want to add, and I want to work on the code some more, but I hope to be done tomorrow. The coolest part is that I’ve included an internal button which drives a pause/resume and speed-controller menu based upon the length of button presses! There’s a lot of awesome stuff I want to write, but once again, I’ll save it for the completed project page.

I rendered the cover sticker wrong and all the LEDs are mislabled. The first LED should be 2^0 (1), and the second should be 2^1 (2), etc. Also, 2^22 and 2^23 are mislabeled - oops! But the thing really does generate, differentiate, and display [only[ prime numbers. Once again, videos (including demonstration of the menus and the programming options) and source code will be posted soon.


Prime Number Generator Prototype

In my quest to build a hardware-based prime number generator I built a rapid prototype to assess how quickly primes can be found with an 8-bit microcontroller. There is a lot of room for improvement, but the code works. Instead of messing with tons of little LEDs, this design displays numbers on an LCD. Interestingly the library to run the LCD takes up about 90% of the memory of the chip leaving only a handful of bytes to write the prime calculation code in!

#define F_CPU 1E6
#include <stdlib.h>
#include <avr/io.h>
#include <math.h>
#include <util/delay.h>
#include "lcd.h"
#include "lcd.c"

const unsigned long int primeMax=pow(2,25);
unsigned long int primeLast=2;
unsigned long int primeTest=0;
unsigned int primeDivy=0;

void wait(void);
void init(void);
void updateDisplay(void);
char *toString(unsigned long int);

int main(void){
    init();
    short maybePrime;
    unsigned int i;
    for(primeTest=2;primeTest<sqrt(primeMax);primeTest++){
        maybePrime=1;
        for (i=2;i<=(sqrt(primeTest)+1);i++){
            primeDivy=i;
            updateDisplay();
            if (primeTest%primeDivy==0){maybePrime=0;break;}
        }
        if (maybePrime==1){primeLast=primeTest;updateDisplay();}
    }
    return 0;
}

void updateDisplay(void){
    lcd_gotoxy(12,0);lcd_puts(toString(primeLast));
    lcd_gotoxy(5,1);lcd_puts(toString(primeTest));
    lcd_gotoxy(16,1);lcd_puts(toString(primeDivy));
    return;
}

void init(void){
    lcd_init(LCD_DISP_ON);
    lcd_puts("PRIME IDENTIFICATIONn");
    _delay_ms(2000);
    lcd_clrscr();
    lcd_puts("LAST PRIME:n");
    lcd_puts("TRY:");
    lcd_gotoxy(14,1);lcd_puts("/");
    return;
}

char *toString(unsigned long int x){
    char s1[8];
    ltoa(x,s1,10);
    return s1;
}

A Prime Idea

I’m completely drained of energy. I visited my wife’s family in Tennessee last week. I left Thursday and came back Tuesday (yesterday). I drove a total of 2,180 miles. The drive to Humboldt, TN (the destination) and back is only 1,658 miles. That means that I drove over 520 miles over the 3 days while at my destination. That’s about 174 miles a day. At 50 MPH average speed that’s about 4 hours in the car. So, 13 hour drive (each way) to get there, then 4 hours in the car every day I was there. That’s a lot of car time!

While speaking with my brother-in-law (who just got a BS in computer science with a minor in mathematics) I learned that a faculty member at the university challenged him to write a computer program which could find the N’th prime number (up to 10^15) for a graduate school project. I was fascinated by the idea project and the various techniques, and workarounds related to it. After working on the theory behind the software (which I tested in Python) for a few hours, I had the idea to attempt to perform a similar task at the microcontroller level.

Here’s the project I want to begin: I want to build a microcontroller-powered prime number generator which displays results in binary. The binary-encoded output is similar to the binary clocks which are nothing new. My project will calculate prime numbers up to 2^25 (33,554,432) and display the results in binary using long strips of 20 LEDs. There will be 3 rows of LEDs. The middle row (red) will simply count from 0 to 2^25. Every time it gets to a new number, the bottom row (blue) counts from 0 to the square root of the middle row. For every number on the bottom row, the remainder (modulus) of the middle/bottom is calculated. If the remainder is 0, the middle (red) number is divisible by the bottom (blue) therefore it is not prime. If the bottom number gets all the way to the square root of the middle number, the middle number is assumed to be prime and it is copied to the top row (green). The top row displays the most recent number determined to be prime.

Technical details of the project further reveal its dual simplicity/complexity nature. I’ll add some buttons/switches for extra features. For example, I want to be able to start the program at a number of my choosing rather than forcing it to start at 0. Also, I want to be able to adjust the speed at which it runs (I don’t want the blue row to just flicker forever). The ATTiny2313 (my microcontroller of choice because I have a few extra of them) has 18 IO pins. If I get creative with my multiplexing techniques, I can probably run 81 LEDs from 18 pins (9 rows of 9 LEDs). I’ve specifically chosen against charlieplexing because I will be lighting many LEDs “simultaneously” and I think the degree of flicker would be far too great to satisfy my sensitive eyes, even though I could do it all with only 10 pins.

I’ve decided to transistorize the entire project to provide a greater and more constant current to the LEDs. I’ll use a set of 9 transistors to set the row that gets power (when the microcontroller powers the base, the row gets power) and another set of 9 transistors to set the LEDs in each row that light up (when the microcontroller powers the base, the LED gets grounded and lights up). To have consistently-bright, non-flickering LEDs which don’t dim as more LEDs illuminate, I will add a resistor to every LED. Maybe I can get creative and utilize 10-pin resistor networks (one for each row) immediately after the row-selecting transistor! That will save me so much time. (I just came up with that idea - just now!) Anyway, that’s my project idea.

I’d love to make this project look nice. All of my other projects were housed in junky plastic or cardboard boxes (if they were housed at all!) and this is something I want to keep. I start dental school soon, and I’ve love to have a fancy-looking piece of artsy/geeky/electrical memorabilia so I’ll never forget who I am, my roots, and my true interests. Plus, it will give me something groovy to stare at when I come home after a long day cleaning the teeth of manikins and wondering why I went to dental school [sigh].

Update (nextday): I’ve been toying over some various layouts for the LEDs. I most like the rectangle and hex-rectangle configurations, and am already working on assembly of the “mini” (prototype). Here are some random images of my thinking process.


Graphing Computer Usage

I enjoy writing Python scripts to analyze and display linear data. One of my favorite blog entries is Linear Data Smoothing with Python, developed for my homemade electrocardiogram project. I installed a program called TimeTrack.exe on my work computer. It basically logs whenever you open or close a program. The data output looks like this:

"Firefox","Prototype of a Digital Biopsy Device - Mozilla Firefox","05/19/2009  9:45a","05/19/2009  9:45a","766ms","0.0"
"Firefox","Dual-Channel Mobile Surface Electromyograph - Mozilla Firefox","05/19/2009  9:46a","05/19/2009  9:46a","797ms","0.0"
"Windows Explorer","","03/24/2008  9:30a","05/19/2009  9:48a","49d 6h 9m","20.7"
"Windows Explorer","09_04_07_RA_SA_AV","05/19/2009  8:48a","05/19/2009  8:48a","1.0s","0.0"
"Windows Explorer","Image003.jpg - Windows Picture and Fax Viewer","05/18/2009  4:03p","05/18/2009  4:03p","1.2s","0.0"

I have a 13 MB file containing lines like this which I parse, condense, analyze, and display with Python. The script finds the first and last entry time and creates a dictionary where keys are the hours between the 1st and last log lines, parses the log, determines which time block each entry belongs to, and increments the integer (value of the dictionary) for its respective key. Something similar is repeated, but with respect to days rather than hours. The result is:

The code I used to generate this graph is:

# This script analyzes data exported from "TimeTrack" (a free computer usage
# monitoring program for windows) and graphs the data visually.

import time, pylab, datetime, numpy

# This is my computer usage data.  Generate yours however you want.
allHours = ['2008_10_29 0', '2009_03_11 5', '2009_04_09 5', '2008_07_04 10',
'2008_12_18 9', '2009_01_30 12', '2008_09_04 7', '2008_05_17 1',
'2008_05_11 5', '2008_11_03 3', '2008_05_21 3', '2009_02_19 11',
'2008_08_15 13', '2008_04_02 4', '2008_07_16 5', '2008_09_16 8',
'2008_04_10 5', '2009_05_10 1', '2008_12_30 4', '2008_06_07 2',
'2008_11_23 0', '2008_08_03 0', '2008_04_30 4', '2008_07_28 9',
'2008_05_19 0', '2009_03_30 7', '2008_06_19 3', '2009_01_24 3',
'2008_08_23 6', '2008_12_01 0', '2009_02_23 6', '2008_11_27 0',
'2008_05_02 5', '2008_10_20 13', '2008_03_27 5', '2009_04_02 9',
'2009_02_21 0', '2008_09_13 1', '2008_12_13 0', '2009_04_14 11',
'2009_01_31 7', '2008_11_04 10', '2008_07_09 6', '2008_10_24 10',
'2009_02_22 0', '2008_09_25 12', '2008_12_25 0', '2008_05_26 4',
'2009_05_01 10', '2009_04_26 11', '2008_08_10 8', '2008_11_08 6',
'2008_07_21 12', '2009_04_21 3', '2009_05_13 8', '2009_02_02 8',
'2008_10_07 2', '2008_06_10 6', '2008_09_21 0', '2009_03_17 9',
'2008_08_30 7', '2008_11_28 4', '2009_02_14 0', '2009_01_22 6',
'2008_10_11 0', '2008_06_22 8', '2008_12_04 0', '2008_03_28 0',
'2009_04_07 2', '2008_09_10 0', '2008_05_15 5', '2008_08_18 12',
'2008_10_31 5', '2009_03_09 7', '2009_02_25 8', '2008_07_02 4',
'2008_12_16 7', '2008_09_06 2', '2009_01_26 5', '2009_04_19 0',
'2008_07_14 13', '2008_11_01 5', '2009_01_18 0', '2009_05_04 0',
'2008_08_13 10', '2009_02_27 3', '2009_01_16 12', '2008_09_18 8',
'2009_02_03 7', '2008_06_01 0', '2008_12_28 0', '2008_07_26 0',
'2008_11_21 1', '2008_08_01 8', '2008_04_28 3', '2009_05_16 0',
'2008_06_13 5', '2008_10_02 11', '2009_03_28 6', '2008_08_21 7',
'2009_01_13 6', '2008_11_25 4', '2008_06_25 1', '2008_10_22 11',
'2008_03_25 6', '2009_02_07 6', '2008_12_11 4', '2009_01_01 4',
'2008_09_15 2', '2009_02_05 12', '2008_07_07 9', '2009_04_12 0',
'2008_04_11 5', '2008_10_26 4', '2008_05_28 3', '2008_09_27 14',
'2009_05_03 0', '2008_12_23 5', '2009_05_12 10', '2008_11_14 3',
'2008_07_19 0', '2009_04_24 8', '2008_04_07 1', '2008_08_08 11',
'2008_06_04 0', '2009_05_15 12', '2009_03_23 13', '2009_02_01 10',
'2008_09_23 11', '2009_02_08 3', '2008_08_28 4', '2008_11_18 9',
'2008_07_31 7', '2008_10_13 0', '2008_06_16 9', '2009_03_27 6',
'2008_12_02 0', '2008_05_01 7', '2009_04_05 1', '2008_08_16 9',
'2009_03_15 0', '2008_04_16 6', '2008_10_17 4', '2008_06_28 5',
'2009_01_28 10', '2008_04_18 0', '2008_12_14 0', '2008_11_07 6',
'2009_04_17 7', '2008_04_14 7', '2008_07_12 0', '2009_01_15 7',
'2009_05_06 8', '2008_12_26 0', '2008_06_03 7', '2008_09_28 0',
'2008_05_25 4', '2008_08_07 8', '2008_04_26 7', '2008_07_24 1',
'2008_04_20 0', '2008_11_11 4', '2009_04_29 0', '2008_10_04 0',
'2009_05_18 9', '2009_03_18 4', '2008_06_15 8', '2009_02_13 6',
'2008_05_04 5', '2009_03_04 2', '2009_03_06 3', '2008_05_06 0',
'2008_08_27 11', '2008_04_22 0', '2009_03_26 6', '2008_03_31 9',
'2008_06_27 5', '2008_10_08 4', '2008_09_09 4', '2008_12_09 3',
'2008_05_10 0', '2008_05_14 5', '2009_04_10 0', '2009_01_11 0',
'2008_07_05 8', '2009_01_05 7', '2008_10_28 0', '2009_02_18 11',
'2009_03_10 7', '2008_05_30 3', '2008_09_05 7', '2008_12_21 6',
'2009_03_02 6', '2008_08_14 5', '2008_11_12 5', '2008_07_17 8',
'2008_04_05 6', '2009_04_22 11', '2009_05_09 0', '2008_06_06 0',
'2009_01_03 0', '2008_09_17 6', '2009_03_21 3', '2009_02_10 7',
'2008_05_08 4', '2008_08_02 0', '2008_11_16 0', '2008_07_29 12',
'2008_10_15 5', '2008_06_18 5', '2009_03_25 2', '2009_01_10 0',
'2009_04_03 5', '2008_08_22 7', '2009_03_13 11', '2008_10_19 0',
'2008_06_30 8', '2008_09_02 9', '2008_05_23 4', '2008_12_12 7',
'2008_07_10 11', '2008_11_05 8', '2008_04_12 4', '2009_04_15 7',
'2008_12_24 1', '2008_09_30 0', '2008_05_27 2', '2008_08_05 10',
'2008_04_24 6', '2009_04_27 6', '2008_07_22 3', '2008_11_09 1',
'2008_06_09 6', '2008_10_06 14', '2009_03_16 7', '2008_05_22 5',
'2009_01_29 12', '2008_11_29 4', '2008_04_09 7', '2008_08_25 12',
'2009_02_15 0', '2008_03_29 7', '2008_06_21 7', '2008_10_10 9',
'2008_05_12 6', '2009_02_16 10', '2008_09_11 11', '2008_12_07 0',
'2008_07_03 6', '2009_04_08 3', '2009_01_23 7', '2009_01_27 5',
'2008_10_30 0', '2009_03_08 0', '2009_01_21 8', '2008_12_19 0',
'2008_05_16 2', '2009_01_25 1', '2009_02_26 5', '2008_09_07 2',
'2008_04_03 1', '2008_08_12 6', '2008_04_13 10', '2008_11_02 0',
'2008_07_15 0', '2009_04_20 3', '2009_02_24 10', '2009_05_11 8',
'2008_12_31 8', '2008_04_15 7', '2008_09_19 10', '2009_01_19 0',
'2008_11_22 3', '2008_07_27 2', '2009_02_04 7', '2009_03_31 1',
'2008_05_24 3', '2008_10_01 8', '2008_06_12 6', '2009_01_12 11',
'2008_11_26 8', '2009_04_01 10', '2009_02_28 0', '2008_08_20 6',
'2008_10_21 10', '2008_06_24 4', '2008_03_26 4', '2008_12_10 0',
'2008_09_12 0', '2008_05_09 7', '2009_02_17 7', '2008_07_08 6',
'2008_10_25 5', '2009_04_13 9', '2009_05_02 0', '2008_12_22 8',
'2008_09_24 9', '2009_01_20 5', '2008_11_15 6', '2009_04_25 10',
'2008_08_11 9', '2008_04_06 8', '2008_07_20 1', '2009_03_22 3',
'2008_06_11 6', '2008_09_20 3', '2009_05_14 10', '2008_11_19 0',
'2008_08_31 2', '2009_02_09 8', '2008_10_12 0', '2008_04_25 5',
'2008_06_23 4', '2009_01_07 8', '2008_08_19 0', '2008_12_05 2',
'2008_07_01 8', '2008_10_16 6', '2009_04_06 3', '2009_03_14 5',
'2008_09_01 2', '2008_12_17 14', '2008_05_18 7', '2008_04_01 2',
'2009_04_18 0', '2008_04_17 0', '2008_07_13 0', '2008_06_02 10',
'2008_09_29 6', '2008_12_29 0', '2009_05_05 8', '2008_04_19 0',
'2009_04_30 8', '2008_08_06 4', '2008_11_20 0', '2008_07_25 6',
'2009_02_06 6', '2009_03_29 3', '2009_05_17 0', '2009_03_19 7',
'2008_10_03 1', '2008_06_14 3', '2008_05_07 5', '2008_08_26 3',
'2008_11_24 9', '2008_04_21 8', '2008_04_23 4', '2008_10_23 11',
'2008_06_26 4', '2008_03_24 8', '2008_12_08 5', '2008_09_14 2',
'2009_01_02 6', '2008_04_08 0', '2008_10_27 6', '2009_04_11 0',
'2008_07_06 0', '2008_12_20 3', '2009_04_23 6', '2008_09_26 9',
'2008_05_31 0', '2008_07_18 4', '2008_11_13 6', '2008_08_09 2',
'2008_04_04 0', '2009_03_20 5', '2008_09_22 7', '2009_05_08 9',
'2008_06_05 7', '2008_07_30 7', '2008_11_17 10', '2008_05_03 0',
'2008_08_29 3', '2009_02_11 12', '2009_01_08 8', '2008_06_17 0',
'2008_10_14 7', '2009_03_24 11', '2008_08_17 6', '2008_12_03 0',
'2009_01_09 4', '2008_05_29 5', '2008_06_29 9', '2008_10_18 5',
'2009_04_04 0', '2008_12_15 10', '2009_03_12 0', '2009_03_05 7',
'2008_05_20 4', '2008_09_03 7', '2009_03_07 8', '2009_01_14 6',
'2008_05_05 5', '2008_11_06 7', '2008_07_11 6', '2009_04_16 9',
'2009_02_20 0', '2008_12_27 0', '2009_01_17 0', '2009_05_07 7',
'2008_11_10 5', '2008_07_23 11', '2009_04_28 0', '2008_04_27 2',
'2008_08_04 0', '2009_03_01 11', '2008_10_05 0', '2008_06_08 8',
'2009_05_19 5', '2008_04_29 4', '2008_11_30 0', '2009_01_06 8',
'2009_02_12 3', '2008_08_24 2', '2009_03_03 10', '2008_10_09 6',
'2008_06_20 2', '2008_05_13 10', '2008_12_06 0', '2008_03_30 7']

def genTimes():
    ## opens  exported timetrack data (CSV) and re-saves a compressed version.
    print "ANALYZING..."
    f=open('timetrack.txt')
    raw=f.readlines()
    f.close()
    times=["05/15/2009 12:00am"] #start time
    for line in raw[1:]:
        if not line.count('","') == 5: continue
        test = line.strip("n")[1:-1].split('","')[-3].replace("  "," ")+"m"
        test = test.replace(" 0:"," 12:")
        times.append(test) #end time
        test = line.strip("n")[1:-1].split('","')[-4].replace("  "," ")+"m"
        test = test.replace(" 0:"," 12:")
        times.append(test) #start time

    times.sort()
    print "WRITING..."
    f=open('times.txt','w')
    f.write(str(times))
    f.close()

def loadTimes():
    ## loads the times from the compressed file.
    f=open("times.txt")
    times = eval(f.read())
    newtimes=[]
    f.close()
    for i in range(len(times)):
        if "s" in times[i]: print times[i]
        newtimes.append(datetime.datetime(*time.strptime(times[i],
                                        "%m/%d/%Y %I:%M%p")[0:5]))
        #if i&gt;1000: break #for debugging
    newtimes.sort()
    return newtimes

def linearize(times):
    ## does all the big math to calculate hours per day.
    for i in range(len(times)):
        times[i]=times[i]-datetime.timedelta(minutes=times[i].minute,
                                             seconds=times[i].second)
    hr = datetime.timedelta(hours=1)
    pos = times[0]-hr
    counts = {}
    days = {}
    lasthr=pos
    lastday=None
    while pos1:counts[pos]=1 #flatten
        if not daypos in days: days[daypos]=0
        if not lasthr == pos:
            if counts[pos]&gt;0:
                days[daypos]=days[daypos]+1
                lasthr=pos
        pos+=hr
    return days #[counts,days]

def genHours(days):
    ## outputs the hours per day as a file.
    out=""
    for day in days:
        print day
        out+="%s %in"%(day.strftime("%Y_%m_%d"),days[day])
    f=open('hours.txt','w')
    f.write(out)
    f.close()
    return

def smoothListGaussian(list,degree=7):
    ## (from an article I wrote) - Google "linear data smoothing with python".
    firstlen=len(list)
    window=degree*2-1
    weight=numpy.array([1.0]*window)
    weightGauss=[]
    for i in range(window):
     i=i-degree+1
     frac=i/float(window)
     gauss=1/(numpy.exp((4*(frac))**2))
     weightGauss.append(gauss)
    weight=numpy.array(weightGauss)*weight
    smoothed=[0.0]*(len(list)-window)
    for i in range(len(smoothed)):
     smoothed[i]=sum(numpy.array(list[i:i+window])*weight)/sum(weight)
    pad_before = [smoothed[0]]*((firstlen-len(smoothed))/2)
    pad_after  = [smoothed[-1]]*((firstlen-len(smoothed))/2+1)
    return pad_before+smoothed+pad_after

### IF YOU USE MY DATA, YOU ONLY USE THE FOLLOWING CODE ###

def graphIt():
    ## Graph the data!
    #f=open('hours.txt')
    #data=f.readlines()
    data=allHours
    data.sort()
    f.close()
    days,hours=[],[]
    for i in range(len(data)):
        day = data[i].split(" ")
        if int(day[1])&lt;4: continue
        days.append(datetime.datetime.strptime(day[0], "%Y_%m_%d"))
        hours.append(int(day[1]))
    fig=pylab.figure(figsize=(14,5))
    pylab.plot(days,smoothListGaussian(hours,1),'.',color='.5',label="single day")
    pylab.plot(days,smoothListGaussian(hours,1),'-',color='.8')
    pylab.plot(days,smoothListGaussian(hours,7),color='b',label="7-day gausian average")
    pylab.axhline(8,color='k',ls=":")
    pylab.title("Computer Usage at Work")
    pylab.ylabel("hours (rounded)")
    pylab.legend()
    pylab.show()
    return

#times = genTimes()
#genHours(linearize(loadTimes()))
graphIt()