The personal website of Scott W Harden
May 20th, 2009

Graphing Computer Usage

I enjoy writing Python scripts to analyze and display linear data. One of my favorite blog entries is Linear Data Smoothing with Python, developed for my homemade electrocardiogram project. I installed a program called TimeTrack.exe on my work computer. It basically logs whenever you open or close a program. The data output looks like this:

"Firefox","Prototype of a Digital Biopsy Device - Mozilla Firefox","05/19/2009  9:45a","05/19/2009  9:45a","766ms","0.0"
"Firefox","Dual-Channel Mobile Surface Electromyograph - Mozilla Firefox","05/19/2009  9:46a","05/19/2009  9:46a","797ms","0.0"
"Windows Explorer","","03/24/2008  9:30a","05/19/2009  9:48a","49d 6h 9m","20.7"
"Windows Explorer","09_04_07_RA_SA_AV","05/19/2009  8:48a","05/19/2009  8:48a","1.0s","0.0"
"Windows Explorer","Image003.jpg - Windows Picture and Fax Viewer","05/18/2009  4:03p","05/18/2009  4:03p","1.2s","0.0"

I have a 13 MB file containing lines like this which I parse, condense, analyze, and display with Python. The script finds the first and last entry time and creates a dictionary where keys are the hours between the 1st and last log lines, parses the log, determines which time block each entry belongs to, and increments the integer (value of the dictionary) for its respective key. Something similar is repeated, but with respect to days rather than hours. The result is:

The code I used to generate this graph is:

# This script analyzes data exported from "TimeTrack" (a free computer usage
# monitoring program for windows) and graphs the data visually.

import time, pylab, datetime, numpy

# This is my computer usage data.  Generate yours however you want.
allHours = ['2008_10_29 0', '2009_03_11 5', '2009_04_09 5', '2008_07_04 10',
'2008_12_18 9', '2009_01_30 12', '2008_09_04 7', '2008_05_17 1',
'2008_05_11 5', '2008_11_03 3', '2008_05_21 3', '2009_02_19 11',
'2008_08_15 13', '2008_04_02 4', '2008_07_16 5', '2008_09_16 8',
'2008_04_10 5', '2009_05_10 1', '2008_12_30 4', '2008_06_07 2',
'2008_11_23 0', '2008_08_03 0', '2008_04_30 4', '2008_07_28 9',
'2008_05_19 0', '2009_03_30 7', '2008_06_19 3', '2009_01_24 3',
'2008_08_23 6', '2008_12_01 0', '2009_02_23 6', '2008_11_27 0',
'2008_05_02 5', '2008_10_20 13', '2008_03_27 5', '2009_04_02 9',
'2009_02_21 0', '2008_09_13 1', '2008_12_13 0', '2009_04_14 11',
'2009_01_31 7', '2008_11_04 10', '2008_07_09 6', '2008_10_24 10',
'2009_02_22 0', '2008_09_25 12', '2008_12_25 0', '2008_05_26 4',
'2009_05_01 10', '2009_04_26 11', '2008_08_10 8', '2008_11_08 6',
'2008_07_21 12', '2009_04_21 3', '2009_05_13 8', '2009_02_02 8',
'2008_10_07 2', '2008_06_10 6', '2008_09_21 0', '2009_03_17 9',
'2008_08_30 7', '2008_11_28 4', '2009_02_14 0', '2009_01_22 6',
'2008_10_11 0', '2008_06_22 8', '2008_12_04 0', '2008_03_28 0',
'2009_04_07 2', '2008_09_10 0', '2008_05_15 5', '2008_08_18 12',
'2008_10_31 5', '2009_03_09 7', '2009_02_25 8', '2008_07_02 4',
'2008_12_16 7', '2008_09_06 2', '2009_01_26 5', '2009_04_19 0',
'2008_07_14 13', '2008_11_01 5', '2009_01_18 0', '2009_05_04 0',
'2008_08_13 10', '2009_02_27 3', '2009_01_16 12', '2008_09_18 8',
'2009_02_03 7', '2008_06_01 0', '2008_12_28 0', '2008_07_26 0',
'2008_11_21 1', '2008_08_01 8', '2008_04_28 3', '2009_05_16 0',
'2008_06_13 5', '2008_10_02 11', '2009_03_28 6', '2008_08_21 7',
'2009_01_13 6', '2008_11_25 4', '2008_06_25 1', '2008_10_22 11',
'2008_03_25 6', '2009_02_07 6', '2008_12_11 4', '2009_01_01 4',
'2008_09_15 2', '2009_02_05 12', '2008_07_07 9', '2009_04_12 0',
'2008_04_11 5', '2008_10_26 4', '2008_05_28 3', '2008_09_27 14',
'2009_05_03 0', '2008_12_23 5', '2009_05_12 10', '2008_11_14 3',
'2008_07_19 0', '2009_04_24 8', '2008_04_07 1', '2008_08_08 11',
'2008_06_04 0', '2009_05_15 12', '2009_03_23 13', '2009_02_01 10',
'2008_09_23 11', '2009_02_08 3', '2008_08_28 4', '2008_11_18 9',
'2008_07_31 7', '2008_10_13 0', '2008_06_16 9', '2009_03_27 6',
'2008_12_02 0', '2008_05_01 7', '2009_04_05 1', '2008_08_16 9',
'2009_03_15 0', '2008_04_16 6', '2008_10_17 4', '2008_06_28 5',
'2009_01_28 10', '2008_04_18 0', '2008_12_14 0', '2008_11_07 6',
'2009_04_17 7', '2008_04_14 7', '2008_07_12 0', '2009_01_15 7',
'2009_05_06 8', '2008_12_26 0', '2008_06_03 7', '2008_09_28 0',
'2008_05_25 4', '2008_08_07 8', '2008_04_26 7', '2008_07_24 1',
'2008_04_20 0', '2008_11_11 4', '2009_04_29 0', '2008_10_04 0',
'2009_05_18 9', '2009_03_18 4', '2008_06_15 8', '2009_02_13 6',
'2008_05_04 5', '2009_03_04 2', '2009_03_06 3', '2008_05_06 0',
'2008_08_27 11', '2008_04_22 0', '2009_03_26 6', '2008_03_31 9',
'2008_06_27 5', '2008_10_08 4', '2008_09_09 4', '2008_12_09 3',
'2008_05_10 0', '2008_05_14 5', '2009_04_10 0', '2009_01_11 0',
'2008_07_05 8', '2009_01_05 7', '2008_10_28 0', '2009_02_18 11',
'2009_03_10 7', '2008_05_30 3', '2008_09_05 7', '2008_12_21 6',
'2009_03_02 6', '2008_08_14 5', '2008_11_12 5', '2008_07_17 8',
'2008_04_05 6', '2009_04_22 11', '2009_05_09 0', '2008_06_06 0',
'2009_01_03 0', '2008_09_17 6', '2009_03_21 3', '2009_02_10 7',
'2008_05_08 4', '2008_08_02 0', '2008_11_16 0', '2008_07_29 12',
'2008_10_15 5', '2008_06_18 5', '2009_03_25 2', '2009_01_10 0',
'2009_04_03 5', '2008_08_22 7', '2009_03_13 11', '2008_10_19 0',
'2008_06_30 8', '2008_09_02 9', '2008_05_23 4', '2008_12_12 7',
'2008_07_10 11', '2008_11_05 8', '2008_04_12 4', '2009_04_15 7',
'2008_12_24 1', '2008_09_30 0', '2008_05_27 2', '2008_08_05 10',
'2008_04_24 6', '2009_04_27 6', '2008_07_22 3', '2008_11_09 1',
'2008_06_09 6', '2008_10_06 14', '2009_03_16 7', '2008_05_22 5',
'2009_01_29 12', '2008_11_29 4', '2008_04_09 7', '2008_08_25 12',
'2009_02_15 0', '2008_03_29 7', '2008_06_21 7', '2008_10_10 9',
'2008_05_12 6', '2009_02_16 10', '2008_09_11 11', '2008_12_07 0',
'2008_07_03 6', '2009_04_08 3', '2009_01_23 7', '2009_01_27 5',
'2008_10_30 0', '2009_03_08 0', '2009_01_21 8', '2008_12_19 0',
'2008_05_16 2', '2009_01_25 1', '2009_02_26 5', '2008_09_07 2',
'2008_04_03 1', '2008_08_12 6', '2008_04_13 10', '2008_11_02 0',
'2008_07_15 0', '2009_04_20 3', '2009_02_24 10', '2009_05_11 8',
'2008_12_31 8', '2008_04_15 7', '2008_09_19 10', '2009_01_19 0',
'2008_11_22 3', '2008_07_27 2', '2009_02_04 7', '2009_03_31 1',
'2008_05_24 3', '2008_10_01 8', '2008_06_12 6', '2009_01_12 11',
'2008_11_26 8', '2009_04_01 10', '2009_02_28 0', '2008_08_20 6',
'2008_10_21 10', '2008_06_24 4', '2008_03_26 4', '2008_12_10 0',
'2008_09_12 0', '2008_05_09 7', '2009_02_17 7', '2008_07_08 6',
'2008_10_25 5', '2009_04_13 9', '2009_05_02 0', '2008_12_22 8',
'2008_09_24 9', '2009_01_20 5', '2008_11_15 6', '2009_04_25 10',
'2008_08_11 9', '2008_04_06 8', '2008_07_20 1', '2009_03_22 3',
'2008_06_11 6', '2008_09_20 3', '2009_05_14 10', '2008_11_19 0',
'2008_08_31 2', '2009_02_09 8', '2008_10_12 0', '2008_04_25 5',
'2008_06_23 4', '2009_01_07 8', '2008_08_19 0', '2008_12_05 2',
'2008_07_01 8', '2008_10_16 6', '2009_04_06 3', '2009_03_14 5',
'2008_09_01 2', '2008_12_17 14', '2008_05_18 7', '2008_04_01 2',
'2009_04_18 0', '2008_04_17 0', '2008_07_13 0', '2008_06_02 10',
'2008_09_29 6', '2008_12_29 0', '2009_05_05 8', '2008_04_19 0',
'2009_04_30 8', '2008_08_06 4', '2008_11_20 0', '2008_07_25 6',
'2009_02_06 6', '2009_03_29 3', '2009_05_17 0', '2009_03_19 7',
'2008_10_03 1', '2008_06_14 3', '2008_05_07 5', '2008_08_26 3',
'2008_11_24 9', '2008_04_21 8', '2008_04_23 4', '2008_10_23 11',
'2008_06_26 4', '2008_03_24 8', '2008_12_08 5', '2008_09_14 2',
'2009_01_02 6', '2008_04_08 0', '2008_10_27 6', '2009_04_11 0',
'2008_07_06 0', '2008_12_20 3', '2009_04_23 6', '2008_09_26 9',
'2008_05_31 0', '2008_07_18 4', '2008_11_13 6', '2008_08_09 2',
'2008_04_04 0', '2009_03_20 5', '2008_09_22 7', '2009_05_08 9',
'2008_06_05 7', '2008_07_30 7', '2008_11_17 10', '2008_05_03 0',
'2008_08_29 3', '2009_02_11 12', '2009_01_08 8', '2008_06_17 0',
'2008_10_14 7', '2009_03_24 11', '2008_08_17 6', '2008_12_03 0',
'2009_01_09 4', '2008_05_29 5', '2008_06_29 9', '2008_10_18 5',
'2009_04_04 0', '2008_12_15 10', '2009_03_12 0', '2009_03_05 7',
'2008_05_20 4', '2008_09_03 7', '2009_03_07 8', '2009_01_14 6',
'2008_05_05 5', '2008_11_06 7', '2008_07_11 6', '2009_04_16 9',
'2009_02_20 0', '2008_12_27 0', '2009_01_17 0', '2009_05_07 7',
'2008_11_10 5', '2008_07_23 11', '2009_04_28 0', '2008_04_27 2',
'2008_08_04 0', '2009_03_01 11', '2008_10_05 0', '2008_06_08 8',
'2009_05_19 5', '2008_04_29 4', '2008_11_30 0', '2009_01_06 8',
'2009_02_12 3', '2008_08_24 2', '2009_03_03 10', '2008_10_09 6',
'2008_06_20 2', '2008_05_13 10', '2008_12_06 0', '2008_03_30 7']

def genTimes():
    ## opens  exported timetrack data (CSV) and re-saves a compressed version.
    print "ANALYZING..."
    f=open('timetrack.txt')
    raw=f.readlines()
    f.close()
    times=["05/15/2009 12:00am"] #start time
    for line in raw[1:]:
        if not line.count('","') == 5: continue
        test = line.strip("n")[1:-1].split('","')[-3].replace("  "," ")+"m"
        test = test.replace(" 0:"," 12:")
        times.append(test) #end time
        test = line.strip("n")[1:-1].split('","')[-4].replace("  "," ")+"m"
        test = test.replace(" 0:"," 12:")
        times.append(test) #start time

    times.sort()
    print "WRITING..."
    f=open('times.txt','w')
    f.write(str(times))
    f.close()

def loadTimes():
    ## loads the times from the compressed file.
    f=open("times.txt")
    times = eval(f.read())
    newtimes=[]
    f.close()
    for i in range(len(times)):
        if "s" in times[i]: print times[i]
        newtimes.append(datetime.datetime(*time.strptime(times[i],
                                        "%m/%d/%Y %I:%M%p")[0:5]))
        #if i>1000: break #for debugging
    newtimes.sort()
    return newtimes

def linearize(times):
    ## does all the big math to calculate hours per day.
    for i in range(len(times)):
        times[i]=times[i]-datetime.timedelta(minutes=times[i].minute,
                                             seconds=times[i].second)
    hr = datetime.timedelta(hours=1)
    pos = times[0]-hr
    counts = {}
    days = {}
    lasthr=pos
    lastday=None
    while pos1:counts[pos]=1 #flatten
        if not daypos in days: days[daypos]=0
        if not lasthr == pos:
            if counts[pos]>0:
                days[daypos]=days[daypos]+1
                lasthr=pos
        pos+=hr
    return days #[counts,days]

def genHours(days):
    ## outputs the hours per day as a file.
    out=""
    for day in days:
        print day
        out+="%s %in"%(day.strftime("%Y_%m_%d"),days[day])
    f=open('hours.txt','w')
    f.write(out)
    f.close()
    return

def smoothListGaussian(list,degree=7):
    ## (from an article I wrote) - Google "linear data smoothing with python".
    firstlen=len(list)
    window=degree*2-1
    weight=numpy.array([1.0]*window)
    weightGauss=[]
    for i in range(window):
     i=i-degree+1
     frac=i/float(window)
     gauss=1/(numpy.exp((4*(frac))**2))
     weightGauss.append(gauss)
    weight=numpy.array(weightGauss)*weight
    smoothed=[0.0]*(len(list)-window)
    for i in range(len(smoothed)):
     smoothed[i]=sum(numpy.array(list[i:i+window])*weight)/sum(weight)
    pad_before = [smoothed[0]]*((firstlen-len(smoothed))/2)
    pad_after  = [smoothed[-1]]*((firstlen-len(smoothed))/2+1)
    return pad_before+smoothed+pad_after

### IF YOU USE MY DATA, YOU ONLY USE THE FOLLOWING CODE ###

def graphIt():
    ## Graph the data!
    #f=open('hours.txt')
    #data=f.readlines()
    data=allHours
    data.sort()
    f.close()
    days,hours=[],[]
    for i in range(len(data)):
        day = data[i].split(" ")
        if int(day[1])<4: continue
        days.append(datetime.datetime.strptime(day[0], "%Y_%m_%d"))
        hours.append(int(day[1]))
    fig=pylab.figure(figsize=(14,5))
    pylab.plot(days,smoothListGaussian(hours,1),'.',color='.5',label="single day")
    pylab.plot(days,smoothListGaussian(hours,1),'-',color='.8')
    pylab.plot(days,smoothListGaussian(hours,7),color='b',label="7-day gausian average")
    pylab.axhline(8,color='k',ls=":")
    pylab.title("Computer Usage at Work")
    pylab.ylabel("hours (rounded)")
    pylab.legend()
    pylab.show()
    return

#times = genTimes()
#genHours(linearize(loadTimes()))
graphIt()
Markdown source code last modified on January 18th, 2021
---
title: Graphing Computer Usage
date: 2009-05-20 08:44:57
tags: python, old
---

# Graphing Computer Usage

__I enjoy writing Python scripts to analyze and display linear data.__ One of my favorite blog entries is [Linear Data Smoothing with Python](http://www.swharden.com/blog/2008-11-17-linear-data-smoothing-in-python/), developed for my [homemade electrocardiogram](http://www.swharden.com/blog/category/diy-ecg-home-made-electrocardiogram/) project. I installed a program called TimeTrack.exe on my work computer. It basically logs whenever you open or close a program. The data output looks like this:

```
"Firefox","Prototype of a Digital Biopsy Device - Mozilla Firefox","05/19/2009  9:45a","05/19/2009  9:45a","766ms","0.0"
"Firefox","Dual-Channel Mobile Surface Electromyograph - Mozilla Firefox","05/19/2009  9:46a","05/19/2009  9:46a","797ms","0.0"
"Windows Explorer","","03/24/2008  9:30a","05/19/2009  9:48a","49d 6h 9m","20.7"
"Windows Explorer","09_04_07_RA_SA_AV","05/19/2009  8:48a","05/19/2009  8:48a","1.0s","0.0"
"Windows Explorer","Image003.jpg - Windows Picture and Fax Viewer","05/18/2009  4:03p","05/18/2009  4:03p","1.2s","0.0"
```

__I have a 13 MB file containing lines like this__ which I parse, condense, analyze, and display with Python. The script finds the first and last entry time and creates a dictionary where keys are the hours between the 1st and last log lines, parses the log, determines which time block each entry belongs to, and increments the integer (value of the dictionary) for its respective key. Something similar is repeated, but with respect to days rather than hours. The result is:

<div class="text-center">

[![](compusage_white_thumb.jpg)](compusage_white.png)

</div>

The code I used to generate this graph is:

```python
# This script analyzes data exported from "TimeTrack" (a free computer usage
# monitoring program for windows) and graphs the data visually.

import time, pylab, datetime, numpy

# This is my computer usage data.  Generate yours however you want.
allHours = ['2008_10_29 0', '2009_03_11 5', '2009_04_09 5', '2008_07_04 10',
'2008_12_18 9', '2009_01_30 12', '2008_09_04 7', '2008_05_17 1',
'2008_05_11 5', '2008_11_03 3', '2008_05_21 3', '2009_02_19 11',
'2008_08_15 13', '2008_04_02 4', '2008_07_16 5', '2008_09_16 8',
'2008_04_10 5', '2009_05_10 1', '2008_12_30 4', '2008_06_07 2',
'2008_11_23 0', '2008_08_03 0', '2008_04_30 4', '2008_07_28 9',
'2008_05_19 0', '2009_03_30 7', '2008_06_19 3', '2009_01_24 3',
'2008_08_23 6', '2008_12_01 0', '2009_02_23 6', '2008_11_27 0',
'2008_05_02 5', '2008_10_20 13', '2008_03_27 5', '2009_04_02 9',
'2009_02_21 0', '2008_09_13 1', '2008_12_13 0', '2009_04_14 11',
'2009_01_31 7', '2008_11_04 10', '2008_07_09 6', '2008_10_24 10',
'2009_02_22 0', '2008_09_25 12', '2008_12_25 0', '2008_05_26 4',
'2009_05_01 10', '2009_04_26 11', '2008_08_10 8', '2008_11_08 6',
'2008_07_21 12', '2009_04_21 3', '2009_05_13 8', '2009_02_02 8',
'2008_10_07 2', '2008_06_10 6', '2008_09_21 0', '2009_03_17 9',
'2008_08_30 7', '2008_11_28 4', '2009_02_14 0', '2009_01_22 6',
'2008_10_11 0', '2008_06_22 8', '2008_12_04 0', '2008_03_28 0',
'2009_04_07 2', '2008_09_10 0', '2008_05_15 5', '2008_08_18 12',
'2008_10_31 5', '2009_03_09 7', '2009_02_25 8', '2008_07_02 4',
'2008_12_16 7', '2008_09_06 2', '2009_01_26 5', '2009_04_19 0',
'2008_07_14 13', '2008_11_01 5', '2009_01_18 0', '2009_05_04 0',
'2008_08_13 10', '2009_02_27 3', '2009_01_16 12', '2008_09_18 8',
'2009_02_03 7', '2008_06_01 0', '2008_12_28 0', '2008_07_26 0',
'2008_11_21 1', '2008_08_01 8', '2008_04_28 3', '2009_05_16 0',
'2008_06_13 5', '2008_10_02 11', '2009_03_28 6', '2008_08_21 7',
'2009_01_13 6', '2008_11_25 4', '2008_06_25 1', '2008_10_22 11',
'2008_03_25 6', '2009_02_07 6', '2008_12_11 4', '2009_01_01 4',
'2008_09_15 2', '2009_02_05 12', '2008_07_07 9', '2009_04_12 0',
'2008_04_11 5', '2008_10_26 4', '2008_05_28 3', '2008_09_27 14',
'2009_05_03 0', '2008_12_23 5', '2009_05_12 10', '2008_11_14 3',
'2008_07_19 0', '2009_04_24 8', '2008_04_07 1', '2008_08_08 11',
'2008_06_04 0', '2009_05_15 12', '2009_03_23 13', '2009_02_01 10',
'2008_09_23 11', '2009_02_08 3', '2008_08_28 4', '2008_11_18 9',
'2008_07_31 7', '2008_10_13 0', '2008_06_16 9', '2009_03_27 6',
'2008_12_02 0', '2008_05_01 7', '2009_04_05 1', '2008_08_16 9',
'2009_03_15 0', '2008_04_16 6', '2008_10_17 4', '2008_06_28 5',
'2009_01_28 10', '2008_04_18 0', '2008_12_14 0', '2008_11_07 6',
'2009_04_17 7', '2008_04_14 7', '2008_07_12 0', '2009_01_15 7',
'2009_05_06 8', '2008_12_26 0', '2008_06_03 7', '2008_09_28 0',
'2008_05_25 4', '2008_08_07 8', '2008_04_26 7', '2008_07_24 1',
'2008_04_20 0', '2008_11_11 4', '2009_04_29 0', '2008_10_04 0',
'2009_05_18 9', '2009_03_18 4', '2008_06_15 8', '2009_02_13 6',
'2008_05_04 5', '2009_03_04 2', '2009_03_06 3', '2008_05_06 0',
'2008_08_27 11', '2008_04_22 0', '2009_03_26 6', '2008_03_31 9',
'2008_06_27 5', '2008_10_08 4', '2008_09_09 4', '2008_12_09 3',
'2008_05_10 0', '2008_05_14 5', '2009_04_10 0', '2009_01_11 0',
'2008_07_05 8', '2009_01_05 7', '2008_10_28 0', '2009_02_18 11',
'2009_03_10 7', '2008_05_30 3', '2008_09_05 7', '2008_12_21 6',
'2009_03_02 6', '2008_08_14 5', '2008_11_12 5', '2008_07_17 8',
'2008_04_05 6', '2009_04_22 11', '2009_05_09 0', '2008_06_06 0',
'2009_01_03 0', '2008_09_17 6', '2009_03_21 3', '2009_02_10 7',
'2008_05_08 4', '2008_08_02 0', '2008_11_16 0', '2008_07_29 12',
'2008_10_15 5', '2008_06_18 5', '2009_03_25 2', '2009_01_10 0',
'2009_04_03 5', '2008_08_22 7', '2009_03_13 11', '2008_10_19 0',
'2008_06_30 8', '2008_09_02 9', '2008_05_23 4', '2008_12_12 7',
'2008_07_10 11', '2008_11_05 8', '2008_04_12 4', '2009_04_15 7',
'2008_12_24 1', '2008_09_30 0', '2008_05_27 2', '2008_08_05 10',
'2008_04_24 6', '2009_04_27 6', '2008_07_22 3', '2008_11_09 1',
'2008_06_09 6', '2008_10_06 14', '2009_03_16 7', '2008_05_22 5',
'2009_01_29 12', '2008_11_29 4', '2008_04_09 7', '2008_08_25 12',
'2009_02_15 0', '2008_03_29 7', '2008_06_21 7', '2008_10_10 9',
'2008_05_12 6', '2009_02_16 10', '2008_09_11 11', '2008_12_07 0',
'2008_07_03 6', '2009_04_08 3', '2009_01_23 7', '2009_01_27 5',
'2008_10_30 0', '2009_03_08 0', '2009_01_21 8', '2008_12_19 0',
'2008_05_16 2', '2009_01_25 1', '2009_02_26 5', '2008_09_07 2',
'2008_04_03 1', '2008_08_12 6', '2008_04_13 10', '2008_11_02 0',
'2008_07_15 0', '2009_04_20 3', '2009_02_24 10', '2009_05_11 8',
'2008_12_31 8', '2008_04_15 7', '2008_09_19 10', '2009_01_19 0',
'2008_11_22 3', '2008_07_27 2', '2009_02_04 7', '2009_03_31 1',
'2008_05_24 3', '2008_10_01 8', '2008_06_12 6', '2009_01_12 11',
'2008_11_26 8', '2009_04_01 10', '2009_02_28 0', '2008_08_20 6',
'2008_10_21 10', '2008_06_24 4', '2008_03_26 4', '2008_12_10 0',
'2008_09_12 0', '2008_05_09 7', '2009_02_17 7', '2008_07_08 6',
'2008_10_25 5', '2009_04_13 9', '2009_05_02 0', '2008_12_22 8',
'2008_09_24 9', '2009_01_20 5', '2008_11_15 6', '2009_04_25 10',
'2008_08_11 9', '2008_04_06 8', '2008_07_20 1', '2009_03_22 3',
'2008_06_11 6', '2008_09_20 3', '2009_05_14 10', '2008_11_19 0',
'2008_08_31 2', '2009_02_09 8', '2008_10_12 0', '2008_04_25 5',
'2008_06_23 4', '2009_01_07 8', '2008_08_19 0', '2008_12_05 2',
'2008_07_01 8', '2008_10_16 6', '2009_04_06 3', '2009_03_14 5',
'2008_09_01 2', '2008_12_17 14', '2008_05_18 7', '2008_04_01 2',
'2009_04_18 0', '2008_04_17 0', '2008_07_13 0', '2008_06_02 10',
'2008_09_29 6', '2008_12_29 0', '2009_05_05 8', '2008_04_19 0',
'2009_04_30 8', '2008_08_06 4', '2008_11_20 0', '2008_07_25 6',
'2009_02_06 6', '2009_03_29 3', '2009_05_17 0', '2009_03_19 7',
'2008_10_03 1', '2008_06_14 3', '2008_05_07 5', '2008_08_26 3',
'2008_11_24 9', '2008_04_21 8', '2008_04_23 4', '2008_10_23 11',
'2008_06_26 4', '2008_03_24 8', '2008_12_08 5', '2008_09_14 2',
'2009_01_02 6', '2008_04_08 0', '2008_10_27 6', '2009_04_11 0',
'2008_07_06 0', '2008_12_20 3', '2009_04_23 6', '2008_09_26 9',
'2008_05_31 0', '2008_07_18 4', '2008_11_13 6', '2008_08_09 2',
'2008_04_04 0', '2009_03_20 5', '2008_09_22 7', '2009_05_08 9',
'2008_06_05 7', '2008_07_30 7', '2008_11_17 10', '2008_05_03 0',
'2008_08_29 3', '2009_02_11 12', '2009_01_08 8', '2008_06_17 0',
'2008_10_14 7', '2009_03_24 11', '2008_08_17 6', '2008_12_03 0',
'2009_01_09 4', '2008_05_29 5', '2008_06_29 9', '2008_10_18 5',
'2009_04_04 0', '2008_12_15 10', '2009_03_12 0', '2009_03_05 7',
'2008_05_20 4', '2008_09_03 7', '2009_03_07 8', '2009_01_14 6',
'2008_05_05 5', '2008_11_06 7', '2008_07_11 6', '2009_04_16 9',
'2009_02_20 0', '2008_12_27 0', '2009_01_17 0', '2009_05_07 7',
'2008_11_10 5', '2008_07_23 11', '2009_04_28 0', '2008_04_27 2',
'2008_08_04 0', '2009_03_01 11', '2008_10_05 0', '2008_06_08 8',
'2009_05_19 5', '2008_04_29 4', '2008_11_30 0', '2009_01_06 8',
'2009_02_12 3', '2008_08_24 2', '2009_03_03 10', '2008_10_09 6',
'2008_06_20 2', '2008_05_13 10', '2008_12_06 0', '2008_03_30 7']

def genTimes():
    ## opens  exported timetrack data (CSV) and re-saves a compressed version.
    print "ANALYZING..."
    f=open('timetrack.txt')
    raw=f.readlines()
    f.close()
    times=["05/15/2009 12:00am"] #start time
    for line in raw[1:]:
        if not line.count('","') == 5: continue
        test = line.strip("n")[1:-1].split('","')[-3].replace("  "," ")+"m"
        test = test.replace(" 0:"," 12:")
        times.append(test) #end time
        test = line.strip("n")[1:-1].split('","')[-4].replace("  "," ")+"m"
        test = test.replace(" 0:"," 12:")
        times.append(test) #start time

    times.sort()
    print "WRITING..."
    f=open('times.txt','w')
    f.write(str(times))
    f.close()

def loadTimes():
    ## loads the times from the compressed file.
    f=open("times.txt")
    times = eval(f.read())
    newtimes=[]
    f.close()
    for i in range(len(times)):
        if "s" in times[i]: print times[i]
        newtimes.append(datetime.datetime(*time.strptime(times[i],
                                        "%m/%d/%Y %I:%M%p")[0:5]))
        #if i&gt;1000: break #for debugging
    newtimes.sort()
    return newtimes

def linearize(times):
    ## does all the big math to calculate hours per day.
    for i in range(len(times)):
        times[i]=times[i]-datetime.timedelta(minutes=times[i].minute,
                                             seconds=times[i].second)
    hr = datetime.timedelta(hours=1)
    pos = times[0]-hr
    counts = {}
    days = {}
    lasthr=pos
    lastday=None
    while pos1:counts[pos]=1 #flatten
        if not daypos in days: days[daypos]=0
        if not lasthr == pos:
            if counts[pos]&gt;0:
                days[daypos]=days[daypos]+1
                lasthr=pos
        pos+=hr
    return days #[counts,days]

def genHours(days):
    ## outputs the hours per day as a file.
    out=""
    for day in days:
        print day
        out+="%s %in"%(day.strftime("%Y_%m_%d"),days[day])
    f=open('hours.txt','w')
    f.write(out)
    f.close()
    return

def smoothListGaussian(list,degree=7):
    ## (from an article I wrote) - Google "linear data smoothing with python".
    firstlen=len(list)
    window=degree*2-1
    weight=numpy.array([1.0]*window)
    weightGauss=[]
    for i in range(window):
     i=i-degree+1
     frac=i/float(window)
     gauss=1/(numpy.exp((4*(frac))**2))
     weightGauss.append(gauss)
    weight=numpy.array(weightGauss)*weight
    smoothed=[0.0]*(len(list)-window)
    for i in range(len(smoothed)):
     smoothed[i]=sum(numpy.array(list[i:i+window])*weight)/sum(weight)
    pad_before = [smoothed[0]]*((firstlen-len(smoothed))/2)
    pad_after  = [smoothed[-1]]*((firstlen-len(smoothed))/2+1)
    return pad_before+smoothed+pad_after

### IF YOU USE MY DATA, YOU ONLY USE THE FOLLOWING CODE ###

def graphIt():
    ## Graph the data!
    #f=open('hours.txt')
    #data=f.readlines()
    data=allHours
    data.sort()
    f.close()
    days,hours=[],[]
    for i in range(len(data)):
        day = data[i].split(" ")
        if int(day[1])&lt;4: continue
        days.append(datetime.datetime.strptime(day[0], "%Y_%m_%d"))
        hours.append(int(day[1]))
    fig=pylab.figure(figsize=(14,5))
    pylab.plot(days,smoothListGaussian(hours,1),'.',color='.5',label="single day")
    pylab.plot(days,smoothListGaussian(hours,1),'-',color='.8')
    pylab.plot(days,smoothListGaussian(hours,7),color='b',label="7-day gausian average")
    pylab.axhline(8,color='k',ls=":")
    pylab.title("Computer Usage at Work")
    pylab.ylabel("hours (rounded)")
    pylab.legend()
    pylab.show()
    return

#times = genTimes()
#genHours(linearize(loadTimes()))
graphIt()
```
May 15th, 2009

Removing Textile Markup From Wordpress Entries

I realized that the C code from yesterday wasn't showing-up properly because of textile, a rapid, inline, tag-based formatting system. While it's fun and convenient to use, it's not always practical. The problem I was having was that in C code, variable names (such as delay) were becoming irrevocably italicized, and nothing I did could prevent textile from ignoring code while styling text. The kicker is that I couldn't disable it easily, because I've been writing in this style for over four years! I decided that the time was now to put my mad Python skills to the test and write code to handle the conversion from textile-format to raw HTML. I accomplished this feat in a number of steps. Yeah, I could have done hours of research to find a "faster way", but it simply wouldn't have been as creative. In a nutshell, I backed-up the SQL database using PHPMyAdmin to a single "x.sql" file. I then wrote a pythons script to parse this [massive] file and output "o.sql", the same data but with all of the textile tags I commonly used replaced by their HTML equivalent. It's not 100% perfect, but it's 99.999% perfect. I'll accept that. The output? You're viewing it! Here's the code I used to do it:

## This Python script removes *SOME* textile formatting from Wordpress
## backups in plain text SQL format (dumped from PHP MyAdmin). Specifically,
## it corrects bold and itallic fonts and corrects links. It should be easy
## to expand if you need to do something else with it.

infile = 'x.sql'

replacements=   ["r"," "],["n"," n "],["*:","* :"],["_:","_ :"],
                ["n","&lt;br&gt;n"],["&gt;*","&gt; *"],["*&lt; ","* &lt;"],
                ["&gt;_","&gt; _"],["_&lt; ","_ &lt;"],
                [" *"," &lt;b&gt;"],["* "," "],[" _"," &lt;i&gt;"],["_ ","&lt;/i&gt; "]
                #These are the easy replacements

def fixLinks(line):
    ## replace ["links":URL] with [&lt;a href="URL"&gt;links&lt;/a&gt;]. ##
    words = line.split(" ")
    for i in range(len(words)):
        word = words[i]
        if '":' in word:
            upto=1
            while (word.count('"')&amp;lt;2):
                word = words[i-upto]+" "+word
                upto+=1
            word_orig = word
            extra=""
            word = word.split('":')
            word[0]=word[0][1:]
            for char in ".),'":
                if word[1][-1]==char: extra=char
            if len(extra)&gt;0: word[1]=word[1][:-1]
            word_new='&lt;a href="%s"&gt;%s&lt;/a&gt;'%(word[1],word[0])+extra
            line=line.replace(word_orig,word_new)
    return line

def stripTextile(orig):
    ## Handle the replacements and link fixing for each line. ##
    if not orig.count("', '") == 13: return orig #non-normal post
    line=orig
    temp = line.split
    line = line.split("', '",5)[2]
    if len(line)&amp;lt;10:return orig #non-normal post
    origline = line
    line = " "+line
    for replacement in replacements:
        line = line.replace(replacement[0],replacement[1])
    line=fixLinks(line)
    line = orig.replace(origline,line)
    return line

f=open(infile)
raw=f.readlines()
f.close
posts=0
for raw_i in range(len(raw)):
    if raw[raw_i][:11]=="INSERT INTO":
        if "wp_posts" in raw[raw_i]: #if it's a post, handle it!
            posts+=1
            print "on post",posts
            raw[raw_i]=stripTextile(raw[raw_i])

print "WRITING..."
out = ""
for line in raw:
    out+=line
f=open('o.sql','w')
f.write(out)
f.close()

I certainly held my breath while the thing ran. As I previously mentioned, this thing modified SQL tables. Therefore, when I uploaded the "corrected" versions, I kept breaking the site until I got all the bugs worked out. Here's an image from earlier today when my site was totally dead (0 blog posts)

Markdown source code last modified on January 18th, 2021
---
title: Removing Textile Markup From Wordpress Entries
date: 2009-05-15 17:56:32
tags: old
---

# Removing Textile Markup From Wordpress Entries

__I realized that the C code from yesterday wasn't showing-up properly__ because of [textile](http://wordpress.org/tags/textile), a rapid, inline, tag-based formatting system.  While it's fun and convenient to use, it's not always practical.  The problem I was having was that in C code, variable names (such as _delay_) were becoming irrevocably italicized, and nothing I did could prevent textile from ignoring code while styling text.  The kicker is that I couldn't disable it easily, because I've been writing in this style for __over four years!__  I decided that the time was now to put my mad Python skills to the test and write code to handle the conversion from textile-format to raw HTML.
__I accomplished this feat__ in a number of steps.  Yeah, I could have done hours of research to find a "faster way", but it simply wouldn't have been as creative.  In a nutshell, I backed-up the SQL database using [PHPMyAdmin](http://en.wikipedia.org/wiki/PhpMyAdmin) to a single "x.sql" file.  I then wrote a pythons script to parse this [massive] file and output "o.sql", the same data but with all of the textile tags I commonly used replaced by their HTML equivalent.  It's not 100% perfect, but it's 99.999% perfect.  I'll accept that.  The output?  You're viewing it!  Here's the code I used to do it:

```python
## This Python script removes *SOME* textile formatting from Wordpress
## backups in plain text SQL format (dumped from PHP MyAdmin). Specifically,
## it corrects bold and itallic fonts and corrects links. It should be easy
## to expand if you need to do something else with it.

infile = 'x.sql'

replacements=   ["r"," "],["n"," n "],["*:","* :"],["_:","_ :"],
                ["n","&lt;br&gt;n"],["&gt;*","&gt; *"],["*&lt; ","* &lt;"],
                ["&gt;_","&gt; _"],["_&lt; ","_ &lt;"],
                [" *"," &lt;b&gt;"],["* "," "],[" _"," &lt;i&gt;"],["_ ","&lt;/i&gt; "]
                #These are the easy replacements

def fixLinks(line):
    ## replace ["links":URL] with [&lt;a href="URL"&gt;links&lt;/a&gt;]. ##
    words = line.split(" ")
    for i in range(len(words)):
        word = words[i]
        if '":' in word:
            upto=1
            while (word.count('"')&amp;lt;2):
                word = words[i-upto]+" "+word
                upto+=1
            word_orig = word
            extra=""
            word = word.split('":')
            word[0]=word[0][1:]
            for char in ".),'":
                if word[1][-1]==char: extra=char
            if len(extra)&gt;0: word[1]=word[1][:-1]
            word_new='&lt;a href="%s"&gt;%s&lt;/a&gt;'%(word[1],word[0])+extra
            line=line.replace(word_orig,word_new)
    return line

def stripTextile(orig):
    ## Handle the replacements and link fixing for each line. ##
    if not orig.count("', '") == 13: return orig #non-normal post
    line=orig
    temp = line.split
    line = line.split("', '",5)[2]
    if len(line)&amp;lt;10:return orig #non-normal post
    origline = line
    line = " "+line
    for replacement in replacements:
        line = line.replace(replacement[0],replacement[1])
    line=fixLinks(line)
    line = orig.replace(origline,line)
    return line

f=open(infile)
raw=f.readlines()
f.close
posts=0
for raw_i in range(len(raw)):
    if raw[raw_i][:11]=="INSERT INTO":
        if "wp_posts" in raw[raw_i]: #if it's a post, handle it!
            posts+=1
            print "on post",posts
            raw[raw_i]=stripTextile(raw[raw_i])

print "WRITING..."
out = ""
for line in raw:
    out+=line
f=open('o.sql','w')
f.write(out)
f.close()

```

__I certainly held my breath while the thing ran.__  As I previously mentioned, this thing modified SQL tables.  Therefore, when I uploaded the "corrected" versions, I kept breaking the site until  I got all the bugs worked out.  Here's an image from earlier today when my site was totally dead (0 blog posts)

<div class="text-center img-border">

[![](hostingwork_thumb.jpg)](hostingwork.jpg)

</div>
May 14th, 2009

Simple Case AVR/PC Serial Communication via MAX232

I recently had the desire to be able to see data from an ATMEL AVR microcontroller (the ATTiny2313) for development and debugging purposes. I wanted an easy way to have my microcontroller talk to my PC (and vise versa) with a minimum number of parts. The easiest way to do this was to utilize the UART capabilities of the ATTiny2313 to talk to my PC through the serial port. One problem is that the ATTiny2313(as with most microcontrollers) puts out 5V for "high" (on) and 0V for "low" (off). The RS-232 standard (which PC serial ports use) required -15V for high and +15v for low! Obviously the microcontroller needs some help to achieve this. The easiest way was to use the MAX232 serial level converter which costs about 3 bucks at DigiKey. Note that it requires a few 10uF capacitors to function properly.

Here's a more general schematic:

I connected my ATTiny2313 to the MAX232 in a very standard way. (photo) MAX232 pins 13 and 14 go to the serial port, and the ATTiny2313 pins 2 and 3 go to the MAX232 pins 12 and 11 respectively. I will note that they used a oscillator value (3.6864MHz) different than mine (9.216MHz).

Determining the speed of serial communication is important. This is dependent on your oscillator frequency! I said I used a 9.216Mhz oscillator. First, a crystal or ceramic oscillator is required over the internal RC oscillator because the internal RC oscillator is not accurate enough for serial communication. The oscillator you select should be a perfect multiple of 1.8432MHz. Mine is 5x this value. Many people use 2x this value (3.6864Mhz) and that's okay! You just have to make sure your microchip knows (1) to use the external oscillator (google around for how to burn the fuses on your chip to do this) and (2) what the frequency of your oscillator is and how fast it should be sending data. This is done by setting the UBRRL value. The formula to do this is here:

The datasheet of your microcontroller may list a lot of common crystal frequencies, bandwidths, and their appropriate UBRR values. However my datasheet lacked an entry for a 9.216MHz crystal, so I had to do the math myself. I Googled around and no "table" is available! Why not make one? (picture, below). Anyway, for my case I determined that if I set the UBRR value to 239, I could transmit data at 2800 baud (bits/second). This is slow enough to ensure accuracy, but fast enough to quickly dump a large amount of text to a PC terminal.

AVR Baud Calculator

This will make your life easier. The page wormfood.net/avrbaudcalc.php has a chart of common crystals and the baud rates they work best with! Try to pick a combination that provides the least error possible...

This is the bare-minimum code to test out my setup. Just load the code (written in C, compiled with avr-gcc) onto your chip and it's ready to go. Be sure you set your fuses to use an external oscillator and that you set your UBRRL value correctly.

#include <avr/io.h>  
#include <avr/interrupt.h>  
#include <util/delay.h>  

int main (void)  
{  
  unsigned char data=0;  
  UBRRL = 239;  
  UCSRB = (1 < < RXEN) | (1 << TXEN);  
  UCSRC = (1 < < UCSZ1) | (1 << UCSZ0);  

  for (;;)  
  {  
    if (data>'Z'||data< 'A')  
    {  
      UDR = 10; UDR = 13; data='A';_delay_ms(100);  
    }  

    UDR = data;  
    data += 1;  
    _delay_ms(100);  
  }  
}  

Once you load it, it's ready to roll! It continuously dumps letters to the serial port. To receive them, open HyperTerminal (on windows, under accessories) or minicom (on Linux, look it up!). Set your baud rate to 2800 (or whatever you selected) and you're in business. This (picture below) is the output of the microcontroller to HyperTerminal on my PC. Forgive the image quality, I photographed the LCD screen instead of taking a screenshot.

This is the circuit which generates the output of the previous image. I have a few extra components. I have an LED which I used for debugging purposes, and also a switch (labeled "R"). The switch (when pressed) grounds pin 1 of the ATTiny2313 which resets it. If I want to program the chip, I hold "R" down and the PC can program it with the inline programmer "parallel port, straight-through, DAPA style). One cable going into the circuit is for the parallel port programmer, one cable is for the serial port (data transfer), and one is for power (5v which I stole from a USB port).

I hope you found this information useful. Feel free to contact me with any questions you may have, but realize that I'm no expert, and I'm merely documenting my successes chronologically on this website.

Markdown source code last modified on January 18th, 2021
---
title: Simple Case AVR/PC Serial Communication via MAX232
date: 2009-05-14 11:00:19
tags: microcontroller, circuit, old
---

# Simple Case AVR/PC Serial Communication via MAX232

__I recently had the desire__ to be able to see data from an ATMEL AVR microcontroller (the [ATTiny2313](http://www.SWHarden.com/blog/images/attiny-2313.gif)) for development and debugging purposes.  I wanted an easy way to have my microcontroller talk to my PC (and vise versa) with a minimum number of parts.  The easiest way to do this was to utilize the [UART](http://en.wikipedia.org/wiki/UART) capabilities of the [ATTiny2313](http://www.SWHarden.com/blog/images/attiny-2313.gif) to talk to my PC through the serial port. One problem is that the [ATTiny2313](http://www.SWHarden.com/blog/images/attiny-2313.gif)(as with most microcontrollers) puts out 5V for "high" (on) and 0V for "low" (off).  The RS-232 standard (which PC serial ports use) required -15V for high and +15v for low!  Obviously the microcontroller needs some help to achieve this.  The easiest way was to use the [MAX232 serial level converter](http://en.wikipedia.org/wiki/MAX232) which [costs about 3 bucks at DigiKey](http://search.digikey.com/scripts/DkSearch/dksus.dll?Detail&amp;name=MAX232CPE%2B-ND). Note that it requires a few 10uF capacitors to function properly.

<div class="text-center img-border">

[![](serialcircuit_thumb.jpg)](serialcircuit.png)

</div>

__Here's a more general schematic:__


<div class="text-center">

![](max232_serial_microcontroller.gif)

</div>

__I connected my ATTiny2313 to the MAX232__ in a very standard way. (photo)   MAX232 pins 13 and 14 go to the serial port, and the ATTiny2313 pins 2 and 3 go to the MAX232 pins 12 and 11 respectively.  I will note that they used a oscillator value (3.6864MHz) different than mine (9.216MHz).

__Determining the speed of serial communication__ is important.  This is dependent on your oscillator frequency!  I said I used a 9.216Mhz oscillator.  First, a crystal or ceramic oscillator is required over the internal RC oscillator because the internal RC oscillator is not accurate enough for serial communication.  The oscillator you select should be a perfect multiple of 1.8432MHz. Mine is 5x this value.  Many people use 2x this value (3.6864Mhz) and that's okay!  You just have to make sure your microchip knows (1) to use the external oscillator (google around for how to burn the fuses on your chip to do this) and (2) what the frequency of your oscillator is and how fast it should be sending data.  This is done by setting the UBRRL value.  The formula to do this is here:

<div class="text-center">

![](ubrrformula.gif)

</div>

__The datasheet of your microcontroller__ may list a lot of common crystal frequencies, bandwidths, and their appropriate UBRR values.  However my datasheet lacked an entry for a 9.216MHz crystal, so I had to do the math myself.  I Googled around and no "table" is available!  Why not make one? (picture, below).  Anyway, for my case I determined that if I set the UBRR value to 239, I could transmit data at 2800 baud (bits/second).  This is slow enough to ensure accuracy, but fast enough to quickly dump a large amount of text to a PC terminal.

<div class="text-center">

![](ubrr-table.gif)

</div>

## AVR Baud Calculator

This will make your life easier. The page <a href="http://www.wormfood.net/avrbaudcalc.php">wormfood.net/avrbaudcalc.php</a> has a chart of common crystals and the baud rates they work best with! Try to pick a combination that provides the least error possible...

__This is the bare-minimum code__ to test out my setup. Just load the code (written in C, compiled with avr-gcc) onto your chip and it's ready to go.  Be sure you set your fuses to use an external oscillator and that you set your UBRRL value correctly.

```c
#include <avr/io.h>  
#include <avr/interrupt.h>  
#include <util/delay.h>  

int main (void)  
{  
  unsigned char data=0;  
  UBRRL = 239;  
  UCSRB = (1 < < RXEN) | (1 << TXEN);  
  UCSRC = (1 < < UCSZ1) | (1 << UCSZ0);  

  for (;;)  
  {  
    if (data>'Z'||data< 'A')  
    {  
      UDR = 10; UDR = 13; data='A';_delay_ms(100);  
    }  
    
    UDR = data;  
    data += 1;  
    _delay_ms(100);  
  }  
}  
```

__Once you load it, it's ready to roll!__  It continuously dumps letters to the serial port.  To receive them, open HyperTerminal (on windows, under accessories) or minicom (on Linux, look it up!).  Set your baud rate to 2800 (or whatever you selected) and you're in business.  This (picture below) is the output of the microcontroller to HyperTerminal on my PC.  Forgive the image quality, I photographed the LCD screen instead of taking a screenshot.

<div class="text-center img-border">

[![](avr_serial_console_thumb.jpg)](avr_serial_console.jpg)

</div>

__This is the circuit__ which generates the output of the previous image.  I have a few extra components.  I have an LED which I used for debugging purposes, and also a switch (labeled "R").  The switch (when pressed) grounds pin 1 of the ATTiny2313 which resets it.  If I want to program the chip, I hold "R" down and the PC can program it with the inline programmer ["parallel port, straight-through, DAPA style](https://wikis.mit.edu/confluence/download/attachments/20512/dapa.png)).  One cable going into the circuit is for the parallel port programmer, one cable is for the serial port (data transfer), and one is for power (5v which I stole from a USB port).

<div class="text-center img-border">

[![](avr_max232_thumb.jpg)](avr_max232.jpg)

</div>

__I hope you found this__ information useful.  Feel free to [contact me](http://www.swharden.com/blog/send-scott-a-message/) with any questions you may have, but realize that I'm no expert, and I'm merely documenting my successes chronologically on this website.
April 30th, 2009

A Foot in the QRSS Door

I've been very busy over the past couple weeks. Last Thursday my boss approached me and asked if I could work over the weekend. He wanted to complete and submit a grant by the deadline (Monday at 5 pm). To make a long story short I worked really hard (really long days) on Friday, Saturday, Sunday, and Monday to accomplish this. Monday afternoon when it was done (at about 4 pm), after which I went home and collapsed from exhaustion. I don't know how my boss does it! He worked on it far more than I did, and over that weekend he didn't sleep much. Anyway, in exchange for my over-weekend work I got Tuesday and Wednesday off.

I knew in advance that I'd have two days off to do whatever I wanted. I prepared ahead of time by ordering a small handful (I think 4?) of ATMEL AVR type ATTiny2313 chips from Digi-Key at $2.26 per chip. They arrived in the mail on Monday. Unlike the simple PICAXE chips which can be programmed a form of BASIC cod from 2 wires of a serial port, the AVR series of chips are usually programmed from assembly-level code. Thankfully, C code can be converted to assembly (thanks to AVR-GCC) and loaded onto these chips. The result is a much faster and more powerful coding platform than the PICAXE chips can delivery. PICAXE seems useful for rapid development (especially if you already know BASIC) but I feel that I'm ready to tackle something new.

I built a straight-through parallel programmer for my ATTiny2313 chips. It was based upon the dapa configuration and connects to the appropriate pins. To be safe I recommend that you protect your parallel port and microcontrollers by installing the proper resisters (~1k?) between the devices, but I didn't do this.

I decided to dive right in to the world of digital RF transmission and should probably go to jail for it. I blatantly violated FCC regulations and simply wired my microcontroller to change the power level given to a 3.579545 MHz oscillator. The antenna is the copper wire sticking vertically out of the breadboard.

These crystals release wide bands of RF not only near the primary frequency (F), but also on the harmonic frequencies (F*n where n=1,2,3...). I was able to pick up the signal on my scanner at its 9th harmonic (32.215905 MHz). I think the harmonic output power is inversely proportional to n. Therefore the frequency I'm listening to represents only a fraction of the RF power the crystal is putting out at its primary frequency. Unfortunately the only listening device I have (currently) is the old scanner, which can only listen above 30 MHz.

Remember when I talked about the illegal part? Yeah, I detected harmonic signals being emitted way up into the high 100s of MHz. I don't think it's a big deal because it's low power and I doubt the signal is getting very far, but I'm always concerned about irritating people (Are people trying to use Morse code at one of the frequencies? Am I jamming my neighbors' TV reception?) so I don't keep it on long. Once I get some more time, I'll build the appropriate receiver circuits (I have another matched crystal) and install a low-pass filter (to eliminate harmonics) and maybe even get a more appropriate radio license (I'm still only technician). But for now, this is a proof-of-concept, and it works. Check out the output of the scanner.

Something I struggled with for half an hour was how to produce a tone with a microcontroller and the oscillator. Simply supplying power to the oscillator produces a strong RF signal, but there is no sound to it. It's just full quieting when it's on, and static noise when it's off. To produce an AM tone, I needed amplitude modulation. I activated the oscillator by supplying power from the microcontroller with one pin (to get it oscillating), and fed it extra juice in the form of timer output from another pin. The fluctuation in power to the oscillator (without power-loss) produced a very strong, loud, clear signal (horizontal lines). I wrote code to make it beep. Frequency can be adjusted by modifying the timer output properties. The code in the screenshot is very primitive, and not current (doesn't use timers to control AM frequency), but it worked. I'm sure I'll write more about it later.

Thoughts from Future Scott (August 2019, 10 years later):

What a good start! But what a bad design =P

Driving a can oscillator's power pin with two microcontroller pins is not a good idea. Also, you were SO CLOSE to getting frequency shift keying to work! Rather than turning the can oscillator on/off with the microcontroller, just leave it on continuously and send a microcontroller pin to the can oscillator's VCO pin. I'm sure I didn't know what that 4th pin does when did when I originally wrote this (and most diagrams of can oscillators online leave that pin disconnected).

Either way, I'm happy this day happened - this was the start of years of hobby radio frequency circuit design!

Markdown source code last modified on January 18th, 2021
---
title: A Foot in the QRSS Door
date: 2009-04-30 09:26:40
tags: qrss, microcontroller, amateur radio, old
---

# A Foot in the QRSS Door

__I've been very busy over the past couple weeks.__ Last Thursday my boss approached me and asked if I could work over the weekend. He wanted to complete and submit a grant by the deadline (Monday at 5 pm). To make a long story short I worked really hard (really long days) on Friday, Saturday, Sunday, and Monday to accomplish this. Monday afternoon when it was done (at about 4 pm), after which I went home and collapsed from exhaustion. I don't know how my boss does it! He worked on it far more than I did, and over that weekend he didn't sleep much. Anyway, in exchange for my over-weekend work I got Tuesday and Wednesday off.

<div class="text-center">

![](attiny2313.jpg)

</div>

__I knew in advance that I'd have two days off to do whatever I wanted.__ I prepared ahead of time by ordering a small handful (I think 4?) of [ATMEL AVR](http://en.wikipedia.org/wiki/Atmel_AVR) type [ATTiny2313 chips from Digi-Key](http://search.digikey.com/scripts/DkSearch/dksus.dll?Detail&amp;name=ATTINY2313-20PU-ND) at $2.26 per chip. They arrived in the mail on Monday. Unlike the simple [PICAXE](http://en.wikipedia.org/wiki/PICAXE) chips which can be programmed a form of BASIC cod from 2 wires of a serial port, the AVR series of chips are usually programmed from assembly-level code. Thankfully, C code can be converted to assembly (thanks to AVR-GCC) and loaded onto these chips. The result is a much faster and more powerful coding platform than the PICAXE chips can delivery. PICAXE seems useful for rapid development (especially if you already know BASIC) but I feel that I'm ready to tackle something new.

__I built a straight-through parallel programmer__ for my ATTiny2313 chips. It was based upon the [dapa configuration](https://wikis.mit.edu/confluence/download/attachments/20512/dapa.png) and connects to the appropriate pins. To be safe I recommend that you protect your parallel port and microcontrollers by installing the proper resisters (~1k?) between the devices, but I didn't do this.

<div class="text-center img-border">

[![](img_1555_thumb.jpg)](img_1555.jpg)

</div>

__I decided to dive right in to the world of digital RF transmission__ and should probably go to jail for it. I blatantly violated FCC regulations and simply wired my microcontroller to change the power level given to a 3.579545 MHz oscillator. The antenna is the copper wire sticking vertically out of the breadboard.

__These crystals release wide bands of RF__ not only near the primary frequency (F), but also on the harmonic frequencies (F\*n where n=1,2,3...). I was able to pick up the signal on my scanner at its 9th harmonic (32.215905 MHz). I think the harmonic output power is inversely proportional to n. Therefore the frequency I'm listening to represents only a fraction of the RF power the crystal is putting out at its primary frequency. Unfortunately the only listening device I have (currently) is the old scanner, which can only listen above 30 MHz.

<div class="text-center img-border">

[![](img_1550_thumb.jpg)](img_1550.jpg)

</div>

__Remember when I talked about the illegal part?__ Yeah, I detected harmonic signals being emitted way up into the high 100s of MHz. I don't think it's a big deal because it's low power and I doubt the signal is getting very far, but I'm always concerned about irritating people (Are people trying to use Morse code at one of the frequencies? Am I jamming my neighbors' TV reception?) so I don't keep it on long. Once I get some more time, I'll build the appropriate receiver circuits (I have another matched crystal) and install a low-pass filter (to eliminate harmonics) and maybe even get a more appropriate radio license (I'm still only technician). But for now, this is a proof-of-concept, and it works. Check out the output of the scanner.

<div class="text-center img-border">

[![](ss_thumb.jpg)](ss.png)

</div>

__Something I struggled with for half an hour__ was how to produce a tone with a microcontroller and the oscillator. Simply supplying power to the oscillator produces a strong RF signal, but there is no sound to it. It's just full quieting when it's on, and static noise when it's off. To produce an AM tone, I needed amplitude modulation. I activated the oscillator by supplying power from the microcontroller with one pin (to get it oscillating), and fed it extra juice in the form of timer output from another pin. The fluctuation in power to the oscillator (without power-loss) produced a very strong, loud, clear signal (horizontal lines). I wrote code to make it beep. Frequency can be adjusted by modifying the timer output properties. The code in the screenshot is very primitive, and not current (doesn't use timers to control AM frequency), but it worked. I'm sure I'll write more about it later.

<blockquote class="wp-block-quote"><p><strong>Thoughts from Future Scott (August 2019, 10 years later):</strong></p><p>What a good start! But what a bad design =P</p><p>Driving a can oscillator's power pin with two microcontroller pins is not a good idea. Also, you were SO CLOSE to getting frequency shift keying to work! Rather than turning the can oscillator on/off with the microcontroller, just leave it on continuously and send a microcontroller pin to the can oscillator's VCO pin. I'm sure I didn't know what that 4th pin does when did when I originally wrote this (and most diagrams of can oscillators online leave that pin disconnected).</p><p>Either way, I'm happy this day happened - this was the start of years of hobby radio frequency circuit design!</p></blockquote>

April 22nd, 2009

Puxing 777 Radio Headset Schematic

I successfully created a speaker/microphone/transmit button circuit for the puxing 777 which probably works for all puxing radios. Instead of simply using circuits found on other websites (always for other radios), I decided to reverse-engineering an earphone/microphone headset that came with the radio to determine how it worked. I can't claim that I'm an expert in electronics theory, but I can say that I faithfully rebuilt the circuitry within the factory-shipped headset and it worked. The result allows me to leave my handheld radio in its charger while casually listening/transmitting with a button that I made instead of having to reach around and awkwardly squeeze the transmit button on the side of the radio. Once again, I built this circuit and it was successful for me, but there may still be a better way to do it.

The microphone is a 20-cent electret microphone with no special modifications. The speaker I used is a standard 8ohm loudspeaker with no special modifications. The switch is a keyboard-style (push-to-talk) switch, and the capacitor I used is good for 10nF.

If you have any ideas for improvements, let me know! I'll post some photos once I have my completed little "base station" set up. My ultimate goal is to turn an el-cheapo handheld VHF radio into a decent desktop transceiver by combining it with a nice antenna (located on a balcony at 30ft) and a convenient, easy-to-use switches/buttons/microphone/speaker/etc on a desktop panel.

Markdown source code last modified on January 18th, 2021
---
title: Puxing 777 Radio Headset Schematic
date: 2009-04-22 11:56:42
tags: amateur radio, circuit, old
---

# Puxing 777 Radio Headset Schematic

__I successfully created a speaker/microphone/transmit button circuit for the puxing 777__ which probably works for all puxing radios.  Instead of simply using circuits found on other websites (always for other radios), I decided to reverse-engineering an earphone/microphone headset that came with the radio to determine how it worked. I can't claim that I'm an expert in electronics theory, but I can say that I faithfully rebuilt the circuitry within the factory-shipped headset and it worked.  The result allows me to leave my handheld radio in its charger while casually listening/transmitting with a button that I made instead of having to reach around and awkwardly squeeze the transmit button on the side of the radio.  Once again, I built this circuit and it was successful for me, but there may still be a better way to do it.

<div class="text-center img-border">

[![](puxing_thumb.jpg)](puxing.png)

</div>

The microphone is a 20-cent [electret microphone](http://en.wikipedia.org/wiki/Electret_microphone) with no special modifications.  The speaker I used is a [standard 8ohm loudspeaker](http://en.wikipedia.org/wiki/File:3.5_Inch_Speaker.jpg) with no special modifications.  The switch is a keyboard-style (push-to-talk) switch, and the capacitor I used is good for 10nF.

<div class="text-center img-border">

![](headset.jpg)

</div>

__If you have any ideas for improvements, let me know!__  I'll post some photos once I have my completed little "base station" set up.  My ultimate goal is to turn an el-cheapo handheld VHF radio into a decent desktop transceiver by combining it with a nice antenna (located on a balcony at 30ft) and a convenient, easy-to-use switches/buttons/microphone/speaker/etc on a desktop panel.
Pages