The personal website of Scott W Harden

PHP-Generated Apache-Style Access Logs

A few months ago I wrote about a way I use PHP to generate apache-style access.log files since my web host blocks access to them. Since then I've forgotten it was even running! I now have some pretty cool-looking graphs generated by Python and Matplotlib. For details (and the messy script) check the original posting.

This image represents the number of requests (php pages) made per hour since I implemented the script. It might be a good idea to perform some linear data smoothing techniques (which I love writing about), but for now I'll leave it as it is so it most accurately reflects the actual data.

This code has been updated: 2009-08-04-generate-apache-style-http-access-logs-via-sql-and-php

Markdown source code last modified on January 18th, 2021
---
title: PHP-Generated Apache-Style Access Logs
date: 2009-05-14 13:33:22
---

# PHP-Generated Apache-Style Access Logs

__A few months ago__ I wrote about a way [I use PHP to generate apache-style access.log files](2009-01-22-using-php-to-create-apache-style-accesslog/) since my web host blocks access to them.  Since then I've forgotten it was even running!  I now have some pretty cool-looking graphs generated by [Python](http://www.python.org) and [Matplotlib](http://matplotlib.sourceforge.net/).  For details (and the messy script) check the original posting.

[![](graph_time_thumb.jpg)](graph_time.png)

__This image represents__ the number of requests (php pages) made per hour since I implemented the script.  It might be a good idea to perform some [linear data smoothing techniques](http://www.swharden.com/blog/2008-11-17-linear-data-smoothing-in-python/) (which I love writing about), but for now I'll leave it as it is so it most accurately reflects the actual data.

This code has been updated:
[2009-08-04-generate-apache-style-http-access-logs-via-sql-and-php](2009-08-04-generate-apache-style-http-access-logs-via-sql-and-php)
May 12th, 2009

Electronic Components have Arrived

I placed another online order to DigiKey yesterday. They have virtually every electronic component available for next-day delivery. How awesome is that? The frustrating thing is that I'm always trying to get my order in before 6pm CMT so that my package will ship same day, so I rush, and I always forget something! It never fails. 30 minutes after I place an order, I always remember something else I wish I'd added to the order (to save on shipping). So, instead of buying it when I want it (now) I wait a few more days to think of some more stuff to add. The last couple weeks I've placed 3 orders totaling about $30. You never think you're spending a lot, but all the sudden you realize that all these 40-cent parts add up when you go to check out. Shipping is usually under $3, which makes me happy. What did I get? I got a couple MAX232 chips so I can easialy shuffle data back and forth from my PC to an AVR chip using the serial port using a simple terminal application like hyperterminal. This will allow me to do some pretty fancy stuff and certainly help with the prototyping/debugging/development steps of my various microcontroller-based projects.

Some of the things which I forgot to order include a few 1F supercapacitors for my solar-powered QRSS beacon project and a random collection of parts needed to build a DIY AVR microcontroller programmer with a USB PC interface and I also wanted to buy at least one ATMega8 microcontroller because they seem to be a good step above the ATTiny 2313 chips I already have and are at the center of so many microcontroller projects. I'm sure there are are few more items I need, but can't think of them right now, so they don't make the list.

A little random, but worth noting is a cool website I found earlier which details (in a very basic way) the various types of HF antennas and does a good job (IMHO) of explaining the basic theory behind them. The page can be found at deltadx.net.

As an aside, I encourage everyone to take a couple minutes out of your day and browse the Wikipedia entry describing the JPEG compression algorithm. Seriously, it's an educational, enlightening, and entertaining read.

Markdown source code last modified on January 18th, 2021
---
title: Random Junk In'da Mail!
date: 2009-05-12 11:49:58
---

# Electronic Components have Arrived

 __I placed another online order__ to [DigiKey](www.digikey.com) yesterday.  They have virtually every electronic component available for next-day delivery.  How awesome is that?  The frustrating thing is that I'm always trying to get my order in before 6pm CMT so that my package will ship same day, so I rush, and I always forget something!  It never fails.  30 minutes after I place an order, I always remember something else I wish I'd added to the order (to save on shipping).  So, instead of buying it when I want it (now) I wait a few more days to think of some more stuff to add.  The last couple weeks I've placed 3 orders totaling about $30.  You never think you're spending a lot, but all the sudden you realize that all these 40-cent parts add up when you go to check out. Shipping is usually under $3, which makes me happy.  What did I get?  I got a couple [MAX232 chips](http://focus.ti.com/lit/ds/symlink/max232.pdf) so I can easialy [shuffle data back and forth from my PC to an AVR chip using the serial port](http://blog.jeffmurry.com/2008/11/14/hello-world--uart--serial-comm-with-pc.aspx) using a simple terminal application like hyperterminal.  This will [allow me to do some pretty fancy stuff](http://www.societyofrobots.com/microcontroller_uart_50_robot.shtml) and certainly help with the prototyping/debugging/development steps of my various microcontroller-based projects.

<div class="text-center">

[![](rs232_03_thumb.jpg)](rs232_03.jpg)

</div>

__Some of the things which I forgot to order__ include [a few 1F supercapacitors for my solar-powered QRSS beacon project](http://www.solarfreaks.com/rc-solar-car-t19-20.html#p439) and a [random collection of parts](http://www.ladyada.net/make/usbtinyisp/parts.html) needed to build [a DIY AVR microcontroller programmer with a USB PC interface](http://www.ladyada.net/make/usbtinyisp/use.html) and I also wanted to buy at least one [ATMega8 microcontroller](http://www.solarbotics.com/assets/images/atmega8l/atmega8l_pl.jpg) because they seem to be a good step above the [ATTiny 2313 chips](http://home.scarlet.be/kvandepitte/Mijn%20afbeeldingen/nixies/attiny2313.jpg) I already have and are at the center of so [many microcontroller projects](http://hackaday.com/tag/atmega/).  I'm sure there are are few more items I need, but can't think of them right now, so they don't make the list.

__A little random, but worth noting__ is a cool website I found earlier which details (in a very basic way) the various types of HF antennas and does a good job (IMHO) of explaining the basic theory behind them.  The page can be found at [deltadx.net](http://www.deltadx.net/ABCDx/Sections/Antennas.htm).

<div class="text-center img-border">

[![](squareant_thumb.jpg)](squareant.jpg)

</div>

__As an aside,__ I encourage everyone to take a couple minutes out of your day [and browse the Wikipedia entry describing the JPEG compression algorithm](http://en.wikipedia.org/wiki/JPEG).  Seriously, it's an educational, enlightening, and entertaining read.
May 7th, 2009

Built a Pixie II

Man, what a long day! Work is so tedious sometimes. This week I've been proofing scientific literature using Office 2003 with "track changes". I make changes, my boss makes changes, I make more changes, and it goes back and forth a few times. I wonder why office 2007 is so bad. Does anybody truly like it, and find it to be a significant improvement upon 2003? ... or Vista over XP? Maybe I'm just getting old, inflexible, and grumpy.

This is what I'm currently working on. The light bubbles on the right are deletions. The dark bubbles on the right are comments. The red text is insertions/modifications I made. Pretty intense, right? Pages and pages of this. I'm starting to grasp the daunting amount of time a scientist must spend writing in the laboratory as opposed to performing actual experiments or even doing literature research.

Last night I assembled a Pixie II circuit similar to the one pictured here. I must say that I'm a little disappointed with the information available on the internet regarding simple RF theory in relation to transceiver circuits. I'm just now starting to get into RF circuitry and the concept looking at circuits and imagining a combination of AC and DC flowing through it is warping my brain. I have everything I need to build a simple Pixie II transceiver (which is supposedly capable of Morse code transmissions over 300 miles, and QRSS applications over 3,000 miles) but I don't want to use it unless I understand how it actually works.

I'm trying to break this circuit down into its primary components. I understand the role of the lowpass filter. I understand the role of the 1st transistor and related circuitry in amplifying the output of the crystal oscillator (left side). I totally get the audio amplifier circuitry (bottom). It's that center transistor (which supposedly handles signal amplification, receiving, and mixing) that I can't get my mind around. Every time I think figure it out for one mode (sending or receiving), I mentally lose the other one. It has me very frustrated because it seems like this should be easier than I'm making it. I selected this circuit because it was simple and I assumed I'd be smart enough to figure it out... maybe I was wrong? I wish I had an oscilloscope so I could probe the RF passing through various stages of this circuit. I guess I should take another stab at reading chapters 5-11 of the ARRL handbook.

Markdown source code last modified on January 18th, 2021
---
title: Built a Pixie II
date: 2009-05-07 15:10:22
---

# Built a Pixie II

__Man, what a long day!__ Work is so tedious sometimes. This week I've been proofing scientific literature using Office 2003 with "track changes". I make changes, my boss makes changes, I make more changes, and it goes back and forth a few times. I wonder why office 2007 is so bad. Does _anybody_ truly like it, and find it to be a significant improvement upon 2003? ... or Vista over XP? Maybe I'm just getting old, inflexible, and grumpy.

<div class="text-center img-border">

[![](proofread_thumb.jpg)](proofread.jpg)

</div>

__This is what I'm currently working on.__ The light bubbles on the right are deletions. The dark bubbles on the right are comments. The red text is insertions/modifications I made. Pretty intense, right? Pages and pages of this. I'm starting to grasp the daunting amount of time a scientist must spend _writing_ in the laboratory as opposed to performing actual experiments or even doing literature research.

<div class="text-center img-border">

![](pixie2.jpg)

</div>

__Last night I assembled a Pixie II__ circuit similar to the one pictured here. I must say that I'm a little disappointed with the information available on the internet regarding simple RF theory in relation to transceiver circuits. I'm just now starting to get into RF circuitry and the concept looking at circuits and imagining a combination of AC and DC flowing through it is warping my brain. I have everything I need to build a simple Pixie II transceiver (which is supposedly capable of Morse code transmissions over 300 miles, and QRSS applications over 3,000 miles) but I don't want to use it unless I understand how it actually works.

<div class="text-center">

![](pixie2transceiver.gif)

</div>

__I'm trying to break this circuit down into its primary components.__ I understand the role of the lowpass filter. I understand the role of the 1st transistor and related circuitry in amplifying the output of the crystal oscillator (left side). I totally get the audio amplifier circuitry (bottom). It's that center transistor (which supposedly handles signal amplification, receiving, and mixing) that I can't get my mind around. Every time I think figure it out for one mode (sending or receiving), I mentally lose the other one. It has me very frustrated because it seems like this should be easier than I'm making it. I selected this circuit because it was simple and I assumed I'd be smart enough to figure it out... maybe I was wrong? I wish I had an oscilloscope so I could probe the RF passing through various stages of this circuit. I guess I should take another stab at reading chapters 5-11 of the ARRL handbook.

May 4th, 2009

AM Transmitter Test

I put the transmitter from the previous post to the test. I changed the circuitry a bit though. I kept the oscillator (50 MHz) is now continuously powered. I programmed the ATTiny 2313 microcontroller (using PWM output) to send an oscillating signal to the base of a transistor (NPN). In this way the microcontroller PWM output didn't supply power to the oscillator, but rather grounded it. I got a big boost in range this way. Yesterday I couldn't even hear the signal in the parking lot of my apartment, whereas today I heard it loud and clear. I decided to take a drive with my scanner, laptop, and Argo to see how far away I could get and still detect the signal. With this bare bones transmitter setup (using a 2M J-pole antenna) I was able to detect it over 4,000 ft away. The receiving antenna was a 2m ~1ft high antenna magnet-mounted on top of my car.

In retrospect, I should have run Argo at my apartment and drove the _transmitter _farther and farther away. I presume that my transmitter is functioning decently, and that if I attached it to a proper antenna (and had a better receiving antenna) I might be able to get some cross-town distance? I'm still learning - this is the point though, right?

This is where I was when the signal died. The red marker (upper right) is my apartment where the transmitter was, and the signal began to die right as I traveled south on Chickasaw past Lake Underhill (~4000 ft away). This immediate loss may be due to the fact that I passed under power lines which parallel Lake Underhill which interrupted the line-of-sight path between my 3rd story apartment balcony and me. If this were the case, supposedly if I kept driving south the signal may have improved.

Markdown source code last modified on January 18th, 2021
---
title: AM Transmitter Test
date: 2009-05-04 22:09:13
---

# AM Transmitter Test

__I put the transmitter from the previous post to the test.__ I changed the circuitry a bit though. I kept the oscillator (50 MHz) is now continuously powered. I programmed the ATTiny 2313 microcontroller (using PWM output) to send an oscillating signal to the base of a transistor (NPN). In this way the microcontroller PWM output didn't supply power to the oscillator, but rather grounded it. I got a big boost in range this way. Yesterday I couldn't even hear the signal in the parking lot of my apartment, whereas today I heard it loud and clear. I decided to take a drive with my scanner, laptop, and Argo to see how far away I could get and still detect the signal. With this bare bones transmitter setup (using a 2M J-pole antenna) I was able to detect it over 4,000 ft away. The receiving antenna was a 2m ~1ft high antenna magnet-mounted on top of my car.

<div class="text-center img-border">

[![](qrss_fade_thumb.jpg)](qrss_fade.png)

</div>

__In retrospect, I should have run Argo at my apartment and drove the _transmitter _farther and farther away.__ I presume that my transmitter is functioning decently, and that if I attached it to a proper antenna (and had a better receiving antenna) I might be able to get some cross-town distance? I'm still learning - this is the point though, right?

<div class="text-center img-border">

[![](firsttransmap_thumb.jpg)](firsttransmap.png)

</div>

__This is where I was when the signal died.__ The red marker (upper right) is my apartment where the transmitter was, and the signal began to die right as I traveled south on Chickasaw past Lake Underhill (~4000 ft away). This immediate loss may be due to the fact that I passed under power lines which parallel Lake Underhill which interrupted the line-of-sight path between my 3rd story apartment balcony and me. If this were the case, supposedly if I kept driving south the signal may have improved.

A Foot in the QRSS Door

I've been very busy over the past couple weeks. Last Thursday my boss approached me and asked if I could work over the weekend. He wanted to complete and submit a grant by the deadline (Monday at 5 pm). To make a long story short I worked really hard (really long days) on Friday, Saturday, Sunday, and Monday to accomplish this. Monday afternoon when it was done (at about 4 pm), after which I went home and collapsed from exhaustion. I don't know how my boss does it! He worked on it far more than I did, and over that weekend he didn't sleep much. Anyway, in exchange for my over-weekend work I got Tuesday and Wednesday off.

I knew in advance that I'd have two days off to do whatever I wanted. I prepared ahead of time by ordering a small handful (I think 4?) of ATMEL AVR type ATTiny2313 chips from Digi-Key at $2.26 per chip. They arrived in the mail on Monday. Unlike the simple PICAXE chips which can be programmed a form of BASIC cod from 2 wires of a serial port, the AVR series of chips are usually programmed from assembly-level code. Thankfully, C code can be converted to assembly (thanks to AVR-GCC) and loaded onto these chips. The result is a much faster and more powerful coding platform than the PICAXE chips can delivery. PICAXE seems useful for rapid development (especially if you already know BASIC) but I feel that I'm ready to tackle something new.

I built a straight-through parallel programmer for my ATTiny2313 chips. It was based upon the dapa configuration and connects to the appropriate pins. To be safe I recommend that you protect your parallel port and microcontrollers by installing the proper resisters (~1k?) between the devices, but I didn't do this.

I decided to dive right in to the world of digital RF transmission and should probably go to jail for it. I blatantly violated FCC regulations and simply wired my microcontroller to change the power level given to a 3.579545 MHz oscillator. The antenna is the copper wire sticking vertically out of the breadboard.

These crystals release wide bands of RF not only near the primary frequency (F), but also on the harmonic frequencies (F*n where n=1,2,3...). I was able to pick up the signal on my scanner at its 9th harmonic (32.215905 MHz). I think the harmonic output power is inversely proportional to n. Therefore the frequency I'm listening to represents only a fraction of the RF power the crystal is putting out at its primary frequency. Unfortunately the only listening device I have (currently) is the old scanner, which can only listen above 30 MHz.

Remember when I talked about the illegal part? Yeah, I detected harmonic signals being emitted way up into the high 100s of MHz. I don't think it's a big deal because it's low power and I doubt the signal is getting very far, but I'm always concerned about irritating people (Are people trying to use Morse code at one of the frequencies? Am I jamming my neighbors' TV reception?) so I don't keep it on long. Once I get some more time, I'll build the appropriate receiver circuits (I have another matched crystal) and install a low-pass filter (to eliminate harmonics) and maybe even get a more appropriate radio license (I'm still only technician). But for now, this is a proof-of-concept, and it works. Check out the output of the scanner.

Something I struggled with for half an hour was how to produce a tone with a microcontroller and the oscillator. Simply supplying power to the oscillator produces a strong RF signal, but there is no sound to it. It's just full quieting when it's on, and static noise when it's off. To produce an AM tone, I needed amplitude modulation. I activated the oscillator by supplying power from the microcontroller with one pin (to get it oscillating), and fed it extra juice in the form of timer output from another pin. The fluctuation in power to the oscillator (without power-loss) produced a very strong, loud, clear signal (horizontal lines). I wrote code to make it beep. Frequency can be adjusted by modifying the timer output properties. The code in the screenshot is very primitive, and not current (doesn't use timers to control AM frequency), but it worked. I'm sure I'll write more about it later.

Thoughts from Future Scott (August 2019, 10 years later):

What a good start! But what a bad design =P

Driving a can oscillator's power pin with two microcontroller pins is not a good idea. Also, you were SO CLOSE to getting frequency shift keying to work! Rather than turning the can oscillator on/off with the microcontroller, just leave it on continuously and send a microcontroller pin to the can oscillator's VCO pin. I'm sure I didn't know what that 4th pin does when did when I originally wrote this (and most diagrams of can oscillators online leave that pin disconnected).

Either way, I'm happy this day happened - this was the start of years of hobby radio frequency circuit design!

Markdown source code last modified on January 18th, 2021
---
title: A Foot in the QRSS Door
date: 2009-04-30 09:26:40
tags: qrss, microcontroller, amateur radio, old
---

# A Foot in the QRSS Door

__I've been very busy over the past couple weeks.__ Last Thursday my boss approached me and asked if I could work over the weekend. He wanted to complete and submit a grant by the deadline (Monday at 5 pm). To make a long story short I worked really hard (really long days) on Friday, Saturday, Sunday, and Monday to accomplish this. Monday afternoon when it was done (at about 4 pm), after which I went home and collapsed from exhaustion. I don't know how my boss does it! He worked on it far more than I did, and over that weekend he didn't sleep much. Anyway, in exchange for my over-weekend work I got Tuesday and Wednesday off.

<div class="text-center">

![](attiny2313.jpg)

</div>

__I knew in advance that I'd have two days off to do whatever I wanted.__ I prepared ahead of time by ordering a small handful (I think 4?) of [ATMEL AVR](http://en.wikipedia.org/wiki/Atmel_AVR) type [ATTiny2313 chips from Digi-Key](http://search.digikey.com/scripts/DkSearch/dksus.dll?Detail&amp;name=ATTINY2313-20PU-ND) at $2.26 per chip. They arrived in the mail on Monday. Unlike the simple [PICAXE](http://en.wikipedia.org/wiki/PICAXE) chips which can be programmed a form of BASIC cod from 2 wires of a serial port, the AVR series of chips are usually programmed from assembly-level code. Thankfully, C code can be converted to assembly (thanks to AVR-GCC) and loaded onto these chips. The result is a much faster and more powerful coding platform than the PICAXE chips can delivery. PICAXE seems useful for rapid development (especially if you already know BASIC) but I feel that I'm ready to tackle something new.

__I built a straight-through parallel programmer__ for my ATTiny2313 chips. It was based upon the [dapa configuration](https://wikis.mit.edu/confluence/download/attachments/20512/dapa.png) and connects to the appropriate pins. To be safe I recommend that you protect your parallel port and microcontrollers by installing the proper resisters (~1k?) between the devices, but I didn't do this.

<div class="text-center img-border">

[![](img_1555_thumb.jpg)](img_1555.jpg)

</div>

__I decided to dive right in to the world of digital RF transmission__ and should probably go to jail for it. I blatantly violated FCC regulations and simply wired my microcontroller to change the power level given to a 3.579545 MHz oscillator. The antenna is the copper wire sticking vertically out of the breadboard.

__These crystals release wide bands of RF__ not only near the primary frequency (F), but also on the harmonic frequencies (F\*n where n=1,2,3...). I was able to pick up the signal on my scanner at its 9th harmonic (32.215905 MHz). I think the harmonic output power is inversely proportional to n. Therefore the frequency I'm listening to represents only a fraction of the RF power the crystal is putting out at its primary frequency. Unfortunately the only listening device I have (currently) is the old scanner, which can only listen above 30 MHz.

<div class="text-center img-border">

[![](img_1550_thumb.jpg)](img_1550.jpg)

</div>

__Remember when I talked about the illegal part?__ Yeah, I detected harmonic signals being emitted way up into the high 100s of MHz. I don't think it's a big deal because it's low power and I doubt the signal is getting very far, but I'm always concerned about irritating people (Are people trying to use Morse code at one of the frequencies? Am I jamming my neighbors' TV reception?) so I don't keep it on long. Once I get some more time, I'll build the appropriate receiver circuits (I have another matched crystal) and install a low-pass filter (to eliminate harmonics) and maybe even get a more appropriate radio license (I'm still only technician). But for now, this is a proof-of-concept, and it works. Check out the output of the scanner.

<div class="text-center img-border">

[![](ss_thumb.jpg)](ss.png)

</div>

__Something I struggled with for half an hour__ was how to produce a tone with a microcontroller and the oscillator. Simply supplying power to the oscillator produces a strong RF signal, but there is no sound to it. It's just full quieting when it's on, and static noise when it's off. To produce an AM tone, I needed amplitude modulation. I activated the oscillator by supplying power from the microcontroller with one pin (to get it oscillating), and fed it extra juice in the form of timer output from another pin. The fluctuation in power to the oscillator (without power-loss) produced a very strong, loud, clear signal (horizontal lines). I wrote code to make it beep. Frequency can be adjusted by modifying the timer output properties. The code in the screenshot is very primitive, and not current (doesn't use timers to control AM frequency), but it worked. I'm sure I'll write more about it later.

<blockquote class="wp-block-quote"><p><strong>Thoughts from Future Scott (August 2019, 10 years later):</strong></p><p>What a good start! But what a bad design =P</p><p>Driving a can oscillator's power pin with two microcontroller pins is not a good idea. Also, you were SO CLOSE to getting frequency shift keying to work! Rather than turning the can oscillator on/off with the microcontroller, just leave it on continuously and send a microcontroller pin to the can oscillator's VCO pin. I'm sure I didn't know what that 4th pin does when did when I originally wrote this (and most diagrams of can oscillators online leave that pin disconnected).</p><p>Either way, I'm happy this day happened - this was the start of years of hobby radio frequency circuit design!</p></blockquote>

Pages