⚠️ WARNING: This page is obsolete
Articles typically receive this designation when the technology they describe is no longer relevant, code
provided is later deemed to be of poor quality, or the topics discussed are better presented in future
articles. Articles like this are retained for the sake of preservation, but their content should be
critically assessed.
I realized that the C code from yesterday wasn’t showing-up properly because of textile, a rapid, inline, tag-based formatting system. While it’s fun and convenient to use, it’s not always practical. The problem I was having was that in C code, variable names (such as delay) were becoming irrevocably italicized, and nothing I did could prevent textile from ignoring code while styling text. The kicker is that I couldn’t disable it easily, because I’ve been writing in this style for over four years! I decided that the time was now to put my mad Python skills to the test and write code to handle the conversion from textile-format to raw HTML.
I accomplished this feat in a number of steps. Yeah, I could have done hours of research to find a “faster way”, but it simply wouldn’t have been as creative. In a nutshell, I backed-up the SQL database using PHPMyAdmin to a single “x.sql” file. I then wrote a pythons script to parse this [massive] file and output “o.sql”, the same data but with all of the textile tags I commonly used replaced by their HTML equivalent. It’s not 100% perfect, but it’s 99.999% perfect. I’ll accept that. The output? You’re viewing it! Here’s the code I used to do it:
## This Python script removes *SOME* textile formatting from Wordpress
## backups in plain text SQL format (dumped from PHP MyAdmin). Specifically,
## it corrects bold and itallic fonts and corrects links. It should be easy
## to expand if you need to do something else with it.
infile = 'x.sql'
replacements= ["r"," "],["n"," n "],["*:","* :"],["_:","_ :"],
["n","<br>n"],[">*","> *"],["*< ","* <"],
[">_","> _"],["_< ","_ <"],
[" *"," <b>"],["* "," "],[" _"," <i>"],["_ ","</i> "]
#These are the easy replacements
def fixLinks(line):
## replace ["links":URL] with [<a href="https://swharden.com/static/2009/05/15/URL">links</a>]. ##
words = line.split(" ")
for i in range(len(words)):
word = words[i]
if '":' in word:
upto=1
while (word.count('"')&lt;2):
word = words[i-upto]+" "+word
upto+=1
word_orig = word
extra=""
word = word.split('":')
word[0]=word[0][1:]
for char in ".),'":
if word[1][-1]==char: extra=char
if len(extra)>0: word[1]=word[1][:-1]
word_new='<a href="https://swharden.com/static/2009/05/15/%s">%s</a>'%(word[1],word[0])+extra
line=line.replace(word_orig,word_new)
return line
def stripTextile(orig):
## Handle the replacements and link fixing for each line. ##
if not orig.count("', '") == 13: return orig #non-normal post
line=orig
temp = line.split
line = line.split("', '",5)[2]
if len(line)&lt;10:return orig #non-normal post
origline = line
line = " "+line
for replacement in replacements:
line = line.replace(replacement[0],replacement[1])
line=fixLinks(line)
line = orig.replace(origline,line)
return line
f=open(infile)
raw=f.readlines()
f.close
posts=0
for raw_i in range(len(raw)):
if raw[raw_i][:11]=="INSERT INTO":
if "wp_posts" in raw[raw_i]: #if it's a post, handle it!
posts+=1
print "on post",posts
raw[raw_i]=stripTextile(raw[raw_i])
print "WRITING..."
out = ""
for line in raw:
out+=line
f=open('o.sql','w')
f.write(out)
f.close()
I certainly held my breath while the thing ran. As I previously mentioned, this thing modified SQL tables. Therefore, when I uploaded the “corrected” versions, I kept breaking the site until I got all the bugs worked out. Here’s an image from earlier today when my site was totally dead (0 blog posts)
A few months ago I wrote about a way I use PHP to generate apache-style access.log files since my web host blocks access to them. Since then I’ve forgotten it was even running! I now have some pretty cool-looking graphs generated by Python and Matplotlib. For details (and the messy script) check the original posting.
This image represents the number of requests (php pages) made per hour since I implemented the script. It might be a good idea to perform some linear data smoothing techniques (which I love writing about), but for now I’ll leave it as it is so it most accurately reflects the actual data.
This code has been updated:
2009-08-04-generate-apache-style-http-access-logs-via-sql-and-php
⚠️ WARNING: This page is obsolete
Articles typically receive this designation when the technology they describe is no longer relevant, code
provided is later deemed to be of poor quality, or the topics discussed are better presented in future
articles. Articles like this are retained for the sake of preservation, but their content should be
critically assessed.
I recently had the desire to be able to see data from an ATMEL AVR microcontroller (the ATTiny2313) for development and debugging purposes. I wanted an easy way to have my microcontroller talk to my PC (and vise versa) with a minimum number of parts. The easiest way to do this was to utilize the UART capabilities of the ATTiny2313 to talk to my PC through the serial port. One problem is that the ATTiny2313(as with most microcontrollers) puts out 5V for “high” (on) and 0V for “low” (off). The RS-232 standard (which PC serial ports use) required -15V for high and +15v for low! Obviously the microcontroller needs some help to achieve this. The easiest way was to use the MAX232 serial level converter which costs about 3 bucks at DigiKey. Note that it requires a few 10uF capacitors to function properly.
Here’s a more general schematic:
I connected my ATTiny2313 to the MAX232 in a very standard way. (photo) MAX232 pins 13 and 14 go to the serial port, and the ATTiny2313 pins 2 and 3 go to the MAX232 pins 12 and 11 respectively. I will note that they used a oscillator value (3.6864MHz) different than mine (9.216MHz).
Determining the speed of serial communication is important. This is dependent on your oscillator frequency! I said I used a 9.216Mhz oscillator. First, a crystal or ceramic oscillator is required over the internal RC oscillator because the internal RC oscillator is not accurate enough for serial communication. The oscillator you select should be a perfect multiple of 1.8432MHz. Mine is 5x this value. Many people use 2x this value (3.6864Mhz) and that’s okay! You just have to make sure your microchip knows (1) to use the external oscillator (google around for how to burn the fuses on your chip to do this) and (2) what the frequency of your oscillator is and how fast it should be sending data. This is done by setting the UBRRL value. The formula to do this is here:
The datasheet of your microcontroller may list a lot of common crystal frequencies, bandwidths, and their appropriate UBRR values. However my datasheet lacked an entry for a 9.216MHz crystal, so I had to do the math myself. I Googled around and no “table” is available! Why not make one? (picture, below). Anyway, for my case I determined that if I set the UBRR value to 239, I could transmit data at 2800 baud (bits/second). This is slow enough to ensure accuracy, but fast enough to quickly dump a large amount of text to a PC terminal.
This will make your life easier. The page wormfood.net/avrbaudcalc.php has a chart of common crystals and the baud rates they work best with! Try to pick a combination that provides the least error possible…
This is the bare-minimum code to test out my setup. Just load the code (written in C, compiled with avr-gcc) onto your chip and it’s ready to go. Be sure you set your fuses to use an external oscillator and that you set your UBRRL value correctly.
#include <avr/io.h>
#include <avr/interrupt.h>
#include <util/delay.h>
int main (void)
{
unsigned char data=0;
UBRRL = 239;
UCSRB = (1 < < RXEN) | (1 << TXEN);
UCSRC = (1 < < UCSZ1) | (1 << UCSZ0);
for (;;)
{
if (data>'Z'||data< 'A')
{
UDR = 10; UDR = 13; data='A';_delay_ms(100);
}
UDR = data;
data += 1;
_delay_ms(100);
}
}
Once you load it, it’s ready to roll! It continuously dumps letters to the serial port. To receive them, open HyperTerminal (on windows, under accessories) or minicom (on Linux, look it up!). Set your baud rate to 2800 (or whatever you selected) and you’re in business. This (picture below) is the output of the microcontroller to HyperTerminal on my PC. Forgive the image quality, I photographed the LCD screen instead of taking a screenshot.
This is the circuit which generates the output of the previous image. I have a few extra components. I have an LED which I used for debugging purposes, and also a switch (labeled “R”). The switch (when pressed) grounds pin 1 of the ATTiny2313 which resets it. If I want to program the chip, I hold “R” down and the PC can program it with the inline programmer “parallel port, straight-through, DAPA style). One cable going into the circuit is for the parallel port programmer, one cable is for the serial port (data transfer), and one is for power (5v which I stole from a USB port).
I hope you found this information useful. Feel free to contact me with any questions you may have, but realize that I’m no expert, and I’m merely documenting my successes chronologically on this website.
I placed another online order to DigiKey yesterday. They have virtually every electronic component available for next-day delivery. How awesome is that? The frustrating thing is that I’m always trying to get my order in before 6pm CMT so that my package will ship same day, so I rush, and I always forget something! It never fails. 30 minutes after I place an order, I always remember something else I wish I’d added to the order (to save on shipping). So, instead of buying it when I want it (now) I wait a few more days to think of some more stuff to add. The last couple weeks I’ve placed 3 orders totaling about $30. You never think you’re spending a lot, but all the sudden you realize that all these 40-cent parts add up when you go to check out. Shipping is usually under $3, which makes me happy. What did I get? I got a couple MAX232 chips so I can easialy shuffle data back and forth from my PC to an AVR chip using the serial port using a simple terminal application like hyperterminal. This will allow me to do some pretty fancy stuff and certainly help with the prototyping/debugging/development steps of my various microcontroller-based projects.
Some of the things which I forgot to order include a few 1F supercapacitors for my solar-powered QRSS beacon project and a random collection of parts needed to build a DIY AVR microcontroller programmer with a USB PC interface and I also wanted to buy at least one ATMega8 microcontroller because they seem to be a good step above the ATTiny 2313 chips I already have and are at the center of so many microcontroller projects. I’m sure there are are few more items I need, but can’t think of them right now, so they don’t make the list.
A little random, but worth noting is a cool website I found earlier which details (in a very basic way) the various types of HF antennas and does a good job (IMHO) of explaining the basic theory behind them. The page can be found at deltadx.net.
As an aside, I encourage everyone to take a couple minutes out of your day and browse the Wikipedia entry describing the JPEG compression algorithm. Seriously, it’s an educational, enlightening, and entertaining read.
Man, what a long day! Work is so tedious sometimes. This week I’ve been proofing scientific literature using Office 2003 with “track changes”. I make changes, my boss makes changes, I make more changes, and it goes back and forth a few times. I wonder why office 2007 is so bad. Does anybody truly like it, and find it to be a significant improvement upon 2003? … or Vista over XP? Maybe I’m just getting old, inflexible, and grumpy.
This is what I’m currently working on. The light bubbles on the right are deletions. The dark bubbles on the right are comments. The red text is insertions/modifications I made. Pretty intense, right? Pages and pages of this. I’m starting to grasp the daunting amount of time a scientist must spend writing in the laboratory as opposed to performing actual experiments or even doing literature research.
Last night I assembled a Pixie II circuit similar to the one pictured here. I must say that I’m a little disappointed with the information available on the internet regarding simple RF theory in relation to transceiver circuits. I’m just now starting to get into RF circuitry and the concept looking at circuits and imagining a combination of AC and DC flowing through it is warping my brain. I have everything I need to build a simple Pixie II transceiver (which is supposedly capable of Morse code transmissions over 300 miles, and QRSS applications over 3,000 miles) but I don’t want to use it unless I understand how it actually works.
I’m trying to break this circuit down into its primary components. I understand the role of the lowpass filter. I understand the role of the 1st transistor and related circuitry in amplifying the output of the crystal oscillator (left side). I totally get the audio amplifier circuitry (bottom). It’s that center transistor (which supposedly handles signal amplification, receiving, and mixing) that I can’t get my mind around. Every time I think figure it out for one mode (sending or receiving), I mentally lose the other one. It has me very frustrated because it seems like this should be easier than I’m making it. I selected this circuit because it was simple and I assumed I’d be smart enough to figure it out… maybe I was wrong? I wish I had an oscilloscope so I could probe the RF passing through various stages of this circuit. I guess I should take another stab at reading chapters 5-11 of the ARRL handbook.
