Quantifying University Network Frustrations

I’m sitting in class frustrated as could be. The Internet in this room is unbelievably annoying. For some reason, everything runs fine, then functionality drops to unusable levels. Downloading files (i.e., PDFs of lectures) occurs at about 0.5kb/s (wow), and Internet browsing is hopeless. At most, I can connect to IRC and enjoy myself in #electronics, #python, and #linux. I decided to channel my frustration into productivity, and wrote a quick Python script to let me visualize the problem.

Notice the massive lag spikes around the time class begins. I think it’s caused by the retarded behavior of windows update and anti-virus software updates being downloaded on a gazillion computers all at the same time which are required to connect to the network on Windows machines. Class start times were 8:30am, 9:35am, and 10:40am. Let’s view it on a logarithmic scale:

Finally, the code. It’s two scripts:

This script pings a website (kernel.org) every few seconds and records the ping time to “pings.txt”:

import socket
import time
import os
import sys
import re


def getping():
    pingaling = os.popen("ping -q -c2 kernel.org")
    sys.stdout.flush()
    while 1:
        line = pingaling.readline()
        if not line:
            break
        line = line.split("n")
        for part in line:
            if "rtt" in part:
                part = part.split(" = ")[1]
                part = part.split('/')[1]
                print part+"ms"
                return part


def add2log(stuff):
    f = open("pings.txt", 'a')
    f.write(stuff+",")
    f.close()


while 1:
    print "pinging...",
    stuff = "[%s,%s]" % (time.time(), getping())
    print stuff
    add2log(stuff)
    time.sleep(1)

This script graphs the results:

import pylab
import time
import datetime
import numpy


def smoothTriangle(data, degree, dropVals=False):
    triangle = numpy.array(range(degree)+[degree]+range(degree)[::-1])+1
    smoothed = []
    for i in range(degree, len(data)-degree*2):
        point = data[i:i+len(triangle)]*triangle
        smoothed.append(sum(point)/sum(triangle))
    if dropVals:
        print "smoothlen:", len(smoothed)
        return smoothed
    while len(smoothed) < len(data):
        smoothed = [None]+smoothed+[None]
    if len(smoothed) > len(data):
        smoothed.pop(-1)
    return smoothed


print "reading"
f = open("pings.txt")
raw = eval("[%s]" % f.read())
f.close()

xs, ys, big = [], [], []
for item in raw:
    t = datetime.datetime.fromtimestamp(item[0])
    maxping = 20000
    if item[1] > maxping or item[1] == None:
        item[1] = maxping
        big.append(t)
    ys.append(float(item[1]))
    xs.append(t)

print "plotting"
fig = pylab.figure(figsize=(10, 7))
pylab.plot(xs, ys, 'k.', alpha=.1)
pylab.plot(xs, ys, 'k-', alpha=.1)
pylab.plot(xs, smoothTriangle(ys, 15), 'b-')
pylab.grid(alpha=.3)
pylab.axis([None, None, None, 2000])
pylab.ylabel("latency (ping kernel.org, ms)")
pylab.title("D3-3 Network Responsiveness")
fig.autofmt_xdate()
pylab.savefig('out.png')
pylab.semilogy()
pylab.savefig('out2.png')
fig.autofmt_xdate()
print "done"

Simple method to send data from ANY microcontroller to a serial port

This weekend I had a need, and I met it with parts I had on hand. Simply put, I wanted to assess whether or not my temperature-controlled crystal heater is doing its job in keeping temperature rock-stable. I wanted to measure temperature by measuring the ADC (analog-to-digital) value at the middle of a voltage divider with a resistor and a thermistor. Using a computer to plot this data, I can see if temperature fluctuates as my apartment AC turns on/off, or if it’s perfectly stable (my goal). The problem is that my only MCU (micro-controller unit) with USART (universal asynchronous receiver/transmitter) built-in is an ATTiny2313, which has no ADC capabilities. I had a lot of ATTiny44A chips on hand, so I had to figure out a way to get the data from my an ATTiny44A to an ATTiny2313 then to a MAX232 chip (voltage driver) so it can be sent to a PC’s serial port.

This is my bare-bones solution to easily sending data from ANY microcontroller to a PC’s serial port using 3 pins to send data to an ATTiny2313 which is interpreted, converted to decimal, then sent to my PC’s serial port. I will keep this little board and use it often to peek at variables inside my microcontroller projects in real time, with minimal coding!

Above is the bare-bones schematic required to send data from an ATTiny2313 to a PC via a serial port. This schematic is improved and documented better on this page than on my previous post Simple Case AVR/PC Serial Communication via MAX232. Note that I’m designing this to be functional, perhaps not well enough to be used in mission-critical systems. Although some schematics suggest extra capacitors, I found that the only one required is between pins 4 and 5 of the MAX232. The role of the MAX232 chip is to act as a voltage pump and relay the incoming signal at increased voltage which the PC’s serial port can read. It doesn’t actually change the data.

UPDATE: in a later project working with an ATMega48 I found that a capacitor was needed between pin 6 and ground - don’t know why! If it’s not working for you (you’re getting garbage) start adding capacitors as shown in this MAX232 circuit

Power supply: Since the thing runs on 5V, we’re golden! Just grab a USB cable, hook up the black (ground) and red (+5V) wires, and you’re good to go! If you want you can add a few capacitors of different values in parallel to the power supply to stabilize fluctuations in voltage, but I’ve been doing just fine without this extra precaution.

Display: The two LEDs are totally optional, but they let me see what’s going on. One of them flashes when the device is waiting for data (to let me know it’s on), and the other one turns on every time a [CLOCK] signal is detected (which is any time data is being sent)

Notes on frequency and crystals. The UBRRL value in the code must be set depending on your micro-controller speed and desired baud rate. I set-up an Excel spreadsheet and did some math determining UBRRL for a combination of different frequencies/rates. The UBRRL values closest to whole numbers are those which should be used to minimize errors. External crystals are often used to increase and stabalize the clock speed of micro-controllers, but I was able to send serial data without a crystal. I set the fuse for “internal 8mhz” clocking, and enabled the “div8” fuse so it actually ran at 1mhz. With these settings at 4800 baud, UBRR [according to the equation UBRR=(freq/(16*baud))-1] is 12.02 (pretty close to 12), so I set UBRRL=12 in the code and it sent data to a PC beautifully without a crystal. However, I had the desire to run the MCU faster to watch for incoming signals. I therefore used a 9.21MHz crystal (I had to set the fuses to enable the external crystal), which can send serial data to a PC reliably at 19200 baud.

Sending data to the ATTiny2313 to be relayed to the PC: Not every MCU has SPI, USI, I2C, TWI, USART, or other “standard” communication methods. If I want to have a Texas Instruments or PIC or PICaxe chip send data to a PC, I have to decipher the datasheet and spend half a day to figure out how (yuk!). Therefore, I developed an ULTRA-SIMPLE protocol of data transfer which can be used by ANY microcontroller. Here’s an example of a sensor microcontroller. Although it’s not shown, there’s a thermistor (or some analog voltage being measured) somewhere. It reads the sensor, then sends its data over the 3 wires [CLOCK], [A], and [B].

Pulling-down the clock: Note that the 100k resistor shown pulling the [CLOCK] line to ground is critical. It doesn’t have to be 100k, it can be virtually any value, it just makes sure that if current is not being sent on the clock line, it quickly goes to 0V. Without this resistor, the clock line might flicker on and off even though no data is being sent.

Sending data this way is easy! The clock line is almost always low. When [clock] goes high, data is read. When data is read, the ATTiny2313 determines the state of [A] and [B]. If A=0 and B=0, a ZERO is sent. If A=1 and B=0, a ONE is sent. If A=0 and B=1, a SPACE is sent (between values). If A=1 and B=1, a LINE BREAK is sent. Values are sent in binary, and when a space or line break is detected, the binary value is converted to decimal and sent to the PC’s serial port. It’s not dependent on speed, so send data as fast (within reason) or slowly as you want from any microcontroller and it will end-up on your computer screen! It’s that easy!

FLAME ALERT: A lot of people will be mad at me for suggesting this method. There are fancier, faster, and more stable ways to data transfer between micro-controllers, but this works well at moderate speeds (maybe 10 measurements a second?) and I can implement this on any microcontroller in seconds, without puzzling over the datasheet.

BREADBOARDED PROTOTYPE

Remember that I’m powering this entirely from USB power. The layout is simple: ATTiny44A measuring ADC of a thermistor on the left (see the little red thing?) sending data with 3 wires (top 3) to an ATTiny2313 intermediate chip (center), which converts this binary data to decimal and sends it to a MAX232 chip (right) where it gets converted to levels suitable for transmission to a serial port.

CODE (AVR-GCC)

This is what runs on the ATTiny2313 intermediate chip:

#include <avr/io.h>
#include <avr/interrupt.h>
#include <util/delay.h>

#define B PB2
#define A PB1
#define clk PB0
unsigned int times = 0;

// THIS RUNS ON THE ATTINY2313
// FUSES SET FOR INTERNAL 8MHZ, DIV/8=TRUE
// LISTENS ON PINS 12 (CLOCK), 13 (DATA), AND 14 (TRANSMIT)
// OUTPUTS TO A MAX232 THEN TO SERIAL PORT
// PC TERMINAL SET TO LISTEN AT 4800 BAUD

int main(void)
{
    UBRRL = 29;                          // value determined for 9.21MHz crystal at 19200 baud
    UCSRB = (1 << RXEN) | (1 << TXEN);   // fire-up USART
    UCSRC = (1 << UCSZ1) | (1 << UCSZ0); // fire-up USART
    DDRB = 0;                            // set all of port b to input
    DDRD = 255;                          // set all of port d to output
    char last = 255;
    int var = 0;
    char pos = 0;
    char i = 0;
    for (;;)
    {
        while
            bit_is_clear(PINB, clk) { blink(); }
        PORTD |= (1 << PD5);
        PORTD &= ~(1 << PD4);
        if (bit_is_set(PINB, A) && bit_is_clear(PINB, B))
        {
            var = (var << 1) + 1;
        } //ONE
        if (bit_is_clear(PINB, A) && bit_is_clear(PINB, B))
        {
            var = (var << 1);
        } //ZERO
        if (bit_is_clear(PINB, A) && bit_is_set(PINB, B))
        {
            show(var);
            var = 0;
            send(32);
        } //SPACE
        if (bit_is_set(PINB, A) && bit_is_set(PINB, B))
        {
            show(var);
            var = 0;
            send(10);
            send(13);
        } //BREAK
        while
            bit_is_set(PINB, clk) { blink(); }
        PORTD &= ~(1 << PD5);
        PORTD |= (1 << PD4);
    }
}
void blink()
{
    // just hanging out
    times++;
    if (times == 10000)
    {
        times == 0;
        PORTD |= (1 << PD3);
    }
    if (times == 20000)
    {
        times == 0;
        PORTD &= ~(1 << PD3);
        times = 0;
    }
}

unsigned int rev(unsigned int b)
{
    unsigned char result = 0;
    while (b)
    {
        result <<= 1;
        result |= b % 2;
        b >>= 1;
    }
    return result;
}

void show(unsigned int val)
{
    // SHOW INDIVIDUAL 1s and 0s?
    // for (char i=0;i<16;i++){
    //   if (val&(1<<i)){send(49);}
    //   else {send(48);}
    // }
    // send(61);

    val = rev(val);
    if (val == 0)
    {
        send(48);
    }
    else
    {
        char started = 0;
        int div = 10000;
        for (char i = 0; i < 5; i++)
        {
            if (val > div)
            {
                started = 1;
            }
            if (started)
            {
                send(val / div + 48);
                val = val - (val / div) * div;
            }
            div = div / 10;
        }
    }
    return;
}

void send(unsigned char data)
{
    while (!(UCSRA & (1 << UDRE)))
        ;       // wait for buffer to be empty
    UDR = data; // send that sucker
}

This is what runs on my ATTiny44a sensor chip. This is what you can replace with ANYTHING, as long as it twiddles the [clock], [a], and [b] pins similarly.

#include <avr/io.h>
#include <avr/interrupt.h>
#include <util/delay.h>

#define dtaprt PORTA
#define clk PA1
#define A PA2
#define B PA0
#define delayms 100 // increase to slow it down

void wait() { _delay_ms(delayms); }
void clockNow()
{
    dtaprt |= (1 << clk);
    wait();
    dtaprt &= ~(1 << clk);
    wait();
}
void sendSpace()
{
    dtaprt = (1 << B);
    clockNow();
}
void sendLine()
{
    dtaprt = (1 << B) | (1 << A);
    clockNow();
}
void sendOne()
{
    dtaprt = (1 << A);
    clockNow();
}
void sendZero()
{
    dtaprt = 0;
    clockNow();
}

// TAKE A READING FROM ADC7 AND SEND IT TO SERIAL CHIP

int main(void)
{
    DDRA |= (1 << clk);
    DDRA |= (1 << A);
    DDRA |= (1 << B);
    ADMUX = (1 << REFS1) | (1 << MUX2) | (1 << MUX1) | (1 << MUX0);    // ADC on ADC7 to 1.1v ref
    ADCSRA = (1 << ADEN) | (1 << ADPS0) | (1 << ADPS1) | (1 << ADPS2); // enable, prescale
    for (;;)
    {
        int data = ReadADC();
        sendData(data);
        sendSpace();
        sendData(data);
        sendLine();
        _delay_ms(1000);
    }
}

void sendData(int data)
{
    char datalen = 16;
    for (char pos = 0; pos < datalen; pos++)
    {
        if ((data >> pos) & 1)
        {
            sendOne();
        }
        else
        {
            sendZero();
        }
    }
}

int ReadADC()
{
    ADCSRA |= (1 << ADSC); // reset value
    while (ADCSRA & (1 << ADSC))
        ; // wait for measurement
    return ADC;
}

UPDATE

I found a simpler way to convert binary numbers into strings ready to be sent in ASCII via USART serial port:

void sendNum(unsigned int num)
{
    char theIntAsString[7];
    int i;
    sprintf(theIntAsString, "%u", num);
    for (i = 0; i < strlen(theIntAsString); i++)
    {
        send(theIntAsString[i]);
    }
}

VD Labs

The VD Labs web page has been published! I hope that the new VD Labs page will be a single location where I can link to descriptions and downloads of useful radio, audio analysis, and QRSS-related software. It will eventually be the home of the next (recorded-from-scratch) version of QRSS VD, but let’s not get too far ahead of ourselves!

Since I ran out of steam from working so much on QRSS VD, I didn’t think I’d be publishing mush more “useful” software, but this one blind-sighted me. People on the Knights QRSS mailing list were talking about dividing QRSS transmissions into images which line up with the period of the transmitters repeated messages and projecting the images together in an attempt to average-out the noise, and boost the signal. It’s a simple idea, and it’s the basis behind how a lot of poor imaging devices can improve their output clarity by software (MRI anyone?). I was overwhelmed by dental school obligations the last few weeks, and it pained me so much to read what people were doing (or at least trying to do) and having to sit it out. Now that I have a free day (yay for weekends!) I sat down and wrote some code. I introduce VD Labs QRSS Stitcher and QRSS Stacker!

Converting Argo captures into continuous images:

example output:

Doing the same thing, with ultra-narrow images:

File produced:

Using QRSS Stacker to project images:

Another example output:

Screenshots:

Hacking Together a Crystal Oven Part 2

With the last post’s promising results, I set out to finish my crystal oven prototype and affix it to my QRSS MEPT prototype. If everything works well, I’m ready to publish the final schematic and parts lists! (and make several MEPTs for people who were interested in having one). I’m not confident my approach to the heater was the best, and am already thinking of ways I could have improved on its performance, but I think this just might work! I’ll test it overnight (Styrofoam-enclosed vs. open air) and see how it does. I wonder if this is good enough to be used outside?

Minimalist Crystal Oven

So I’m working on building a crystal oven to keep my QRSS MEPT (radio transmitter) at an extremely stable frequency. Even inside a thick Styrofoam box, slight changes in my apartment temperature caused by the AC turning on and off is enough to change the crystal temperature of the transmitter, slightly modifying its oscillation frequency. For a device that vibrates exactly 10,140,070 times a second, even 3 to many or too few vibrations per second is too much. Keeping in the spirit of hacking things together with a minimum of parts, this is what I came up with!

It uses a thermistor, potentiometer, and comparator of a microcontroller (ATTiny44a) to tightly sense and regulate temperature. The heater element is a junk MOSFET I found in an old battery backup system. I simply have pass a ton of current (turned on/off by the gate) to generate heat, transferred into a piece of steel for smooth regulation. One of the unexpected advantages is that the light flickers rapidly near equilibrium, which is great because it has the ability to turn the heater on a little or a lot based upon the averaging effect of the flicker. Here is the code I wrote on the microcontroller to handle the comparator. It couldn’t be simpler!

#include <avr/io.h>
#include <util/delay.h>

int main(void)
{
    DDRA = 0;   // all inputs
    DDRB = 255; // all outputs

    while (1)
    {
        if (ACSR & _BV(ACO))
        {
            /* AIN0 is more positive than AIN1 right now */
            PORTB |= (1 << PB0);
            PORTB &= ~(1 << PB1);
        }
        else
        {
            /* AIN0 is more negative than AIN1 */
            PORTB |= (1 << PB1);
            PORTB &= ~(1 << PB0);
        }
    }
}