The personal website of Scott W Harden
December 17th, 2022

Divide 10 MHz to 1PPS with a Microcontroller

I often find it useful to gate frequency counters using a 1 pulse per second (1PPS) signal derived from a 10 MHz precision frequency reference. However, a divide-by-10-million circuit isn't trivial to implement. Counter ICs exist which enable divide-by-100 by combining multiple divide-by-2 and divide-by-5 stages (e.g., MC74HC390A around $0.85 each), but dividing 10 MHz all the way down to 1Hz would require at least 4 of these chips and a lot of wiring.

You can clock a microcontroller at 10 MHz and use its timer and interrupt systems to generate 1PPS. For example, an ATTiny202 in an 8-pin SOIC package is available from Mouser (>50k stocked) for $0.51 each. Note that modern series AVRs require a special UDPI programmer.

ATTiny202 ($0.51) ATTint826 ($0.95)

This page documents a divide-by-10-million circuit achieved with a single microcontroller to scale a 10MHz frequency reference down to 1PPS. I'm using an ATTiny826 because that is what I have on hand, but these concepts apply to any microcontroller with a 16-bit timer.

ATTiny Breakout Board

Some AVRs come in DIP packages but their pin numbers may be different than the same chip in a SMT package. To facilitate prototyping using designs and code that will work identically across a breadboard prototype and a PCB containing SMT chips, I prefer to build DIP breakout boards using whatever SMT package I intend to include in my final builds. In this case it's ATTint826 in a SOIC-20 package, and I can easily use this in a breadboard by soldering them onto SOIC-to-DIP breakout boards.

I assembled the breakout board by hand using a regular soldering iron. When working with small packages it helps so much to coat the pins with a little tack flux to facilitate wetting and prevent solder bridges. I'm presently using Chip Quik NC191. Even if flux is advertized as "no-clean", it's good practice and makes the boards look much nicer to remove remaining flux with acetone and Q-tips or brushes.

Circuit

  • FTDI breakout board for power: To test this design I'm using a FT232 breakout board just to provide easy access to GND and Vcc (5V from the USB rail).

  • 10 MHz can oscillator: It's not ovenized or GPS disciplined, but I'm using this as a stand-in for whatever high-precision 10 MHz frequency standard will eventually be used in this circuit. The important thing is just to know that it outputs 0-5V square waves at 10 MHz going into the EXTCLK pin of the microcontroller

  • UPDI Programmer: I'm using the Atmel-ICE, but a MPLAB Snap would also work here. See Programming Modern AVR Microcontrollers for more information.

  • Output: A LED on an output pin will visualize the 1pps signal

Configuration Change Protection (CCP)

Traditional AVR microcontrollers used fuse bits to set the clock source, but modern series chips can change the clock source from within code. However, modifying the clock source requires temporarily disabling the configuration change protection (CCP) system.

Disabling the CCP only lasts four clock cycles, so the immediate next statement must be assignment of the new value. I use the following function to facilitate this action.

/* Write a value to a CCP-protected register */
void ccp_write(volatile register8_t* address, uint8_t value){
    CCP = CCP_IOREG_gc;
    *address = value;
}
// Use internal 20 MHz clock with CKOUT pin enabled
ccp_write(&CLKCTRL.MCLKCTRLA, CLKCTRL.MCLKCTRLA | CLKCTRL_CLKOUT_bm);

Do not use compound statements when writing to the CCP register. The code below fails to change clock as one may expect by looking at the code, presumably because the combined OR operation with the assignment exceeds four clock cycles. Instead of direct assignment, use the ccp_write function described above.

// WARNING: This code does not actually change the clock source
CCP = CCP_IOREG_gc;
CLKCTRL.MCLKCTRLA = CLKCTRL.MCLKCTRLA | CLKCTRL_CLKOUT_bm;

Configuring the Clock Source

Internal 10 MHz clock

This is the configuration I use to achieve a 10 CPU clock using the built-in 20 MHz oscillator.

void configure_clock_internal_10mhz(){
    ccp_write(&CLKCTRL.MCLKCTRLA, CLKCTRL.MCLKCTRLA | CLKCTRL_CLKOUT_bm); // 20 MHz internal clock, enable CKOUT
    ccp_write(&CLKCTRL.MCLKCTRLB, CLKCTRL_PEN_bm); // enable divide-by-2 clock prescaler
}

External 10 MHz clock

This is the configuration I use to clock the CPU from an external 10 MHz clock source applied to the EXTCLK pin.

void configure_clock_external(){
    ccp_write(&CLKCTRL.MCLKCTRLA, CLKCTRL_CLKSEL_EXTCLK_gc | CLKCTRL_CLKOUT_bm); // external clock, enable CKOUT
    ccp_write(&CLKCTRL.MCLKCTRLB, 0); // disable prescaler
}

Configuring the 16-Bit Timer

This is how I configured my ATTiny826's TCA0 16-bit timer to fire an interrupt every 200 ms.

  • Prescale: By enabling a divide-by-64 prescaler, my 10 MHz input becomes 156,250 Hz.

  • Top: By setting the top of my 16-bit counter at 31,250, I achieve exactly 5 overflows per second (once every 200 ms).

  • Interrupt: By enabling an overflow interrupt, I am able to call a function every 200 ms.

void configure_1pps(){
    // 10 MHz system clock with div64 prescaler is 156,250 Hz.
    // Setting a 16-bit timer's top to 31,250 means 5 overflows per second.
    TCA0.SINGLE.INTCTRL = TCA_SINGLE_OVF_bm; // overflow interrupt
    TCA0.SINGLE.CTRLB = TCA_SINGLE_WGMODE_NORMAL_gc; // normal mode
    TCA0.SINGLE.PER = 31249UL; // control timer period by setting timer top
    TCA0.SINGLE.CTRLA |= TCA_SINGLE_CLKSEL_DIV64_gc; // set clock source
    TCA0.SINGLE.CTRLA |= TCA_SINGLE_ENABLE_bm; // start timer
}

Alternatively, multiple timers could be cascaded to achieve a similar effect. Modern AVR series microcontrollers have sections in their datasheet describing considerations for cascading two 16-bit timers to create a single 32-bit timer. Using this strategy one could set the top of the counter to 5 million and arrange an interrupt to toggle an LED, resulting in a 1Hz signal with 50% duty.

Configuring the Interrupt System

This method is called whenever the timer's overflow interrupt is triggered. Since it's called 5 times per second, I just need a global counter to count the number of times it was called, and set an output pin to high on every 5th invocation.

uint8_t overflow_count;

ISR(TCA0_OVF_vect)
{
    overflow_count++;
    if (overflow_count == 5){
        overflow_count = 0;
        PORTB.OUT = PIN1_bm;
    } else {
        PORTB.OUT = 0;
    }

    TCA0.SINGLE.INTFLAGS = TCA_SINGLE_OVF_bm; // indicate interrupt was handled
}

Do not forget to enable global interrupts in your start-up sequence! This is an easy mistake to make, and without calling this function the overflow function will never be invoked.

sei(); // enable global interrupts

Results

We have achieved a light that blinks exactly once per second with roughly the same precision as the 10 MHz frequency reference used to clock the microcontroller. This output signal is ready to use for precision measurement purposes, such as toggling the gate of a discrete frequency counter.

Resources

Markdown source code last modified on December 23rd, 2022
---
Title: Divide 10 MHz to 1PPS with a Microcontroller
Description: How to use a microcontroller to inexpensively scale down a 10 MHz reference clock into a one pulse per second (1pps) signal
Date: 2022-12-17 1:09PM EST
Tags: circuit, microcontroller
---

# Divide 10 MHz to 1PPS with a Microcontroller

**I often find it useful to gate frequency counters using a 1 pulse per second (1PPS) signal derived from a 10 MHz precision frequency reference.** However, a divide-by-10-million circuit isn't trivial to implement. Counter ICs exist which enable divide-by-100 by combining multiple divide-by-2 and divide-by-5 stages (e.g., [MC74HC390A](https://www.onsemi.com/pdf/datasheet/mc74hc390a-d.pdf) around $0.85 each), but dividing 10 MHz all the way down to 1Hz would require at least 4 of these chips and a lot of wiring.

**You can clock a microcontroller at 10 MHz and use its timer and interrupt systems to generate 1PPS.** For example, an [ATTiny202](https://www.mouser.com/datasheet/2/268/ATtiny202_402_AVR_MCU_with_Core_Independent_Periph-1384964.pdf) in an 8-pin SOIC package is available from Mouser (>50k stocked) for $0.51 each. Note that [modern series AVRs require a special UDPI programmer](https://swharden.com/blog/2022-12-09-avr-programming). 

ATTiny202 ($0.51) | ATTint826 ($0.95)
---|---
<img src="ATTINY202-SOIC-8.png">|<img src="ATTINY826-SOIC-20.png">

**This page documents a divide-by-10-million circuit achieved with a single microcontroller to scale a 10MHz frequency reference down to 1PPS.** I'm using an [ATTiny826](https://www.mouser.com/datasheet/2/268/ATtiny424_426_427_824_826_827_DataSheet_DS40002311-2887739.pdf) because that is what I have on hand, but these concepts apply to any microcontroller with a 16-bit timer.

![](1pps-breadboard2.jpg)

## ATTiny Breakout Board

**Some AVRs come in DIP packages but their pin numbers may be different than the same chip in a SMT package.** To facilitate prototyping using designs and code that will work identically across a breadboard prototype and a PCB containing SMT chips, I prefer to build DIP breakout boards using whatever SMT package I intend to include in my final builds. In this case it's ATTint826 in a SOIC-20 package, and I can easily use this in a breadboard by soldering them onto [SOIC-to-DIP breakout boards](https://www.amazon.com/s?k=soic+dip+breakout).

![](breakout1.jpg)

**I assembled the breakout board by hand using a regular soldering iron.** When working with small packages it helps _so much_ to coat the pins with a little tack flux to facilitate wetting and prevent solder bridges. I'm presently using [Chip Quik NC191](https://www.amazon.com/s?k=NC191). Even if flux is advertized as "no-clean", it's good practice and makes the boards look much nicer to remove remaining flux with acetone and Q-tips or brushes.

![](breakout2.jpg)
![](breakout3.jpg)

## Circuit

* **FTDI breakout board for power:** To test this design I'm using a FT232 breakout board just to provide easy access to `GND` and `Vcc` (5V from the USB rail).

* **10 MHz can oscillator:** It's not ovenized or GPS disciplined, but I'm using this as a stand-in for whatever high-precision 10 MHz frequency standard will eventually be used in this circuit. The important thing is just to know that it outputs 0-5V square waves at 10 MHz going into the `EXTCLK` pin of the microcontroller

* **UPDI Programmer:** I'm using the Atmel-ICE, but a MPLAB Snap would also work here. See [Programming Modern AVR Microcontrollers](https://swharden.com/blog/2022-12-09-avr-programming) for more information.

* **Output:** A LED on an output pin will visualize the 1pps signal

![](1pps-breadboard.jpg)

## Configuration Change Protection (CCP)

**Traditional AVR microcontrollers used fuse bits to set the clock source, but modern series chips can change the clock source from within code.** However, modifying the clock source requires temporarily disabling the configuration change protection (CCP) system. 

Disabling the CCP only lasts four clock cycles, so the immediate next statement must be assignment of the new value. I use the following function to facilitate this action.

```c
/* Write a value to a CCP-protected register */
void ccp_write(volatile register8_t* address, uint8_t value){
	CCP = CCP_IOREG_gc;
	*address = value;
}
```

```c
// Use internal 20 MHz clock with CKOUT pin enabled
ccp_write(&CLKCTRL.MCLKCTRLA, CLKCTRL.MCLKCTRLA | CLKCTRL_CLKOUT_bm);
```

**Do not use compound statements when writing to the CCP register.**  The code below fails to change clock as one may expect by looking at the code, presumably because the combined OR operation with the assignment exceeds four clock cycles. Instead of direct assignment, use the `ccp_write` function described above.

```c
// WARNING: This code does not actually change the clock source
CCP = CCP_IOREG_gc;
CLKCTRL.MCLKCTRLA = CLKCTRL.MCLKCTRLA | CLKCTRL_CLKOUT_bm;
```

## Configuring the Clock Source


### Internal 10 MHz clock

This is the configuration I use to achieve a 10 CPU clock using the built-in 20 MHz oscillator.

```c
void configure_clock_internal_10mhz(){
	ccp_write(&CLKCTRL.MCLKCTRLA, CLKCTRL.MCLKCTRLA | CLKCTRL_CLKOUT_bm); // 20 MHz internal clock, enable CKOUT
	ccp_write(&CLKCTRL.MCLKCTRLB, CLKCTRL_PEN_bm); // enable divide-by-2 clock prescaler
}
```

### External 10 MHz clock

This is the configuration I use to clock the CPU from an external 10 MHz clock source applied to the `EXTCLK` pin.

```c
void configure_clock_external(){
	ccp_write(&CLKCTRL.MCLKCTRLA, CLKCTRL_CLKSEL_EXTCLK_gc | CLKCTRL_CLKOUT_bm); // external clock, enable CKOUT
	ccp_write(&CLKCTRL.MCLKCTRLB, 0); // disable prescaler
}
```

## Configuring the 16-Bit Timer

This is how I configured my ATTiny826's TCA0 16-bit timer to fire an interrupt every 200 ms.

* **Prescale:** By enabling a divide-by-64 prescaler, my 10 MHz input becomes 156,250 Hz.

* **Top:** By setting the top of my 16-bit counter at 31,250, I achieve exactly 5 overflows per second (once every 200 ms).

* **Interrupt:** By enabling an overflow interrupt, I am able to call a function every 200 ms.

```c
void configure_1pps(){
	// 10 MHz system clock with div64 prescaler is 156,250 Hz.
	// Setting a 16-bit timer's top to 31,250 means 5 overflows per second.
	TCA0.SINGLE.INTCTRL = TCA_SINGLE_OVF_bm; // overflow interrupt
	TCA0.SINGLE.CTRLB = TCA_SINGLE_WGMODE_NORMAL_gc; // normal mode
	TCA0.SINGLE.PER = 31249UL; // control timer period by setting timer top
	TCA0.SINGLE.CTRLA |= TCA_SINGLE_CLKSEL_DIV64_gc; // set clock source
	TCA0.SINGLE.CTRLA |= TCA_SINGLE_ENABLE_bm; // start timer
}
```

**Alternatively, multiple timers could be cascaded to achieve a similar effect.** Modern AVR series microcontrollers have sections in their datasheet describing considerations for cascading two 16-bit timers to create a single 32-bit timer. Using this strategy one could set the top of the counter to 5 million and arrange an interrupt to toggle an LED, resulting in a 1Hz signal with 50% duty.

## Configuring the Interrupt System

**This method is called whenever the timer's overflow interrupt is triggered.** Since it's called 5 times per second, I just need a global counter to count the number of times it was called, and set an output pin to high on every 5th invocation.

```c
uint8_t overflow_count;

ISR(TCA0_OVF_vect)
{
	overflow_count++;
	if (overflow_count == 5){
		overflow_count = 0;
		PORTB.OUT = PIN1_bm;
    } else {
		PORTB.OUT = 0;
	}
    
	TCA0.SINGLE.INTFLAGS = TCA_SINGLE_OVF_bm; // indicate interrupt was handled
}
```

**Do not forget to enable global interrupts in your start-up sequence!** This is an easy mistake to make, and without calling this function the overflow function will never be invoked.

```c
sei(); // enable global interrupts
```

## Results

**We have achieved a light that blinks exactly once per second** with roughly the same precision as the 10 MHz frequency reference used to clock the microcontroller. This output signal is ready to use for precision measurement purposes, such as toggling the gate of a discrete frequency counter.

<video playsinline autoplay muted loop class="border border-dark shadow-sm img-fluid">
  <source src="1pps.webm" type="video/webm">
</video>

## Resources

* Full source code: [ATTiny826 1pps project on GitHub](https://github.com/swharden/AVR-projects/tree/master/ATTiny826%20Timer%201pps) and specifically [main.c](https://github.com/swharden/AVR-projects/blob/master/ATTiny826%20Timer%201pps/ATTiny826%20Clock%20and%20Timer/main.c)

* Inspecting the header file `iotn826.h` in my Program Files / Atmel folder was very useful for identifying named bit masks stored as enums. There is a similarly named file for every supported AVR microcontroller.

* EEVblog forum: [Divide by 10000000](https://www.eevblog.com/forum/projects/divide-by-10000000/)

* EEVblog forum: [Divide by 10 prescaler for frequency counter](https://www.eevblog.com/forum/rf-microwave/divide-by-10-prescaler-for-frequency-counter/)

* EEVblog forum: [10MHz to 1pps divider](https://www.eevblog.com/forum/projects/10mhz-to-1pps-divider/)

* EEVblog forum: [Easiest way to divide 10MHz to 1MHz?](https://www.eevblog.com/forum/projects/easiest-way-to-divide-10mhz-to-1mhz/)

* YouTube: [Build a DIY Frequency Divider](https://www.youtube.com/watch?v=GlKWexGWoXw)

* [picDIV: Single Chip Frequency Divider](http://www.leapsecond.com/pic/picdiv.htm) (2011)

* [PICDIV on GitHub](https://github.com/aewallin/PICDIV)

* All About Circuits thread: [How to convert 10 MHz sine wave to 1Hz TTL (PPS)?](https://forum.allaboutcircuits.com/threads/convert-10-mhz-sine-wave-to-1hz-ttl-pps.54085/)

* [10 MHz to 1 Hz frequency divider using discrete 74HC4017D stages](http://www.perdrix.co.uk/FrequencyDivider/) by David C. Partridge
December 9th, 2022

Programming Modern AVR Microcontrollers

This page describes how to program Microchip's newest series of AVR microcontrollers using official programming gear and software. I spent many years programming the traditional series of Atmel chips, but now several years after Microchip acquired Atmel I am interested in exploring the capabilities of the latest series of AVR microcontrollers (especially the new AVR DD family). Currently the global chip shortage makes it difficult to source traditional ATMega and STM32 chips, but the newest series of AVR microcontrollers feature an impressive set of peripherals for the price and are available from all the major vendors.

TLDR

  • Older AVR microcontrollers are programmed using in-circuit serial programming (ICSP) through the RESET, SCK, MISO, and MOSI pins using cheap programmers like USBtiny. However, serial programming is not supported on newer AVR microcontrollers.

  • New AVR microcontrollers are programmed using the unified program and debug interface (UDPI) exclusively through the UDPI pin. UDPI is a Microchip proprietary interface requiring a UDPI-capable programmer.

  • Official UDPI programmers include Atmel-ICE ($129) and MPLAB Snap ($35). The Atmel-ICE is expensive but it is very well supported. The MPLAB Snap is hacky, requires re-flashing, and has a physical design flaw requiring a hardware modification before it can program AVR series chips.

  • There are notable attempts to create alternative programmers (e.g., jtag2updi and pymcuprog), but this journey into the land of unofficial programmer designs is fraught with abandoned GitHub repositories and a lot of added complexity and brittleness (e.g., SpenceKonde/AVR-Guidance), so to save yourself frustration in the future I highly recommend just buying an officially supported programmer. It's also nice when you can program and debug your microcontroller from within your IDE.

  • UDPI programmers have a Vcc pin that is used to sense supply voltage (but not provide it), so you must power your board yourself while using one of these new programmers.

Blinking a LED is the "Hello, World" of microcontroller programming. Let's take a look at the code necessary to blink a LED on pin 2 of an ATTiny286. It is compiled and programmed onto the chip using Microchip Studio.

#define F_CPU 3333333UL
#include <avr/io.h>
#include <util/delay.h>

int main(void)
{
    PORTA.DIR = 0xFF;
    while (1)
    {
        PORTA.OUT = 255;
        _delay_ms(500);
        PORTA.OUT = 0;
        _delay_ms(500);
    }
}
  • PORTA.DIR sets the direction of pins on port A (0xFF means all outputs)
  • PORTA.OUT sets output voltage on pins of port A (0xFF means all high)
  • Using _delay_ms() requires including delay.h
  • Including delay.h requires defining F_CPU (the CPU frequency)
  • The ATTiny286 datasheet section 11.3.3 indicates the default clock is 20 MHz with a /6 prescaler, so the default clock is 3333333 Hz (3.3 MHz). This behavior can be customized using the Oscillator Configuration Fuse (FUSE.OSCCFG).

ATTiny826 Pinout

From page 14 of the ATTiny826 datasheet

SMT ATTiny Breakout Board

Many of the newer AVR series microcontrollers are not available in breadboard-friendly DIP packages. I find SOIC-to-DIP breakout boards (available on Amazon and eBay) to be useful for experimenting with chips in SOIC packages. Here I added extra power and PA4 (pin 2) LEDs and 10 kΩ current limiting resistors.

I power the device from the 3.3V or 5V pins on a FT232 USB breakout board. Although the topic is out of scope for this article, I find it convenient to use FTDI chips to exchange small amounts of data or debug messages between a microcontroller and a modern PC over USB without requiring special drivers.

Why is programming modern AVRs so difficult?

I'm surprised how little information there is online about how to reliably program modern AVR series microcontrollers. In late 2022 there is a surprisingly large amount of "advice" on this topic which leads to dead ends and broken or abandoned projects. After looking into it for a while, here is my opinionated assessment. Mouser and Digikey have links to expensive programmers, and Amazon has links to similar items but reviews are littered with comments like "arrived bricked" and "can not program AVR series chips". DIY options typically involve abandoned (or soon-to-be abandoned?) GitHub repositories, or instructions for Arduino-related programming. I seek to consolidate and distill the most useful information onto this page, and I hope others will find it useful.

Atmel-ICE: Expensive but Effective

After using $5 ICSP programmers for the last decade I almost fell out of my chair when I saw Microchip's recommended entry-level programmer is over $180! Digikey sells a "basic" version without cables for $130, but that still seems crazy to me. Also, $50 for a ribbon cable?

I found a kit on Amazon that sells the programmer with a cable for $126. It was hard for me to press that buy button, but I figured the time I would save by having access to modern and exotic chips during the present global chip shortage would make it worth it. After a couple days of free Prime shipping, it arrived. It was smaller than I thought it would be from the product photos.

The cable that came with the device seemed a bit hacky at first, but I'm happy to have it. The female 2.54 mm socket is easy to insert breadboard jumpers into.

I'm glad this thing is idiot proof. The very first thing I did after unboxing this programmer was hook it up to my power supply rails using reverse polarity. I misread the pin diagram and confused the socket with the connector (they are mirror images of one another). This is an easy mistake to make though, so here's a picture of the correct orientation. Note the location of the tab on the side of the connector.

Atmel ICE Pinout Programming Connection
  • Black: GND
  • Red: Vcc - This line is used to sense power and not deliver it, so you are responsible for externally powering your board.
  • Blue: UPDI pin - Although a pull-up resistor on the UPDI pin is recommended, I did not find it was required to program my chip on the breadboard in this configuration.

The AVR Ice was easy to use with Microchip Studio. My programmer was detected immediately, a window popped-up and walked me through updating the firmware, and my LED was blinking in no time.

MPLAB Snap: Cheap and Convoluted

Did I really need to spend $126 for an AVR programmer? Amazon carries the MPLAB Snap for $34, but lots of reviews say it doesn't work. After easily getting the Atmel-ICE programmer up and running I thought it would be a similarly easy experience setting-up the MPLAB Snap for AVR UPDI programming, but boy was I wrong. Now that I know the steps to get this thing working it's not so bad, but the information here was only gathered after hours of frustration.

Here are the steps you can take to program modern AVR microcontrollers with UPDI using a MPLAB Snap:

Step 1: Setup MPLAB

  • The MPLAB Snap ships with obsolete firmware and must be re-flashed immediately upon receipt.

  • Microchip Studio's firmware upgrade tool does not actually work with the MPLAB Snap. It shows the board with version 0.00 software and it hangs (with several USB disconnect/reconnect sounds) if you try to upgrade it.

  • You can only re-flash the MPLAB Snap using the MPLAB X IDE. Download the 1.10 GB MPLAB setup executable and install the MPLAB IDE software which occupies a cool 9.83 GB.

Step 2: Re-Flash the Programmer

  • In the MPLAB IDE select Tools and select Hardware Tool Emergency Boot Firmware Recovery. At least this tool is helpful. It walks you through how to reset the device and program the latest firmware.

Step 3: Acknowledge Your Programmer is Defective

Defective may be a strong word, but let's just say the hardware was not designed to enable programming AVR chips using UPDI. Microchip Studio will detect the programmer but if you try to program an AVR you'll get a pop-up error message that provides surprisingly little useful information.

Atmel Studio was unable to start your debug session. Please verify device selection, interface settings, target power and connections to the target device. Look in the details section for more information. StatusCode: 131107 ModuleName: TCF (TCF command: Processes:launch failed.) An illegal configuration parameter was used. Debugger command Activate physical failed.

Step 4: Fix Your Programmer

The reason MPLAB Snap cannot program AVR microcontrollers is because the UPDI pin should be pulled high, but the MPLAB Snap comes from the factory with its UPDI pin pulled low with a 4.7 kΩ resistor to ground. You can try to overpower this resistor by adding a low value pull-up resistor to Vcc (1 kΩ worked for me on a breadboard), but the actual fix is to fire-up the soldering iron and remove that tiny surface-mount pull-down resistor labeled R48.

Have your glasses? R48 is here:

These photos were taken after I removed the resistor. I didn't use hot air. I just touched it a for a few seconds with a soldering iron and wiped it off then threw it away.

You don't need a microscope, but I'm glad I had one.

Step 5: Reflect

You can now program AVR microcontrollers using UPDI with your MPLAB Snap! Blink, LED, blink.

Can you believe this is the officially recommended action? According to the official Microchip Engineering Technical Note ETN #36: MPLAB Snap AVR Interface Modification

  • Symptom: Programming and debugging fails with AVR microcontroller devices that use the UPDI/PDI/TPI interfaces. MPLAB SNAP, Assembly #02-10381-R1 requires an external pull-up resistor for AVR microcontroller devices that use these interfaces.

  • Problem: AVR microcontroller devices that use the UPDI/PDI/TPI interfaces require the idle state of inactivity to be at a logic high level. Internally, the AVR devices have a weak (50-100K) pull-up resistor that attempts to keep the line high. An external and stronger pull-up resistor may be enough to mitigate this issue and bring voltages to acceptable VDD levels. In some cases, this may not be enough and the pull-down resistor that is part of the ICSP protocol can be removed for these AVR microcontroller applications.

  • Solution: If most of the applications are AVR-centric, consider removing the R48 resistor as shown below. This completely isolates any loading on the programming data line. Additionally, a pull-up resistor to VDD in the range of 1K to 10K should be used for robustness. Pin 4 of J4 is the TPGD data line used for ICSP interfaces and it also doubles as the DAT signal for UPDI/PDI and TPI interfaces. The pull-up resistor can be mounted directly from TVDD (J4-2) to TPGD/DAT (J4-4). Alternatively, the resistor can be mounted on the application side of the circuit for convenience.

I feel genuinely sorry for the Amazon sellers who are getting poor reviews because they sell this product. It really isn't their fault. I hope Google directs people here so that they can get their boards working and leave positive reviews that point more people to this issue.

UPDI Programming with a Serial Adapter

There is no official support for UPDI programming using a serial adapter, but it seems some people have figured out how to do it in some capacity. There was a promising pyupdi project, but it is now deprecated. At the time of writing the leading project aiming to enable UPDI programming without official hardware is pymcuprog, but its repository has a single commit dated two months ago and no activity since. Interestingly, that commit was made by buildmaster@microchip.com (an unverified email address), so it may not be fair to refer to it as an "unofficial" solution. The long term support of the pymcuprog project remains uncertain, but regardless let's take a closer look at how it works.

To build a programmer you just need a TTL USB serial adapter and a 1kΩ resistor. These are the steps I used to program a LED blink program using this strategy:

  • Use a generic FT232 breakout board to achieve a USB-controlled serial port on my breadboard.

  • Connect the programmer as shown with the RX pin directly to the UPDI pin of the microcontroller and the resistor between the RX and TX pins.

  • Ensure a modern version of Python is installed on your system

  • pip install pymcuprog

  • Use the device manager to identify the name of the COM port representing your programmer. In my case it's COM12.

  • I then interacted with the microcontroller by running pymcuprog from a terminal

Ping the Microcontroller

This command returns the device ID (1E9328 for my ATtiny826) indicating the UPDI connection is working successfully

pymcuprog ping -d attiny826 -t uart -u com12
Connecting to SerialUPDI
Pinging device...
Ping response: 1E9328

Write a Hex File

I used Microchip Studio to compile my C code and generate the hex file. Now I can use pymcuprog to load those hex files onto the chip. It's slower to program and inconvenient to drop to a terminal whenever I want to program a chip, but it works.

pymcuprog write -f LedBlink.hex -d attiny826 -t uart -u com12
Connecting to SerialUPDI
Pinging device...
Ping response: 1E9328
Writing from hex file...
Writing flash...

Conclusions

  • The new AVR series microcontrollers have lots of cool peripherals for the price and are available during a chip shortage that threatens availability of the more common traditional microcontrollers.

  • The Atmel-ICE is expensive, but the most convenient and reliable way to program modern AVR microcontrollers using UPDI.

  • The MPLAB Snap can program modern AVRs using UPDI after a software flash and a hardware modification, but its support for AVRs seems like an afterthought rather than its design priority.

  • You can create a makeshift unofficial UPDI programmer from a USB serial adapter, but the added complexity, lack of debugging capabilities, increased friction during the development loop, and large number of abandoned projects in this space make this an unappealing long term solution in my opinion.

Resources

Markdown source code last modified on December 25th, 2022
---
Title: Programming Modern AVR Microcontrollers
Description: Blink a LED on a modern series AVR using Atmel-ICS or MPLAB Snap UPDI programmers.
Date: 2022-12-09 11:45PM EST
Tags: circuit, microcontroller
---

# Programming Modern AVR Microcontrollers

**This page describes how to program Microchip's newest series of AVR microcontrollers using official programming gear and software.** I spent many years programming the traditional series of Atmel chips, but now several years after Microchip acquired Atmel I am interested in exploring the capabilities of the latest series of AVR microcontrollers (especially the new AVR DD family). Currently the global chip shortage makes it difficult to source traditional ATMega and STM32 chips, but the newest series of AVR microcontrollers feature an impressive set of peripherals for the price and are available from all the major vendors.

<div class="w-75 mx-auto">

![](https://www.youtube.com/embed/M-myqg-2c5s)

</div>

## TLDR

* Older AVR microcontrollers are programmed using _in-circuit serial programming_ (ICSP) through the `RESET`, `SCK`, `MISO`, and `MOSI` pins using cheap programmers like [USBtiny](https://learn.adafruit.com/usbtinyisp). However, serial programming is not supported on newer AVR microcontrollers.

* New AVR microcontrollers are programmed using the _unified program and debug interface_ (UDPI) exclusively through the `UDPI` pin. UDPI is a Microchip proprietary interface requiring a UDPI-capable programmer.

* Official UDPI programmers include [Atmel-ICE](https://www.digikey.com/en/products/detail/microchip-technology/ATATMEL-ICE-BASIC/4753381) ($129) and [MPLAB Snap](https://www.digikey.com/en/products/detail/microchip-technology/PG164100/9562532) ($35). The Atmel-ICE is expensive but it is very well supported. The MPLAB Snap is hacky, requires re-flashing, and has a physical design flaw requiring a hardware modification before it can program AVR series chips.

* There are notable attempts to create alternative programmers (e.g., [jtag2updi](https://github.com/ElTangas/jtag2updi) and [pymcuprog](https://github.com/microchip-pic-avr-tools/pymcuprog)), but this journey into the land of unofficial programmer designs is fraught with abandoned GitHub repositories and a lot of added complexity and brittleness (e.g., [SpenceKonde/AVR-Guidance](https://github.com/SpenceKonde/AVR-Guidance/blob/master/UPDI/jtag2updi.md)), so to save yourself frustration in the future I highly recommend just buying an officially supported programmer. It's also nice when you can program and debug your microcontroller from within your IDE.

* UDPI programmers have a `Vcc` pin that is used to _sense_ supply voltage (but not provide it), so you must power your board yourself while using one of these new programmers.

## ATTiny826 LED Blink

**Blinking a LED is the "Hello, World" of microcontroller programming.** Let's take a look at the code necessary to blink a LED on pin 2 of an [ATTiny286](https://ww1.microchip.com/downloads/en/DeviceDoc/ATtiny424-426-427-824-826-827-DataSheet-DS40002311A.pdf). It is compiled and programmed onto the chip using [Microchip Studio](https://www.microchip.com/en-us/tools-resources/develop/microchip-studio).

```c
#define F_CPU 3333333UL
#include <avr/io.h>
#include <util/delay.h>

int main(void)
{
	PORTA.DIR = 0xFF;
	while (1)
	{
		PORTA.OUT = 255;
		_delay_ms(500);
		PORTA.OUT = 0;
		_delay_ms(500);
	}
}
```

* `PORTA.DIR` sets the direction of pins on port A (`0xFF` means all outputs)
* `PORTA.OUT` sets output voltage on pins of port A (`0xFF` means all high)
* Using `_delay_ms()` requires including `delay.h`
* Including `delay.h` requires defining `F_CPU` (the CPU frequency)
* The [ATTiny286 datasheet section 11.3.3](https://ww1.microchip.com/downloads/en/DeviceDoc/ATtiny424-426-427-824-826-827-DataSheet-DS40002311A.pdf) indicates the default clock is 20 MHz with a /6 prescaler, so the default clock is `3333333 Hz` (3.3 MHz). This behavior can be customized using the Oscillator Configuration Fuse (FUSE.OSCCFG).

## ATTiny826 Pinout

From page 14 of the [ATTiny826 datasheet](https://ww1.microchip.com/downloads/en/DeviceDoc/ATtiny424-426-427-824-826-827-DataSheet-DS40002311A.pdf)

<img src="attiny826-pinout.png" class="img-fluid mx-auto d-block w-75" />

## SMT ATTiny Breakout Board

**Many of the newer AVR series microcontrollers are not available in breadboard-friendly DIP packages.** I find SOIC-to-DIP breakout boards (available on Amazon and eBay) to be useful for experimenting with chips in SOIC packages. Here I added extra power and PA4 (pin 2) LEDs and 10 kΩ current limiting resistors.

<a href="photos/leds2.jpg">
<img src="photos/leds2.jpg" class="img-fluid border shadow my-4" />
</a>

<a href="photos/scope1.jpg">
<img src="photos/scope1.jpg" class="img-fluid border shadow my-4" />
</a>

**I power the device from the 3.3V or 5V pins on a FT232 USB breakout board.** Although the topic is out of scope for this article, I find it convenient to use FTDI chips to exchange small amounts of data or debug messages between a microcontroller and a modern PC over USB without requiring special drivers.

<img src="ft232-breadboard.jpg" class="img-fluid mx-auto d-block w-50" />

## Why is programming modern AVRs so difficult?

**I'm surprised how little information there is online about how to _reliably_ program modern AVR series microcontrollers.** In late 2022 there is a surprisingly large amount of "advice" on this topic which leads to dead ends and broken or abandoned projects. After looking into it for a while, here is my opinionated assessment. Mouser and Digikey have links to expensive programmers, and Amazon has links to similar items but reviews are littered with comments like "arrived bricked" and "can not program AVR series chips". DIY options typically involve abandoned (or soon-to-be abandoned?) GitHub repositories, or instructions for Arduino-related programming. I seek to consolidate and distill the most useful information onto this page, and I hope others will find it useful.

## Atmel-ICE: Expensive but Effective

**After using $5 ICSP programmers for the last decade I almost fell out of my chair when I saw Microchip's recommended entry-level programmer is over $180!** Digikey sells a "basic" version without cables for $130, but that still seems crazy to me. Also, $50 for a ribbon cable?

<a href="avr-ice.webp">
<img src="avr-ice.webp" class="img-fluid w-50 d-block mx-auto my-5" />
</a>

**I found a kit on Amazon that sells the programmer with a cable for $126.** It was hard for me to press that buy button, but I figured the time I would save by having access to modern and exotic chips during the present global chip shortage would make it worth it. After a couple days of free Prime shipping, it arrived. It was smaller than I thought it would be from the product photos.

<a href="photos/atmel-ice-1.jpg">
<img src="photos/atmel-ice-1.jpg" class="img-fluid border shadow my-4" />
</a>

**The cable that came with the device seemed a bit hacky at first, but I'm happy to have it.** The female 2.54 mm socket is easy to insert breadboard jumpers into.

<a href="photos/atmel-ice-2.jpg">
<img src="photos/atmel-ice-2.jpg" class="img-fluid border shadow my-4" />
</a>

**I'm glad this thing is idiot proof.** The very first thing I did after unboxing this programmer was hook it up to my power supply rails using reverse polarity. I misread the pin diagram and confused the _socket_ with the _connector_ (they are mirror images of one another). This is an easy mistake to make though, so here's a picture of the correct orientation. Note the location of the tab on the side of the connector.

Atmel ICE Pinout | Programming Connection
---|---
<a href="atmel-ice-pinout.png"><img src="atmel-ice-pinout.png" class="img-fluid"></a>|<a href="photos/atmel-ice-3.jpg"><img src="photos/atmel-ice-3.jpg" class="img-fluid"></a>

* Black: `GND`
* Red: `Vcc` - This line is used to _sense_ power and not _deliver_ it, so you are responsible for externally powering your board.
* Blue: `UPDI` pin - Although a pull-up resistor on the UPDI pin is recommended, I did not find it was required to program my chip on the breadboard in this configuration.

**The AVR Ice was easy to use with Microchip Studio.** My programmer was detected immediately, a window popped-up and walked me through updating the firmware, and my LED was blinking in no time.

<a href="photos/atmel-ice-4.jpg">
<img src="photos/atmel-ice-4.jpg" class="img-fluid border shadow my-4" />
</a>

## MPLAB Snap: Cheap and Convoluted

**Did I really need to spend $126 for an AVR programmer? Amazon carries the MPLAB Snap for $34, but lots of reviews say it doesn't work.** After easily getting the Atmel-ICE programmer up and running I thought it would be a similarly easy experience setting-up the MPLAB Snap for AVR UPDI programming, but boy was I wrong. Now that I know the steps to get this thing working it's not so bad, but the information here was only gathered after hours of frustration. 

<a href="mplab-snap.webp">
<img src="mplab-snap.webp" class="img-fluid d-block mx-auto w-75" />
</a>

<a href="photos/mplab-snap-1.jpg">
<img src="photos/mplab-snap-1.jpg" class="img-fluid border shadow my-4" />
</a>

Here are the steps you can take to program modern AVR microcontrollers with UPDI using a MPLAB Snap:

### Step 1: Setup MPLAB

* The MPLAB Snap ships with obsolete firmware and must be re-flashed immediately upon receipt.

* Microchip Studio's firmware upgrade tool does not actually work with the MPLAB Snap. It shows the board with version 0.00 software and it hangs (with several USB disconnect/reconnect sounds) if you try to upgrade it.

* You can only re-flash the MPLAB Snap using the MPLAB X IDE. Download the 1.10 GB [MPLAB setup](https://www.microchip.com/en-us/tools-resources/develop/mplab-x-ide) executable and install the MPLAB IDE software which occupies a cool 9.83 GB.

### Step 2: Re-Flash the Programmer

* In the MPLAB IDE select `Tools` and select `Hardware Tool Emergency Boot Firmware Recovery`. At least this tool is helpful. It walks you through how to reset the device and program the latest firmware.

### Step 3: Acknowledge Your Programmer is Defective

Defective may be a strong word, but let's just say the hardware was not designed to enable programming AVR chips using UPDI. Microchip Studio will detect the programmer but if you try to program an AVR you'll get a pop-up error message that provides surprisingly little useful information.

<img src="verify.png" class="mx-auto d-block img-fluid" />

> Atmel Studio was unable to start your debug session. Please verify device selection, interface settings, target power and connections to the target device. Look in the details section for more information.
> StatusCode:	131107
> ModuleName:	TCF (TCF command: Processes:launch failed.)
> An illegal configuration parameter was used. Debugger command Activate physical failed.

### Step 4: Fix Your Programmer

**The reason MPLAB Snap cannot program AVR microcontrollers is because the UPDI pin should be pulled _high_, but the MPLAB Snap comes from the factory with its UPDI pin pulled _low_ with a 4.7 kΩ resistor to ground.** You can try to overpower this resistor by adding a low value pull-up resistor to Vcc (1 kΩ worked for me on a breadboard), but the actual fix is to fire-up the soldering iron and remove that tiny surface-mount pull-down resistor labeled `R48`.

Have your glasses? R48 is here:

<a href="photos/mplab-snap-fix.jpg">
<img src="photos/mplab-snap-fix.jpg" class="img-fluid border shadow my-4" />
</a>


**These photos were taken after I removed the resistor.** I didn't use hot air. I just touched it a for a few seconds with a soldering iron and wiped it off then threw it away.

<a href="photos/scope2.jpg">
<img src="photos/scope2.jpg" class="img-fluid border shadow my-4" />
</a>

You don't need a microscope, but I'm glad I had one.

### Step 5: Reflect

You can now program AVR microcontrollers using UPDI with your MPLAB Snap! Blink, LED, blink.

<a href="photos/mplab-snap-2.jpg">
<img src="photos/mplab-snap-2.jpg" class="img-fluid border shadow my-4" />
</a>

**Can you believe this is the officially recommended action?** According to the official Microchip Engineering Technical Note [ETN #36](http://ww1.microchip.com/downloads/en/DeviceDoc/ETN36_MPLAB%20Snap%20AVR%20Interface%20Modification.pdf): MPLAB Snap AVR Interface Modification

* **Symptom:** Programming and debugging fails with AVR microcontroller devices that use the UPDI/PDI/TPI interfaces. MPLAB SNAP, Assembly #02-10381-R1 requires an external pull-up resistor for AVR microcontroller
devices that use these interfaces.

* **Problem:** AVR microcontroller devices that use the UPDI/PDI/TPI interfaces require the idle state of inactivity to be at a logic high level. Internally, the AVR devices have a weak (50-100K) pull-up resistor that attempts to keep the line high. An external and stronger pull-up resistor may be enough to mitigate this issue and bring voltages to acceptable VDD levels. In some cases, this may not be enough and the pull-down resistor that is part of the ICSP protocol can be removed for these AVR microcontroller applications.

* **Solution:** If most of the applications are AVR-centric, consider removing the R48 resistor as shown below. This completely isolates any loading on the programming data line. Additionally, a pull-up resistor to VDD in the range of 1K to 10K should be used for robustness. Pin 4 of J4 is the TPGD data line used for ICSP interfaces and it also doubles as the DAT signal for UPDI/PDI and TPI interfaces. The pull-up resistor can be mounted directly from TVDD (J4-2) to TPGD/DAT (J4-4). Alternatively, the resistor can be mounted on the application side of the circuit
for convenience.

**I feel genuinely sorry for the Amazon sellers who are getting poor reviews because they sell this product.** It really isn't their fault. I hope Google directs people here so that they can get their boards working and leave positive reviews that point more people to this issue.

## UPDI Programming with a Serial Adapter

There is no official support for UPDI programming using a serial adapter, but it seems some people have figured out how to do it in some capacity. There was a promising [pyupdi](https://github.com/mraardvark/pyupdi) project, but it is now deprecated. At the time of writing the leading project aiming to enable UPDI programming without official hardware is [pymcuprog](https://github.com/microchip-pic-avr-tools/pymcuprog), but its repository has a single commit dated two months ago and no activity since. Interestingly, [that commit](https://github.com/microchip-pic-avr-tools/pymcuprog/commit/593afdc8e089e39a4fed9f4fb19ae81f5f51e9a5.patch) was made by buildmaster@microchip.com (an unverified email address), so it may not be fair to refer to it as an "unofficial" solution. The long term support of the pymcuprog project remains uncertain, but regardless let's take a closer look at how it works.

![](updi-ftdi-serial-programmer.png)

To build a programmer you just need a TTL USB serial adapter and a 1kΩ resistor. These are the steps I used to program a LED blink program using this strategy:

* Use a generic [FT232 breakout board](https://www.amazon.com/s?k=ft232+breakout) to achieve a USB-controlled serial port on my breadboard.

* Connect the programmer as shown with the RX pin directly to the UPDI pin of the microcontroller and the resistor between the RX and TX pins.

* Ensure a [modern version of Python](https://www.python.org/) is installed on your system

* `pip install pymcuprog`

* Use the device manager to identify the name of the COM port representing your programmer. In my case it's `COM12`.

* I then interacted with the microcontroller by running `pymcuprog` from a terminal

### Ping the Microcontroller

This command returns the device ID (1E9328 for my ATtiny826) indicating the UPDI connection is working successfully

```bash
pymcuprog ping -d attiny826 -t uart -u com12
```

```
Connecting to SerialUPDI
Pinging device...
Ping response: 1E9328
```

### Write a Hex File

I used Microchip Studio to compile my C code and generate the hex file. Now I can use `pymcuprog` to load those hex files onto the chip. It's slower to program and inconvenient to drop to a terminal whenever I want to program a chip, but it works.

```
pymcuprog write -f LedBlink.hex -d attiny826 -t uart -u com12
```

```
Connecting to SerialUPDI
Pinging device...
Ping response: 1E9328
Writing from hex file...
Writing flash...
```

## Conclusions

* The new AVR series microcontrollers have lots of cool peripherals for the price and are available during a chip shortage that threatens availability of the more common traditional microcontrollers.

* The Atmel-ICE is expensive, but the most convenient and reliable way to program modern AVR microcontrollers using UPDI.

* The MPLAB Snap can program modern AVRs using UPDI after a software flash and a hardware modification, but its support for AVRs seems like an afterthought rather than its design priority.

* You can create a makeshift unofficial UPDI programmer from a USB serial adapter, but the added complexity, lack of debugging capabilities, increased friction during the development loop, and large number of abandoned projects in this space make this an unappealing long term solution in my opinion.

## Resources

* Atmel-ICE
  * [Atmel-ICE on Mouser](https://www.mouser.com/ProductDetail/Microchip-Technology-Atmel/ATATMEL-ICE?qs=sGAEpiMZZMuRZxwUfDU0miN4udwF8GpUanrVt%252BDSn9Q4SZQ5wSGB4Q%3D%3D) (currently $180.64)
  * [Atmel-ICE on DigiKey](https://www.digikey.com/en/products/detail/microchip-technology/ATATMEL-ICE/4753379) (currently $180.62)
  * [Atmel-ICE on Amazon](https://www.amazon.com/s?k=atmel+ice) (currently $126.49)
  * [Atmel-ICE on eBay](https://www.ebay.com/sch/i.html?_nkw=atmel-ice) (currently $135.00)
  * [Atmel-ICE datasheet](https://ww1.microchip.com/downloads/en/DeviceDoc/Atmel-ICE_UserGuide.pdf)
* MPLAB Snap
  * [MPLAB Snap on Mouser](https://www.mouser.com/ProductDetail/Microchip-Technology-Atmel/PG164100?qs=w%2Fv1CP2dgqoaLDDBjfzhMQ%3D%3D) (currently $34.77)
  * [MPLAB Snap on DigiKey](https://www.digikey.com/en/products/detail/microchip-technology/PG164100/9562532) (currently $34.76)
  * [MPLAB Snap on Amazon](https://www.amazon.com/s?k=mplab+snap) (currently $34.09)
  * [MPLAB Snap datasheet](https://ww1.microchip.com/downloads/en/DeviceDoc/50002787C.pdf)
  * [MPLAB Snap AVR UPDI modification](http://ww1.microchip.com/downloads/en/DeviceDoc/ETN36_MPLAB%20Snap%20AVR%20Interface%20Modification.pdf)
* [ATTiny826 datasheet](https://ww1.microchip.com/downloads/en/DeviceDoc/ATtiny424-426-427-824-826-827-DataSheet-DS40002311A.pdf)
* [UPDI Physical Interface](https://onlinedocs.microchip.com/pr/GUID-DDB0017E-84E3-4E77-AAE9-7AC4290E5E8B-en-US-4/index.html)
* [Contact me](https://swharden.com/about/) if you have suggestions or updated information
November 15th, 2022

Interfacing HX710 Differential ADC with Arduino

This page demonstrates how to read differential voltage from a HX710 ADC using Arduino. I recently obtained some pressure sensor boards from Amazon for less than $3 each under names like 6pcs 3.3-5V Digital Barometric Air Pressure Sensor Module Liquid Water Level Controller Board 0-40KPa that use this ADC. Several years ago I worked on a precision pressure meter project based on an I2C temperature and pressure sensor (MS5611), and now that I see new inexpensive SPI pressure sensor modules on the consumer market I'm interested to learn more about their capabilities.

Analog-to-Digital Converter IC

The ADC chip is easily identified as a HX710B 24-Bit Analog-to-Digital Converter (ADC) with Built-in Temperature Sensor. According to the datasheet it can be powered by a 3.3V or 5V supply, and the value it reports is the differential voltage between two input pins.

The datasheet indicates this device can be run from a 3.3V or 5V supply, it uses a built-in fixed-gain (128x) differential amplifier, and it can read up to 40 samples per second. The datasheet provides an example circuit demonstrating how this ADC can be used to measure weight from a scale sensor:

Pressure Sensor

To get a better idea of how this sensor works it would be helpful to locate its product number. I had a hunch it was beneath the part so I desoldered it, and indeed I found part identification information.

The pressure sensor is labeled as a PSG010S but unfortunately I struggled to find a quality datasheet for it. I did find some now-deleted images from an AliExpress listing showing the differences between the base model and the R and S variants. I found this PSG010R datasheet (curiously written in Comic Sans) indicating that maximum voltage is 5V and that the gauge pressure is 0 - 40KPa (0 - 5.8 PSI). This seems to be a fairly standard differential pressure sensor design using a pair of voltage dividers where the pressure is a function of the difference in voltage at the two mid-points (a Wheatstone bridge).

Update (2022-12-23): I received an email from somebody offering additional information about this component:

The PSG010 reports positive and negative pressures and can easily have its range shifted to almost double in one direction with almost none in the other. All that is needed is to lift the +V (2) or ground pin (5) and insert a surface mount 75R ±15R under it. Lifting the ground side by 75R makes it double positive, while pushing the applied +V down makes it double negative (vacuum).
-- bruceg

Read HX710B with Arduino

This code demonstrates how to measure HX710B values using Arduino and display the readings in the serial terminal sufficient to graph in real time using the serial plotter. The animated plot is what it looks like when I blow puffs of air on the sensor.

const int HX_OUT_PIN = 2;
const int HX_SCK_PIN = 3;

enum HX_MODE { NONE, DIFF_10Hz, TEMP_40Hz, DIFF_40Hz};
const byte HX_MODE = DIFF_40Hz;

void setup() {
  pinMode(HX_SCK_PIN, OUTPUT);
  pinMode(HX_OUT_PIN, INPUT);
  Serial.begin(9600);
}

void loop() {
  Serial.println(readHX());
}

unsigned long readHX() {

  // pulse clock line to start a reading
  for (char i = 0; i < HX_MODE; i++) {
    digitalWrite(HX_SCK_PIN, HIGH);
    digitalWrite(HX_SCK_PIN, LOW);
  }

  // wait for the reading to finish
  while (digitalRead(HX_OUT_PIN)) {}

  // read the 24-bit pressure as 3 bytes using SPI
  byte data[3];
  for (byte j = 3; j--;) {
    data[j] = shiftIn(HX_OUT_PIN, HX_SCK_PIN, MSBFIRST);
  }

  data[2] ^= 0x80;  // see note

  // shift the 3 bytes into a large integer
  long result;
  result += (long)data[2] << 16;
  result += (long)data[1] << 8;
  result += (long)data[0];

  return result;
}

Note: This code flips the most significant bit of the sensor reading. The sensor always returns this bit as 1, except for the case of an out-of-range error (see excerpt from datasheet below). By simply flipping the bit our reported values are a continuous range from 0 to 2^14-1, with the edge values representing out-of-range errors.

The output 24 bits of data is in 2’s complement format. When input differential signal goes out of the 24 bit range, the output data will be saturated at 0x800000 (MIN) or 0x7FFFFF (MAX) until the input signal comes back to the input range.
-- HX710 datasheet

Update (2022-12-23): I received an email from someone offering feedback about this code:

This code works in a loop, but perhaps by accident. The strongly worded statements in the HX710 datasheet about 25 - 27 clocks per readout imply that it is risky to rely on this. It may be that hanging clocks induce unwanted sleep modes or over-run into the next read cycle, etc. There is simply no real explanation in what is shown, so best to be safe - always set the next mode immediately AFTER collecting a reading and then always poll for new data ready before attempting a collection. Your 'pulse clock line to start a reading' loop before a reading should be 'add next mode' after a reading to comply with the timing specification. This will ensure that the next conversion will be available rather than the next scheduled conversion AFTER the mode is eventually sent.
-- bruceg

Open-Source HX710B Libraries

Although some libraries are available which facilitate interacting with the HX710, here I engage with it discretely to convey each step of the conversion and measurement process. I found that many libraries use the 10 Hz mode by default, whereas I certainly prefer the 40 Hz mode. More frustratingly, code in many libraries refer to this as gain, which is incorrect. The datasheet indicates gain is fixed at 128 and cannot be changed in software.

Update (2022-12-23): I received an email explaining why people often use "gain" and "mode" when referring to the HX710:

The HX711 is similar to the HX710 but it has user selectable gain AND user selectable sample rates BUT only certain combinations are allowed, so setting mode WILL also select its matched gain value. The HX710 uses most of the same internals, but with just 3 modes - reading the Wheatstone Bridge always using 128 gain at 10 or 40Hz while swapping to Avolt (HX710A) or internal Temperature (HX710B) uses a lower gain and less digits. So for people familiar with the HX711 there is no ambiguity in mixing mode and gain.
-- bruceg

Resources

Markdown source code last modified on December 24th, 2022
---
Title: Interfacing HX710 Differential ADC with Arduino
Description: How to read differential voltage from a HX710 ADC using Arduino
Date: 2022-11-15 12:30AM EST
Tags: circuit, microcontroller
---

# Interfacing HX710 Differential ADC with Arduino

**This page demonstrates how to read differential voltage from a HX710 ADC using Arduino.** I recently obtained some pressure sensor boards from Amazon for less than $3 each under names like _6pcs 3.3-5V Digital Barometric Air Pressure Sensor Module Liquid Water Level Controller Board 0-40KPa_ that use this ADC. Several years ago I worked on a [precision pressure meter project](https://swharden.com/blog/2017-04-29-precision-pressure-meter-project/) based on an [I2C](https://en.wikipedia.org/wiki/I%C2%B2C)  temperature and pressure sensor ([MS5611](https://www.te.com/commerce/DocumentDelivery/DDEController?Action=showdoc&DocId=Data+Sheet%7FMS5611-01BA03%7FB3%7Fpdf%7FEnglish%7FENG_DS_MS5611-01BA03_B3.pdf%7FCAT-BLPS0036)), and now that I see new inexpensive [SPI](https://en.wikipedia.org/wiki/Serial_Peripheral_Interface) pressure sensor modules on the consumer market I'm interested to learn more about their capabilities.

<img src="hx710b-pressure-board.jpg" class="my-5 border border-dark shadow img-fluid w-75 mx-auto d-block">

## Analog-to-Digital Converter IC

**The ADC chip is easily identified as a [HX710B](https://www.electronicscomp.com/datasheet/hx710b-ic-datasheet.pdf) _24-Bit Analog-to-Digital Converter (ADC) with Built-in Temperature Sensor_.** According to the datasheet it can be powered by a 3.3V or 5V supply, and the value it reports is the differential voltage between two input pins. 

<img src="hx710b-pinout.jpg" class="my-5 img-fluid w-75 mx-auto d-block">

The datasheet indicates this device can be run from a 3.3V or 5V supply, it uses a built-in fixed-gain (128x) differential amplifier, and it can read up to 40 samples per second. The datasheet provides an example circuit demonstrating how this ADC can be used to measure weight from a scale sensor:

<img src="hx710-datasheet.jpg" class="my-5 img-fluid w-75 mx-auto d-block">

## Pressure Sensor

To get a better idea of how this sensor works it would be helpful to locate its product number. I had a hunch it was beneath the part so I desoldered it, and indeed I found part identification information.

<img src="hx710b-pressure-psg010s.jpg" class="my-5 border border-dark shadow img-fluid w-75 mx-auto d-block">

**The pressure sensor is labeled as a PSG010S** but unfortunately I struggled to find a quality datasheet for it. I did find some now-deleted images from an AliExpress listing showing the differences between the base model and the R and S variants. 
I found [this PSG010R datasheet](https://www.katranji.com/tocimages/files/536845-544144.pdf) (curiously written in Comic Sans) indicating that maximum voltage is 5V and that the gauge pressure is 0 - 40KPa (0 - 5.8 PSI). This seems to be a fairly standard [differential pressure sensor](https://www.avnet.com/wps/portal/abacus/solutions/technologies/sensors/pressure-sensors/measurement-types/differential/) design using a pair of voltage dividers where the pressure is a function of the difference in voltage at the two mid-points (a [Wheatstone bridge](https://en.wikipedia.org/wiki/Wheatstone_bridge)).

**Update (2022-12-23):** I received an email from somebody offering additional information about this component:

> The PSG010 reports positive and negative pressures and can easily have its range shifted to almost double in one direction with almost none in the other.  All that is needed is to lift the +V (2) or ground pin (5) and insert a surface mount 75R ±15R under it. 
Lifting the ground side by 75R makes it double positive, while pushing the applied +V down makes it double negative (vacuum).<br>
> -- <cite class="text-end">bruceg</cite>


<img src="psg-pressure-sensor.jpg" class="my-5 border border-dark shadow img-fluid w-75 mx-auto d-block">

## Read HX710B with Arduino

**This code demonstrates how to measure HX710B values using Arduino** and display the readings in the serial terminal sufficient to graph in real time using the serial plotter. The animated plot is what it looks like when I blow puffs of air on the sensor.

<img src="hx710-arduino-plot.gif" class="my-5 img-fluid mx-auto d-block">

```c
const int HX_OUT_PIN = 2;
const int HX_SCK_PIN = 3;

enum HX_MODE { NONE, DIFF_10Hz, TEMP_40Hz, DIFF_40Hz};
const byte HX_MODE = DIFF_40Hz;

void setup() {
  pinMode(HX_SCK_PIN, OUTPUT);
  pinMode(HX_OUT_PIN, INPUT);
  Serial.begin(9600);
}

void loop() {
  Serial.println(readHX());
}

unsigned long readHX() {

  // pulse clock line to start a reading
  for (char i = 0; i < HX_MODE; i++) {
    digitalWrite(HX_SCK_PIN, HIGH);
    digitalWrite(HX_SCK_PIN, LOW);
  }

  // wait for the reading to finish
  while (digitalRead(HX_OUT_PIN)) {}

  // read the 24-bit pressure as 3 bytes using SPI
  byte data[3];
  for (byte j = 3; j--;) {
    data[j] = shiftIn(HX_OUT_PIN, HX_SCK_PIN, MSBFIRST);
  }
  
  data[2] ^= 0x80;  // see note

  // shift the 3 bytes into a large integer
  long result;
  result += (long)data[2] << 16;
  result += (long)data[1] << 8;
  result += (long)data[0];

  return result;
}
```

**Note: This code flips the most significant bit of the sensor reading.** The sensor always returns this bit as `1`, except for the case of an out-of-range error (see excerpt from datasheet below). By simply flipping the bit our reported values are a continuous range from `0` to `2^14-1`, with the edge values representing out-of-range errors.

> The output 24 bits of data is in [2’s complement format](https://en.wikipedia.org/wiki/Two%27s_complement).
> When input differential signal goes out of the 24 bit range, the output data will be saturated at `0x800000` (MIN) or `0x7FFFFF` (MAX)
> until the input signal comes back to the input range.<br>
> -- <cite class="text-end"><a href='https://www.electronicscomp.com/datasheet/hx710b-ic-datasheet.pdf'>HX710 datasheet</a></cite>

**Update (2022-12-23):** I received an email from someone offering feedback about this code:

> This code works in a loop, but perhaps by accident. The strongly worded statements in the [HX710 datasheet](https://www.electronicscomp.com/datasheet/hx710b-ic-datasheet.pdf) about 25 - 27 clocks per readout imply that it is risky to rely on this.  It may be that hanging clocks induce unwanted sleep modes or over-run into the next read cycle, etc.  There is simply no real explanation in what is shown, so best to be safe - always set the next mode immediately AFTER collecting a reading and then always poll for new data ready before attempting a collection. Your 'pulse clock line to start a reading' loop before a reading should be 'add next mode' after a reading to comply with the timing specification.  This will ensure that the next conversion will be available rather than the next scheduled conversion AFTER the mode is eventually sent.<br>
> -- <cite class="text-end">bruceg</cite>

## Open-Source HX710B Libraries

Although some libraries are available which facilitate interacting with the HX710, here I engage with it discretely to convey each step of the conversion and measurement process. I found that many libraries use the 10 Hz mode by default, whereas I certainly prefer the 40 Hz mode. More frustratingly, code in many libraries refer to this as _gain_, which is incorrect. The datasheet indicates gain is fixed at 128 and cannot be changed in software.

**Update (2022-12-23):** I received an email explaining why people often use "gain" and "mode" when referring to the HX710:

> The [HX711](https://www.digikey.com/htmldatasheets/production/1836471/0/0/1/HX711.pdf) is similar to the [HX710](https://www.electronicscomp.com/datasheet/hx710b-ic-datasheet.pdf) but it has user selectable gain AND user selectable sample rates BUT only certain combinations are allowed, so setting mode WILL also select its matched gain value.
The HX710 uses most of the same internals, but with just 3 modes - reading the Wheatstone Bridge always using 128 gain at 10 or 40Hz while swapping to Avolt (HX710A) or internal Temperature (HX710B) uses a lower gain and less digits. So for people familiar with the HX711 there is no ambiguity in mixing mode and gain.<br>
> -- <cite class="text-end">bruceg</cite>

## Resources

* [Pressure Sensor Guide](https://www.electroschematics.com/pressure-sensor-guide/) by T.K. HAREENDRAN - A similar write-up that goes into additional detail. They didn't de-solder the pressure sensor to identify the component name, but there's lots of good information on this page.

* [bogde/HX711 on GitHub](https://github.com/bogde/HX711) - An Arduino library to interface the Avia Semiconductor HX711 24-Bit Analog-to-Digital Converter (ADC) for reading load cells / weight scales. Code on this page does not use this library, but others may find it helpful.

* [HX710 datasheet (English)](https://www.electronicscomp.com/datasheet/hx710b-ic-datasheet.pdf)

* [Differential pressure sensors](https://www.avnet.com/wps/portal/abacus/solutions/technologies/sensors/pressure-sensors/measurement-types/differential/) - Article about the topic which includes a good example of an instrumentation amplifier.

* [Design tips for a resistive-bridge pressure sensor in industrial process-control systems](https://www.ti.com/lit/an/slyt640/slyt640.pdf) - Texas Instruments application note

* [The Wheatstone Bridge](https://meritsensor.com/the-wheatstone-bridge/) by Michael Daily
November 7th, 2022

Creating Bitmaps from Scratch in C#

This project how to represent bitmap data in a plain old C object (POCO) to create images from scratch using C# and no dependencies. Common graphics libraries like SkiaSharp, ImageSharp, System.Drawing, and Maui.Graphics can read and write bitmaps in memory, so a POCO that stores image data and converts it to a bitmap byte allows creation of platform-agnostic APIs that can be interfaced from any graphics library.

This page demonstrates how to use C# (.NET 6.0) to create bitmap images from scratch. Bitmap images can then be saved to disk and viewed with any image editing program, or they can consumed as a byte array in memory by a graphics library. There are various bitmap image formats (grayscale, indexed colors, 16-bit, 32-bit, transparent, etc.) but code here demonstrates the simplest common case (8-bit RGB color).

Representing Color

The following struct represents RGB color as 3 byte values and has helper methods for creating new colors.

public struct RawColor
{
    public readonly byte R, G, B;

    public RawColor(byte r, byte g, byte b)
    {
        (R, G, B) = (r, g, b);
    }

    public static RawColor Random(Random rand)
    {
        byte r = (byte)rand.Next(256);
        byte g = (byte)rand.Next(256);
        byte b = (byte)rand.Next(256);
        return new RawColor(r, g, b);
    }

    public static RawColor Gray(byte value)
    {
        return new RawColor(value, value, value);
    }
}

A color class like this could be extended to support additional niceties. Refer to SkiaSharp's SKColor.cs, System.Drawing's Color.cs, and Maui.Graphics' Color.cs for examples and implementation details. I commonly find the following features useful include when writing a color class:

  • A static class with named colors e.g., RawColors.Blue
  • Conversion to/from ARGB e.g., RawColor.FromAGRB(123456)
  • Conversion to/from HTML e.g., RawColor.FromHtml(#003366)
  • Conversion between RGB and HSL/HSV
  • Helper functions to Lighten() and Darken()
  • Helper functions to ShiftHue()
  • Extension methods to convert to common other formats like SKColor

Representing the Bitmap Image

This is the entire image class and it serves a few specific roles:

  • Store image data in a byte array arranged identically to how it will be exported in the bitmap
  • Provide helper methods to get/set pixel color
  • Provide a method to return the image as a bitmap by adding a minimal header
public class RawBitmap
{
    public readonly int Width;
    public readonly int Height;
    private readonly byte[] ImageBytes;

    public RawBitmap(int width, int height)
    {
        Width = width;
        Height = height;
        ImageBytes = new byte[width * height * 4];
    }

    public void SetPixel(int x, int y, RawColor color)
    {
        int offset = ((Height - y - 1) * Width + x) * 4;
        ImageBytes[offset + 0] = color.B;
        ImageBytes[offset + 1] = color.G;
        ImageBytes[offset + 2] = color.R;
    }

    public byte[] GetBitmapBytes()
    {
        const int imageHeaderSize = 54;
        byte[] bmpBytes = new byte[ImageBytes.Length + imageHeaderSize];
        bmpBytes[0] = (byte)'B';
        bmpBytes[1] = (byte)'M';
        bmpBytes[14] = 40;
        Array.Copy(BitConverter.GetBytes(bmpBytes.Length), 0, bmpBytes, 2, 4);
        Array.Copy(BitConverter.GetBytes(imageHeaderSize), 0, bmpBytes, 10, 4);
        Array.Copy(BitConverter.GetBytes(Width), 0, bmpBytes, 18, 4);
        Array.Copy(BitConverter.GetBytes(Height), 0, bmpBytes, 22, 4);
        Array.Copy(BitConverter.GetBytes(32), 0, bmpBytes, 28, 2);
        Array.Copy(BitConverter.GetBytes(ImageBytes.Length), 0, bmpBytes, 34, 4);
        Array.Copy(ImageBytes, 0, bmpBytes, imageHeaderSize, ImageBytes.Length);
        return bmpBytes;
    }

    public void Save(string filename)
    {
        byte[] bytes = GetBitmapBytes();
        File.WriteAllBytes(filename, bytes);
    }
}

Generate Images from Scratch

The following code uses the bitmap class and color struct above to create test images

Random Colors

RawBitmap bmp = new(400, 300);
Random rand = new();
for (int x = 0; x < bmp.Width; x++)
    for (int y = 0; y < bmp.Height; y++)
        bmp.SetPixel(x, y, RawColor.Random(rand));
bmp.Save("random-rgb.bmp");

Rainbow

RawBitmap bmp = new(400, 300);
Random rand = new();
for (int x = 0; x < bmp.Width; x++)
{
    for (int y = 0; y < bmp.Height; y++)
    {
        byte r = (byte)(255.0 * x / bmp.Width);
        byte g = (byte)(255.0 * y / bmp.Height);
        byte b = (byte)(255 - 255.0 * x / bmp.Width);
        RawColor color = new(r, g, b);
        bmp.SetPixel(x, y, color);
    }
}
bmp.Save("rainbow.bmp");

Rectangles

RawBitmap bmp = new(400, 300);
Random rand = new();
for (int i = 0; i < 1000; i++)
{
    int rectX = rand.Next(bmp.Width);
    int rectY = rand.Next(bmp.Height);
    int rectWidth = rand.Next(50);
    int rectHeight = rand.Next(50);
    RawColor color = RawColor.Random(rand);

    for (int x = rectX; x < rectX + rectWidth; x++)
    {
        for (int y = rectY; y < rectY + rectHeight; y++)
        {
            if (x < 0 || x >= bmp.Width) continue;
            if (y < 0 || y >= bmp.Height) continue;
            bmp.SetPixel(x, y, color);
        }
    }
}
bmp.Save("rectangles.bmp");

Interfacing Graphics Libraries

The following code demonstrates how to load the bitmap byte arrays generated above into common graphics libraries and save the result as a JPEG file. Although the bitmap byte array can be written directly to disk as a .bmp file, these third-party libraries are required to encode images in additional formats like JPEG.

System.Drawing

using System.Drawing;

static void SaveBitmap(byte[] bytes, string filename = "demo.jpg")
{
    using MemoryStream ms = new(bytes);
    using Image img = Bitmap.FromStream(ms);
    img.Save(filename);
}

SkiaSharp

using SkiaSharp;

static void SaveBitmap(byte[] bytes, string filename = "demo.jpg")
{
    using SKBitmap bmp = SKBitmap.Decode(bytes);
    using SKFileWStream fs = new(filename);
    bmp.Encode(fs, SKEncodedImageFormat.Jpeg, quality: 95);
}

ImageSharp

using SixLabors.ImageSharp;
using SixLabors.ImageSharp.Formats.Jpeg;

static void SaveBitmap(byte[] bytes, string filename = "demo.jpg")
{
    using Image img = Image.Load(bytes);
    JpegEncoder encoder = new() { Quality = 95 };
    img.Save(filename, encoder);
}

Resources

Markdown source code last modified on November 8th, 2022
---
Title: Creating Bitmaps from Scratch in C#
Description: How to create a bitmap and set pixel colors in memory and save the result to disk or convert it to a traditional image format
Date: 2022-11-07 18:45PM EST
Tags: csharp, graphics
---

# Creating Bitmaps from Scratch in C# 

**This project how to represent bitmap data in a plain old C object (POCO) to create images from scratch using C# and no dependencies.** 
Common graphics libraries like [SkiaSharp](https://swharden.com/csdv/skiasharp/), [ImageSharp](https://swharden.com/csdv/platforms/imagesharp/), [System.Drawing](https://swharden.com/csdv/system.drawing/), and [Maui.Graphics](https://swharden.com/csdv/maui.graphics/) can read and write bitmaps in memory, so a [POCO](https://en.wikipedia.org/wiki/POCO) that stores image data and converts it to a bitmap byte allows creation of platform-agnostic APIs that can be interfaced from any graphics library.

**This page demonstrates how to use C# (.NET 6.0) to create bitmap images from scratch.** Bitmap images can then be saved to disk and viewed with any image editing program, or they can consumed as a byte array in memory by a graphics library. There are various [bitmap image formats](https://learn.microsoft.com/en-us/dotnet/desktop/winforms/advanced/types-of-bitmaps?view=netframeworkdesktop-4.8) (grayscale, indexed colors, 16-bit, 32-bit, transparent, etc.) but code here demonstrates the simplest common case (8-bit RGB color).

## Representing Color

The following `struct` represents RGB color as 3 `byte` values and has helper methods for creating new colors. 

```cs
public struct RawColor
{
    public readonly byte R, G, B;

    public RawColor(byte r, byte g, byte b)
    {
        (R, G, B) = (r, g, b);
    }

    public static RawColor Random(Random rand)
    {
        byte r = (byte)rand.Next(256);
        byte g = (byte)rand.Next(256);
        byte b = (byte)rand.Next(256);
        return new RawColor(r, g, b);
    }

    public static RawColor Gray(byte value)
    {
        return new RawColor(value, value, value);
    }
}
```

A color class like this could be extended to support additional niceties. Refer to [SkiaSharp's `SKColor.cs`](https://github.com/mono/SkiaSharp/blob/main/binding/Binding/SKColor.cs), [System.Drawing's `Color.cs`](https://github.com/dotnet/runtime/blob/main/src/libraries/System.Drawing.Primitives/src/System/Drawing/Color.cs), and [Maui.Graphics' `Color.cs`](https://github.com/dotnet/maui/blob/main/src/Graphics/src/Graphics/Color.cs) for examples and implementation details. I commonly find the following features useful include when writing a color class:

* A static class with named colors e.g., `RawColors.Blue`
* Conversion to/from ARGB e.g., `RawColor.FromAGRB(123456)`
* Conversion to/from HTML e.g., `RawColor.FromHtml(#003366)`
* Conversion between RGB and [HSL/HSV](https://en.wikipedia.org/wiki/HSL_and_HSV)
* Helper functions to `Lighten()` and `Darken()`
* Helper functions to `ShiftHue()`
* Extension methods to convert to common other formats like `SKColor`

## Representing the Bitmap Image

This is the entire image class and it serves a few specific roles:

* Store image data in a byte array arranged identically to how it will be exported in the bitmap
* Provide helper methods to get/set pixel color
* Provide a method to return the image as a bitmap by adding a minimal header

```cs
public class RawBitmap
{
    public readonly int Width;
    public readonly int Height;
    private readonly byte[] ImageBytes;

    public RawBitmap(int width, int height)
    {
        Width = width;
        Height = height;
        ImageBytes = new byte[width * height * 4];
    }

    public void SetPixel(int x, int y, RawColor color)
    {
        int offset = ((Height - y - 1) * Width + x) * 4;
        ImageBytes[offset + 0] = color.B;
        ImageBytes[offset + 1] = color.G;
        ImageBytes[offset + 2] = color.R;
    }

    public byte[] GetBitmapBytes()
    {
        const int imageHeaderSize = 54;
        byte[] bmpBytes = new byte[ImageBytes.Length + imageHeaderSize];
        bmpBytes[0] = (byte)'B';
        bmpBytes[1] = (byte)'M';
        bmpBytes[14] = 40;
        Array.Copy(BitConverter.GetBytes(bmpBytes.Length), 0, bmpBytes, 2, 4);
        Array.Copy(BitConverter.GetBytes(imageHeaderSize), 0, bmpBytes, 10, 4);
        Array.Copy(BitConverter.GetBytes(Width), 0, bmpBytes, 18, 4);
        Array.Copy(BitConverter.GetBytes(Height), 0, bmpBytes, 22, 4);
        Array.Copy(BitConverter.GetBytes(32), 0, bmpBytes, 28, 2);
        Array.Copy(BitConverter.GetBytes(ImageBytes.Length), 0, bmpBytes, 34, 4);
        Array.Copy(ImageBytes, 0, bmpBytes, imageHeaderSize, ImageBytes.Length);
        return bmpBytes;
    }

    public void Save(string filename)
    {
        byte[] bytes = GetBitmapBytes();
        File.WriteAllBytes(filename, bytes);
    }
}
```

## Generate Images from Scratch

The following code uses the bitmap class and color struct above to create test images

### Random Colors

```cs
RawBitmap bmp = new(400, 300);
Random rand = new();
for (int x = 0; x < bmp.Width; x++)
    for (int y = 0; y < bmp.Height; y++)
        bmp.SetPixel(x, y, RawColor.Random(rand));
bmp.Save("random-rgb.bmp");
```

![](SkiaSharp-random-rgb.jpg)

### Rainbow
```cs
RawBitmap bmp = new(400, 300);
Random rand = new();
for (int x = 0; x < bmp.Width; x++)
{
    for (int y = 0; y < bmp.Height; y++)
    {
        byte r = (byte)(255.0 * x / bmp.Width);
        byte g = (byte)(255.0 * y / bmp.Height);
        byte b = (byte)(255 - 255.0 * x / bmp.Width);
        RawColor color = new(r, g, b);
        bmp.SetPixel(x, y, color);
    }
}
bmp.Save("rainbow.bmp");
```

![](SkiaSharp-rainbow.jpg)

### Rectangles
```cs
RawBitmap bmp = new(400, 300);
Random rand = new();
for (int i = 0; i < 1000; i++)
{
    int rectX = rand.Next(bmp.Width);
    int rectY = rand.Next(bmp.Height);
    int rectWidth = rand.Next(50);
    int rectHeight = rand.Next(50);
    RawColor color = RawColor.Random(rand);

    for (int x = rectX; x < rectX + rectWidth; x++)
    {
        for (int y = rectY; y < rectY + rectHeight; y++)
        {
            if (x < 0 || x >= bmp.Width) continue;
            if (y < 0 || y >= bmp.Height) continue;
            bmp.SetPixel(x, y, color);
        }
    }
}
bmp.Save("rectangles.bmp");
```

![](SkiaSharp-rectangles.jpg)

## Interfacing Graphics Libraries

**The following code demonstrates how to load the bitmap byte arrays generated above into common graphics libraries and save the result as a JPEG file.** Although the bitmap byte array can be written directly to disk as a .bmp file, these third-party libraries are required to encode images in additional formats like JPEG.


### System.Drawing

```cs
using System.Drawing;

static void SaveBitmap(byte[] bytes, string filename = "demo.jpg")
{
    using MemoryStream ms = new(bytes);
    using Image img = Bitmap.FromStream(ms);
    img.Save(filename);
}
```

### SkiaSharp

```cs
using SkiaSharp;

static void SaveBitmap(byte[] bytes, string filename = "demo.jpg")
{
    using SKBitmap bmp = SKBitmap.Decode(bytes);
    using SKFileWStream fs = new(filename);
    bmp.Encode(fs, SKEncodedImageFormat.Jpeg, quality: 95);
}
```

### ImageSharp

```cs
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.Formats.Jpeg;

static void SaveBitmap(byte[] bytes, string filename = "demo.jpg")
{
    using Image img = Image.Load(bytes);
    JpegEncoder encoder = new() { Quality = 95 };
    img.Save(filename, encoder);
}
```

# Resources

* [C# Data Visualization](https://swharden.com/csdv/) - Resources for visualizing data using C# and the .NET platform

* [SkiaSharp: `SKColor.cs`](https://github.com/mono/SkiaSharp/blob/main/binding/Binding/SKColor.cs)

* [System.Drawing: `Color.cs`](https://github.com/dotnet/runtime/blob/main/src/libraries/System.Drawing.Primitives/src/System/Drawing/Color.cs)

* [Maui.Graphics: `Color.cs`](https://github.com/dotnet/maui/blob/main/src/Graphics/src/Graphics/Color.cs)
October 16th, 2022

Experiments in PSK-31 Synthesis

PSK-31 is a narrow-bandwidth digital mode which encodes text as an audio tone that varies phase at a known rate. To learn more about this digital mode and solve a challenging programming problem, I'm going to write a PSK-31 encoder and decoder from scratch using the C# programming language. All code created for this project is open-source, available from my PSK Experiments GitHub Repository, and released under the permissive MIT license. This page documents my progress and notes things I learn along the way.

Encoding Bits as Phase Shifts

  • PSK31 messages have a continuous carrier tone

  • Symbols are represented by "symbols", each 1/31.25 seconds long

  • If a symbol changes phase from its previous symbol it is a 0, otherwise it is a 1

Amplitude Modulation Silences Phase Transitions

Although a continuous phase-shifting tone of constant amplitude can successfully transmit PSK31 data, the abrupt phase transitions will cause splatter. If you transmit this you will be heard, but those trying to communicate using adjacent frequencies will be highly disappointed.

Hard phase transitions (splatter) Soft phase transitions (cleaner)

To reduce spectral artifacts that result from abruptly changing phase, phase transitions are silenced by shaping the waveform envelope as a sine wave so it is silent at the transition. This way the maximum rate of phase shifts is a sine wave with a period of half the baud rate. This is why the opening of a PSK31 message (a series of logical 0 bits) sounds like two tones: It's the carrier sine wave with an envelope shaped like a sine wave with a period of 31.25/2 Hz. These two tones separated by approximately 15 Hz are visible in the spectrogram (waterfall).

Encoding Text as Bits

Unlike ASCII (8 bits per character) and RTTY (5 bits per character), BPSK uses Varicode (1-10 bits per character) to encode text. Consecutive zeros 00 separate characters, so character codes must not contain 00 and they must start ane end with a 1 bit. Messages are flanked by a preamble (repeated 0 bits) and a postamble (repeated 1 bits).

NUL 1010101011
SOH 1011011011
STX 1011101101
ETX 1101110111
EOT 1011101011
ENQ 1101011111
ACK 1011101111
BEL 1011111101
BS  1011111111
HT  11101111
LF  11101
VT  1101101111
FF  1011011101
CR  11111
SO  1101110101
SI  1110101011
DLE 1011110111
DC1 1011110101
DC2 1110101101
DC3 1110101111
DC4 1101011011
NAK 1101101011
SYN 1101101101
ETB 1101010111
CAN 1101111011
EM  1101111101
SUB 1110110111
ESC 1101010101
FS  1101011101
GS  1110111011
RS  1011111011
US  1101111111
SP  1
!   111111111
"   101011111
#   111110101
$   111011011
%   1011010101
&   1010111011
'   101111111
(   11111011
)   11110111
*   101101111
+   111011111
,   1110101
-   110101
.   1010111
/   110101111
0   10110111
1   10111101
2   11101101
3   11111111
4   101110111
5   101011011
6   101101011
7   110101101
8   110101011
9   110110111
:   11110101
;   110111101
<   111101101
=   1010101
>   111010111
?   1010101111
@   1010111101
A   1111101
B   11101011
C   10101101
D   10110101
E   1110111
F   11011011
G   11111101
H   101010101
I   1111111
J   111111101
K   101111101
L   11010111
M   10111011
N   11011101
O   10101011
P   11010101
Q   111011101
R   10101111
S   1101111
T   1101101
U   101010111
V   110110101
X   101011101
Y   101110101
Z   101111011
[   1010101101
\   111110111
]   111101111
^   111111011
_   1010111111
.   101101101
/   1011011111
a   1011
b   1011111
c   101111
d   101101
e   11
f   111101
g   1011011
h   101011
i   1101
j   111101011
k   10111111
l   11011
m   111011
n   1111
o   111
p   111111
q   110111111
r   10101
s   10111
t   101
u   110111
v   1111011
w   1101011
x   11011111
y   1011101
z   111010101
{   1010110111
|   110111011
}   1010110101
~   1011010111
DEL 1110110101

How to Generate a PSK Waveform

Now that we've covered the major steps of PSK31 message composition and modulation, let's go through the steps ot generate a PSK31 message in code.

Step 1: Convert a Message to Varicode

Here's the gist of how I store my varicode table in code. Note that the struct has an additional Description field which is useful for decoding and debugging.

public struct VaricodeSymbol
{
    public readonly string Symbol;
    public string BitString;
    public int[] Bits;
    public readonly string Description;

    public VaricodeSymbol(string symbol, string bitString, string? description = null)
    {
        Symbol = symbol;
        BitString = bitString;
        Bits = bitString.ToCharArray().Select(x => x == '1' ? 1 : 0).ToArray();
        Description = description ?? string.Empty;
    }
}
static VaricodeSymbol[] GetAllSymbols() => new VaricodeSymbol[]
{
    new("NUL", "1010101011", "Null character"),
    new("LF", "11101", "Line feed"),
    new("CR", "11111", "Carriage return"),
    new("SP", "1", "Space"),
    new("a", "1011"),
    new("b", "1011111"),
    new("c", "101111"),
    // etc...
};

I won't show how I do the message-to-varicode lookup, but it's trivial. Here's the final function I use to generate Varicode bits from a string:

static int[] GetVaricodeBits(string message)
{
    List<int> bits = new();

    // add a preamble of repeated zeros
    for (int i=0; i<20; i++)
        bits.Add(0);

    // encode each character of a message
    foreach (char character in message)
    {
        VaricodeSymbol symbol = Lookup(character);
        bits.AddRange(symbol.Bits);
        bits.AddRange(CharacterSeparator);
    }

    // add a postamble of repeated ones
    for (int i=0; i<20; i++)
        bits.Add(1);

    return bits.ToArray();
}

Step 2: Determine Phase Shifts

Now that we have our Varicode bits, we need to generate an array to indicate phase transitions. A transition occurs every time a bit changes value form the previous bit. This code returns phase as an array of double given the bits from a Varicode message.

public static double[] GetPhaseShifts(int[] bits, double phase1 = 0, double phase2 = Math.PI)
{
    double[] phases = new double[bits.Length];
    for (int i = 0; i < bits.Length; i++)
    {
        double previousPhase = i > 0 ? phases[i - 1] : phase1;
        double oppositePhase = previousPhase == phase1 ? phase2 : phase1;
        phases[i] = bits[i] == 1 ? previousPhase : oppositePhase;
    }
    return phases;
}

Step 3: Generate the Waveform

These constants will be used to define the shape of the waveform:

public const int SampleRate = 8000;
public const double Frequency = 1000;
public const double BaudRate = 31.25;

This minimal code generates a decipherable PSK-31 message, but it does not silence the phase transitions so it produces a lot of splatter. This function must be refined to shape the waveform such that phase transitions are silenced.

public double[] GetWaveformBPSK(double[] phases)
{
    int totalSamples = (int)(phases.Length * SampleRate / BaudRate);
    double[] wave = new double[totalSamples];
    for (int i = 0; i < wave.Length; i++)
    {
        double time = (double)i / SampleRate;
        int frame = (int)(time * BaudRate);
        double phaseShift = phases[frame];
        wave[i] = Math.Cos(2 * Math.PI * Frequency * time + phaseShift);
    }
    return wave;
}

Step 4: Generate the Waveform with Amplitude Modulation

This is the same function as above, but with extra logic for amplitude-modulating the waveform in the shape of a sine wave to silence phase transitions.

public double[] GetWaveformBPSK(double[] phases)
{
    int baudSamples = (int)(SampleRate / BaudRate);
    double samplesPerBit = SampleRate / BaudRate;
    int totalSamples = (int)(phases.Length * SampleRate / BaudRate);
    double[] wave = new double[totalSamples];

    // create the amplitude envelope sized for a single bit
    double[] envelope = new double[(int)samplesPerBit];
    for (int i = 0; i < envelope.Length; i++)
        envelope[i] = Math.Sin((i + .5) * Math.PI / envelope.Length);

    for (int i = 0; i < wave.Length; i++)
    {
        // phase modulated carrier
        double time = (double)i / SampleRate;
        int frame = (int)(time * BaudRate);
        double phaseShift = phases[frame];
        wave[i] = Math.Cos(2 * Math.PI * Frequency * time + phaseShift);

        // envelope at phase transitions
        int firstSample = (int)(frame * SampleRate / BaudRate);
        int distanceFromFrameStart = i - firstSample;
        int distanceFromFrameEnd = baudSamples - distanceFromFrameStart + 1;
        bool isFirstHalfOfFrame = distanceFromFrameStart < distanceFromFrameEnd;
        bool samePhaseAsLast = frame == 0 ? false : phases[frame - 1] == phases[frame];
        bool samePhaseAsNext = frame == phases.Length - 1 ? false : phases[frame + 1] == phases[frame];
        bool rampUp = isFirstHalfOfFrame && !samePhaseAsLast;
        bool rampDown = !isFirstHalfOfFrame && !samePhaseAsNext;

        if (rampUp)
            wave[i] *= envelope[distanceFromFrameStart];

        if (rampDown)
            wave[i] *= envelope[distanceFromFrameEnd];
    }

    return wave;
}

PSK31 Encoder Program

I wrapped the functionality above in a Windows Forms GUI that allows the user to type a message, specify frequency, baud rate, and whether or not to refine the envelope to reduce splatter, then either play or save the result. An interactive ScottPlot Chart allows the user to inspect the waveform.

Sample PSK-31 Transmissions

These audio files encode the text The Quick Brown Fox Jumped Over The Lazy Dog 1234567890 Times! in 1kHz BPSK at various baud rates.

Non-Standard Baud Rates

Let's see what PSK-3 sounds like. This mode encodes data at a rate of 3 bits per second. Note that Varicode characters may require up to ten bits, so this is pretty slow. On the other hand the side tones are closer to the carrier and the total bandwidth is much smaller. The message here has been shortened to just my callsign, AJ4VD.

Encode PSK-31 In Your Browser

After implementing the C# encoder described above I created a JavaScript version (as per Atwood's Law).

Decoding PSK-31

Considering all the steps for encoding PSK-31 transmissions are already described on this page, it doesn't require too much additional effort to create a decoder using basic software techniques. Once symbol phases are detected it's easy to work backwards: detect phase transitions (logical 0s) or repeats (logical 1s), treat consecutive zeros as a character separator, then look-up characters according to the Varicode table. The tricky bit is analyzing the source audio to generate the array of phase offsets.

The simplest way to decode PSK-31 transmissions leans on the fact that we already know the baud rate: 31.25 symbols per second, or one symbol every 256 samples at 8kHz sample rate. The audio signal can be segregated into many 256-sample bins, and processed by FFT. Once the center frequency is determined, the FFT power at this frequency can be calculated for each bin. Signal offset can be adjusted to minimize the imaginary component of the FFTs at the carrier frequency, then the real component will be strongly positive or negative, allowing phase transitions to be easily detected.

There are more advanced techniques to improve BPSK decoding, such as continuously adjusting frequency and phase alignment (synchronization). A Costas loop can help lock onto the carrier frequency while preserving its phase. Visit Simple BPSK31 Decoding with Python for an excellent demonstration of how to decode BPSK31 using these advanced techniques.

A crude C# implementation of a BPSK decoded is available on GitHub in the PSK Experiments repository

Encoding PSK-31 in Hardware

Since BPSK is just a carrier that applies periodic 180º phase-shifts, it's easy to generate in hardware by directly modulating the signal source. A good example of this is KA7OEI's PSK31 transmitter which feeds the output of an oscillator through an even or odd number of NAND gates (from a 74HC00) to produce two signals of opposite phase.

Quadrature Phase Shift Keying (QPSK)

Unlike the 0º and 180º phases of binary phase shift keying (BPSK), quadrature phase shift keying (QPSK) encodes extra data into each symbol by uses a larger number of phases. When QPSK-31 is used in amateur radio these extra bits aren't used to send messages faster but instead send them more reliably using convolutional coding and error correction. These additional features come at a cost (an extra 3 dB SNR is required), and in practice QPSK is not used as much by amateur radio operators.

QPSK encoding/decoding and convolutional encoding/decoding are outside the scope of this page, but excellent information exists on the Wikipedia: QPSK and in the US Naval Academy's EC314 Lesson 23: Digital Modulation document.

PSK-31 in 2022

After all that, it turns out PSK-31 isn't that popular anymore. These days it seems the FT-8 digital mode with WSJT-X software is orders of magnitude more popular ??

Resources

Markdown source code last modified on December 17th, 2022
---
Title: Experiments in PSK-31 Synthesis
Description: How to encode and decode PSK-31 messages using C#
Date: 2022-10-16 23:26PM EST
Tags: amateur radio, csharp
---

# Experiments in PSK-31 Synthesis

**PSK-31 is a narrow-bandwidth digital mode which encodes text as an audio tone that varies phase at a known rate.** To learn more about this digital mode and solve a challenging programming problem, I'm going to write a PSK-31 encoder and decoder from scratch using the C# programming language. All code created for this project is open-source, available from my [PSK Experiments GitHub Repository](https://github.com/swharden/psk-experiments), and released under the permissive MIT license. This page documents my progress and notes things I learn along the way.

<a href="psk-waterfall2.jpg"><img src="psk-waterfall2.jpg"></a>

## Encoding Bits as Phase Shifts

<!--
<img src="bpsk.png" class="w-75 d-block mx-auto my-3">
-->

* PSK31 messages have a continuous carrier tone

* Symbols are represented by "symbols", each 1/31.25 seconds long

* If a symbol changes phase from its previous symbol it is a `0`, otherwise it is a `1`

<a href="theory.png"><img src="theory.png" class="d-block mx-auto"></a>

## Amplitude Modulation Silences Phase Transitions

Although a continuous phase-shifting tone of constant amplitude can successfully transmit PSK31 data, the abrupt phase transitions will cause splatter. If you transmit this you will be heard, but those trying to communicate using adjacent frequencies will be highly disappointed.

<div class="text-center">

Hard phase transitions (splatter) | Soft phase transitions (cleaner)
---|---
<img src="splatter.png">|<img src="no-splatter.png">

</div>

To reduce spectral artifacts that result from abruptly changing phase, phase transitions are silenced by shaping the waveform envelope as a sine wave so it is silent at the transition. This way the maximum rate of phase shifts is a sine wave with a period of half the baud rate. This is why the opening of a PSK31 message (a series of logical `0` bits) sounds like two tones: It's the carrier sine wave with an envelope shaped like a sine wave with a period of 31.25/2 Hz. These two tones separated by approximately 15 Hz are visible in the spectrogram (waterfall).

![](modulation.png)

## Encoding Text as Bits

**Unlike ASCII (8 bits per character) and RTTY (5 bits per character), BPSK uses [Varicode](https://en.wikipedia.org/wiki/Varicode) (1-10 bits per character) to encode text.** Consecutive zeros `00` separate characters, so character codes must not contain `00` and they must start ane end with a `1` bit. Messages are flanked by a preamble (repeated `0` bits) and a postamble (repeated `1` bits).

<table cellpadding=30 style="white-space: nowrap; font-size: .8em;" class="m-0 p-0 mx-auto"><tr><td valign=top>
<TT>NUL 1010101011</TT>
<BR><TT>SOH 1011011011</TT>
<BR><TT>STX 1011101101</TT>
<BR><TT>ETX 1101110111</TT>
<BR><TT>EOT 1011101011</TT>
<BR><TT>ENQ 1101011111</TT>
<BR><TT>ACK 1011101111</TT>
<BR><TT>BEL 1011111101</TT>
<BR><TT>BS&nbsp; 1011111111</TT>
<BR><TT>HT&nbsp; 11101111</TT>
<BR><TT>LF&nbsp; 11101</TT>
<BR><TT>VT&nbsp; 1101101111</TT>
<BR><TT>FF&nbsp; 1011011101</TT>
<BR><TT>CR&nbsp; 11111</TT>
<BR><TT>SO&nbsp; 1101110101</TT>
<BR><TT>SI&nbsp; 1110101011</TT>
<BR><TT>DLE 1011110111</TT>
<BR><TT>DC1 1011110101</TT>
<BR><TT>DC2 1110101101</TT>
<BR><TT>DC3 1110101111</TT>
<BR><TT>DC4 1101011011</TT>
<BR><TT>NAK 1101101011</TT>
<BR><TT>SYN 1101101101</TT>
<BR><TT>ETB 1101010111</TT>
<BR><TT>CAN 1101111011</TT>
<BR><TT>EM&nbsp; 1101111101</TT>
<BR><TT>SUB 1110110111</TT>
<BR><TT>ESC 1101010101</TT>
<BR><TT>FS&nbsp; 1101011101</TT>
<BR><TT>GS&nbsp; 1110111011</TT>
<BR><TT>RS&nbsp; 1011111011</TT>
<BR><TT>US&nbsp; 1101111111</TT>
<BR><TT>SP&nbsp; 1</TT>
<BR><TT>!&nbsp;&nbsp; 111111111</TT>
<BR><TT>"&nbsp;&nbsp; 101011111</TT>
<BR><TT>#&nbsp;&nbsp; 111110101</TT>
<BR><TT>$&nbsp;&nbsp; 111011011</TT>
<BR><TT>%&nbsp;&nbsp; 1011010101</TT>
<BR><TT>&amp;&nbsp;&nbsp; 1010111011</TT>
<BR><TT>'&nbsp;&nbsp; 101111111</TT>
<BR><TT>(&nbsp;&nbsp; 11111011</TT>
<BR><TT>)&nbsp;&nbsp; 11110111</TT>
<BR><TT>*&nbsp;&nbsp; 101101111</TT>
</td><td valign=top width="33%" >
<TT>+&nbsp;&nbsp; 111011111</TT>
<BR><TT>,&nbsp;&nbsp; 1110101</TT>
<BR><TT>-&nbsp;&nbsp; 110101</TT>
<BR><TT>.&nbsp;&nbsp; 1010111</TT>
<BR><TT>/&nbsp;&nbsp; 110101111</TT>
<BR><TT>0&nbsp;&nbsp; 10110111</TT>
<BR><TT>1&nbsp;&nbsp; 10111101</TT>
<BR><TT>2&nbsp;&nbsp; 11101101</TT>
<BR><TT>3&nbsp;&nbsp; 11111111</TT>
<BR><TT>4&nbsp;&nbsp; 101110111</TT>
<BR><TT>5&nbsp;&nbsp; 101011011</TT>
<BR><TT>6&nbsp;&nbsp; 101101011</TT>
<BR><TT>7&nbsp;&nbsp; 110101101</TT>
<BR><TT>8&nbsp;&nbsp; 110101011</TT>
<BR><TT>9&nbsp;&nbsp; 110110111</TT>
<BR><TT>:&nbsp;&nbsp; 11110101</TT>
<BR><TT>;&nbsp;&nbsp; 110111101</TT>
<BR><TT>&lt;&nbsp;&nbsp; 111101101</TT>
<BR><TT>=&nbsp;&nbsp; 1010101</TT>
<BR><TT>>&nbsp;&nbsp; 111010111</TT>
<BR><TT>?&nbsp;&nbsp; 1010101111</TT>
<BR><TT>@&nbsp;&nbsp; 1010111101</TT>
<BR><TT>A&nbsp;&nbsp; 1111101</TT>
<BR><TT>B&nbsp;&nbsp; 11101011</TT>
<BR><TT>C&nbsp;&nbsp; 10101101</TT>
<BR><TT>D&nbsp;&nbsp; 10110101</TT>
<BR><TT>E&nbsp;&nbsp; 1110111</TT>
<BR><TT>F&nbsp;&nbsp; 11011011</TT>
<BR><TT>G&nbsp;&nbsp; 11111101</TT>
<BR><TT>H&nbsp;&nbsp; 101010101</TT>
<BR><TT>I&nbsp;&nbsp; 1111111</TT>
<BR><TT>J&nbsp;&nbsp; 111111101</TT>
<BR><TT>K&nbsp;&nbsp; 101111101</TT>
<BR><TT>L&nbsp;&nbsp; 11010111</TT>
<BR><TT>M&nbsp;&nbsp; 10111011</TT>
<BR><TT>N&nbsp;&nbsp; 11011101</TT>
<BR><TT>O&nbsp;&nbsp; 10101011</TT>
<BR><TT>P&nbsp;&nbsp; 11010101</TT>
<BR><TT>Q&nbsp;&nbsp; 111011101</TT>
<BR><TT>R&nbsp;&nbsp; 10101111</TT>
<BR><TT>S&nbsp;&nbsp; 1101111</TT>
<BR><TT>T&nbsp;&nbsp; 1101101</TT>
<BR><TT>U&nbsp;&nbsp; 101010111</TT>
</td><td valign=top width="33%" >
<TT>V&nbsp;&nbsp; 110110101</TT>
<BR><TT>X&nbsp;&nbsp; 101011101</TT>
<BR><TT>Y&nbsp;&nbsp; 101110101</TT>
<BR><TT>Z&nbsp;&nbsp; 101111011</TT>
<BR><TT>[&nbsp;&nbsp; 1010101101</TT>
<BR><TT>\&nbsp;&nbsp; 111110111</TT>
<BR><TT>]&nbsp;&nbsp; 111101111</TT>
<BR><TT>^&nbsp;&nbsp; 111111011</TT>
<BR><TT>_&nbsp;&nbsp; 1010111111</TT>
<BR><TT>.&nbsp;&nbsp; 101101101</TT>
<BR><TT>/&nbsp;&nbsp; 1011011111</TT>
<BR><TT>a&nbsp;&nbsp; 1011</TT>
<BR><TT>b&nbsp;&nbsp; 1011111</TT>
<BR><TT>c&nbsp;&nbsp; 101111</TT>
<BR><TT>d&nbsp;&nbsp; 101101</TT>
<BR><TT>e&nbsp;&nbsp; 11</TT>
<BR><TT>f&nbsp;&nbsp; 111101</TT>
<BR><TT>g&nbsp;&nbsp; 1011011</TT>
<BR><TT>h&nbsp;&nbsp; 101011</TT>
<BR><TT>i&nbsp;&nbsp; 1101</TT>
<BR><TT>j&nbsp;&nbsp; 111101011</TT>
<BR><TT>k&nbsp;&nbsp; 10111111</TT>
<BR><TT>l&nbsp;&nbsp; 11011</TT>
<BR><TT>m&nbsp;&nbsp; 111011</TT>
<BR><TT>n&nbsp;&nbsp; 1111</TT>
<BR><TT>o&nbsp;&nbsp; 111</TT>
<BR><TT>p&nbsp;&nbsp; 111111</TT>
<BR><TT>q&nbsp;&nbsp; 110111111</TT>
<BR><TT>r&nbsp;&nbsp; 10101</TT>
<BR><TT>s&nbsp;&nbsp; 10111</TT>
<BR><TT>t&nbsp;&nbsp; 101</TT>
<BR><TT>u&nbsp;&nbsp; 110111</TT>
<BR><TT>v&nbsp;&nbsp; 1111011</TT>
<BR><TT>w&nbsp;&nbsp; 1101011</TT>
<BR><TT>x&nbsp;&nbsp; 11011111</TT>
<BR><TT>y&nbsp;&nbsp; 1011101</TT>
<BR><TT>z&nbsp;&nbsp; 111010101</TT>
<BR><TT>{&nbsp;&nbsp; 1010110111</TT>
<BR><TT>|&nbsp;&nbsp; 110111011</TT>
<BR><TT>}&nbsp;&nbsp; 1010110101</TT>
<BR><TT>~&nbsp;&nbsp; 1011010111</TT>
<BR><TT>DEL 1110110101</TT>
</td></tr></table>

## How to Generate a PSK Waveform

Now that we've covered the major steps of PSK31 message composition and modulation, let's go through the steps ot generate a PSK31 message in code.

### Step 1: Convert a Message to Varicode

Here's the gist of how I store my varicode table in code. Note that the `struct` has an additional `Description` field which is useful for decoding and debugging.

```cs
public struct VaricodeSymbol
{
    public readonly string Symbol;
    public string BitString;
    public int[] Bits;
    public readonly string Description;

    public VaricodeSymbol(string symbol, string bitString, string? description = null)
    {
        Symbol = symbol;
        BitString = bitString;
        Bits = bitString.ToCharArray().Select(x => x == '1' ? 1 : 0).ToArray();
        Description = description ?? string.Empty;
    }
}
```

```cs
static VaricodeSymbol[] GetAllSymbols() => new VaricodeSymbol[]
{
    new("NUL", "1010101011", "Null character"),
    new("LF", "11101", "Line feed"),
    new("CR", "11111", "Carriage return"),
    new("SP", "1", "Space"),
    new("a", "1011"),
    new("b", "1011111"),
    new("c", "101111"),
    // etc...
};
```

I won't show how I do the message-to-varicode lookup, but it's trivial. Here's the final function I use to generate Varicode bits from a string:

```cs
static int[] GetVaricodeBits(string message)
{
    List<int> bits = new();

    // add a preamble of repeated zeros
    for (int i=0; i<20; i++)
        bits.Add(0);

    // encode each character of a message
    foreach (char character in message)
    {
        VaricodeSymbol symbol = Lookup(character);
        bits.AddRange(symbol.Bits);
        bits.AddRange(CharacterSeparator);
    }

    // add a postamble of repeated ones
    for (int i=0; i<20; i++)
        bits.Add(1);

    return bits.ToArray();
}
```

### Step 2: Determine Phase Shifts

Now that we have our Varicode bits, we need to generate an array to indicate phase transitions. A transition occurs every time a bit changes value form the previous bit. This code returns phase as an array of `double` given the bits from a Varicode message.

```cs
public static double[] GetPhaseShifts(int[] bits, double phase1 = 0, double phase2 = Math.PI)
{
    double[] phases = new double[bits.Length];
    for (int i = 0; i < bits.Length; i++)
    {
        double previousPhase = i > 0 ? phases[i - 1] : phase1;
        double oppositePhase = previousPhase == phase1 ? phase2 : phase1;
        phases[i] = bits[i] == 1 ? previousPhase : oppositePhase;
    }
    return phases;
}
```

### Step 3: Generate the Waveform

These constants will be used to define the shape of the waveform:

```cs
public const int SampleRate = 8000;
public const double Frequency = 1000;
public const double BaudRate = 31.25;
```

This minimal code generates a decipherable PSK-31 message, but it does not silence the phase transitions so it produces a lot of splatter. This function must be refined to shape the waveform such that phase transitions are silenced.

```cs
public double[] GetWaveformBPSK(double[] phases)
{
    int totalSamples = (int)(phases.Length * SampleRate / BaudRate);
    double[] wave = new double[totalSamples];
    for (int i = 0; i < wave.Length; i++)
    {
        double time = (double)i / SampleRate;
        int frame = (int)(time * BaudRate);
        double phaseShift = phases[frame];
        wave[i] = Math.Cos(2 * Math.PI * Frequency * time + phaseShift);
    }
    return wave;
}
```

### Step 4: Generate the Waveform with Amplitude Modulation

This is the same function as above, but with extra logic for amplitude-modulating the waveform in the shape of a sine wave to silence phase transitions.

```cs
public double[] GetWaveformBPSK(double[] phases)
{
    int baudSamples = (int)(SampleRate / BaudRate);
    double samplesPerBit = SampleRate / BaudRate;
    int totalSamples = (int)(phases.Length * SampleRate / BaudRate);
    double[] wave = new double[totalSamples];

    // create the amplitude envelope sized for a single bit
    double[] envelope = new double[(int)samplesPerBit];
    for (int i = 0; i < envelope.Length; i++)
        envelope[i] = Math.Sin((i + .5) * Math.PI / envelope.Length);

    for (int i = 0; i < wave.Length; i++)
    {
        // phase modulated carrier
        double time = (double)i / SampleRate;
        int frame = (int)(time * BaudRate);
        double phaseShift = phases[frame];
        wave[i] = Math.Cos(2 * Math.PI * Frequency * time + phaseShift);

        // envelope at phase transitions
        int firstSample = (int)(frame * SampleRate / BaudRate);
        int distanceFromFrameStart = i - firstSample;
        int distanceFromFrameEnd = baudSamples - distanceFromFrameStart + 1;
        bool isFirstHalfOfFrame = distanceFromFrameStart < distanceFromFrameEnd;
        bool samePhaseAsLast = frame == 0 ? false : phases[frame - 1] == phases[frame];
        bool samePhaseAsNext = frame == phases.Length - 1 ? false : phases[frame + 1] == phases[frame];
        bool rampUp = isFirstHalfOfFrame && !samePhaseAsLast;
        bool rampDown = !isFirstHalfOfFrame && !samePhaseAsNext;

        if (rampUp)
            wave[i] *= envelope[distanceFromFrameStart];

        if (rampDown)
            wave[i] *= envelope[distanceFromFrameEnd];
    }

    return wave;
}
```

## PSK31 Encoder Program

I wrapped the functionality above in a Windows Forms GUI that allows the user to type a message, specify frequency, baud rate, and whether or not to refine the envelope to reduce splatter, then either play or save the result. An interactive [ScottPlot Chart](https://scottplot.net) allows the user to inspect the waveform.

* **Download PSK31 Encoder: [PSK31-encoder.zip](PSK31-encoder.zip)**

* **PSK31 Encoder Source Code: [PSK Experiments on GitHub](https://github.com/swharden/psk-experiments)**

![](screenshot.png)

## Sample PSK-31 Transmissions

These audio files encode the text _The Quick Brown Fox Jumped Over The Lazy Dog 1234567890 Times!_ in 1kHz BPSK at various baud rates.

* PSK-31: [dog31.wav](dog31.wav)
* PSK-63: [dog63.wav](dog63.wav)
* PSK-125: [dog125.wav](dog125.wav)
* PSK-256: [dog256.wav](dog256.wav)

### Non-Standard Baud Rates

**Let's see what PSK-3 sounds like.** This mode encodes data at a rate of 3 bits per second. Note that Varicode characters may require up to ten bits, so this is pretty slow. On the other hand the side tones are closer to the carrier and the total bandwidth is much smaller. The message here has been shortened to just my callsign, AJ4VD.

* PSK-3: [psk3.wav](psk3.wav)

## Encode PSK-31 In Your Browser

After implementing the C# encoder described above I created a JavaScript version (as per <a href="https://en.wikipedia.org/wiki/Jeff_Atwood">Atwood's Law</a>).

* Try it on your phone or computer! [**Launch PskJS**](pskjs)

<a href="pskjs"><img src="pskjs.png" class="d-block mx-auto my-3"></a>

## Decoding PSK-31

Considering all the steps for _encoding_ PSK-31 transmissions are already described on this page, it doesn't require too much additional effort to create a _decoder_ using basic software techniques. Once symbol phases are detected it's easy to work backwards: detect phase transitions (logical 0s) or repeats (logical 1s), treat consecutive zeros as a character separator, then look-up characters according to the Varicode table. The tricky bit is analyzing the source audio to generate the array of phase offsets.

The simplest way to decode PSK-31 transmissions leans on the fact that we already know the baud rate: 31.25 symbols per second, or one symbol every 256 samples at 8kHz sample rate. The audio signal can be segregated into many 256-sample bins, and processed by FFT. Once the center frequency is determined, the FFT power at this frequency can be calculated for each bin. Signal offset can be adjusted to minimize the imaginary component of the FFTs at the carrier frequency, then the real component will be strongly positive or negative, allowing phase transitions to be easily detected.

<div class="row">
	<div class="col">
		<a href="psk31-receiver.html"><img src="py-fft.png" class="img-fluid"></a>
	</div>
	<div class="col">
		<a href="psk31-receiver.html"><img src="py-iq.png" class="img-fluid"></a>
	</div>
	<div class="col">
		<a href="psk31-receiver.html"><img src="py-eyediagram.png" class="img-fluid"></a>
	</div>
</div>

There are more advanced techniques to improve BPSK decoding, such as continuously adjusting frequency and phase alignment (synchronization). A [Costas loop](https://en.wikipedia.org/wiki/Costas_loop) can help lock onto the carrier frequency while preserving its phase. Visit [**Simple BPSK31 Decoding with Python**](psk31-receiver.html) for an excellent demonstration of how to decode BPSK31 using these advanced techniques.

A crude C# implementation of a BPSK decoded is available on GitHub in the [PSK Experiments](https://github.com/swharden/psk-experiments) repository

![](psk-decode.png)

## Encoding PSK-31 in Hardware

Since BPSK is just a carrier that applies periodic 180º phase-shifts, it's easy to generate in hardware by directly modulating the signal source. A good example of this is [KA7OEI's PSK31 transmitter](http://www.ka7oei.com/psk_bm_tx.html) which feeds the output of an oscillator through an even or odd number of NAND gates (from a [74HC00](https://www.mouser.com/datasheet/2/308/74HC00-105628.pdf)) to produce two signals of opposite phase.

![](pic-psk31.png)

## Quadrature Phase Shift Keying (QPSK)

**Unlike the 0º and 180º phases of binary phase shift keying (BPSK), quadrature phase shift keying (QPSK) encodes extra data into each symbol by uses a larger number of phases.** When QPSK-31 is used in amateur radio these extra bits aren't used to send messages faster but instead send them more reliably using convolutional coding and error correction. These additional features come at a cost (an extra 3 dB SNR is required), and in practice QPSK is not used as much by amateur radio operators.

QPSK encoding/decoding and convolutional encoding/decoding are outside the scope of this page, but excellent information exists on the [Wikipedia: QPSK](https://en.wikipedia.org/wiki/Phase-shift_keying#Quadrature_phase-shift_keying_(QPSK)) and in the US Naval Academy's [EC314 Lesson 23: Digital Modulation](https://www.usna.edu/ECE/ec312/Lessons/wireless/EC312_Lesson_23_Digital_Modulation_Course_Notes.pdf) document.

<a href="qpsk.png"><img src="qpsk.png" class="mx-auto d-block"></a>

## PSK-31 in 2022

After all that, it turns out PSK-31 isn't that popular anymore. These days it seems the [FT-8 digital mode](https://en.wikipedia.org/wiki/FT8) with [WSJT-X software](https://physics.princeton.edu/pulsar/k1jt/wsjtx.html) is orders of magnitude more popular ??

## Resources

* [PSK Experiments](https://github.com/swharden/psk-experiments) (GitHub) - Source code for the project shown on this page

* Software: [digipan](https://www.apkfollow.com/articles/2020/06/digipan.net.html) - A Freeware Program for PSK31 and PSK63

* Software: [fldigi](http://www.w1hkj.com/) - Supports PSK31 and other digital modes

* Software: [Digital Master 780](https://swharden.com/blog/2022-10-14-ham-radio-deluxe) - A Windows application that supports PSK31 and other digital modes bundled with Ham Radio Deluxe 5.0 (the last free version)

* Software: [WinPSK](https://www.moetronix.com/ae4jy/winpsk.htm) - open source PSK31 software for Windows

* Software: [PSKCore DLL](http://www.moetronix.com/ae4jy/pskcoredll.htm) - A Windows DLL that can be included in other software to add support for PSK31

* Software: [jacobwgillespie/psk31](https://github.com/jacobwgillespie/psk31) - Example PSK31 message generate using JavaScript

* [Digital Modulation](https://www.usna.edu/ECE/ec312/Lessons/wireless/EC312_Lesson_23_Digital_Modulation_Course_Notes.pdf) (US Naval Academy, EC314 Lesson 23) - A good description of quadrature PSK and higher order phase-shift encoding.

* [PSK-31 Specification](http://www.arrl.org/psk31-spec) (ARRL) - theory, varicode table, and convolutional code table.

* [PSK31 Description](http://aintel.bi.ehu.es/psk31.html) by G3PLX is the original / official description of the digital mode.

* [PSK31: A New Radio-Teletype Mode](http://www.arrl.org/files/file/Technology/tis/info/pdf/x9907003.pdf) (1999) by Peter Martinez, G3PLX

* [PSK31 The Easy Way](https://www.vic.wicen.org.au/wp-content/uploads/2012/05/psk31.pdf) (1999) by Alan Gibbs, VK6PG

* [Wikipedia: Varicode](https://en.wikipedia.org/wiki/Varicode) includes a table of all symbols

* [Wikipedia: QPSK](https://en.wikipedia.org/wiki/Phase-shift_keying#Quadrature_phase-shift_keying_(QPSK))

* [PSK31 Fundamentals](http://aintel.bi.ehu.es/psk31theory.html) and [PSK31 Setup](https://myplace.frontier.com/~nb6z/psk31.htm) by Peter Martinez, G3PLX

* [Varicode](http://math0.wvstateu.edu/~baker/cs240/info/varicode.html) - West Virginia State University CS240

* [Introduction to PSK31](https://sites.google.com/site/psk31matlabproject/introduction-to-psk) by engineering students at Walla Walla University

* [GNURadio PSK31 Decoder](https://sdradventure.wordpress.com/2011/10/15/gnuradio-psk31-decoder-part-1/) by VA7STH

* [Simple BPSK31](psk31-receiver.html) - a fantastic Jupyter notebook demonstrating BPSK decoding with Python

* [PySDR: Digital Modulation](https://pysdr.org/content/digital_modulation.html) - a summary of signal modulation types

* [A PIC-Based PSK31 exciter using a Balanced Modulator](http://www.ka7oei.com/psk_bm_tx.html) by Clint Turner, KA7OEI
Pages