SWHarden.com

The personal website of Scott W Harden

Treemapping with C#

How to create a treemap diagram using C#

Treemap diagrams display a series of positive numbers using rectangles sized proportional to the value of each number. This page demonstrates how to calculate the size and location of rectangles to create a tree map diagram using C#. Although the following uses System.Drawing to save the tree map as a Bitmap image, these concepts may be combined with information on the C# Data Visualization page to create treemap diagrams using SkiaSharp, WPF, or other graphics technologies.

The tree map above was generated from random data using the following C# code:

// Create sample data. Data must be sorted large to small.
double[] sortedValues = Enumerable.Range(0, 40)
    .Select(x => (double)Random.Shared.Next(10, 100))
    .OrderByDescending(x => x)
    .ToArray();

// Create an array of labels in the same order as the sorted data.
string[] labels = sortedValues.Select(x => x.ToString()).ToArray();

// Calculate the size and position of all rectangles in the tree map
int width = 600;
int height = 400;
RectangleF[] rectangles = TreeMap.GetRectangles(sortedValues, width, height);

// Create an image to draw on (with 1px extra to make room for the outline)
using Bitmap bmp = new(width + 1, height + 1);
using Graphics gfx = Graphics.FromImage(bmp);
using Font fnt = new("Consolas", 8);
using SolidBrush brush = new(Color.Black);
gfx.Clear(Color.White);

// Draw and label each rectangle
for (int i = 0; i < rectangles.Length; i++)
{
    brush.Color = Color.FromArgb(
        red: Random.Shared.Next(150, 250),
        green: Random.Shared.Next(150, 250),
        blue: Random.Shared.Next(150, 250));

    gfx.FillRectangle(brush, rectangles[i]);
    gfx.DrawRectangle(Pens.Black, rectangles[i]);
    gfx.DrawString(labels[i], fnt, Brushes.Black, rectangles[i].X, rectangles[i].Y);
}

// Save the output
bmp.Save("treemap.bmp");

Treemap Logic

The previous code block focuses on data generation and display, but hides the tree map calculations behind the TreeMap class. Below is the code for that class. It is self-contained static class and exposes a single static method which takes a pre-sorted array of values and returns tree map rectangles ready to display on an image.

💡 Although the System.Drawing.Common is a Windows-only library (as of .NET 7), System.Drawing.Primitives is a cross-platform package that provides the RectangleF structure used in the tree map class. See the SkiaSharp Quickstart to learn how to create image files using cross-platform .NET code.

public static class TreeMap
{
    public static RectangleF[] GetRectangles(double[] values, int width, int height)
    {
        for (int i = 1; i < values.Length; i++)
            if (values[i] > values[i - 1])
                throw new ArgumentException("values must be ordered large to small");

        var slice = GetSlice(values, 1, 0.35);
        var rectangles = GetRectangles(slice, width, height);
        return rectangles.Select(x => x.ToRectF()).ToArray();
    }

    private class Slice
    {
        public double Size { get; }
        public IEnumerable<double> Values { get; }
        public Slice[] Children { get; }

        public Slice(double size, IEnumerable<double> values, Slice sub1, Slice sub2)
        {
            Size = size;
            Values = values;
            Children = new Slice[] { sub1, sub2 };
        }

        public Slice(double size, double finalValue)
        {
            Size = size;
            Values = new double[] { finalValue };
            Children = Array.Empty<Slice>();
        }
    }

    private class SliceResult
    {
        public double ElementsSize { get; }
        public IEnumerable<double> Elements { get; }
        public IEnumerable<double> RemainingElements { get; }

        public SliceResult(double elementsSize, IEnumerable<double> elements, IEnumerable<double> remainingElements)
        {
            ElementsSize = elementsSize;
            Elements = elements;
            RemainingElements = remainingElements;
        }
    }

    private class SliceRectangle
    {
        public Slice Slice { get; set; }
        public float X { get; set; }
        public float Y { get; set; }
        public float Width { get; set; }
        public float Height { get; set; }
        public SliceRectangle(Slice slice) => Slice = slice;
        public RectangleF ToRectF() => new(X, Y, Width, Height);
    }

    private static Slice GetSlice(IEnumerable<double> elements, double totalSize, double sliceWidth)
    {
        if (elements.Count() == 1)
            return new Slice(totalSize, elements.Single());

        SliceResult sr = GetElementsForSlice(elements, sliceWidth);
        Slice child1 = GetSlice(sr.Elements, sr.ElementsSize, sliceWidth);
        Slice child2 = GetSlice(sr.RemainingElements, 1 - sr.ElementsSize, sliceWidth);
        return new Slice(totalSize, elements, child1, child2);
    }

    private static SliceResult GetElementsForSlice(IEnumerable<double> elements, double sliceWidth)
    {
        var elementsInSlice = new List<double>();
        var remainingElements = new List<double>();
        double current = 0;
        double total = elements.Sum();

        foreach (var element in elements)
        {
            if (current > sliceWidth)
                remainingElements.Add(element);
            else
            {
                elementsInSlice.Add(element);
                current += element / total;
            }
        }

        return new SliceResult(current, elementsInSlice, remainingElements);
    }

    private static IEnumerable<SliceRectangle> GetRectangles(Slice slice, int width, int height)
    {
        SliceRectangle area = new(slice) { Width = width, Height = height };

        foreach (var rect in GetRectangles(area))
        {
            if (rect.X + rect.Width > area.Width)
                rect.Width = area.Width - rect.X;

            if (rect.Y + rect.Height > area.Height)
                rect.Height = area.Height - rect.Y;

            yield return rect;
        }
    }

    private static IEnumerable<SliceRectangle> GetRectangles(SliceRectangle sliceRectangle)
    {
        var isHorizontalSplit = sliceRectangle.Width >= sliceRectangle.Height;
        var currentPos = 0;
        foreach (var subSlice in sliceRectangle.Slice.Children)
        {
            var subRect = new SliceRectangle(subSlice);
            int rectSize;

            if (isHorizontalSplit)
            {
                rectSize = (int)Math.Round(sliceRectangle.Width * subSlice.Size);
                subRect.X = sliceRectangle.X + currentPos;
                subRect.Y = sliceRectangle.Y;
                subRect.Width = rectSize;
                subRect.Height = sliceRectangle.Height;
            }
            else
            {
                rectSize = (int)Math.Round(sliceRectangle.Height * subSlice.Size);
                subRect.X = sliceRectangle.X;
                subRect.Y = sliceRectangle.Y + currentPos;
                subRect.Width = sliceRectangle.Width;
                subRect.Height = rectSize;
            }

            currentPos += rectSize;

            if (subSlice.Values.Count() > 1)
            {
                foreach (var sr in GetRectangles(subRect))
                {
                    yield return sr;
                }
            }
            else if (subSlice.Values.Count() == 1)
            {
                yield return subRect;
            }
        }
    }
}

Source Code Complexity Analysis

A few days ago I wrote an article describing how to programmatically generate .NET source code analytics using C#. Using these tools I analyzed the source code for all classes in a large project (ScottPlot.NET). The following tree map displays every class in the project as a rectangle sized according to number of lines of code and colored according to maintainability.

In this diagram large rectangles represent classes with the most code, and red color indicates classes that are difficult to maintain.

💡 I’m using a perceptually uniform colormap (similar to Turbo)provided by the ScottPlot provided by the ScottPlot NuGet package. See ScottPlot’s colormaps gallery for all available colormaps.

💡 The Maintainability Index is a value between 0 (worst) and 100 (best) that represents the relative ease of maintaining the code. It’s calculated from a combination of Halstead complexity (size of the compiled code), Cyclomatic complexity (number of paths that can be taken through the code), and the total number of lines of code.

Conclusions

References


.NET Source Code Analysis

How to analyze source code metrics of .NET assemblies from a console application

This page describes how to use the Microsoft.CodeAnalysis.Metrics package to perform source code analysis of .NET assemblies from a console application. Visual Studio users can perform source code analysis by clicking the “Analyze” dropdown menu and selecting “Calculate Code Metrics”, but I sought to automate this process so I can generate custom code analysis reports from console applications as part of my CI pipeline.

Performing Code Analysis

Step 1: Add the Microsoft.CodeAnalysis.Metrics package to your project:

dotnet add package Microsoft.CodeAnalysis.Metrics

Step 2: Perform code analysis:

dotnet build -target:Metrics

Note that multi-targeted projects must append --framework net6.0 to specify a single platform target to use for code analysis.

Step 3: Inspect analysis results in ProjectName.Metrics.xml

<?xml version="1.0" encoding="utf-8"?>
<CodeMetricsReport Version="1.0">
  <Targets>
    <Target Name="ScottPlot.csproj">
      <Assembly Name="ScottPlot, Version=4.1.61.0">
        <Metrics>
          <Metric Name="MaintainabilityIndex" Value="81" />
          <Metric Name="CyclomaticComplexity" Value="6324" />
          <Metric Name="ClassCoupling" Value="664" />
          <Metric Name="DepthOfInheritance" Value="3" />
          <Metric Name="SourceLines" Value="35360" />
          <Metric Name="ExecutableLines" Value="10208" />
        </Metrics>
        <Namespaces>
        ...

Parsing the Analysis XML File

The code analysis XML contains information about every assembly, namespace, type, and function in the whole code base! There is a lot of possible information to extract, but the code below is enough to get us started extracting basic metric information for every type in the code base.

using System;
using System.IO;
using System.Linq;
using System.Text;
using System.Xml.Linq;
using System.Collections.Generic;

/// <summary>
/// Display a particular metric for every type in an assembly.
/// </summary>
void RankTypes(string xmlFilePath, string metricName = "CyclomaticComplexity", bool highToLow = true)
{
    string xmlText = File.ReadAllText(xmlFilePath);
    XDocument doc = XDocument.Parse(xmlText);
    XElement assembly = doc.Descendants("Assembly").First();

    var rankedTypes = GetMetricByType(assembly, metricName).OrderBy(x => x.Value).ToArray();
    if (highToLow)
        Array.Reverse(rankedTypes);

    Console.WriteLine($"Types ranked by {metricName}:");
    foreach (var type in rankedTypes)
        Console.WriteLine($"{type.Value:N0}\t{type.Key}");
}

Dictionary<string, int> GetMetricByType(XElement assembly, string metricName)
{
    Dictionary<string, int> metricByType = new();

    foreach (XElement namespaceElement in assembly.Element("Namespaces")!.Elements("Namespace"))
    {
        foreach (XElement namedType in namespaceElement.Elements("Types").Elements("NamedType"))
        {
            XElement metric = namedType.Element("Metrics")!.Elements("Metric")
                .Where(x => x.Attribute("Name")!.Value == metricName)
                .Single();
            string typeName = namedType.Attribute("Name")!.Value;
            string namespaceName = namespaceElement.Attribute("Name")!.Value;
            string fullTypeName = $"{namespaceName}.{typeName}";
            metricByType[fullTypeName] = int.Parse(metric.Attribute("Value")!.Value!.ToString());
        }
    }

    return metricByType;
}

Querying Code Analysis Results

Specific metrics of interest will vary, but here are some code examples demonstrating how to parse the code metrics file to display useful information. For these examples I run the code analysis command above to generate ScottPlot.Metrics.xml from the ScottPlot code base and use the code above to generate various reports.

Rank Types by Cyclomatic Complexity

Cyclomatic complexity is a measure of the number of different paths that can be taken through a computer program, and it is often used as an indicator for difficult-to-maintain code. Some CI systems even prevent the merging of pull requests if their cyclomatic complexity exceeds a predefined threshold! Although I don’t intend to gate pull requests by complexity at this time, I would like to gain insight into which classes are the most complex as a way to quantitatively target my code maintenance and efforts.

RankTypes("ScottPlot.Metrics.xml", "CyclomaticComplexity");
517     ScottPlot.Plot
218     ScottPlot.Plottable.SignalPlotBase<T>
173     ScottPlot.Plottable.ScatterPlot
139     ScottPlot.Settings
120     ScottPlot.Ticks.TickCollection
118     ScottPlot.Renderable.Axis
114     ScottPlot.Drawing.Colormap
113     ScottPlot.Control.ControlBackEnd
109     ScottPlot.DataGen
99      ScottPlot.Plottable.AxisLineVector
98      ScottPlot.Plottable.Heatmap
95      ScottPlot.Tools
93      ScottPlot.Plottable.RepeatingAxisLine
91      ScottPlot.Plottable.PopulationPlot
85      ScottPlot.Plottable.AxisLine
83      ScottPlot.Plottable.AxisSpan
77      ScottPlot.Plottable.RadialGaugePlot
...

Rank Types by Lines of Code

Similarly, ranking all my project’s types by how many lines of code they contain can give me insight into which types may benefit most from refactoring.

RankTypes("ScottPlot.Metrics.xml", "SourceLines");
Types ranked by SourceLines:
4,155   ScottPlot.Plot
1,182   ScottPlot.DataGen
954     ScottPlot.Plottable.SignalPlotBase<T>
726     ScottPlot.Control.ControlBackEnd
670     ScottPlot.Ticks.TickCollection
670     ScottPlot.Settings
630     ScottPlot.Plottable.ScatterPlot
600     ScottPlot.Renderable.Axis
477     ScottPlot.Statistics.Common
454     ScottPlot.Tools
451     ScottPlot.Plottable.PopulationPlot
432     ScottPlot.Drawing.GDI
343     ScottPlot.Plottable.SignalPlotXYGeneric<TX, TY>
336     ScottPlot.Plottable.RepeatingAxisLine
335     ScottPlot.Drawing.Colormap
332     ScottPlot.Plottable.AxisLineVector
...

Rank Types by Maintainability

The Maintainability Index is a value between 0 (worst) and 100 (best) that represents the relative ease of maintaining the code. It’s calculated from a combination of Halstead complexity (size of the compiled code), Cyclomatic complexity (number of paths that can be taken through the code), and the total number of lines of code.

MaintainabilityIndex = 171 
  - 5.2 * Math.Log(HalsteadVolume) 
  - 0.23 * CyclomaticComplexity
  - 16.2 * Math.Log(LinesOfCCode);

The maintainability index is calculated by Microsoft.CodeAnalysis.Metrics so we don’t have to. I don’t know how Microsoft arrived at their weights for this formula, but the overall idea is described here.

RankTypes("ScottPlot.Metrics.xml", "MaintainabilityIndex", highToLow: false);
43      ScottPlot.Drawing.Tools
48      ScottPlot.Statistics.Interpolation.Cubic
48      ScottPlot.Statistics.Interpolation.PeriodicSpline
49      ScottPlot.Statistics.Interpolation.EndSlopeSpline
49      ScottPlot.Statistics.Interpolation.NaturalSpline
50      ScottPlot.Renderable.AxisTicksRender
54      ScottPlot.Statistics.Interpolation.CatmullRom
55      ScottPlot.Statistics.Interpolation.SplineInterpolator
57      ScottPlot.DataGen
58      ScottPlot.DataStructures.SegmentedTree<T>
58      ScottPlot.MarkerShapes.Hashtag
58      ScottPlot.Ticks.TickCollection
59      ScottPlot.MarkerShapes.Asterisk
59      ScottPlot.Plottable.SignalPlotXYGeneric<TX, TY>
59      ScottPlot.Statistics.Interpolation.Bezier
60      ScottPlot.Statistics.Interpolation.Chaikin
61      ScottPlot.Generate
61      ScottPlot.Plot
61      ScottPlot.Statistics.Finance
...

Create Custom HTML Reports

With a little more effort you can generate HTML reports that use tables and headings to highlight useful code metrics and draw attention to types that could benefit from refactoring to improve maintainability.

Conclusions

Microsoft’s official Microsoft.CodeAnalysis.Metrics NuGet package is a useful tool for analyzing assemblies, navigating through namespaces, types, properties, and methods, and evaluating their metrics. Since these analyses can be performed using console applications, they can be easily integrated into CI pipelines or used to create standalone code analysis applications. Future projects can build on the concepts described here to create graphical visualizations of code metrics in large projects.

Resources


Divide 10 MHz to 1PPS with a Microcontroller

How to use a microcontroller to inexpensively scale down a 10 MHz reference clock into a one pulse per second (1pps) signal

I often find it useful to gate frequency counters using a 1 pulse per second (1PPS) signal derived from a 10 MHz precision frequency reference. However, a divide-by-10-million circuit isn’t trivial to implement. Counter ICs exist which enable divide-by-100 by combining multiple divide-by-2 and divide-by-5 stages (e.g., MC74HC390A around $0.85 each), but dividing 10 MHz all the way down to 1Hz would require at least 4 of these chips and a lot of wiring.

You can clock a microcontroller at 10 MHz and use its timer and interrupt systems to generate 1PPS. For example, an ATTiny202 in an 8-pin SOIC package is available from Mouser (>50k stocked) for $0.51 each. Note that modern series AVRs require a special UDPI programmer.

ATTiny202 ($0.51) ATTint826 ($0.95)

This page documents a divide-by-10-million circuit achieved with a single microcontroller to scale a 10MHz frequency reference down to 1PPS. I’m using an ATTiny826 because that is what I have on hand, but these concepts apply to any microcontroller with a 16-bit timer.

ATTiny Breakout Board

Some AVRs come in DIP packages but their pin numbers may be different than the same chip in a SMT package. To facilitate prototyping using designs and code that will work identically across a breadboard prototype and a PCB containing SMT chips, I prefer to build DIP breakout boards using whatever SMT package I intend to include in my final builds. In this case it’s ATTint826 in a SOIC-20 package, and I can easily use this in a breadboard by soldering them onto SOIC-to-DIP breakout boards.

I assembled the breakout board by hand using a regular soldering iron. When working with small packages it helps so much to coat the pins with a little tack flux to facilitate wetting and prevent solder bridges. I’m presently using Chip Quik NC191. Even if flux is advertized as “no-clean”, it’s good practice and makes the boards look much nicer to remove remaining flux with acetone and Q-tips or brushes.

Circuit

Configuration Change Protection (CCP)

Traditional AVR microcontrollers used fuse bits to set the clock source, but modern series chips can change the clock source from within code. However, modifying the clock source requires temporarily disabling the configuration change protection (CCP) system.

Disabling the CCP only lasts four clock cycles, so the immediate next statement must be assignment of the new value. I use the following function to facilitate this action.

/* Write a value to a CCP-protected register */
void ccp_write(volatile register8_t* address, uint8_t value){
	CCP = CCP_IOREG_gc;
	*address = value;
}
// Use internal 20 MHz clock with CKOUT pin enabled
ccp_write(&CLKCTRL.MCLKCTRLA, CLKCTRL.MCLKCTRLA | CLKCTRL_CLKOUT_bm);

Do not use compound statements when writing to the CCP register. The code below fails to change clock as one may expect by looking at the code, presumably because the combined OR operation with the assignment exceeds four clock cycles. Instead of direct assignment, use the ccp_write function described above.

// WARNING: This code does not actually change the clock source
CCP = CCP_IOREG_gc;
CLKCTRL.MCLKCTRLA = CLKCTRL.MCLKCTRLA | CLKCTRL_CLKOUT_bm;

Configuring the Clock Source

Internal 10 MHz clock

This is the configuration I use to achieve a 10 CPU clock using the built-in 20 MHz oscillator.

void configure_clock_internal_10mhz(){
	ccp_write(&CLKCTRL.MCLKCTRLA, CLKCTRL.MCLKCTRLA | CLKCTRL_CLKOUT_bm); // 20 MHz internal clock, enable CKOUT
	ccp_write(&CLKCTRL.MCLKCTRLB, CLKCTRL_PEN_bm); // enable divide-by-2 clock prescaler
}

External 10 MHz clock

This is the configuration I use to clock the CPU from an external 10 MHz clock source applied to the EXTCLK pin.

void configure_clock_external(){
	ccp_write(&CLKCTRL.MCLKCTRLA, CLKCTRL_CLKSEL_EXTCLK_gc | CLKCTRL_CLKOUT_bm); // external clock, enable CKOUT
	ccp_write(&CLKCTRL.MCLKCTRLB, 0); // disable prescaler
}

Configuring the 16-Bit Timer

This is how I configured my ATTiny826’s TCA0 16-bit timer to fire an interrupt every 200 ms.

void configure_1pps(){
	// 10 MHz system clock with div64 prescaler is 156,250 Hz.
	// Setting a 16-bit timer's top to 31,250 means 5 overflows per second.
	TCA0.SINGLE.INTCTRL = TCA_SINGLE_OVF_bm; // overflow interrupt
	TCA0.SINGLE.CTRLB = TCA_SINGLE_WGMODE_NORMAL_gc; // normal mode
	TCA0.SINGLE.PER = 31249UL; // control timer period by setting timer top
	TCA0.SINGLE.CTRLA |= TCA_SINGLE_CLKSEL_DIV64_gc; // set clock source
	TCA0.SINGLE.CTRLA |= TCA_SINGLE_ENABLE_bm; // start timer
}

Alternatively, multiple timers could be cascaded to achieve a similar effect. Modern AVR series microcontrollers have sections in their datasheet describing considerations for cascading two 16-bit timers to create a single 32-bit timer. Using this strategy one could set the top of the counter to 5 million and arrange an interrupt to toggle an LED, resulting in a 1Hz signal with 50% duty.

Configuring the Interrupt System

This method is called whenever the timer’s overflow interrupt is triggered. Since it’s called 5 times per second, I just need a global counter to count the number of times it was called, and set an output pin to high on every 5th invocation.

uint8_t overflow_count;

ISR(TCA0_OVF_vect)
{
	overflow_count++;
	if (overflow_count == 5){
		overflow_count = 0;
		PORTB.OUT = PIN1_bm;
    } else {
		PORTB.OUT = 0;
	}
    
	TCA0.SINGLE.INTFLAGS = TCA_SINGLE_OVF_bm; // indicate interrupt was handled
}

Do not forget to enable global interrupts in your start-up sequence! This is an easy mistake to make, and without calling this function the overflow function will never be invoked.

sei(); // enable global interrupts

Results

We have achieved a light that blinks exactly once per second with roughly the same precision as the 10 MHz frequency reference used to clock the microcontroller. This output signal is ready to use for precision measurement purposes, such as toggling the gate of a discrete frequency counter.

Resources


Programming Modern AVR Microcontrollers

How to program modern AVR chips using Atmel ICE, MPLAB Snap, or Serial Port UPDI programmers

This page describes how to program Microchip’s newest series of AVR microcontrollers using official programming gear and software. I spent many years programming the traditional series of Atmel chips, but now several years after Microchip acquired Atmel I am interested in exploring the capabilities of the latest series of AVR microcontrollers (especially the new AVR DD family). Currently the global chip shortage makes it difficult to source traditional ATMega and STM32 chips, but the newest series of AVR microcontrollers feature an impressive set of peripherals for the price and are available from all the major vendors. pressive set of peripherals for the price and are available from all the major vendors.

TLDR

Blinking a LED is the “Hello, World” of microcontroller programming. Let’s take a look at the code necessary to blink a LED on pin 2 of an ATTiny286. It is compiled and programmed onto the chip using Microchip Studio.

#define F_CPU 3333333UL
#include <avr/io.h>
#include <util/delay.h>

int main(void)
{
	PORTA.DIR = 0xFF;
	while (1)
	{
		PORTA.OUT = 255;
		_delay_ms(500);
		PORTA.OUT = 0;
		_delay_ms(500);
	}
}

ATTiny826 Pinout

From page 14 of the ATTiny826 datasheet

SMT ATTiny Breakout Board

Many of the newer AVR series microcontrollers are not available in breadboard-friendly DIP packages. I find SOIC-to-DIP breakout boards (available on Amazon and eBay) to be useful for experimenting with chips in SOIC packages. Here I added extra power and PA4 (pin 2) LEDs and 10 kΩ current limiting resistors.

I power the device from the 3.3V or 5V pins on a FT232 USB breakout board. Although the topic is out of scope for this article, I find it convenient to use FTDI chips to exchange small amounts of data or debug messages between a microcontroller and a modern PC over USB without requiring special drivers.

Why is programming modern AVRs so difficult?

I’m surprised how little information there is online about how to reliably program modern AVR series microcontrollers. In late 2022 there is a surprisingly large amount of “advice” on this topic which leads to dead ends and broken or abandoned projects. After looking into it for a while, here is my opinionated assessment. Mouser and Digikey have links to expensive programmers, and Amazon has links to similar items but reviews are littered with comments like “arrived bricked” and “can not program AVR series chips”. DIY options typically involve abandoned (or soon-to-be abandoned?) GitHub repositories, or instructions for Arduino-related programming. I seek to consolidate and distill the most useful information onto this page, and I hope others will find it useful.

Atmel-ICE: Expensive but Effective

After using $5 ICSP programmers for the last decade I almost fell out of my chair when I saw Microchip’s recommended entry-level programmer is over $180! Digikey sells a “basic” version without cables for $130, but that still seems crazy to me. Also, $50 for a ribbon cable?

I found a kit on Amazon that sells the programmer with a cable for $126. It was hard for me to press that buy button, but I figured the time I would save by having access to modern and exotic chips during the present global chip shortage would make it worth it. After a couple days of free Prime shipping, it arrived. It was smaller than I thought it would be from the product photos.

The cable that came with the device seemed a bit hacky at first, but I’m happy to have it. The female 2.54 mm socket is easy to insert breadboard jumpers into.

I’m glad this thing is idiot proof. The very first thing I did after unboxing this programmer was hook it up to my power supply rails using reverse polarity. I misread the pin diagram and confused the socket with the connector (they are mirror images of one another). This is an easy mistake to make though, so here’s a picture of the correct orientation. Note the location of the tab on the side of the connector.

Atmel ICE Pinout Programming Connection

You can break-out power and programming pins so you can power your device with this programmer and also not need the stupid ribbon cable for programming it! See my article Hack an Atmel ICE to Deliver Power for more information.

The Atmel ICE was easy to use with Microchip Studio. My programmer was detected immediately, a window popped-up and walked me through updating the firmware, and my LED was blinking in no time.

MPLAB Snap: Cheap and Convoluted

Did I really need to spend $126 for an AVR programmer? Amazon carries the MPLAB Snap for $34, but lots of reviews say it doesn’t work. After easily getting the Atmel-ICE programmer up and running I thought it would be a similarly easy experience setting-up the MPLAB Snap for AVR UPDI programming, but boy was I wrong. Now that I know the steps to get this thing working it’s not so bad, but the information here was only gathered after hours of frustration.

Here are the steps you can take to program modern AVR microcontrollers with UPDI using a MPLAB Snap:

Step 1: Setup MPLAB

Step 2: Re-Flash the Programmer

Step 3: Acknowledge Your Programmer is Defective

Defective may be a strong word, but let’s just say the hardware was not designed to enable programming AVR chips using UPDI. Microchip Studio will detect the programmer but if you try to program an AVR you’ll get a pop-up error message that provides surprisingly little useful information.

Atmel Studio was unable to start your debug session. Please verify device selection, interface settings, target power and connections to the target device. Look in the details section for more information. StatusCode: 131107 ModuleName: TCF (TCF command: Processes:launch failed.) An illegal configuration parameter was used. Debugger command Activate physical failed.

Step 4: Fix Your Programmer

The reason MPLAB Snap cannot program AVR microcontrollers is because the UPDI pin should be pulled high, but the MPLAB Snap comes from the factory with its UPDI pin pulled low with a 4.7 kΩ resistor to ground. You can try to overpower this resistor by adding a low value pull-up resistor to Vcc (1 kΩ worked for me on a breadboard), but the actual fix is to fire-up the soldering iron and remove that tiny surface-mount pull-down resistor labeled R48.

Have your glasses? R48 is here:

These photos were taken after I removed the resistor. I didn’t use hot air. I just touched it a for a few seconds with a soldering iron and wiped it off then threw it away.

You don’t need a microscope, but I’m glad I had one.

Step 5: Reflect

You can now program AVR microcontrollers using UPDI with your MPLAB Snap! Blink, LED, blink.

Can you believe this is the officially recommended action? According to the official Microchip Engineering Technical Note ETN #36: MPLAB Snap AVR Interface Modification

I feel genuinely sorry for the Amazon sellers who are getting poor reviews because they sell this product. It really isn’t their fault. I hope Google directs people here so that they can get their boards working and leave positive reviews that point more people to this issue.

UPDI Programming with a Serial Adapter

There is no official support for UPDI programming using a serial adapter, but it seems some people have figured out how to do it in some capacity. There was a promising pyupdi project, but it is now deprecated. At the time of writing the leading project aiming to enable UPDI programming without official hardware is pymcuprog, but its repository has a single commit dated two months ago and no activity since. Interestingly, that commit was made by buildmaster@microchip.com (an unverified email address), so it may not be fair to refer to it as an “unofficial” solution. The long term support of the pymcuprog project remains uncertain, but regardless let’s take a closer look at how it works.

To build a programmer you just need a TTL USB serial adapter and a 1kΩ resistor. These are the steps I used to program a LED blink program using this strategy:

Ping the Microcontroller

This command returns the device ID (1E9328 for my ATtiny826) indicating the UPDI connection is working successfully

pymcuprog ping -d attiny826 -t uart -u com12
Connecting to SerialUPDI
Pinging device...
Ping response: 1E9328

Write a Hex File

I used Microchip Studio to compile my C code and generate the hex file. Now I can use pymcuprog to load those hex files onto the chip. It’s slower to program and inconvenient to drop to a terminal whenever I want to program a chip, but it works.

pymcuprog write -f LedBlink.hex -d attiny826 -t uart -u com12
Connecting to SerialUPDI
Pinging device...
Ping response: 1E9328
Writing from hex file...
Writing flash...

Conclusions

Resources


Interfacing HX710 Differential ADC with Arduino

How to read differential voltage from a HX710 ADC using Arduino

This page demonstrates how to read differential voltage from a HX710 ADC using Arduino. I recently obtained some pressure sensor boards from Amazon for less than $3 each under names like 6pcs 3.3-5V Digital Barometric Air Pressure Sensor Module Liquid Water Level Controller Board 0-40KPa that use this ADC. Several years ago I worked on a precision pressure meter project based on an I2C temperature and pressure sensor (MS5611), and now that I see new inexpensive SPI pressure sensor modules on the consumer market I’m interested to learn more about their capabilities.

Analog-to-Digital Converter IC

The ADC chip is easily identified as a HX710B 24-Bit Analog-to-Digital Converter (ADC) with Built-in Temperature Sensor. According to the datasheet it can be powered by a 3.3V or 5V supply, and the value it reports is the differential voltage between two input pins.

The datasheet indicates this device can be run from a 3.3V or 5V supply, it uses a built-in fixed-gain (128x) differential amplifier, and it can read up to 40 samples per second. The datasheet provides an example circuit demonstrating how this ADC can be used to measure weight from a scale sensor:

Pressure Sensor

To get a better idea of how this sensor works it would be helpful to locate its product number. I had a hunch it was beneath the part so I desoldered it, and indeed I found part identification information.

The pressure sensor is labeled as a PSG010S but unfortunately I struggled to find a quality datasheet for it. I did find some now-deleted images from an AliExpress listing showing the differences between the base model and the R and S variants. I found this PSG010R datasheet (curiously written in Comic Sans) indicating that maximum voltage is 5V and that the gauge pressure is 0 - 40KPa (0 - 5.8 PSI). This seems to be a fairly standard differential pressure sensor design using a pair of voltage dividers where the pressure is a function of the difference in voltage at the two mid-points (a Wheatstone bridge).

I used a multimeter to determine the resistor network inside this pressure sensor. Interestingly I found that pin 4 was not connected, which is opposite of many other designs I find online (where pin 5 is not connected). It is critically important to get this right if your goal is to design a PCB around this component. I guess connecting pins 1 & 6 and also 3 & 4 could be an option for covering all your bases? I also question if people are getting confused and reporting the mirror image pinout because they mistake the vent hole for the registration marker. Either way, I made the following diagram which I found helpful:

Update (2022-12-23): I received an email from somebody offering additional information about this component:

The PSG010 reports positive and negative pressures and can easily have its range shifted to almost double in one direction with almost none in the other. All that is needed is to lift the VCC or GND pin and insert a surface mount 75R ±15R under it. Lifting the ground side by 75R makes it double positive, while pushing the applied +V down makes it double negative (vacuum).
bruceg

Read HX710B with Arduino

This code demonstrates how to measure HX710B values using Arduino and display the readings in the serial terminal sufficient to graph in real time using the serial plotter. The animated plot is what it looks like when I blow puffs of air on the sensor.

void setup() {
  pinMode(2, INPUT);   // Connect HX710 OUT to Arduino pin 2
  pinMode(3, OUTPUT);  // Connect HX710 SCK to Arduino pin 3
  Serial.begin(9600);
}

void loop() {
  // wait for the current reading to finish
  while (digitalRead(2)) {}

  // read 24 bits
  long result = 0;
  for (int i = 0; i < 24; i++) {
    digitalWrite(3, HIGH);
    digitalWrite(3, LOW);
    result = result << 1;
    if (digitalRead(2)) {
      result++;
    }
  }

  // get the 2s compliment
  result = result ^ 0x800000;

  // pulse the clock line 3 times to start the next pressure reading
  for (char i = 0; i < 3; i++) {
    digitalWrite(3, HIGH);
    digitalWrite(3, LOW);
  }

  // display pressure
  Serial.println(result);
}

Open-Source HX710B Libraries

Although some libraries are available which facilitate interacting with the HX710, here I engage with it discretely to convey each step of the conversion and measurement process. I found that many libraries use the 10 Hz mode by default, whereas I certainly prefer the 40 Hz mode. More frustratingly, code in many libraries refer to this as gain, which is incorrect. The datasheet indicates gain is fixed at 128 and cannot be changed in software.

Update (2022-12-23): I received an email explaining why people often use “gain” and “mode” when referring to the HX710:

The HX711 is similar to the HX710 but it has user selectable gain AND user selectable sample rates BUT only certain combinations are allowed, so setting mode WILL also select its matched gain value. The HX710 uses most of the same internals, but with just 3 modes - reading the Wheatstone Bridge always using 128 gain at 10 or 40Hz while swapping to Avolt (HX710A) or internal Temperature (HX710B) uses a lower gain and less digits. So for people familiar with the HX711 there is no ambiguity in mixing mode and gain.
bruceg

Resources