The personal website of Scott W Harden
September 10th, 2021

Drawing with Maui.Graphics

.NET MAUI (Multi-Platform Application User Interface) is a new framework for creating cross-platform apps using C#. MAUI will be released as part of .NET 6 in November 2021 and it is expected to come with Maui.Graphics, a cross-platform drawing library superior to System.Drawing in many ways. Although System.Drawing.Common currently supports rendering in Linux and MacOS, cross-platform support for System.Drawing will sunset over the next few releases and begin throwing a PlatformNotSupportedException in .NET 6.

By creating a graphics model using only Maui.Graphics dependencies, users can share drawing code across multiple rendering technologies (GDI, Skia, SharpDX, etc.), operating systems (Windows, Linux, MacOS, etc.), and application frameworks (WinForms, WPF, Maui, WinUI, etc.). This page demonstrates how to create a platform-agnostic graphics model and render it using Windows Forms and WPF. Resources in the Maui.Graphics namespace can be used by any modern .NET application (not just Maui apps).

⚠️ WARNING: Maui.Graphics is still a pre-release experimental library (as noted on their GitHub page). Although the code examples on this page work presently, the API may change between now and the official release.

1. Create a new Project

For this example I will start by creating a a .NET 5.0 WinForms project from scratch. Later we will extend the solution to include a WPF project that uses the same graphics model.

2. Add References to Maui.Graphics

We need to get the windows forms MAUI control and all its dependencies. At the time of writing (September, 2021) these packages are not yet available on NuGet, but they can be downloaded from the Microsoft.Maui.Graphics GitHub page.

💡 Tip: If you're developing a desktop application you can improve the "rebuild all" time by editing the csproj files of your dependencies so TargetFrameworks only includes .NET Standard targets.

  • Add the Microsoft.Maui.Graphics project to your solution and add a reference to it from your project.

  • Windows Forms: Add the Microsoft.Maui.Graphics.GDI.Winforms project to your solution and add a reference to it in your WinForms project. A GDIGraphicsView control should appear in the toolbox.

  • WPF: Add the Microsoft.Maui.Graphics.Skia.WPF project to your solution and add a reference to it in your WPF project. A WDSkiaGraphicsView control should appear in the toolbox.

3. Create a Drawable Object

A drawable is a class that implements IDrawable, has a Draw() method, and can be rendered anywhere Maui.Graphics is supported. By only depending on Maui.Graphics it's easy to create a graphics model that can be used on any operating system using any supported graphical framework.

using Microsoft.Maui.Graphics;

This drawable fills the image blue and renders 1,000 randomly-placed anti-aliased semi-transparent white lines on it. Note that the location of the lines depends on the size of the render field (passed-in as an argument).

public class RandomLines : IDrawable
{
    public void Draw(ICanvas canvas, RectangleF dirtyRect)
    {
        canvas.FillColor = Color.FromArgb("#003366");
        canvas.FillRectangle(dirtyRect);

        canvas.StrokeSize = 1;
        canvas.StrokeColor = Color.FromRgba(255, 255, 255, 100);
        Random Rand = new();
        for (int i = 0; i < 1000; i++)
        {
            canvas.DrawLine(
                x1: (float)Rand.NextDouble() * dirtyRect.Width,
                y1: (float)Rand.NextDouble() * dirtyRect.Height,
                x2: (float)Rand.NextDouble() * dirtyRect.Width,
                y2: (float)Rand.NextDouble() * dirtyRect.Height);
        }
    }
}

4. Add a GraphicsView Control

Drag/drop a GraphicsView control from the Toolbox onto your application and assign your graphics model to its Drawable field.

For animations you can use a timer to invalidate the control (forcing a redraw) automatically every 20 ms.

Windows Forms (Rendering with GDI)

Windows Forms applications may also want to intercept SizeChanged events to force redraws as the window is resized.

public partial class Form1 : Form
{
    public Form1()
    {
        InitializeComponent();
        gdiGraphicsView1.Drawable = new RandomLines();
    }

    private void GdiGraphicsView1_SizeChanged(object? sender, EventArgs e) =>
        gdiGraphicsView1.Invalidate();

    private void timer1_Tick(object sender, EventArgs e) =>
        gdiGraphicsView1.Invalidate();
}

I found performance to be quite adequate. On my system 1,000 lines rendered on an 800x600 window at ~60 fps. Like System.Drawing this system slows down as a function of image size, so full-screen 1920x1080 animation was much slower (~10 fps).

WPF (Rendering with SkiaSharp)

<Skia:WDSkiaGraphicsView Name="MyGraphicsView" />
public partial class MainWindow : Window
{
    public MainWindow()
    {
        InitializeComponent();
        MyGraphicsView.Drawable = new RandomLines();

        DispatcherTimer timer = new();
        timer.Interval = TimeSpan.FromMilliseconds(20);
        timer.Tick += Timer_Tick; ;
        timer.Start();
    }

    private void Timer_Tick(object sender, EventArgs e) =>
        MyGraphicsView.Invalidate();
}

Extend the Graphics Model

Since our project is configured to display the same graphics model with both WinForms and WPF, it's easy to edit the model in one place and it's updated everywhere. We can replace the random lines model with one that manages randomly colored and sized balls that bounce off the edges of the window as the model advances.

BallField.cs

public class BallField : IDrawable
{
    private readonly Ball[] Balls;

    public BallField(int ballCount)
    {
        Balls = new Ball[ballCount];
    }

    public void Draw(ICanvas canvas, RectangleF dirtyRect)
    {
        canvas.FillColor = Colors.Navy;
        canvas.FillRectangle(dirtyRect);

        foreach (Ball ball in Balls)
        {
            ball?.Draw(canvas);
        }
    }

    public void Randomize(double width, double height)
    {
        Random rand = new();
        for (int i = 0; i < Balls.Length; i++)
        {
            Balls[i] = new()
            {
                X = rand.NextDouble() * width,
                Y = rand.NextDouble() * height,
                Radius = rand.NextDouble() * 5 + 5,
                XVel = rand.NextDouble() - .5,
                YVel = rand.NextDouble() - .5,
                R = (byte)rand.Next(50, 255),
                G = (byte)rand.Next(50, 255),
                B = (byte)rand.Next(50, 255),
            };
        }
    }

    public void Advance(double timeDelta, double width, double height)
    {
        foreach (Ball ball in Balls)
        {
            ball?.Advance(timeDelta, width, height);
        }
    }
}

Ball.cs

public class Ball
{
    public double X;
    public double Y;
    public double Radius = 5;
    public double XVel;
    public double YVel;
    public byte R, G, B;

    public void Draw(ICanvas canvas)
    {
        canvas.FillColor = Color.FromRgb(R, G, B);
        canvas.FillCircle((float)X, (float)Y, (float)Radius);
    }

    public void Advance(double timeDelta, double width, double height)
    {
        MoveForward(timeDelta);
        Bounce(width, height);
    }

    private void MoveForward(double timeDelta)
    {
        X += XVel * timeDelta;
        Y += YVel * timeDelta;
    }

    private void Bounce(double width, double height)
    {
        double minX = Radius;
        double minY = Radius;
        double maxX = width - Radius;
        double maxY = height - Radius;

        if (X < minX)
        {
            X = minX + (minX - X);
            XVel = -XVel;
        }
        else if (X > maxX)
        {
            X = maxX - (X - maxX);
            XVel = -XVel;
        }

        if (Y < minY)
        {
            Y = minY + (minY - Y);
            YVel = -YVel;
        }
        else if (Y > maxY)
        {
            Y = maxY - (Y - maxY);
            YVel = -YVel;
        }
    }
}

Download This Project

To build this project from source code you currently have to download Maui.Graphics source from GitHub and edit the solution file to point to the correct directory containing these projects. This will get a lot easier after Microsoft puts their WinForms and WPF controls on NuGet.

Coding Challenge

Can you recreate this classic screensaver using Maui.Graphics? Bonus points if the user can customize the number of shapes, the number of corners each shape has, and the number of lines drawn in each shape's history. It's a fun problem and I encourage you to give it a go! Here's how I did it: mystify-maui.zip

Resources

Markdown source code last modified on September 14th, 2021
---
Title: Drawing with Maui.Graphics
Description: How to use Maui.Graphics to draw and create animations in a Windows Forms and WPF applications
Date: 2021-09-10 10:30PM EST
Tags: csharp
---

# Drawing with Maui.Graphics

**[.NET MAUI](https://docs.microsoft.com/en-us/dotnet/maui/what-is-maui) (Multi-Platform Application User Interface) is a new framework for creating cross-platform apps using C#.** MAUI will be released as part of .NET 6 in November 2021 and it is expected to come with [`Maui.Graphics`](https://github.com/dotnet/Microsoft.Maui.Graphics), a cross-platform drawing library superior to [`System.Drawing`](https://docs.microsoft.com/en-us/dotnet/api/system.drawing?view=net-5.0#remarks) in many ways. Although [`System.Drawing.Common`](https://www.nuget.org/packages/System.Drawing.Common) currently supports rendering in Linux and MacOS, [cross-platform support for System.Drawing will sunset](https://github.com/dotnet/designs/blob/main/accepted/2021/system-drawing-win-only/system-drawing-win-only.md) over the next few releases and begin throwing a `PlatformNotSupportedException` in .NET 6.

By creating a graphics model using only `Maui.Graphics` dependencies, users can share drawing code across multiple rendering technologies (GDI, Skia, SharpDX, etc.), operating systems (Windows, Linux, MacOS, etc.), and application frameworks (WinForms, WPF, Maui, WinUI, etc.). **This page demonstrates how to create a platform-agnostic graphics model and render it using Windows Forms and WPF.** Resources in the `Maui.Graphics` namespace can be used by any modern .NET application (not just Maui apps).

<div class="text-center">

![](maui-graphics-balls.gif)

</div>

> ⚠️ **WARNING:** `Maui.Graphics` is still a pre-release experimental library (as noted on [their GitHub page](https://github.com/dotnet/Microsoft.Maui.Graphics)). Although the code examples on this page work presently, the API may change between now and the official release.

## 1. Create a new Project

For this example I will start by creating a a .NET 5.0 WinForms project from scratch. Later we will extend the solution to include a WPF project that uses the same graphics model.

## 2. Add References to Maui.Graphics

We need to get the windows forms MAUI control and all its dependencies. At the time of writing (September, 2021) these packages are not yet available on NuGet, but they can be downloaded from the [Microsoft.Maui.Graphics GitHub page](https://github.com/dotnet/Microsoft.Maui.Graphics).

> 💡 **Tip:** If you're developing a desktop application you can improve the "rebuild all" time by editing the csproj files of your dependencies so `TargetFrameworks` only includes .NET Standard targets.

* Add the `Microsoft.Maui.Graphics` project to your solution and add a reference to it from your project.

* **Windows Forms:** Add the `Microsoft.Maui.Graphics.GDI.Winforms` project to your solution and add a reference to it in your WinForms project. A `GDIGraphicsView` control should appear in the toolbox.

* **WPF:** Add the `Microsoft.Maui.Graphics.Skia.WPF` project to your solution and add a reference to it in your WPF project. A `WDSkiaGraphicsView` control should appear in the toolbox.

<div class="text-center img-border">

![](vs-maui.png)

</div>

## 3. Create a Drawable Object

**A _drawable_ is a class that implements `IDrawable`, has a `Draw()` method, and can be rendered anywhere Maui.Graphics is supported.** By only depending on `Maui.Graphics` it's easy to create a graphics model that can be used on any operating system using any supported graphical framework.

```cs
using Microsoft.Maui.Graphics;
```

This _drawable_ fills the image blue and renders 1,000 randomly-placed anti-aliased semi-transparent white lines on it. Note that the location of the lines depends on the size of the render field (passed-in as an argument).

```cs
public class RandomLines : IDrawable
{
    public void Draw(ICanvas canvas, RectangleF dirtyRect)
    {
        canvas.FillColor = Color.FromArgb("#003366");
        canvas.FillRectangle(dirtyRect);

        canvas.StrokeSize = 1;
        canvas.StrokeColor = Color.FromRgba(255, 255, 255, 100);
        Random Rand = new();
        for (int i = 0; i < 1000; i++)
        {
            canvas.DrawLine(
                x1: (float)Rand.NextDouble() * dirtyRect.Width,
                y1: (float)Rand.NextDouble() * dirtyRect.Height,
                x2: (float)Rand.NextDouble() * dirtyRect.Width,
                y2: (float)Rand.NextDouble() * dirtyRect.Height);
        }
    }
}
```

## 4. Add a GraphicsView Control

Drag/drop a GraphicsView control from the Toolbox onto your application and assign your graphics model to its `Drawable` field.

For animations you can use a timer to invalidate the control (forcing a redraw) automatically every 20 ms.

### Windows Forms (Rendering with GDI)

Windows Forms applications may also want to intercept SizeChanged events to force redraws as the window is resized.

```cs
public partial class Form1 : Form
{
    public Form1()
    {
        InitializeComponent();
        gdiGraphicsView1.Drawable = new RandomLines();
    }

    private void GdiGraphicsView1_SizeChanged(object? sender, EventArgs e) =>
        gdiGraphicsView1.Invalidate();

    private void timer1_Tick(object sender, EventArgs e) =>
        gdiGraphicsView1.Invalidate();
}
```

<div class="text-center">

![](maui-graphics-winforms.gif)

</div>

**I found performance to be quite adequate.** On my system 1,000 lines rendered on an 800x600 window at ~60 fps. Like `System.Drawing` this system slows down as a function of image size, so full-screen 1920x1080 animation was much slower (~10 fps).

### WPF  (Rendering with SkiaSharp)

```xml
<Skia:WDSkiaGraphicsView Name="MyGraphicsView" />
```

```cs
public partial class MainWindow : Window
{
    public MainWindow()
    {
        InitializeComponent();
        MyGraphicsView.Drawable = new RandomLines();

        DispatcherTimer timer = new();
        timer.Interval = TimeSpan.FromMilliseconds(20);
        timer.Tick += Timer_Tick; ;
        timer.Start();
    }

    private void Timer_Tick(object sender, EventArgs e) =>
        MyGraphicsView.Invalidate();
}
```

<div class="text-center">

![](maui-graphics-wpf.gif)

</div>

## Extend the Graphics Model

Since our project is configured to display the same graphics model with both WinForms and WPF, it's easy to edit the model in one place and it's updated everywhere. We can replace the random lines model with one that manages randomly colored and sized balls that bounce off the edges of the window as the model advances.

<div class="text-center">

![](maui-graphics-balls.gif)

</div>

### BallField.cs

```cs
public class BallField : IDrawable
{
    private readonly Ball[] Balls;

    public BallField(int ballCount)
    {
        Balls = new Ball[ballCount];
    }

    public void Draw(ICanvas canvas, RectangleF dirtyRect)
    {
        canvas.FillColor = Colors.Navy;
        canvas.FillRectangle(dirtyRect);

        foreach (Ball ball in Balls)
        {
            ball?.Draw(canvas);
        }
    }

    public void Randomize(double width, double height)
    {
        Random rand = new();
        for (int i = 0; i < Balls.Length; i++)
        {
            Balls[i] = new()
            {
                X = rand.NextDouble() * width,
                Y = rand.NextDouble() * height,
                Radius = rand.NextDouble() * 5 + 5,
                XVel = rand.NextDouble() - .5,
                YVel = rand.NextDouble() - .5,
                R = (byte)rand.Next(50, 255),
                G = (byte)rand.Next(50, 255),
                B = (byte)rand.Next(50, 255),
            };
        }
    }

    public void Advance(double timeDelta, double width, double height)
    {
        foreach (Ball ball in Balls)
        {
            ball?.Advance(timeDelta, width, height);
        }
    }
}
```

### Ball.cs

```cs
public class Ball
{
    public double X;
    public double Y;
    public double Radius = 5;
    public double XVel;
    public double YVel;
    public byte R, G, B;

    public void Draw(ICanvas canvas)
    {
        canvas.FillColor = Color.FromRgb(R, G, B);
        canvas.FillCircle((float)X, (float)Y, (float)Radius);
    }

    public void Advance(double timeDelta, double width, double height)
    {
        MoveForward(timeDelta);
        Bounce(width, height);
    }

    private void MoveForward(double timeDelta)
    {
        X += XVel * timeDelta;
        Y += YVel * timeDelta;
    }

    private void Bounce(double width, double height)
    {
        double minX = Radius;
        double minY = Radius;
        double maxX = width - Radius;
        double maxY = height - Radius;

        if (X < minX)
        {
            X = minX + (minX - X);
            XVel = -XVel;
        }
        else if (X > maxX)
        {
            X = maxX - (X - maxX);
            XVel = -XVel;
        }

        if (Y < minY)
        {
            Y = minY + (minY - Y);
            YVel = -YVel;
        }
        else if (Y > maxY)
        {
            Y = maxY - (Y - maxY);
            YVel = -YVel;
        }
    }
}
```

## Download This Project 

* **Source Code:** [**Balls.zip**](2021-09-12-Balls.zip) (10 kB)

* **WinForms (GDI) Demo:** [**Balls-WinForms.exe**](2021-09-12-Balls.exe-WinForms.zip) (237 kB)

* **WPF (Skia) Demo:** [**Balls-WPF.exe**](2021-09-12-Balls.exe-WPF.zip) (13 MB)

> To build this project from source code you currently have to download [Maui.Graphics source from GitHub](https://github.com/dotnet/Microsoft.Maui.Graphics) and edit the solution file to point to the correct directory containing these projects. This will get a lot easier after Microsoft puts their WinForms and WPF controls on NuGet.

## Coding Challenge

**Can you recreate this classic screensaver using Maui.Graphics?** Bonus points if the user can customize the number of shapes, the number of corners each shape has, and the number of lines drawn in each shape's history. It's a fun problem and I encourage you to give it a go! Here's how I did it: [mystify-maui.zip](2021-09-13-mystify-maui.zip)

![](maui-mystify.mp4)

## Resources

* [Draw with Maui.Graphics and Skia in a C# Console Application](https://swharden.com/blog/2021-08-01-maui-skia-console/)

* [Microsoft.Maui.Graphics on GitHub](https://github.com/dotnet/Microsoft.Maui.Graphics)

* [Maui.Graphics on NuGet](https://www.nuget.org/packages?q=Maui.Graphics)

* [Maui on GitHub](https://github.com/dotnet/maui)

* [C# Data Visualization](https://swharden.com/CsharpDataVis/) - Notes for visualizing data using C# and the .NET platform. I started preparing these using `System.Drawing` but will likely rework them to use `Maui.Graphics` and share them on [maui.graphics](https://maui.graphics)
August 1st, 2021

Draw with Maui.Graphics and Skia in a C# Console Application

Microsoft's System.Drawing.Common package is commonly used for cross-platform graphics in .NET Framework and .NET Core applications, but according to the dotnet roadmap System.Drawing will soon only support Windows. As Microsoft sunsets cross-platform support for System.Drawing they will be simultaneously developing Microsoft.Maui.Graphics, a cross-platform graphics library for iOS, Android, Windows, macOS, Tizen and Linux completely in C#.

The Maui.Graphics library can be used in any .NET application (not just MAUI applications). This page documents how I used the Maui.Drawing package to render graphics in memory (using a Skia back-end) and save them as static images from a console application.

I predict Maui.Graphics will eventually evolve to overtake System.Drawing in utilization. It has many advantages for performance and memory management (discussed extensively elsewhere on the internet), but it is still early in development. As of today (July 2021) the Maui.Graphics GitHub page warns "This is an experimental library ... There is no official support. Use at your own Risk."

Maui Graphics Skia Console Quickstart

This program will create an image, fill it with blue, add 1,000 random lines, then draw some text. It is written as a .NET 5 top-level console application and requires the Microsoft.Maui.Graphics and Microsoft.Maui.Graphics.Skia NuGet packages (both are currently in preview).

We use SkiaSharp to create a canvas, but importantly that canvas implements Microsoft.Maui.Graphics.ICanvas (it's not Skia-specific) so all the methods that draw on it can be agnostic to which rendering system was used. This makes it easy to write generic rendering methods now and have the option to switch the rendering system later.

Program.cs

using System;
using System.IO;
using Microsoft.Maui.Graphics;
using Microsoft.Maui.Graphics.Skia;

// Use Skia to create a Maui graphics context and canvas
BitmapExportContext bmpContext = SkiaGraphicsService.Instance.CreateBitmapExportContext(600, 400);
SizeF bmpSize = new(bmpContext.Width, bmpContext.Height);
ICanvas canvas = bmpContext.Canvas;

// Draw on the canvas with abstract methods that are agnostic to the renderer
ClearBackground(canvas, bmpSize, Colors.Navy);
DrawRandomLines(canvas, bmpSize, 1000);
DrawBigTextWithShadow(canvas, "This is Maui.Graphics with Skia");
SaveFig(bmpContext, Path.GetFullPath("quickstart.jpg"));

static void ClearBackground(ICanvas canvas, SizeF bmpSize, Color bgColor)
{
    canvas.FillColor = Colors.Navy;
    canvas.FillRectangle(0, 0, bmpSize.Width, bmpSize.Height);
}

static void DrawRandomLines(ICanvas canvas, SizeF bmpSize, int count = 1000)
{
    Random rand = new();
    for (int i = 0; i < count; i++)
    {
        canvas.StrokeSize = (float)rand.NextDouble() * 10;

        canvas.StrokeColor = new Color(
            red: (float)rand.NextDouble(),
            green: (float)rand.NextDouble(),
            blue: (float)rand.NextDouble(),
            alpha: .2f);

        canvas.DrawLine(
            x1: (float)rand.NextDouble() * bmpSize.Width,
            y1: (float)rand.NextDouble() * bmpSize.Height,
            x2: (float)rand.NextDouble() * bmpSize.Width,
            y2: (float)rand.NextDouble() * bmpSize.Height);
    }
}

static void DrawBigTextWithShadow(ICanvas canvas, string text)
{
    canvas.FontSize = 36;
    canvas.FontColor = Colors.White;
    canvas.SetShadow(offset: new SizeF(2, 2), blur: 1, color: Colors.Black);
    canvas.DrawString(text, 20, 50, HorizontalAlignment.Left);
}

static void SaveFig(BitmapExportContext bmp, string filePath)
{
    bmp.WriteToFile(filePath);
    Console.WriteLine($"WROTE: {filePath}");
}

MauiGraphicsDemo.csproj

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net5.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.Maui.Graphics" Version="6.0.100-preview.6.299" />
    <PackageReference Include="Microsoft.Maui.Graphics.Skia" Version="6.0.100-preview.6.299" />
  </ItemGroup>

</Project>

Resources

Markdown source code last modified on September 14th, 2021
---
Title: Draw with Maui.Graphics and Skia in a C# Console Application
Description: This page describes how to draw graphics in a console application with Maui Graphics and Skia
Date: 2021-08-01 7:15PM EST
Tags: csharp
---

# Draw with Maui.Graphics and Skia in a C# Console Application

**Microsoft's `System.Drawing.Common` package is commonly used for cross-platform graphics in .NET Framework and .NET Core applications, but according to the dotnet roadmap [System.Drawing will soon only support Windows](https://github.com/dotnet/designs/blob/main/accepted/2021/system-drawing-win-only/system-drawing-win-only.md).** As Microsoft sunsets cross-platform support for `System.Drawing` they will be simultaneously developing [`Microsoft.Maui.Graphics`](https://github.com/dotnet/Microsoft.Maui.Graphics), a cross-platform graphics library for iOS, Android, Windows, macOS, Tizen and Linux completely in C#.

**The `Maui.Graphics` library can be used in any .NET application (not just MAUI applications).** This page documents how I used the Maui.Drawing package to render graphics in memory (using a Skia back-end) and save them as static images from a console application.

**I predict `Maui.Graphics` will eventually evolve to overtake `System.Drawing` in utilization.** It has many advantages for performance and memory management (discussed extensively elsewhere on the internet), but it is still early in development. As of today (July 2021) [the Maui.Graphics GitHub page](https://github.com/dotnet/Microsoft.Maui.Graphics) warns "This is an experimental library ... There is no official support. Use at your own Risk."

## Maui Graphics Skia Console Quickstart

This program will create an image, fill it with blue, add 1,000 random lines, then draw some text. It is written as a .NET 5 top-level console application and requires the [Microsoft.Maui.Graphics](https://www.nuget.org/packages/Microsoft.Maui.Graphics) and [Microsoft.Maui.Graphics.Skia](https://www.nuget.org/packages/Microsoft.Maui.Graphics.Skia) NuGet packages (both are currently in preview). 

We use SkiaSharp to create a canvas, but importantly that canvas implements `Microsoft.Maui.Graphics.ICanvas` (it's not Skia-specific) so all the methods that draw on it can be agnostic to which rendering system was used. This makes it easy to write generic rendering methods now and have the option to switch the rendering system later.

<div class="text-center">

![](maui-graphics-quickstart.jpg)

</div>

### Program.cs
```cs
using System;
using System.IO;
using Microsoft.Maui.Graphics;
using Microsoft.Maui.Graphics.Skia;

// Use Skia to create a Maui graphics context and canvas
BitmapExportContext bmpContext = SkiaGraphicsService.Instance.CreateBitmapExportContext(600, 400);
SizeF bmpSize = new(bmpContext.Width, bmpContext.Height);
ICanvas canvas = bmpContext.Canvas;

// Draw on the canvas with abstract methods that are agnostic to the renderer
ClearBackground(canvas, bmpSize, Colors.Navy);
DrawRandomLines(canvas, bmpSize, 1000);
DrawBigTextWithShadow(canvas, "This is Maui.Graphics with Skia");
SaveFig(bmpContext, Path.GetFullPath("quickstart.jpg"));

static void ClearBackground(ICanvas canvas, SizeF bmpSize, Color bgColor)
{
    canvas.FillColor = Colors.Navy;
    canvas.FillRectangle(0, 0, bmpSize.Width, bmpSize.Height);
}

static void DrawRandomLines(ICanvas canvas, SizeF bmpSize, int count = 1000)
{
    Random rand = new();
    for (int i = 0; i < count; i++)
    {
        canvas.StrokeSize = (float)rand.NextDouble() * 10;

        canvas.StrokeColor = new Color(
            red: (float)rand.NextDouble(),
            green: (float)rand.NextDouble(),
            blue: (float)rand.NextDouble(),
            alpha: .2f);

        canvas.DrawLine(
            x1: (float)rand.NextDouble() * bmpSize.Width,
            y1: (float)rand.NextDouble() * bmpSize.Height,
            x2: (float)rand.NextDouble() * bmpSize.Width,
            y2: (float)rand.NextDouble() * bmpSize.Height);
    }
}

static void DrawBigTextWithShadow(ICanvas canvas, string text)
{
    canvas.FontSize = 36;
    canvas.FontColor = Colors.White;
    canvas.SetShadow(offset: new SizeF(2, 2), blur: 1, color: Colors.Black);
    canvas.DrawString(text, 20, 50, HorizontalAlignment.Left);
}

static void SaveFig(BitmapExportContext bmp, string filePath)
{
    bmp.WriteToFile(filePath);
    Console.WriteLine($"WROTE: {filePath}");
}
```

### MauiGraphicsDemo.csproj
```xml
<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net5.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.Maui.Graphics" Version="6.0.100-preview.6.299" />
    <PackageReference Include="Microsoft.Maui.Graphics.Skia" Version="6.0.100-preview.6.299" />
  </ItemGroup>

</Project>
```

## Resources
* [Animated Rendering with SkiaSharp and OpenGL](https://swharden.com/CsharpDataVis/)
* [Microsoft.Maui.Graphics on GitHub](https://github.com/dotnet/Microsoft.Maui.Graphics)
* [Microsoft.Maui.Graphics on NuGet](https://www.nuget.org/packages/Microsoft.Maui.Graphics/)
* [SkiaSharp Graphics in Xamarin.Forms](https://docs.microsoft.com/en-us/xamarin/xamarin-forms/user-interface/graphics/skiasharp/)
* [Maui.Graphics](https://maui.graphics)
July 3rd, 2021

C# Microphone Level Monitor

This page demonstrates how to continuously monitor microphone input using C#. Code here may be a helpful reference for developers interested in working with mono or stereo data captured from an audio device in real time. This project uses NAudio to provide simple access to the microphone on Windows platforms.

Mono Stereo

Full source code is available on GitHub (Program.cs)

Configure the Audio Input Device

This program starts by creating a WaveInEvent with a WaveFormat that specifies the sample rate, bit depth, and number of channels (1 for mono, 2 for stereo).

We can create a function to handle incoming data and add it to the DataAvailable event handler:

var waveIn = new NAudio.Wave.WaveInEvent
{
    DeviceNumber = 0, // customize this to select your microphone device
    WaveFormat = new NAudio.Wave.WaveFormat(rate: 44100, bits: 16, channels: 1),
    BufferMilliseconds = 50
};
waveIn.DataAvailable += ShowPeakMono;
waveIn.StartRecording();

Analyze Mono Audio Data

This method is called when the incoming audio buffer is filled. One of the arguments gives you access to the raw bytes in the buffer, and it's up to you to convert them to the appropriate data format.

This example is suitable for 16-bit (two bytes per sample) mono input.

private static void ShowPeakMono(object sender, NAudio.Wave.WaveInEventArgs args)
{
    float maxValue = 32767;
    int peakValue = 0;
    int bytesPerSample = 2;
    for (int index = 0; index < args.BytesRecorded; index += bytesPerSample)
    {
        int value = BitConverter.ToInt16(args.Buffer, index);
        peakValue = Math.Max(peakValue, value);
    }

    Console.WriteLine("L=" + GetBars(peakValue / maxValue));
}

This method converts a level (fraction) into bars suitable to display in the console:

private static string GetBars(double fraction, int barCount = 35)
{
    int barsOn = (int)(barCount * fraction);
    int barsOff = barCount - barsOn;
    return new string('#', barsOn) + new string('-', barsOff);
}

Analyze Stereo Audio Data

When the WaveFormat is configured for 2 channels, bytes in the incoming audio buffer will have left and right channel values interleaved (2 bytes for left, two bytes for right, then repeat). Left and right channels must be treated separately to display independent levels for stereo audio inputs.

This example is suitable for 16-bit (two bytes per sample) stereo input.

private static void ShowPeakStereo(object sender, NAudio.Wave.WaveInEventArgs args)
{
    float maxValue = 32767;
    int peakL = 0;
    int peakR = 0;
    int bytesPerSample = 4;
    for (int index = 0; index < args.BytesRecorded; index += bytesPerSample)
    {
        int valueL = BitConverter.ToInt16(args.Buffer, index);
        peakL = Math.Max(peakL, valueL);
        int valueR = BitConverter.ToInt16(args.Buffer, index + 2);
        peakR = Math.Max(peakR, valueR);
    }

    Console.Write("L=" + GetBars(peakL / maxValue));
    Console.Write(" ");
    Console.Write("R=" + GetBars(peakR / maxValue));
    Console.Write("\n");
}

Resources

Markdown source code last modified on September 12th, 2021
---
Title: C# Microphone Level Monitor
Description: How to continuously monitor the level of an audio input (mono or stereo) with C#
Date: 2021-07-03 10:42PM EST
Tags: csharp
---

# C# Microphone Level Monitor

**This page demonstrates how to continuously monitor microphone input using C#.** Code here may be a helpful reference  for developers interested in working with mono or stereo data captured from an audio device in real time. This project uses [NAudio](https://www.nuget.org/packages/NAudio) to provide simple access to the microphone on Windows platforms.

Mono | Stereo
---|---
<img src='microphone-mono.gif'>|<img src='microphone-stereo.gif'>

 Full source code is available on GitHub ([Program.cs](https://github.com/swharden/Csharp-Data-Visualization/blob/master/examples/2021-07-03-console-microphone/Program.cs))



## Configure the Audio Input Device

This program starts by creating a `WaveInEvent` with a `WaveFormat` that specifies the sample rate, bit depth, and number of channels (1 for mono, 2 for stereo).

We can create a function to handle incoming data and add it to the `DataAvailable` event handler:

```cs
var waveIn = new NAudio.Wave.WaveInEvent
{
    DeviceNumber = 0, // customize this to select your microphone device
    WaveFormat = new NAudio.Wave.WaveFormat(rate: 44100, bits: 16, channels: 1),
    BufferMilliseconds = 50
};
waveIn.DataAvailable += ShowPeakMono;
waveIn.StartRecording();
```

## Analyze Mono Audio Data

This method is called when the incoming audio buffer is filled. One of the arguments gives you access to the raw bytes in the buffer, and it's up to you to convert them to the appropriate data format. 

This example is suitable for 16-bit (two bytes per sample) mono input.

```cs
private static void ShowPeakMono(object sender, NAudio.Wave.WaveInEventArgs args)
{
    float maxValue = 32767;
    int peakValue = 0;
    int bytesPerSample = 2;
    for (int index = 0; index < args.BytesRecorded; index += bytesPerSample)
    {
        int value = BitConverter.ToInt16(args.Buffer, index);
        peakValue = Math.Max(peakValue, value);
    }

    Console.WriteLine("L=" + GetBars(peakValue / maxValue));
}
```

This method converts a level (fraction) into bars suitable to display in the console:

```cs
private static string GetBars(double fraction, int barCount = 35)
{
    int barsOn = (int)(barCount * fraction);
    int barsOff = barCount - barsOn;
    return new string('#', barsOn) + new string('-', barsOff);
}
```

<div class="text-center">

![](microphone-mono.gif)

</div>

## Analyze Stereo Audio Data

When the `WaveFormat` is configured for 2 channels, bytes in the incoming audio buffer will have left and right channel values interleaved (2 bytes for left, two bytes for right, then repeat). Left and right channels must be treated separately to display independent levels for stereo audio inputs.

This example is suitable for 16-bit (two bytes per sample) stereo input.

```cs
private static void ShowPeakStereo(object sender, NAudio.Wave.WaveInEventArgs args)
{
    float maxValue = 32767;
    int peakL = 0;
    int peakR = 0;
    int bytesPerSample = 4;
    for (int index = 0; index < args.BytesRecorded; index += bytesPerSample)
    {
        int valueL = BitConverter.ToInt16(args.Buffer, index);
        peakL = Math.Max(peakL, valueL);
        int valueR = BitConverter.ToInt16(args.Buffer, index + 2);
        peakR = Math.Max(peakR, valueR);
    }

    Console.Write("L=" + GetBars(peakL / maxValue));
    Console.Write(" ");
    Console.Write("R=" + GetBars(peakR / maxValue));
    Console.Write("\n");
}
```

<div class="text-center">

![](microphone-stereo.gif)

</div>

## Resources

* [Realtime Audio Visualization in Python](https://swharden.com/blog/2016-07-19-realtime-audio-visualization-in-python/) - a similar project using Python and the [pyaudio](http://people.csail.mit.edu/hubert/pyaudio/) library

* [NuGet: NAudio](https://www.nuget.org/packages/NAudio)

* [GitHub: console-microphone/Program.cs](https://github.com/swharden/Csharp-Data-Visualization/blob/master/examples/2021-07-03-console-microphone/Program.cs)

* [GitHub: C# Data Visualization](https://github.com/swharden/Csharp-Data-Visualization)
June 8th, 2021

Working with 16-bit Images in CSharp

Scientific image analysis frequently involves working with 12-bit and 14-bit sensor data stored in 16-bit TIF files. Images commonly encountered on the internat are 24-bit or 32-bit RGB images (where each pixel is represented by 8 bits each for red, green, blue, and possibly alpha). Typical image analysis libraries and documentation often lack information about how to work with 16-bit image data.

This page summarizes how I work with 16-bit TIF file data in C#. I prefer Magick.NET (an ImageMagick wrapper) when working with many different file formats, and LibTiff.Net whenever I know my source files will all be identically-formatted TIFs or multidimensional TIFs (stacks).

ImageMagick

ImageMagick is a free and open-source cross-platform software suite for displaying, creating, converting, modifying, and editing raster images. Although ImageMagick is commonly used at the command line, .NET wrappers exist to make it easy to use ImageMagick from within C# applications.

ImageMagick has many packages on NuGet and they are described on ImageMagick's documentation GitHub page. TLDR: Install the Q16 package (not HDRI) to allow you to work with 16-bit data without losing precision.

ImageMagick is free and distributed under the Apache 2.0 license, so it can easily be used in commercial projects.

An advantage of loading images with ImageMagick is that it will work easily whether the source file is a JPG, PNG, GIF, TIF, or something different. ImageMagick supports over 100 file formats!

Load a 16-bit TIF File as a Pixel Value Array

// Load pixel values from a 16-bit TIF using ImageMagick (Q16)
MagickImage image = new MagickImage("16bit.tif");
ushort[] pixelValues = image.GetPixels().GetValues();

That's it! The pixelValues array will contain one value per pixel from the original image. The length of this array will equal the image's height times its width.

Load an 8-bit TIF File as a Pixel Value Array

Since the Q16 package was installed, 2 bytes will be allocated for each pixel (16-bit) even if it only requires one byte (8-bit). In this case you must collect just the bytes you are interested in:

MagickImage image = new MagickImage("8bit.tif");
ushort[] pixelValues = image.GetPixels().GetValues();
int[] values8 = Enumerable.Range(0, pixelValues.Length / 2).Select(x => (int)pixelValues[x * 2 + 1]).ToArray();

Load a 32-bit TIF File as a Pixel Value Array

For this you'll have to install the high dynamic range (HDRI) Q16 package, then your GetValues() method will return a float[] instead of a ushort[]. Convert these values to proper pixel intensity values by dividing by 2^16.

MagickImage image = new MagickImage("32bit.tif");
float[] pixels = image.GetPixels().GetValues();
for (int i = 0; i < pixels.Length; i++)
    pixels[i] = (long)pixels[i] / 65535;

LibTiff

LibTiff is a pure C# (.NET Standard) TIF file reader. Although it doesn't support all the image formats that ImageMagick does, it's really good at working with TIFs. It has a more intuitive interface for working with TIF-specific features such as multi-dimensional images (color, Z position, time, etc.).

LibTiff gives you a lower-level access to the bytes that underlie image data, so it's on you to perform the conversion from a byte array to the intended data type. Note that some TIFs are little-endian encoded and others are big-endian encoded, and endianness can be read from the header.

LibTiff is distributed under a BSD 3-clause license, so it too can be easily used in commercial projects.

Load a 16-bit TIF as a Pixel Value Array

// Load pixel values from a 16-bit TIF using LibTiff
using Tiff image = Tiff.Open("16bit.tif", "r");

// get information from the header
int width = image.GetField(TiffTag.IMAGEWIDTH)[0].ToInt();
int height = image.GetField(TiffTag.IMAGELENGTH)[0].ToInt();
int bytesPerPixel = image.GetField(TiffTag.BITSPERSAMPLE)[0].ToInt() / 8;

// read the image data bytes
int numberOfStrips = image.NumberOfStrips();
byte[] bytes = new byte[numberOfStrips * image.StripSize()];
for (int i = 0; i < numberOfStrips; ++i)
    image.ReadRawStrip(i, bytes, i * image.StripSize(), image.StripSize());

// convert the data bytes to a double array
if (bytesPerPixel != 2)
    throw new NotImplementedException("this is only for 16-bit TIFs");
double[] data = new double[bytes.Length / bytesPerPixel];
for (int i = 0; i < data.Length; i++)
{
    if (image.IsBigEndian())
        data[i] = bytes[i * 2 + 1] + (bytes[i * 2] << 8);
    else
        data[i] = bytes[i * 2] + (bytes[i * 2 + 1] << 8);
}

Routines for detecting and converting data from 8-bit, 24-bit, and 32-bit TIF files can be created by inspecting bytesPerPixel. LibTiff has documentation describing how to work with RGB TIF files and multi-frame TIFs.

Convert a Pixel Array to a 2D Array

I often prefer to work with scientific image data as a 2D arrays of double values. I write my analysis routines to pass double[,] between methods so the file I/O can be encapsulated in a static class.

// Load pixel values from a 16-bit TIF using ImageMagick (Q16)
MagickImage image = new MagickImage("16bit.tif");
ushort[] pixelValues = image.GetPixels().GetValues();

// create a 2D array of pixel values
double[,] imageData = new double[image.Height, image.Width];
for (int i = 0; i < image.Height; i++)
    for (int j = 0; j < image.Width; j++)
        imageData[i, j] = pixelValues[i * image.Width + j];

Other Libraries

ImageProcessor

According to ImageProcessor's GitHub page, "ImageProcessor is, and will only ever be supported on the .NET Framework running on a Windows OS" ... it doesn't appear to be actively maintained and is effectively labeled as deprecated, so I won't spend much time looking further into it.

ImageSharp

As of the time of writing, ImageSharp does not support TIF format, but it appears likely to be supported in a future release.

System.Drawing

Although this library can save images at different depths, it can only load image files with 8-bit depths. System.Drawing does not support loading 16-bit TIFs, so another library must be used to work with these file types.

Resources

Markdown source code last modified on September 12th, 2021
---
Title: Working with 16-bit Images in CSharp
Description: A summary of how I work with 16-bit TIF file data in C# (using ImageMagick and LibTiff).
Date: 2021-06-08 11PM EST
Tags: csharp
---

# Working with 16-bit Images in CSharp

**Scientific image analysis frequently involves working with 12-bit and 14-bit sensor data stored in 16-bit TIF files.** Images commonly encountered on the internat are 24-bit or 32-bit RGB images (where each pixel is represented by 8 bits each for red, green, blue, and possibly alpha). Typical image analysis libraries and documentation often lack information about how to work with 16-bit image data. 

**This page summarizes how I work with 16-bit TIF file data in C#.** I prefer [Magick.NET](https://www.nuget.org/packages?q=magick) (an ImageMagick wrapper) when working with many different file formats, and [LibTiff.Net](https://bitmiracle.com/libtiff) whenever I know my source files will all be identically-formatted TIFs or multidimensional TIFs (stacks).

[](https://github.com/swharden/Csharp-Image-Analysis/blob/main/dev/notes.md)

## ImageMagick

[ImageMagick](https://imagemagick.org/index.php) is a free and open-source cross-platform software suite for displaying, creating, converting, modifying, and editing raster images. Although ImageMagick is commonly used at the command line, .NET wrappers exist to make it easy to use ImageMagick from within C# applications.

ImageMagick has [many packages on NuGet](https://www.nuget.org/packages?q=imagemagick) and they are described on ImageMagick's [documentation](https://github.com/dlemstra/Magick.NET/tree/main/docs) GitHub page. **TLDR: Install the Q16 package (not HDRI)** to allow you to work with 16-bit data without losing precision.

ImageMagick is free and distributed under the Apache 2.0 license, so it can easily be used in commercial projects.

An advantage of loading images with ImageMagick is that it will work easily whether the source file is a JPG, PNG, GIF, TIF, or something different. ImageMagick supports over 100 file formats!

### Load a 16-bit TIF File as a Pixel Value Array

```cs
// Load pixel values from a 16-bit TIF using ImageMagick (Q16)
MagickImage image = new MagickImage("16bit.tif");
ushort[] pixelValues = image.GetPixels().GetValues();
```

That's it! The `pixelValues` array will contain one value per pixel from the original image. The length of this array will equal the image's height times its width.

### Load an 8-bit TIF File as a Pixel Value Array
Since the `Q16` package was installed, 2 bytes will be allocated for each pixel (16-bit) even if it only requires one byte (8-bit). In this case you must collect just the bytes you are interested in:

```cs
MagickImage image = new MagickImage("8bit.tif");
ushort[] pixelValues = image.GetPixels().GetValues();
int[] values8 = Enumerable.Range(0, pixelValues.Length / 2).Select(x => (int)pixelValues[x * 2 + 1]).ToArray();
```

### Load a 32-bit TIF File as a Pixel Value Array

For this you'll have to install the high dynamic range (HDRI) Q16 package, then your `GetValues()` method will return a `float[]` instead of a `ushort[]`. Convert these values to proper pixel intensity values by dividing by 2^16.

```cs
MagickImage image = new MagickImage("32bit.tif");
float[] pixels = image.GetPixels().GetValues();
for (int i = 0; i < pixels.Length; i++)
    pixels[i] = (long)pixels[i] / 65535;
```

## LibTiff

**LibTiff is a pure C# (.NET Standard) TIF file reader.** Although it doesn't support all the image formats that ImageMagick does, it's really good at working with TIFs. It has a more intuitive interface for working with TIF-specific features such as multi-dimensional images (color, Z position, time, etc.). 

LibTiff gives you a lower-level access to the bytes that underlie image data, so it's on you to perform the conversion from a byte array to the intended data type. Note that some TIFs are little-endian encoded and others are big-endian encoded, and endianness can be read from the header.

LibTiff is distributed under a BSD 3-clause license, so it too can be easily used in commercial projects.

### Load a 16-bit TIF as a Pixel Value Array

```cs
// Load pixel values from a 16-bit TIF using LibTiff
using Tiff image = Tiff.Open("16bit.tif", "r");

// get information from the header
int width = image.GetField(TiffTag.IMAGEWIDTH)[0].ToInt();
int height = image.GetField(TiffTag.IMAGELENGTH)[0].ToInt();
int bytesPerPixel = image.GetField(TiffTag.BITSPERSAMPLE)[0].ToInt() / 8;

// read the image data bytes
int numberOfStrips = image.NumberOfStrips();
byte[] bytes = new byte[numberOfStrips * image.StripSize()];
for (int i = 0; i < numberOfStrips; ++i)
    image.ReadRawStrip(i, bytes, i * image.StripSize(), image.StripSize());

// convert the data bytes to a double array
if (bytesPerPixel != 2)
    throw new NotImplementedException("this is only for 16-bit TIFs");
double[] data = new double[bytes.Length / bytesPerPixel];
for (int i = 0; i < data.Length; i++)
{
    if (image.IsBigEndian())
        data[i] = bytes[i * 2 + 1] + (bytes[i * 2] << 8);
    else
        data[i] = bytes[i * 2] + (bytes[i * 2 + 1] << 8);
}
```

Routines for detecting and converting data from 8-bit, 24-bit, and 32-bit TIF files can be created by inspecting `bytesPerPixel`. LibTiff has [documentation](https://bitmiracle.com/libtiff/) describing how to work with RGB TIF files and multi-frame TIFs.

## Convert a Pixel Array to a 2D Array

I often prefer to work with scientific image data as a 2D arrays of `double` values. I write my analysis routines to pass `double[,]` between methods so the file I/O can be encapsulated in a static class.

```cs
// Load pixel values from a 16-bit TIF using ImageMagick (Q16)
MagickImage image = new MagickImage("16bit.tif");
ushort[] pixelValues = image.GetPixels().GetValues();

// create a 2D array of pixel values
double[,] imageData = new double[image.Height, image.Width];
for (int i = 0; i < image.Height; i++)
    for (int j = 0; j < image.Width; j++)
        imageData[i, j] = pixelValues[i * image.Width + j];
```

## Other Libraries

### ImageProcessor

According to [ImageProcessor's GitHub page](https://github.com/JimBobSquarePants/ImageProcessor), "ImageProcessor is, and will **only ever be supported on the .NET Framework running on a Windows OS**" ... it doesn't appear to be actively maintained and is effectively labeled as deprecated, so I won't spend much time looking further into it.

### ImageSharp

As of the time of writing, [ImageSharp](https://docs.sixlabors.com/articles/imagesharp/imageformats.html) does not support TIF format, but it appears likely to be supported in a future release.

### System.Drawing

Although this library can _save_ images at different depths, it can only _load_ image files with 8-bit depths. System.Drawing does not support loading 16-bit TIFs, so another library must be used to work with these file types.

## Resources

* [C# Image Analysis](https://github.com/swharden/Csharp-Image-Analysis) (GitHub) - a collection of code examples for working with image data as 2D arrays
* [LibTiff.Net](https://bitmiracle.com/libtiff/) - The .NET version of original libtiff library
* [dlemstra/Magick.NET](https://github.com/dlemstra/Magick.NET) - The .NET library for ImageMagick
* [ImageSharp](https://docs.sixlabors.com/articles/imagesharp/imageformats.html) - Supported image formats
June 3rd, 2021

Representing Images in Memory

This page is a quick reference for programmers interested in working with image data in memory (byte arrays). This topic is straightforward overall, but there are a few traps that aren't necessarily intuitive so I try my best to highlight those here.

This article assumes you have some programming experience working with byte arrays in a C-type language and have an understanding of what is meant by 32-bit, 24-bit, 16-bit, and 8-bit integers.

Pixel Values

An image is composed of a 2D grid of square pixels, and the type of image greatly influences how much memory each pixel occupies and what format its data is in.

Bits per pixel (bpp) is the number of bits it takes to represent the value a single pixel. This is typically a multiple of 8 bits (1 byte).

Common Pixel Formats

  • 8-bit (1 byte) Pixel Formats
    • Gray 8 - Specifies one of 2^8 (256) shades of gray.
    • Indexed 8 - The pixel data contains color-indexed values, which means the values are an index to colors in the system color table, as opposed to individual color values.
  • 16-bit (2-byte) Pixel Formats
    • ARGB 1555 - Specifies one of 2^15 (32,768) shades of color (5 bits each for red, green, and blue) and 1 bit for alpha.
    • Gray 16 - Specifies one of 2^16 (65,536) shades of gray.
    • RGB 555 - 5 bits each are used for the red, green, and blue components. The remaining bit is not used.
    • RGB 565 - 5 bits are used for the red component, 6 bits are used for the green component, and 5 bits are used for the blue component. The additional green bit doubles the number of gradations and improves image perception in most humans.
  • 24-bit (3 byte) Pixel Formats
    • RGB 888 - 8 bits each are used for the red, green, and blue components.
  • 32-bit (4-byte) Pixel Formats
    • ARGB - 8 bits each are used for the alpha, red, green, and blue components. This is the most common pixel format.

There are others (e.g., 64-bit RGB images), but these are the most typically encountered pixel formats.

Endianness

Endianness describes the order of bytes in a multi-byte value uses to store its data:

  • big-endian: the smallest address contains the most significant byte

  • little-endian: the smallest address contains the least significant byte

Assuming array index values ascend from left to right, 32-bit (4-byte) pixel data can be represented using either of these two formats in memory:

  • 4 bpp little-endian: [A, B, G, R] (most common)

  • 4 bpp big-endian: [R, G, B, A]

Bitmap images use little-endian integer format! New programmers may expect the bytes that contain "RGB" values to be sequenced in the order "R, G, B", but this is not the case.

Premultiplied Alpha

Premultiplication refers to the relationship between color (R, G, B) and transparency (alpha). In transparent images the alpha channel may be straight (unassociated) or premultiplied (associated).

With straight alpha, the RGB components represent the full-intensity color of the object or pixel, disregarding its opacity. Later R, G, and B will each be multiplied by the alpha to adjust intensity and transparency.

With premultiplied alpha, the RGB components represent the emission (color and intensity) of each pixel, and the alpha only represents transparency (occlusion of what is behind it). This reduces the computational performance for image processing if transparency isn't actually used.

In C# using System.Drawing premultiplied alpha is not enabled by default. This must be defined when creating new Bitmap as seen here:

var bmp = new Bitmap(400, 300, PixelFormat.Format32bppPArgb);

Benchmarking reveals the performance enhancement of drawing on bitmaps in memory using premultiplied alpha pixel format. In this test I'm using .NET 5 with the System.Drawing.Common NuGet package. Anti-aliasing is off in this example, but similar results were obtained with it enabled.

Random rand = new(0);
int width = 600;
int height = 400;
var bmp = new Bitmap(600, 400, PixelFormat.Format32bppPArgb);
var gfx = Graphics.FromImage(bmp);
var pen = new Pen(Color.Black);
gfx.Clear(Color.Magenta);
var sw = Stopwatch.StartNew();
for (int i = 0; i < 1e6; i++)
{
    pen.Color = Color.FromArgb(rand.Next());
    gfx.DrawLine(pen, rand.Next(width), rand.Next(height), rand.Next(width), rand.Next(height));
}
Console.WriteLine(sw.Elapsed);
bmp.Save("benchmark.png", ImageFormat.Png);

Time to render 1 million frames:

  • Standard ARGB: 6.77 ± 0.02 sec
  • Premultiplied ARGB: 5.83 ± 0.03 sec (14% faster)

At the end you have a beautiful figure:

Pixel Locations in Space and Memory

A 2D image is composed of pixels, but addressing them in memory isn't as trivial as it may seem. The dimensions of bitmaps are stored in their header, and the arrangement of pixels forms rows (left-to-right) then columns (top-to-bottom).

Width and height are the dimensions (in pixels) of the visible image, but...

⚠️ Image size in memory is not just width * height * bytesPerPixel

Because of old hardware limitations, bitmap widths in memory (also called the stride) must be multiplies of 4 bytes. This is effortless when using ARGB formats because each pixel is already 4 bytes, but when working with RGB images it's possible to have images with an odd number of bytes in each row, requiring data to be padded such that the stride length is a multiple of 4.

// calculate stride length of a bitmap row in memory
int stride = 4 * ((imageWidth * bytesPerPixel + 3) / 4);

Working with Bitmap Bytes in C#

This example demonstrates how to convert a 3D array (X, Y, C) into a flat byte array ready for copying into a bitmap. Notice this code adds padding to the image width to ensure the stride is a multiple of 4 bytes. Notice also the integer encoding is little endian.

public static byte[] GetBitmapBytes(byte[,,] input)
{
    int height = input.GetLength(0);
    int width = input.GetLength(1);
    int bpp = input.GetLength(2);
    int stride = 4 * ((width * bpp + 3) / 4);

    byte[] pixelsOutput = new byte[height * stride];
    byte[] output = new byte[height * stride];

    for (int y = 0; y < height; y++)
        for (int x = 0; x < width; x++)
            for (int z = 0; z < bpp; z++)
                output[y * stride + x * bpp + (bpp - z - 1)] = input[y, x, z];

    return output;
}

For completeness, here's the complimentary code that converts a flat byte array from bitmap memory to a 3D array (assuming we know the image dimensions and bytes per pixel from reading the image header):

public static byte[,,] GetBitmapBytes3D(byte[] input, int width, int height, int bpp)
{
    int stride = 4 * ((width * bpp + 3) / 4);

    byte[,,] output = new byte[height, width, bpp];
    for (int y = 0; y < height; y++)
        for (int x = 0; x < width; x++)
            for (int z = 0; z < bpp; z++)
                output[y, x, z] = input[stride * y + x * bpp + (bpp - z - 1)];

    return output;
}

Marshalling Bytes in and out of Bitmaps

The code examples above are intentionally simple to focus on the location of pixels in memory and the endianness of their values. To actually convert between byte[] and System.Drawing.Bitmap you must use Marshall.Copy as shown:

public static byte[] BitmapToBytes(Bitmap bmp)
{
    Rectangle rect = new(0, 0, bmp.Width, bmp.Height);
    BitmapData bmpData = bmp.LockBits(rect, ImageLockMode.ReadWrite, bmp.PixelFormat);
    int byteCount = Math.Abs(bmpData.Stride) * bmp.Height;
    byte[] bytes = new byte[byteCount];
    Marshal.Copy(bmpData.Scan0, bytes, 0, byteCount);
    bmp.UnlockBits(bmpData);
    return bytes;
}
public static Bitmap BitmapFromBytes(byte[] bytes, PixelFormat bmpFormat)
{
    Bitmap bmp = new(width, height, bmpFormat);
    var rect = new Rectangle(0, 0, width, height);
    BitmapData bmpData = bmp.LockBits(rect, ImageLockMode.ReadOnly, bmpFormat);
    Marshal.Copy(bytes, 0, bmpData.Scan0, bytes.Length);
    bmp.UnlockBits(bmpData);
    return bmp;
}

Reference

Markdown source code last modified on September 12th, 2021
---
Title: Representing Images in Memory
Description: quick reference for programmers interested in working with image data in memory (byte arrays)
Date: 2021-06-03 00:03:00 EST
Tags: csharp
---

# Representing Images in Memory

**This page is a quick reference for programmers interested in working with image data in memory (byte arrays).** This topic is straightforward overall, but there are a few traps that aren't necessarily intuitive so I try my best to highlight those here. 

This article assumes you have some programming experience working with byte arrays in a C-type language and have an understanding of what is meant by 32-bit, 24-bit, 16-bit, and 8-bit integers.

## Pixel Values

**An image is composed of a 2D grid of square pixels,** and the type of image greatly influences how much memory each pixel occupies and what format its data is in.

**Bits per pixel (bpp)** is the number of bits it takes to represent the value a single pixel. This is typically a multiple of 8 bits (1 byte).

### Common Pixel Formats

* **8-bit (1 byte)  Pixel Formats**
  * **`Gray 8`** - Specifies one of 2^8 (256) shades of gray.
  * **`Indexed 8`** - The pixel data contains color-indexed values, which means the values are an index to colors in the system color table, as opposed to individual color values.
* **16-bit (2-byte) Pixel Formats**
  * **`ARGB 1555`** - Specifies one of 2^15 (32,768) shades of color (5 bits each for red, green, and blue) and 1 bit for alpha.
  * **`Gray 16`** - Specifies one of 2^16 (65,536) shades of gray.
  * **`RGB 555`** - 5 bits each are used for the red, green, and blue components. The remaining bit is not used.
  * **`RGB 565`** - 5 bits are used for the red component, 6 bits are used for the green component, and 5 bits are used for the blue component. The additional green bit doubles the number of gradations and improves image perception in most humans.
* **24-bit (3 byte) Pixel Formats**
  * **`RGB 888`** - 8 bits each are used for the red, green, and blue components.
* **32-bit (4-byte) Pixel Formats**
  * **`ARGB`** - 8 bits each are used for the alpha, red, green, and blue components. This is the most common pixel format.

There are others (e.g., 64-bit RGB images), but these are the most typically encountered pixel formats.

### Endianness

[Endianness](https://en.wikipedia.org/wiki/Endianness) describes the order of bytes in a multi-byte value uses to store its data:

* **big-endian:** the smallest address contains the most significant byte

* **little-endian:** the smallest address contains the least significant byte

Assuming array index values ascend from left to right, 32-bit (4-byte) pixel data can be represented using either of these two formats in memory:

* 4 bpp little-endian: **`[A, B, G, R]`** (most common)

* 4 bpp big-endian: **`[R, G, B, A]`**

**Bitmap images use little-endian integer format!** New programmers may expect the bytes that contain "RGB" values to be sequenced in the order "R, G, B", but this is not the case.

### Premultiplied Alpha

**Premultiplication refers to the relationship between color (R, G, B) and transparency (alpha).** In transparent images the alpha channel may be straight (unassociated) or premultiplied (associated).

With **straight alpha**, the RGB components represent the full-intensity color of the object or pixel, disregarding its opacity. Later R, G, and B will each be multiplied by the alpha to adjust intensity and transparency.

With **premultiplied alpha**, the RGB components represent the emission (color and intensity) of each pixel, and the alpha only represents transparency (occlusion of what is behind it). This reduces the computational performance for image processing if transparency isn't actually used.

**In C# using `System.Drawing` premultiplied alpha is not enabled by default.** This must be defined when creating new `Bitmap` as seen here:

```cs
var bmp = new Bitmap(400, 300, PixelFormat.Format32bppPArgb);
```

**Benchmarking reveals the performance enhancement** of drawing on bitmaps in memory using premultiplied alpha pixel format. In this test I'm using .NET 5 with the [System.Drawing.Common](https://www.nuget.org/packages/System.Drawing.Common) NuGet package. Anti-aliasing is off in this example, but similar results were obtained with it enabled.

```cs
Random rand = new(0);
int width = 600;
int height = 400;
var bmp = new Bitmap(600, 400, PixelFormat.Format32bppPArgb);
var gfx = Graphics.FromImage(bmp);
var pen = new Pen(Color.Black);
gfx.Clear(Color.Magenta);
var sw = Stopwatch.StartNew();
for (int i = 0; i < 1e6; i++)
{
    pen.Color = Color.FromArgb(rand.Next());
    gfx.DrawLine(pen, rand.Next(width), rand.Next(height), rand.Next(width), rand.Next(height));
}
Console.WriteLine(sw.Elapsed);
bmp.Save("benchmark.png", ImageFormat.Png);
```

Time to render 1 million frames:
* Standard ARGB: 6.77 ± 0.02 sec
* Premultiplied ARGB: 5.83 ± 0.03 sec (14% faster)

At the end you have a beautiful figure:

<div class="text-center img-border">

![](benchmark.png)

</div>

## Pixel Locations in Space and Memory

**A 2D image is composed of pixels, but addressing them in memory isn't as trivial as it may seem.** The dimensions of bitmaps are stored in their header, and the arrangement of pixels forms rows (left-to-right) then columns (top-to-bottom). 

**Width** and **height** are the dimensions (in pixels) of the visible image, but...

⚠️ **Image size in memory is _not_ just `width * height * bytesPerPixel`**

Because of old hardware limitations, bitmap widths _in memory_ (also called the **stride**) must be multiplies of 4 bytes. This is effortless when using ARGB formats because each pixel is already 4 bytes, but when working with RGB images it's possible to have images with an odd number of bytes in each row, requiring data to be **padded** such that the stride length is a multiple of 4.

<div class="text-center">

![](image-byte-position.png)

</div>

```cs
// calculate stride length of a bitmap row in memory
int stride = 4 * ((imageWidth * bytesPerPixel + 3) / 4);
```

### Working with Bitmap Bytes in C# 

**This example demonstrates how to convert a 3D array (X, Y, C) into a flat byte array ready for copying into a bitmap.** Notice this code adds padding to the image width to ensure the stride is a multiple of 4 bytes. Notice also the integer encoding is little endian.

```cs
public static byte[] GetBitmapBytes(byte[,,] input)
{
    int height = input.GetLength(0);
    int width = input.GetLength(1);
    int bpp = input.GetLength(2);
    int stride = 4 * ((width * bpp + 3) / 4);

    byte[] pixelsOutput = new byte[height * stride];
    byte[] output = new byte[height * stride];

    for (int y = 0; y < height; y++)
        for (int x = 0; x < width; x++)
            for (int z = 0; z < bpp; z++)
                output[y * stride + x * bpp + (bpp - z - 1)] = input[y, x, z];

    return output;
}
```

For completeness, here's the complimentary code that converts a flat byte array from bitmap memory to a 3D array (assuming we know the image dimensions and bytes per pixel from reading the image header):

```cs
public static byte[,,] GetBitmapBytes3D(byte[] input, int width, int height, int bpp)
{
    int stride = 4 * ((width * bpp + 3) / 4);

    byte[,,] output = new byte[height, width, bpp];
    for (int y = 0; y < height; y++)
        for (int x = 0; x < width; x++)
            for (int z = 0; z < bpp; z++)
                output[y, x, z] = input[stride * y + x * bpp + (bpp - z - 1)];

    return output;
}
```

### Marshalling Bytes in and out of Bitmaps

The code examples above are intentionally simple to focus on the location of pixels in memory and the endianness of their values. To actually convert between `byte[]` and `System.Drawing.Bitmap` you must use `Marshall.Copy` as shown:

```cs
public static byte[] BitmapToBytes(Bitmap bmp)
{
    Rectangle rect = new(0, 0, bmp.Width, bmp.Height);
    BitmapData bmpData = bmp.LockBits(rect, ImageLockMode.ReadWrite, bmp.PixelFormat);
    int byteCount = Math.Abs(bmpData.Stride) * bmp.Height;
    byte[] bytes = new byte[byteCount];
    Marshal.Copy(bmpData.Scan0, bytes, 0, byteCount);
    bmp.UnlockBits(bmpData);
    return bytes;
}
```

```cs
public static Bitmap BitmapFromBytes(byte[] bytes, PixelFormat bmpFormat)
{
    Bitmap bmp = new(width, height, bmpFormat);
    var rect = new Rectangle(0, 0, width, height);
    BitmapData bmpData = bmp.LockBits(rect, ImageLockMode.ReadOnly, bmpFormat);
    Marshal.Copy(bytes, 0, bmpData.Scan0, bytes.Length);
    bmp.UnlockBits(bmpData);
    return bmp;
}
```

## Reference

* [A programmer's view on digital images: the essentials](https://www.collabora.com/news-and-blog/blog/2016/02/16/a-programmers-view-on-digital-images-the-essentials/)
* [Microsoft: `System.Drawing.PixelFormat` Enumeration](https://docs.microsoft.com/en-us/dotnet/api/system.drawing.imaging.pixelformat?view=net-5.0) - describes pixel formats supported by this common drawing library
* [Creighton University: Tip of the week (June 2014) - ](https://medschool.creighton.edu/fileadmin/user/medicine/Departments/Biomedical_Sciences/CUIBIF/Tip_of_the_week/June_2014/4096_Shades_of_Gray.pdf) inspired the title of this article.
* [8, 12, 14 vs 16-Bit Depth: What Do You Really Need?!](https://petapixel.com/2018/09/19/8-12-14-vs-16-bit-depth-what-do-you-really-need/)
* [Premultiplied alpha](https://shawnhargreaves.com/blog/premultiplied-alpha.html) by Shawn Hargreaves (2009)
* [Straight versus premultiplied alpha](https://en.wikipedia.org/wiki/Alpha_compositing#Straight_versus_premultiplied) on Wikipedia
Pages