The personal website of Scott W Harden
April 4th, 2022

Mystify your Mind with SkiaSharp

This article explores my recreation of the classic screensaver Mystify your Mind implemented using C#. I used SkiaSharp to draw graphics and FFMpegCore to encode frames into high definition video files suitable for YouTube.

The Mystify Sandbox application has advanced options allowing exploration of various configurations outside the capabilities of the original screensaver. Interesting configurations can be exported as video (x264-encoded MP4 or WebM format) or viewed in full-screen mode resembling an actual screensaver.

Download

Programming Strategy

  • Corner - tracks point that bounces around the edges of the screen
    • Has Position and Velocity fields
    • Has Advance() to move points collide with edges
  • Wire - represents a single polygon that moves around the screen
    • Contains List<Corner> and a Color which all change over time
    • Has Advance() which advances all corner and cycles Color.
    • Contains List<WireSnapshot> to record history
  • WireSnapshot - represents properties of a Wire at an instant in time
    • Contains Point[] and Color and is intended to be immutable
    • Can draw itself using a Draw() method that accepts a SKCanvas
  • Field - represents the whole animation
    • Contains List<Wire> and has Width and Height
    • Has Advance() which advances all wires
    • Can draw itself using a Draw() method that accepts a SKCanvas

Original Behavior

Close inspection of video from the original Mystify screensaver revealed notable behaviors.

Broken Lines

The original Mystify implementation did not clear the screen and between every frame. With GDI large fills (clearing the background) are expensive, and drawing many polygons probably challenged performance in the 90s. Instead only the leading wire was drawn, and the trailing wire was drawn-over using black. This strategy results in lines which appear to have single pixel breaks on a black background (magenta arrow). It may not have been particularly visible on CRT monitors available in the 90s, but it is quite noticeable on LCD screens today.

Bouncing Changes Speed

Observing videos of the classic screensaver I noticed that corners don't bounce symmetrically off edges. After every bounce they change their speed slightly. This can be seen by observing the history of corners which reflect off edges of the screen demonstrating their change in speed (green arrow). I recreated this behavior using a weighted random number generator.

Programming Notes

Color Cycling

I used a HSL-to-RGB method to generate colors from hue (variable), saturation (always 100%), and luminosity (always 50%). By repeatedly ramping hue from 0% to 100% slowly I achieved a rainbow gradient effect. Increasing the color change speed (% change for every new wire) cycles the colors faster, and very high values produce polygons whose visible history spans a gradient of colors. Fade effect is achieved by increasing alpha of wire snapshots as they are drawn from old to new.

Encoding video with C

The FFMpegCore package is a C# wrapper for FFMpeg that can encode video from frames piped into it. Using this strategy required creation of a SkiaSharp.SKBitmap wrapper that implements FFMpegCore.Pipes.IVideoFrame. For a full explaination and example code see C# Data Visualization: Render Video with SkiaSharp.

Performance

It's amusing to see retro screensavers running on modern gear! I can run this graphics model simulation at full-screen resolutions using thousands of wires at real-time frame rates. The most natural density of shapes for my 3440x1440 display was 20 wires with a history of 5.

Rendering the 2D image and encoding HD video using the x264 codec occupies all my CPU cores and runs a little above 500 frames per second. Encoding 24 hours of video (over 2 million frames) took this system 1 hour and 12 minutes and produced a 15.3 GB MP4 file. Encoding WebM format is considerably slower, with the same system only achieving an encoding rate of 12 frames per second.

Simulations

Traditional Behavior

The classic screensaver is typically run with two 4-cornered polygons that slowly change color.

Rainbow

Increasing the rate of color transition produces a rainbow effect within the visible history of polygons. The effect is made more striking by increasing the history length and decreasing the speed so the historical lines are closer together.

Solid

If the speed is greatly decreased and the number of historical records is greatly increased the resulting shape has little or no gap between historical traces and appears like a solid object. If fading is enabled (where opacity of older traces fades to transparent) the resulting effect is very interesting.

Chaos

Adding 100 shapes produces a chaotic but interesting effect. This may be the first time the world has seen Mystify like this!

EDIT: All these lines are very stressful on the video encoder and produce large file sizes to achieve high quality (25 MB for 10 seconds). I'm showing this one as a JPEG but click here to view mystify-100.webm if you're on a good internet connection.

YouTube

Resources

Markdown source code last modified on April 9th, 2022
---
title: Mystify your Mind with SkiaSharp
description: My implementation of the classic screensaver using SkiaSharp, OpenGL, and FFMpeg
date: 2022-04-04 18:34:00
tags: csharp, graphics
---

# Mystify your Mind with SkiaSharp

**This article explores my recreation of the classic screensaver _Mystify your Mind_ implemented using C#.** I used [SkiaSharp](https://github.com/mono/SkiaSharp) to draw graphics and [FFMpegCore](https://github.com/rosenbjerg/FFMpegCore) to encode frames into high definition video files suitable for YouTube.

<div class="text-center">

![](mystify.gif)

</div>

**The Mystify Sandbox application has advanced options** allowing exploration of various configurations outside the capabilities of the original screensaver. Interesting configurations can be exported as video (x264-encoded MP4 or WebM format) or viewed in full-screen mode resembling an actual screensaver. 

![](mystify-advanced.jpg)

## Download
* The [Releases page](https://github.com/swharden/Mystify/releases) has a click-to-run EXE for Windows
* [GitHub.com/swharden/Mystify](https://github.com/swharden/Mystify/) contains project source code (C#/.NET6)

## Programming Strategy

* `Corner` - tracks point that bounces around the edges of the screen
  * Has `Position` and `Velocity` fields
  * Has `Advance()` to move points collide with edges
* `Wire` - represents a single polygon that moves around the screen
  * Contains `List<Corner>` and a `Color` which all change over time
  * Has `Advance()` which advances all corner and cycles `Color`.
  * Contains `List<WireSnapshot>` to record history
* `WireSnapshot` - represents properties of a `Wire` at an instant in time
  * Contains `Point[]` and `Color` and is intended to be immutable
  * Can draw itself using a `Draw()` method that accepts a `SKCanvas`
* `Field` - represents the whole animation
  * Contains `List<Wire>` and has `Width` and `Height`
  * Has `Advance()` which advances all wires
  * Can draw itself using a `Draw()` method that accepts a `SKCanvas`

## Original Behavior

Close inspection of [video from the original](https://youtu.be/SaBvcHHdlGE) Mystify screensaver revealed notable behaviors.

<img src="mystify-inspection.jpg" class="d-block shadow mx-auto my-5">

### Broken Lines
The original Mystify implementation did not clear the screen and between every frame. With GDI large fills (clearing the background) are expensive, and drawing many polygons probably challenged performance in the 90s. Instead only the leading wire was drawn, and the trailing wire was drawn-over using black. This strategy results in lines which appear to have single pixel breaks on a black background (magenta arrow). It may not have been particularly visible on CRT monitors available in the 90s, but it is quite noticeable on LCD screens today.

### Bouncing Changes Speed
Observing videos of the classic screensaver I noticed that corners don't bounce symmetrically off edges. After every bounce they change their speed slightly. This can be seen by observing the history of corners which reflect off edges of the screen demonstrating their change in speed (green arrow). I recreated this behavior using a weighted random number generator.

## Programming Notes

### Color Cycling
I used a HSL-to-RGB method to generate colors from hue (variable), saturation (always 100%), and luminosity (always 50%). By repeatedly ramping hue from 0% to 100% slowly I achieved a rainbow gradient effect. Increasing the color change speed (% change for every new wire) cycles the colors faster, and very high values produce polygons whose visible history spans a gradient of colors. Fade effect is achieved by increasing alpha of wire snapshots as they are drawn from old to new.

### Encoding video with C#
The FFMpegCore package is a C# wrapper for FFMpeg that can encode video from frames piped into it. Using this strategy required creation of a `SkiaSharp.SKBitmap` wrapper that implements `FFMpegCore.Pipes.IVideoFrame`. For a full explaination and example code see [C# Data Visualization: Render Video with SkiaSharp](https://swharden.com/csdv/skiasharp/video/).

### Performance

**It's amusing to see retro screensavers running on modern gear!** I can run this graphics model simulation at full-screen resolutions using thousands of wires at real-time frame rates. The most natural density of shapes for my 3440x1440 display was 20 wires with a history of 5.

<img src="desk.jpg" class="d-block shadow mx-auto my-5">

Rendering the 2D image and encoding HD video using the x264 codec occupies all my CPU cores and runs a little above 500 frames per second. Encoding 24 hours of video (over 2 million frames) took this system 1 hour and 12 minutes and produced a 15.3 GB MP4 file. Encoding WebM format is considerably slower, with the same system only achieving an encoding rate of 12 frames per second.

<img src="cpu.png" class="d-block mx-auto my-5">


## Simulations

### Traditional Behavior

The classic screensaver is typically run with two 4-cornered polygons that slowly change color.

<video width="759" height="470" controls class="d-block mx-auto my-5 shadow" style="max-width: 100%; height: 100%;">
  <source src="mystify-01-standard.webm" type="video/mp4">
</video>

### Rainbow

Increasing the rate of color transition produces a rainbow effect within the visible history of polygons. The effect is made more striking by increasing the history length and decreasing the speed so the historical lines are closer together.

<video width="759" height="470" controls class="d-block mx-auto my-5 shadow" style="max-width: 100%; height: 100%;">
  <source src="mystify-02-rainbow.webm" type="video/mp4">
</video>

### Solid

If the speed is greatly decreased and the number of historical records is greatly increased the resulting shape has little or no gap between historical traces and appears like a solid object. If fading is enabled (where opacity of older traces fades to transparent) the resulting effect is very interesting.

<video width="759" height="470" controls class="d-block mx-auto my-5 shadow" style="max-width: 100%; height: 100%;">
  <source src="mystify-03-solid.webm" type="video/mp4">
</video>

### Chaos

Adding 100 shapes produces a chaotic but interesting effect. This may be the first time the world has seen Mystify like this!

_EDIT: All these lines are very stressful on the video encoder and produce large file sizes to achieve high quality (25 MB for 10 seconds). I'm showing this one as a JPEG but [click here to view mystify-100.webm](mystify-04-100.webm) if you're on a good internet connection._

<a href='mystify-04-100.webm'><img src="mystify-04-100.jpg" class="d-block mx-auto my-5 shadow"></a>

## YouTube

<div class="text-center">

![](https://youtu.be/queN9r3Leis)

</div>

## Resources
* A click-to-run EXE can be downloaded from the [Releases Page](https://github.com/swharden/Mystify/releases)
* Source Code is available on https://github.com/swharden/Mystify
* Implementation Details: [C# Data Visualization: Mystify](https://swharden.com/csdv/simulations/mystify/)
* [C# Data Visualization: Render Video with SkiaSharp](https://swharden.com/csdv/skiasharp/video/)
* GitHub: [SkiaSharp](https://github.com/mono/SkiaSharp)
* GitHub: [FFMpegCore](https://github.com/rosenbjerg/FFMpegCore) 
* Windows 3.1 Mystify (video): https://youtu.be/osCZyfoScFg?t=370
* Windows 95 Mystify (video): https://youtu.be/SaBvcHHdlGE