Curve Fitting in C# using Particle Swarm Optimization

This article describes how to fit arbitrary curves with any number of parameters to X/Y data points using C#. The strategy described here uses particle swarm optimization to iteratively work toward an optimal solution. Unlike other curve fitting strategies, the first derivative of the error function does not need to be calculated mathematically, making it particularly useful for problems where the solution landscape is noisy, nonlinear, or completely unknown. Users only need to provide a function and the limits for each parameter, and the swarm will converge on the optimal solution.

SwarmFit is a free and open-source NuGet package that makes it easy to fit arbitrary curves to data. I created SwarmFit after researching the topic of particle swarm optimization in preparation for this article. Although this page details how particle swarm optimization works under the hood, SwarmFit be used without any knowledge of the topic.

Quickstart

• Create a new .NET project: dotnet new console

• Add the SwarmFit NuGet package: dotnet add SwarmFit

• Add the following to Program.cs then dotnet run

// data points to fit
double[] xs = [1, 2, 3, 4, 5];
double[] ys = [304, 229, 174, 134, 111];

// a custom function to fit to the data using any number of parameters
static double MyFitFunc(double x, double[] parameters)
{
	double a = parameters[0];
	double b = parameters[1];
	double c = parameters[2];
	return a + b * Math.Exp(x * c);
}

// the minimum and maximum value for each parameter
double[] minVars = [-100, -5000, -10];
double[] maxVars = [100, 5000, 10];

// perform the fit with general purpose settings
double[] solution = QuickFit.Solve(xs, ys, MyFitFunc, minVars, maxVars);

// display the solution
double a = solution[0];
double b = solution[1];
double c = solution[2];
Console.WriteLine($"Y = {a} + {b} * e^(x * {c})");

Advanced users may wish to use F12 to peek inside the QuickFit.Solve() method to see how it creates a SwarmFitter, initializes it, and runs the solver. The SwarmFitter has many properties which may be adjusted to change the behavior of the fitter for enhanced logging, improved performance, or potentially better fits. See the SwarmFit GitHub repository for additional information.

Theory

Particle swarm optimization is a type of swarm intelligence algorithm that represents candidate solutions as particles in a swarm that all work together to minimize the error function. In this strategy, a set of parameter values is represented by a particle, and a small swarm of particles iteratively works toward a solution by each of their parameters to minimize the total difference between the data points and the curve generated by those parameters.

Particles move toward the optimal solution by combining their own values and recent experience with information gathered from neighboring particles. The combination of self-guided and socially guided behavior allows the swarm to explore a broad range of solutions while progressively converging onto the best one. Randomness is added to the system by randomly killing particles and spawning new ones, increasing the chances that particles will find the global minimum and not get trapped in a local one.

Implementation

Here’s a summary of how particle swarm optimization may be implemented in C#. Full source code may be reviewed from the SwarmFit GitHub repository and additional information is available in the resources section at the bottom of the page.

Preparation

• Data points: Start with a collection of X/Y points represented by xs[] and ys[] (two arrays of equal length).

• Curve formula: The user must create a function which returns Y given an X position and a collection of parameters. In this example we’ll assume parameters are in an array named vars[]. In C# a static function can be represented by Func<> making it easy to pass into a fitter class as a parameter.

• Error function: The fitter must be able to evaluate how well a set of vars[] performs, so predict the Y for every X in the dataset. This can be achieved by returning the sum of the differences between each X’s actual Y value and the predicted Y value obtained from the formula given that X and the current parameters.

• Parameter limits: The minimum and maximum possible limits for each parameter must be prepared by the user. These limits are used when initializing the swarm (placing particles randomly across the whole field) and also when adding randomness to the system (randomly displacing particles by a distance relative to the field size).

Optimization Steps

• Initialize the swarm: Create a population of particles randomly placed on the field and given a random movement direction.

• Evaluate fitness: For each particle, calculate how well the curve defined by its parameters fits the data.

• Track best positions: Each particle keeps track of its own best known position (the one with the smallest error), and the optimizer keeps track of the best known position across all particles.

• Update movement vector: Adjust the direction and speed of each particle based on its own best solution and the best solution found by the swarm.

• Move particles: Move each particle forward according to its new direction and speed. This new set of particles represents new and potentially improved solutions.

• Add randomness: Randomly kill particles and add new ones. This step is optional, but I find it particularly interesting. It is recommended to place new particles a random distance from the population mean which is proportional to the variance of the population.

• Iterate: Repeat the process of adjusting movement, updating positions, and evaluating fitness over several iterations, allowing the swarm to progressively improve its collective solution.

• Return the best solution: Once a certain number of iterations is reached, the best set of curve parameters is returned as the final solution.

Resources

• SwarmFit source code on GitHub

• SwarmFit NuGet Package

• ScottPlot is a useful library for creating 2D charts in C#

• Particle Swarm Optimization Using C# - Visual Studio Magazine article by James McCaffrey published in 2013

• Swarm Intelligence Optimization (YouTube) Talk by James McCaffrey at PyData Seattle 2015

• Particle Swarm Optimisation (YouTube) by Craig Ferguson

• Particle Swarm Optimization (YouTube) by Ali Mirjalili

• ParticleSwarmSharp - A .NET library for building apps that use particle swarm optimization. Warning: this project has a GPL license

• Boids in C# - A C# implementation of the Boids flocking algorithm

• Blazor Boids (launch) - A Blazor app that demonstrates the Boids flocking algorithm in your browser

• Simple Exponential Fit in C# using linear least squares optimization