Parallel design in C#

Photo by Glenn Carstens-Peters on Unsplash

Parallel design is one of the methods that using in multi-core CPU applications. Here have some experiment result share with you.

What is the difference between parallel design and normal sequential design in C#.

Sequential type

for (int i = 0; i < 500; i++)
{
   DoSomeStuff();
}

Parallel type

Parallel.For(0, 500, i =>
{
   DoSomeStuff();
});

let’s do some experiments to see does it works or not.

• Let’s check the processing time

in DoSomeStuff, I do something to make the system busy

double DoStuffs(int cycle)
{
    var ret = .0;
    for (int i = 0; i < cycle; i++)
    {
        ret += Math.PI;
    }
    return ret;
}

Sequential code will be 2 loops and counting the elapsed time

var stopwatch = new Stopwatch();
stopwatch.Start();
double ret = 0;
for (int i = 0; i < 500; i++)
{
    for (int j = 0; j < 500; j++)
    {
        ret += DoStuffs(cycle);
    }
}
stopwatch.Stop();
Console.WriteLine($"{ret}, {stopwatch.ElapsedMilliseconds}");

Same as Parallel code, it just replaces the first loop as a parallel method.

var stopwatch = new Stopwatch();
stopwatch.Start();
double ret = 0;
Parallel.For(0, 500, i =>
{
    for (int j = 0; j < 500; j++)
    {
        ret += DoStuffs(cycle);
    }
});
stopwatch.Stop();
Console.WriteLine($"{ret}, {stopwatch.ElapsedMilliseconds}");

The elapsed time of results is as below

╔═══════════╦════════════════════╦═══════════════╗
║ Cycling   ║  Sequential (ms)   ║ Parallel(ms)  ║
╠═══════════╬════════════════════╬═══════════════╣
║ 100       ║     98             ║     245       ║
║ 500       ║     432            ║     827       ║
║ 1000      ║     1131           ║     425       ║
║ 5000      ║     4214           ║     1133      ║
╚═══════════╩════════════════════╩═══════════════╝

In cycling time < 500, the Sequential time is even better than the Parallel one. the reason is that DoSomeStuff is only getting the Math.PI data and add up. In Parallel method even needs resources to prepare the thread for processing. Let’s try another thing, make the DoSomeStuff busier and verify again.

double DoStuffs(int cycle)
{
    var ret = .0;
    for (int i = 0; i < cycle; i++)
    {
        ret += Math.Sin(i) + Math.Cos(i);
    }
    return ret;
}
╔═══════════╦════════════════════╦═══════════════╗
║ Cycling   ║  Sequential (ms)   ║ Parallel(ms)  ║
╠═══════════╬════════════════════╬═══════════════╣
║ 100       ║     792            ║     655       ║
║ 500       ║     3486           ║     1116      ║
║ 1000      ║     6097           ║     1886      ║
║ 5000      ║     24915          ║     10674     ║
╚═══════════╩════════════════════╩═══════════════╝

The Parallel method in 100 cycling time is already lowered than the Sequential one.

Let’s check the CPU status, I make the Sequential run first. After finishing the method, the system will wait 5 seconds and then execute Parallel one.

As for the CPU usage, the first two peaks are contributed by the Sequential one the other peak is by the Parallel one. Both methods cause a high peak, however, the Parallel one can be a shorter time effect.

You have think what not also change second loop as Parallel type as below:

Parallel.For(0, 500, i =>
{
    Parallel.For(0, 500, j=>
    {
        ret += DoStuffs(cycle);
    });
});

Bascilly, it already does not have too much different compare with one Parallel type. System already optimate in your first Parallel type.

There have a point needs very carefully. If you do more test, you can find the “ret” that is not the same between Sequential and Parallel.

Let’s sample “DoStuffs” it and give a try. As the result, I only get the same result in first run.

double DoStuffs(int cycle)
{
    var ret = .0;
    for (int i = 0; i < cycle; i++)
    {
        ret = 1;
    }
    return ret;
}
╔═══════════╦════════════════════╦═══════════════╗
║ Cycling   ║  Sequential (ret)  ║ Parallel(ret) ║
╠═══════════╬════════════════════╬═══════════════╣
║ 2         ║     250000         ║    250000     ║
║ 2         ║     250000         ║    104705     ║
║ 2         ║     250000         ║    108740     ║
║ 2         ║     250000         ║    89855      ║
╚═══════════╩════════════════════╩═══════════════╝

This behavior is due to the looping have data depenance. When different threads are working parallel, it may cause the “race condition” for the “ret”.

In order to fix this issue, you can add “lock” for protection data as below:

Parallel.For(0, 500, i =>
{
    for (int j = 0; j < 500; j++)
    {
        var data = DoStuffs(cycle);
        lock(_lock)
        {
            ret += data;
        }
    }
});

Hope those information is good for you, enjoy~