Monday 10 August 2009

Quick look at STM.NET performance

STM as an idea looks interesting and definitely has a lot of advantages over regular locking. It helps prevent deadlocks, makes easier to compose components that implement some kind of synchronization, helps eliminate nested locks, supports retries and so on. You can find the full list of goodness here.
When I found out that STM.NET(based on .NET 4.0) is out my first question was related to its performance. I put together a simple test and I have to say that in most scenarios STM.NET is an overkill. Sure, it’s a research project and it hasn’t been productized which means it’s not optimized and it’s not ready for prime time. But still, on average it’s 20x slower than the regular lock statement. That’s a lot taking into account that in most cases developers use very simple locking strategies and try to avoid nested locks. If Microsoft can bring it down to 2x then it is a completely different story.
The results:

The code of the benchmark:

using System;
using System.Threading;
using System.Collections;
using System.TransactionalMemory;
using System.Collections.Generic;
using System.Diagnostics;
class Program
{
public delegate void CustomAction();
[AtomicNotSupported]
static void Main(string[] args)
{
var iterations = 1000000;
Console.WriteLine("Number of Threads | STM.NET [ms] |" +
"Regular locking [ms] | Times slower");
for (var i =1 ; i < 15; i++)
{
Test(i, iterations);
}

Console.ReadLine();
}
private static void Test(int numberOfThreads, int iterations)
{
var storage = new Storage();
CustomAction action = () =>
{
Atomic.Do(() =>
{
storage.UpdateByOne();
});
};
var stm = Execute(numberOfThreads, iterations, action, storage);
storage = new Storage();
CustomAction action1 = () =>
{
lock (storage)
{
storage.UpdateByOne();
}
};
var locking = Execute(numberOfThreads, iterations, action1, storage);
int slower = (int)(stm.ElapsedMilliseconds / locking.ElapsedMilliseconds);
var message = String.Format("{0}\t\t\t{1}\t\t\t{2}\t\t\t{3}", numberOfThreads,
stm.ElapsedMilliseconds,
locking.ElapsedMilliseconds, slower);
Console.WriteLine(message);
}
private static Stopwatch Execute(int numberOfThreads, int iterations,
CustomAction action, Storage storage)
{
var stopwatch = Stopwatch.StartNew();
var threads = new List<Thread>();
for (var i = 0; i < numberOfThreads; i++)
{
var thread = new Thread(() =>
{
for (var j = 0; j < iterations; j++)
{
action();
}
});
threads.Add(thread);
}
foreach (var thread in threads)
{
thread.Start();
}
foreach (var thread in threads)
{
thread.Join();
}
stopwatch.Stop();
if (numberOfThreads * iterations != storage.Result)
{
var message = String.Format("Incorrect value. Expected: {0} Actual: {1}",
numberOfThreads*iterations, storage.Result);
throw new ApplicationException(message);
}
return stopwatch;
}
}
public class Storage
{
private long value;
public void UpdateByOne()
{
value = value + 1;
}
public long Result
{
get { return value; }
}
}

1 comment: