I saw lots of discussion on Twitter around this benchmark that does some comparisons between various different ORMs on Net Core and Framework.
Although I was happy to see ServiceStack on the list (albeit and older version), I was hoping for it to have a better showing performance and memory wise.
Is there anything that can be gleamed from this bench that can be improved on in newer versions of OrmLite to improve perf and memory usage?
The problem with Benchmarks is that they’re gamed by anyone who profiles against the benchmark. Profiling against a benchmark is mostly just going to improve numbers against specific micro benchmarks, not for real world performance & I’m not interested in chasing every benchmark that’s ever created. If there was a single standard benchmark that all ORMs evaluated against where the runs & results were automated, i.e. they’re run by a CI task and results immediately available (instead of manually handpicked runs) I’d invest time in profiling against it. But not someone else’s benchmark, that’s why I don’t create my own benchmarks anymore, the results would be meaningless as you’re the only one who’s going to be profiling against the chosen scenarios.
That benchmark is not great. Two obvious reasons:
It has an agenda. I noticed that without looking at the readme - because it starts dropping frameworks for specific feature tests mid way through the test. Then I looked at the author. Oh.
Second reason: The warm up procedure is terrible, particularly for SQL Server. There is no way you warm up the database to steady state that quickly. Later tests would run faster, always.