Possible performance problem with Ormlite SingleAsync<T> on SQL Server

Hi,
I have a scenario where Ormlites SingleAsync<T> is about 50 times slower than Single<T> when fetching data from a table with a nvarchar(max) field containg a lot of chars.

This happens with SqlServer2012Dialect, but not with SqliteDialect.

Note: I have found a workaround using Dappers QuerySingleAsync method, which is fast in this scenario.

When testing with a string length of 1000000 against an otherwise empty table, these are typical results:

Testing with text length 1000000 using SqliteOrmLiteDialectProvider
db.Single<T>: 35ms
db.Single<T>: 2ms
db.Single<T>: 2ms
db.Single<T>: 0ms
db.Single<T>: 0ms
await db.SingleAsync<T>: 5ms
await db.SingleAsync<T>: 0ms
await db.SingleAsync<T>: 0ms
await db.SingleAsync<T>: 0ms
await db.SingleAsync<T>: 0ms

Testing with text length 1000000 using SqlServer2012OrmLiteDialectProvider
db.Single<T>: 14ms
db.Single<T>: 7ms
db.Single<T>: 6ms
db.Single<T>: 6ms
db.Single<T>: 5ms
await db.SingleAsync<T>: 433ms
await db.SingleAsync<T>: 385ms
await db.SingleAsync<T>: 391ms
await db.SingleAsync<T>: 334ms
await db.SingleAsync<T>: 292ms

I am not sure how I best can provide something for you to reproduce, but below is all the code needed to reproduce in a .net 8 Console app, using ServiceStack 8.4.0, if you have access to SQL server in some form:

I have tested this on SQL Server LocalDB, SQL Server 2022 on Docker Linux and SQL Server 2022 on Windows.

//Program.cs
using System.Diagnostics;
using ServiceStack.DataAnnotations;
using ServiceStack.OrmLite;

var text = new string('x', 1000000);
var iterations = 5;
var connectionString = @"Data Source=(localdb)\MSSQLLocalDB;Initial Catalog=TestDb;";

var factories = new List<OrmLiteConnectionFactory> {
    new (":memory:", SqliteDialect.Provider),
    new(connectionString, SqlServer2012Dialect.Provider),
 };

foreach (var dbFactory in factories)
{
    await RunAsync(text, iterations, dbFactory);
}

static async Task RunAsync(string text, int iterations, OrmLiteConnectionFactory dbFactory)
{
    Console.WriteLine($"Testing with text length {text.Length} using {dbFactory.DialectProvider.GetType().Name}");

    using var db = dbFactory.Open();

    if (dbFactory.DialectProvider.GetType() == SqliteDialect.Provider.GetType())
    {
        db.CreateTable<MyTable>();
    }

    var id = db.Insert(new MyTable { Contents = text }, selectIdentity: true);

    var sw = new Stopwatch();
    var timerResults = new List<string>();

    for (var i = 0; i < iterations; i++)
    {
        sw.Restart();
        var result = db.Single<MyTable>(x => x.Id == id);
        sw.Stop();
        var errror = (result?.Contents.Length ?? 0) == 0 ? "NO CONTENT!" : "";
        timerResults.Add($"db.Single<T>: {sw.ElapsedMilliseconds}ms {errror}");
    }

    for (var i = 0; i < iterations; i++)
    {
        sw.Restart();
        var result = await db.SingleAsync<MyTable>(x => x.Id == id);
        sw.Stop();
        var errror = (result?.Contents.Length ?? 0) == 0 ? "NO CONTENT!" : "";
        timerResults.Add($"await db.SingleAsync<T>: {sw.ElapsedMilliseconds}ms {errror}");
    }

    db.Delete<MyTable>(x => x.Id == id);

    Console.WriteLine($"{string.Join(Environment.NewLine, timerResults)}\n");
}

public class MyTable
{
    [AutoIncrement] public int Id { get; set; }

    [StringLength(StringLengthAttribute.MaxText)]
    public string Contents { get; set; }
}
/*
CREATE TABLE [dbo].[MyTable](
	[ID] [int] IDENTITY(1,1) NOT NULL,
	[Contents] [nvarchar](max) NOT NULL,
 CONSTRAINT [PK_MyTable] PRIMARY KEY CLUSTERED 
(
	[ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
GO
*/

Best regards, Anders

Thanks we’ll have a look, although In Memory SQLite isn’t a good comparison here since the Async APIs aren’t true async APIs as they’re just pseudo async wrappers over SQLite’s blocking APIs which doesn’t even hit disk since it’s in memory.

Which SQL Server NuGet package are you using? If you’re using ServiceStack.OrmLite.SqlServer can you switch to the recommended ServiceStack.OrmLite.SqlServer.Data NuGet package for ASP .NET Core as it uses the actively maintained Microsoft.Data.SqlClient library.

I’ve added a ServiceStack.OrmLite.Tests.Benchmarks BenchmarkDotNet project to try and reproduce the difference between sync/async APIs but they’re all essentially the same across SQLServer, MySQL and PostgreSQL:

Run with:

 dotnet run -c Release
| Method             | DbName | Mean      | Error     | StdDev    | Gen0      | Gen1      | Gen2      | Allocated |
|------------------- |------- |----------:|----------:|----------:|----------:|----------:|----------:|----------:|
| TextContents_Sync  | mssql  | 27.729 ms | 3.3070 ms | 0.1813 ms | 2468.7500 | 2468.7500 | 2468.7500 |  14.85 MB |
| TextContents_Async | mssql  | 27.555 ms | 5.8326 ms | 0.3197 ms | 2555.5556 | 2444.4444 | 2444.4444 |  20.04 MB |
| TextContents_Sync  | mysql  |  2.830 ms | 0.5069 ms | 0.0278 ms | 1648.4375 | 1648.4375 | 1648.4375 |   9.64 MB |
| TextContents_Async | mysql  |  3.066 ms | 0.4808 ms | 0.0264 ms | 1648.4375 | 1648.4375 | 1648.4375 |   9.65 MB |
| TextContents_Sync  | pgsql  | 10.013 ms | 0.6599 ms | 0.0362 ms | 1656.2500 | 1656.2500 | 1656.2500 |    9.6 MB |
| TextContents_Async | pgsql  | 10.580 ms | 1.8836 ms | 0.1032 ms | 1062.5000 | 1062.5000 | 1062.5000 |   9.58 MB |

All databases are running against a RDBMS in a local Docker instance on Linux which isn’t optimal for SQL Server but it should still show a highlight differences between Sync/Async APIs. I’ve ran it a few times and in all cases for SQL Server db.SingleAsync<T> performs marginally faster than db.Single<T>.

Can’t really recommend anything other than using ServiceStack.OrmLite.SqlServer.Data NuGet package if you’re not already and to run the benchmark in Release mode.

Thanks for having a look and pointing out the correct Nuget package!

I was using ServiceStack.OrmLite.SqlServer.
Switching to ServiceStack.OrmLite.SqlServer.Data seems to speed things up quite a bit.

I have run the app several times after the package change and the results are pretty consistent.
Still quite a bit of difference, but since I have a workaround, you don’t need to dig further into this.
Unless you really want to :wink: and in that case, let me know if there is anything I can do to help.

This is the latest run

Testing with text length 1000000 using SqlServer2012OrmLiteDialectProvider
db.Single<T>: 12ms
db.Single<T>: 7ms
db.Single<T>: 6ms
db.Single<T>: 7ms
db.Single<T>: 6ms
await db.SingleAsync<T>: 102ms
await db.SingleAsync<T>: 64ms
await db.SingleAsync<T>: 59ms
await db.SingleAsync<T>: 57ms
await db.SingleAsync<T>: 57ms

Update: Just ran it with Release mode, but results are essentially the same.

FYI: We discovered this during QA testing when users requested images stored in a SQL Server Db, via our ServiceStack-based API.
A 7 MB image took about 35 seconds to get from Db and pass on to the response.

I’ve changed the benchmark to run 1 iteration of 7MB and it does start to show changes discrepancy between Sync/Async for SQL Server which now takes 116ms:

| Method             | DbName | Mean       | Error      | StdDev     | Gen0     | Gen1     | Gen2     | Allocated |
|------------------- |------- |-----------:|-----------:|-----------:|---------:|---------:|---------:|----------:|
| TextContents_Sync  | mssql  |  45.605 ms |  79.887 ms |  4.3789 ms | 777.7778 | 777.7778 | 777.7778 |  20.76 MB |
| TextContents_Async | mssql  | 116.730 ms | 229.891 ms | 12.6011 ms | 500.0000 | 500.0000 | 500.0000 |  28.02 MB |
| TextContents_Sync  | mysql  |   3.288 ms |   2.026 ms |  0.1111 ms | 156.2500 | 156.2500 | 156.2500 |   13.4 MB |
| TextContents_Async | mysql  |   3.335 ms |   1.921 ms |  0.1053 ms | 156.2500 | 156.2500 | 156.2500 |  13.41 MB |
| TextContents_Sync  | pgsql  |  14.083 ms |   2.888 ms |  0.1583 ms | 484.3750 | 484.3750 | 484.3750 |   13.4 MB |
| TextContents_Async | pgsql  |  14.291 ms |   1.015 ms |  0.0556 ms | 156.2500 | 156.2500 | 156.2500 |  13.36 MB |

10 iterations (avg 120ms):

| Method             | DbName | Mean        | Error        | StdDev     | Gen0       | Gen1      | Gen2      | Allocated |
|------------------- |------- |------------:|-------------:|-----------:|-----------:|----------:|----------:|----------:|
| TextContents_Sync  | mssql  |   587.34 ms | 3,454.699 ms | 189.364 ms |  3000.0000 | 3000.0000 | 3000.0000 | 207.53 MB |
| TextContents_Async | mssql  | 1,202.01 ms | 3,583.196 ms | 196.407 ms | 10000.0000 | 9000.0000 | 8000.0000 | 280.29 MB |
| TextContents_Sync  | mysql  |    31.34 ms |    14.020 ms |   0.768 ms |  1531.2500 | 1531.2500 | 1531.2500 | 133.77 MB |
| TextContents_Async | mysql  |    31.06 ms |     5.292 ms |   0.290 ms |  1500.0000 | 1500.0000 | 1500.0000 | 133.83 MB |
| TextContents_Sync  | pgsql  |   140.86 ms |    74.658 ms |   4.092 ms |  2000.0000 | 2000.0000 | 2000.0000 | 134.05 MB |
| TextContents_Async | pgsql  |   140.19 ms |    16.638 ms |   0.912 ms |  1750.0000 | 1750.0000 | 1750.0000 | 133.59 MB |

So yeah SQL Server does slow down for larger columns but shouldn’t be anywhere near 35s unless your RDBMS is under heavy load or under resourced.

Although I’d personally only store the path to large files in the DB and the file contents in managed storage which is something our Managed File Uploads provides good support for.

Thanks, I’ll try and run from your Benchmark om my machine and will post the results.

Although the allocated resources for the db server in QA were not great, I was later able to replicate the problem on the db server in my company’s Staging environment, as well as against my LocalDB and a Docker hosted DB on my machine (32 GB RAM, Dell XPS 15 9520, Core i7) .

I did try going from nvarchar to varchar and toggling the DialogProviders Unicode settings and that seemed to have a significant impact (improvement when using varchar, but this may just be the different in bytes per char), but I don’t have any good tests that highlights these differences at the moment. Just thought it might be worth mentioning.

1 Like