DatabaseJobs - Exceptions in startup, due to a null key exception

ServiceStack
1

I have successfully migrated to ServiceStack DatabaseJobs, using a PostgreSQL DB. The process is working well for my multi-instance deployment in AWS AppRunner. However, I am now seeing Exceptions in app startup, due to a null key exception occurring in await jobs.StartAsync(stoppingToken); in my JobsHostedService.

I traced the issue back to a NULL value in the Command column in the JobSummary table. This is occurring because I am using jobs.EnqueueApi() to asynchronously invoke some API operations. When I delete the database records with the NULL values the app starts up without errors. Is there something I should be doing differently when calling jobs.EnqueueApi() to make the Command column non-NULL, or another workaround I can use?

2

I’ll need some way to be able to identify the issue, what’s the Exception StackTrace?

3

Here is the stack trace for the Exception thrown in jobs.StartAsync():

System.ArgumentNullException: Value cannot be null. (Parameter 'key')
   at System.Collections.Generic.Dictionary'2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
   at System.Collections.Generic.Dictionary'2.Add(TKey key, TValue value)
   at ServiceStack.OrmLite.OrmLiteReadCommandExtensions.Dictionary[K,V](IDataReader reader, IOrmLiteDialectProvider dialectProvider) in /home/runner/work/ServiceStack/ServiceStack/ServiceStack.OrmLite/src/ServiceStack.OrmLite/OrmLiteReadCommandExtensions.cs:line 855
   at ServiceStack.OrmLite.OrmLiteResultsFilterExtensions.Dictionary[K,V](IDbCommand dbCmd, ISqlExpression expression) in /home/runner/work/ServiceStack/ServiceStack/ServiceStack.OrmLite/src/ServiceStack.OrmLite/OrmLiteResultsFilterExtensions.cs:line 297
   at ServiceStack.OrmLite.OrmLiteReadApi.<>c__DisplayClass36_0'2.<Dictionary>b__0(IDbCommand dbCmd) in /home/runner/work/ServiceStack/ServiceStack/ServiceStack.OrmLite/src/ServiceStack.OrmLite/OrmLiteReadApi.cs:line 346
   at ServiceStack.OrmLite.OrmLiteExecFilter.Exec[T](IDbConnection dbConn, Func'2 filter) in /home/runner/work/ServiceStack/ServiceStack/ServiceStack.OrmLite/src/ServiceStack.OrmLite/OrmLiteExecFilter.cs:line 70
   at ServiceStack.OrmLite.OrmLiteReadExpressionsApi.Exec[T](IDbConnection dbConn, Func'2 filter) in /home/runner/work/ServiceStack/ServiceStack/ServiceStack.OrmLite/src/ServiceStack.OrmLite/OrmLiteReadExpressionsApi.cs:line 19
   at ServiceStack.OrmLite.OrmLiteReadApi.Dictionary[K,V](IDbConnection dbConn, ISqlExpression query) in /home/runner/work/ServiceStack/ServiceStack/ServiceStack.OrmLite/src/ServiceStack.OrmLite/OrmLiteReadApi.cs:line 346
   at ServiceStack.DbJobs.LoadJobQueue() in /home/runner/work/ServiceStack/ServiceStack/ServiceStack/src/ServiceStack.Server/DbJobs.cs:line 1002
   at ServiceStack.DbJobs.StartAsync(CancellationToken stoppingToken) in /home/runner/work/ServiceStack/ServiceStack/ServiceStack/src/ServiceStack.Server/DbJobs.cs:line 971
   at CTV1.JobsHostedService.ExecuteAsync(CancellationToken stoppingToken) in   C:\repos\web-api\CTV1\Configure.BackgroundJobs.cs:line 75
   at Microsoft.Extensions.Hosting.Internal.Host.TryExecuteBackgroundServiceAsync(BackgroundService backgroundService)

I traced the error to LoadJobQueue() and this specific query:

var apiDurations = db.Dictionary<string, int>(
    db.From<JobSummary>()
        .Where(j => Sql.In(j.Id,
            db.From<JobSummary>()
                .Where(x => x.State == BackgroundJobState.Completed
                            && x.DurationMs > 0
                            && x.RequestType == CommandResult.Api)
                .GroupBy(x => new { x.Id, x.Command, x.Worker })
                .SelectDistinct(x => x.Id)))
        .GroupBy(x => new { x.Command, x.Worker })
        .Select(x => new {
            Command = Sql.Custom($"CASE WHEN {columns.Worker} is null THEN {columns.Command} ELSE {sqlCommandWorker} END"), 
            DurationMs = Sql.Custom($"CASE WHEN SUM({columns.DurationMs}) > {int.MaxValue} THEN {int.MaxValue} ELSE SUM({columns.DurationMs}) END"),
        }));

The NULL value in the Command column is causing the ArgumentNullException.

I have been able to work around this by adding the following lines in Configure.BackgroundJobs.cs:

using var db = jobs.OpenDb();`
await db.DeleteAsync<JobSummary>(x => x.Command == null);`
await jobs.StartAsync(stoppingToken);
4

Thanks for the StackTrace, the query is wrong as API should be querying Request instead of the Command column which should be resolved from this commit.

This change is available from v10.0.5+ that’s now available in the pre-release packages.

FYI, to quote a block of code in markdown, use triple back ticks, e.g:

```
Log files, Exception Stacktraces, Adhoc text blocks.
```

For marking up code with syntax highlighting, add the name of the programming language, e.g:

```csharp
Console.WriteLine("Hello C#");
```
5

Thank you for the quick response, @mythz ! Thanks also for the markdown tips - I’m rusty. :slightly_smiling_face:

1 Like