Unexpected end of Stream when uploading to aspnet core

.NET Core
1

We’ve been struggling the last week with several issues after moving our api to dotnet core. One of those has shown up when trying to POST a file.

It’s a simple endpoint expecting the standard multipart/form-data upload. However, we get the following error now.

{"responseStatus":{"errorCode":"IOException","message":"Unexpected end of Stream, the content may have already been read by another component. ","stackTrace":"   at Microsoft.AspNetCore.WebUtilities.MultipartReaderStream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)\n   at Microsoft.AspNetCore.WebUtilities.StreamHelperExtensions.DrainAsync(Stream stream, ArrayPool`1 bytePool, Nullable`1 limit, CancellationToken cancellationToken)\n   at Microsoft.AspNetCore.WebUtilities.MultipartReader.ReadNextSectionAsync(CancellationToken cancellationToken)\n   at Microsoft.AspNetCore.Http.Features.FormFeature.InnerReadFormAsync(CancellationToken cancellationToken)\n   at Microsoft.AspNetCore.Http.Features.FormFeature.ReadForm()\n   at ServiceStack.Host.NetCore.NetCoreRequest.get_FormData()\n   at ServiceStack.HttpRequestExtensions.GetFlattenedRequestParams(IRequest request)\n   at ServiceStack.Host.RestHandler.CreateRequestAsync(IRequest httpReq, IRestPath restPath)\n   at ServiceStack.Host.RestHandler.ProcessRequestAsync(IRequest req, IResponse httpRes, String operationName)"}}

The only thing we could find is this. It doesn’t really pertain since we’re not using mvc here but servicestack.

2

It’s an internal Kestrel Exception when trying to access the .NET Core Request Form Data APIs. There’s a number of issues on Google on this Exception, the first answer suggests enabling Rewind:

app.Use(async (context, next) => {
    context.Request.EnableRewind();
        await next();
    });

Otherwise it’s not clear what the cause or the solution to the Inner Exception is. Maybe its due to another feature configured in your App that’s in conflict.

3

THanks, will try. Guess my googling skillz have been in decline as I didn’t find that one at all. :frowning: sorry.

4

Well, that didn’t work. And, to my mistake I should have mentioned this is only when running on linux. On windows it runs fine without issue.

I’ll keep poking around.

5

So this took a really really long time to figure out - although rather simple in the end.

It turns out it’s the request logger! It will log the filedata without resetting it. Our fix was to just skip this endpoint for logging for now. Long term we’d like to log it, but not the file contents.

1 Like
6

So, @mythz to make it a little more clear here as to the problem and why I think something should be done. It’s specific to .net core and isn’t a problem in framework.

When enabling the RequestLogsFeature.EnableRequestBodyTracking the UseBufferedStream gets set. Doing this then causes the problem as the ListnerRequest buffers the stream but then Microsoft.AspNetCore.WebUtilities.MultipartReaderStream.ReadAsync fails on the POST as it’s already been read (multipart file content).

Initially we tried to exclude the DTO in logging, but that doesn’t work as it’s already been read at that point. The only option we had (that I know of) was to add a request filter.

PreRequestFilters.Insert(0, (httpReq, httpRes) =>
            {
                if (!(httpReq.ContentType.ToUpper().StartsWith("MULTIPART/")))
                    httpReq.UseBufferedStream = true;
            });

I think there should be a cleaner way to deal with this.

7

What workaround are you suggesting? Enabling Stream buffering is already at the start of the ServiceStack Request pipeline, ServiceStack plugins can’t register it any sooner.

You can enable buffering in .NET Core before ServiceStack gets the request with the .NET Core middleware:

app.Use(next => context => {
    context.Request.EnableRewind();

    return next(context);
});

app.UseServiceStack(...);
8

That was your first suggestion and it doesn’t work. Although it looks slightly different - so I’ll give it a try.

I don’t really have a specific suggestion but what I had to do doesn’t seem like the best solution. If someone turns on request logging and enables request body tracking, I don’t expect them (myself) to have to know that if it’s multipart request content, the logger is going to read and and not rewind, so further down the pipe it fails with the error posted initially. It was a PITA to track down and no obvious solution.

My expectation is that I shouldn’t be concerned with the inner workings of SS or .net core in order to enable request logging. I just want it to work.

If it’s not something that has a solution that as a user of SS is just automatic for such a situation, it shouldn’t be an option that takes down a service and leaves me in the dark trying to figure out what’s going on. Just as concerning is this works fine in Framework version.

9

Update, this doesn’t work:

app.Use(next => context => {
context.Request.EnableRewind();

return next(context);
});
10

Well if multipart/formdata requests can’t be buffered in .NET Core than it’s going to be incompatible with being able to log the Request Body so I’ve disabled it for .NET Core in this commit.

Change now available from v4.5.1 that’s now available on MyGet.