Need some help with binary data transfer

ServiceStack
1

I try to download a ZIP file form my server and need some clarification regarding the client side to use.
Infrastructure:
Client: WPF application written against .NET 4.6.1
Server: .NETCore 2.1 running as Docker container on RHEL / CENTOS 7

My server code looks as follows:

[Route("/v1/reports/invoicetemplates/download/{TenantCode}/{ReportName}", "GET", Summary = "Request to return all report template files compressed to one ZIP file.")]
public class DownloadReportTemplate : IReturn<IHttpResult>
{
	public string TenantCode { get; set; }
	public string ReportName { get; set; }
}

Implementation:

public object Get(DownloadReportTemplate request)
{
    var templatePath = $$"/data/{request.TenantCode}/System/ReportTemplates/BizBusInvoice/{request.ReportName}";
    Logger.Debug($$"The report template files should be at: {templatePath}");
    if (!Directory.Exists(templatePath))
    {
        var msg = $$"Could not find any '{request.ReportName}' report files for tenant '{request.TenantCode}'.";
        Logger.Error(msg);
        throw HttpError.NotFound(msg);
    }
    
    // code to retrieve and ZIP files
    var fileStream = new FileStream(zippedReportTemplates, FileMode.Open, FileAccess.Read);
    var ms = new MemoryStream();
    fileStream.CopyTo(ms);
    File.Delete(zippedReportTemplates);
    
    var doc4Client = new ServerDocument(ms, "ZIP", $$"{request.ReportName}ReportFiles.zip");

    return doc4Client;  
}

public class ServerDocument : IHasOptions, IStreamWriterAsync
{
    private static readonly ILogger Logger = Log.ForContext<ServerDocument>();
    private readonly Stream _responseStream;
    public IDictionary<string, string> Options { get; }

    public ServerDocument(Stream responseStream, string filetype, string filename)
    {
        _responseStream = responseStream;
        Options = new Dictionary<string, string>
        {
            {"Content-Type", "application/octet-stream"},
            {"X-Api-DocName", filename},
            {"X-Api-DocType", filetype},
            {"X-Api-DocLength", responseStream.Length.ToString()}
        };
    }

    public Task WriteToAsync(Stream responseStream, CancellationToken token = new CancellationToken())
    {
        if (_responseStream == null)
            return Task.CompletedTask;

        _responseStream.WriteTo(responseStream);
        responseStream.Flush();
        responseStream.Dispose();
	
        return Task.CompletedTask;
    }
}

The problem starts on the Client side:

//public JsonHttpClient BizBus4WindowsClient { get; private set; }    <===== result is a garbage string
public JsonServiceClient BizBus4WindowsClient { get; private set; }  <===== result is OK as expected

private static async Task GetInvoiceReportTemplate(string reportTemplateFile)
{
            // some code....
    var apiUri = $$"/v1/reports/invoicetemplates/download/{currentApp.ActiveTenant.TenantDto.TenantCode}/Invoice";
    Logger.Debug($$"Calling server: ssc.GetAsync<HttpWebResponse>({apiUri})");
    HttpWebResponse response;

    try
    {
        var ssc = currentApp.BizBus4WindowsClient;
        response = await ssc.GetAsync<HttpWebResponse>(apiUri);
    }
    catch (WebServiceException we)
    {
        var msg = $$"Failed to download invoice report templates! Error: {we.ErrorMessage}";
        Logger.Error(msg);
        throw new BizBusException(msg);
    }
    catch (Exception ex)
    {
        var msg = $$"Failed to download invoice report templates! Error: {ex.Message}";
        Logger.Error(msg);
        throw new BizBusException(msg);
    }

     // read and process the response stream by creating a new FileStream
    using (var stream = response.GetResponseStream())
    {
        try
        {
            using (var fs = new FileStream(zippedReportTemplates, FileMode.Create))
            {
                try
                {
                    fs.Write(stream.ToBytes(), 0, (int)length);
                }
                catch (Exception exception)
                {
                    Logger.Error($"Failed to write FileStream to file. Error: {exception}");
                }
            }
        }
        catch (Exception ex)
        {
        }
    }
}

QUESTIONS

  • When should I use JsonHttpClient and when JsonServiceClient? (With .NETCore on UNIX as the backend)
  • 99% of my REST calls use simple POCOs and work just fine with JsonHTTPClient but seem also to work with JsonServiceClient. Is it save to use JsonServiceClient for everything? I use PushMessaging as well…
  • Is it recommended to create an instance of JsonServiceClient in the (rare) cases where I transfer binary data and use JsonHttpClient for the majority of calls?

What is recommended best practice and most stable implementation?

Thanks a lot for your feedback!

2

Our recommendation from the docs:

If running on .NET Core we recommend using HttpClient ServiceClient due to .NET’s HttpWebRequest having a suboptimal implementation wrapper over HttpClient which is much slower than the .NET Framework implementation.

They should work in both, it’s just more optimal to use JsonHttpClient in .NET Core and I personally use JsonServiceClient in .NET Fx since it doesn’t require an additional dependency. Also if you need sync in .NET Fx use JsonServiceClient, don’t use sync APIs in .NET Core (which uses sync over async).

Why wouldn’t you use it for all calls? The docs show how you can get raw Stream / byte[] Responses from Service Clients.

3

Hi Demis,
Yes, that is exactly what I have done when I ported my Windows stuff to .NETCore, moved to JsonHttpClient. But when I use JsonHttpClient with binary data I get the following exception:

Exception: Failed to download invoice report templates! Error: Type definitions should start with a '{', expecting serialized type 'HttpWebResponse', got string starting with: PK????zo3Kb???X????????InvoicePositions_DE., Exception: Type definitions should start with a '{', expecting serialized type 'HttpWebResponse', got string starting with: PK????zo3Kb???X????????InvoicePositions_DE. " Exception thrown: 'BizBus4Windows.Util.BizBusException' in BizBus4Windows.exe

4

The exception stacktrace indicates it’s trying to deserialize the binary data as JSON.

What’s the exact code you’re using consume the binary response?

Also HTTP Utils is a good alternative for downloading binary data (still use *Async APIs in .NET Core).

5

It is basically what I have pasted above:

In the App class of my WPF application I have a global variable:

public JsonHttpClient BizBus4WindowsClient { get; private set; }

which is initialized:

BizBus4WindowsClient = new JsonHttpClient(bbinConnectionUrl);

The method which consumes the binary response:

private static async Task GetInvoiceReportTemplate(string reportTemplateFile)
{
	var currentApp = (App)Application.Current;
	HttpWebResponse response;

	try
	{
		var ssc = currentApp.BizBus4WindowsClient;
		response = await ssc.GetAsync<HttpWebResponse>(apiUri);  <== HERE THE EXCEPTION IS THROWN
	}
	catch (WebServiceException we)
	{
		var msg = $$"Failed to download invoice report templates! Error: {we.ErrorMessage}, Exception: {we.GetAllExceptions()}";
		Logger.Error(msg);
		throw new BizBusException(msg);
	}
	catch (Exception ex)
	{
		var msg = $$"Failed to download invoice report templates! Error: {ex.Message}, Exception: {ex.GetAllExceptions()}";
		Logger.Error(msg);
		throw new BizBusException(msg);
	}

	if (!long.TryParse(response.GetResponseHeader("X-Api-DocLength"), out var length))
		length = -1;
	var docName = response.GetResponseHeader("X-Api-DocName");
	var docType = response.GetResponseHeader("X-Api-DocType");
	var zippedReportTemplates = Path.Combine(templateFolder, docName);

	using (var stream = response.GetResponseStream())
	{
		try
		{
			using (var fs = new FileStream(zippedReportTemplates, FileMode.Create))
			{
				try
				{
					fs.Write(stream.ToBytes(), 0, (int)length);
				}
				catch (Exception exception)
				{
					// error processing...
				}
			}

		}
		catch (Exception ex)
		{
			// error processing
		}
	}

	// code to decompress and store downloaded file...
}

If I replace JsonHttpClient with JsonServiceClient this code works without problems.

6

You can’t use HttpWebResponse in HttpClient implementations, that’s the response class for .NET WebRequest’s. .NET’s HttpClient is a completely different implementation, you can only read raw responses as string, byte[] or Stream.

If you need the HTTP Headers you can use ResponseFilter to intercept and inspect HttpClient’s HttpResponseMessage type.

Note: you need to escape ‘$’ with ‘$$’ in discourse otherwise it gets expanded into some weird uuid

7

Hmm, do you have a code snippet or example somewhere? And is this suitable for larger files also (could be several 100MB ZIP file).

Additionally I have to implement an upload feature soon, where users can drag and drop files to the WPF application, there I will ZIP the files and must upload them on the server. This can also be larger files.

I was shortly reading your protobuf document (not in detail and without a try). Could that be an alternative for my scenario?

8

I’ve previously linked to accessing raw service responses, use a Stream if you want avoid buffering the response in a byte[].

I’m assuming you’re returning a raw binary response? In which case you wouldn’t be encoding the response in any serialization format as you’d just be accessing the raw response binary data.

9

Hi Demis, thanks for your help but I cannot get it going, I tried most of the possible ways.

The only thing that currently works is to use JsonServiceClient instead of JsonHttpClient. So I create an instance of JsonServiceClient just for the REST calls I do to download files. After successfully processing the received stream I dispose that client instance. For standard POCO REST calls I use the JsonHttpClient instance which lives as long as the application does.

I have to look deeper into that issue once I have more time. I switched to .NETcore 2.1 a couple of days ago and I will upgrade to 2.2 soon. So maybe it has to do with that…