Virtual File System and Global Request Handlers

#1

I try to solve a problem with the behavior of VFS.
In AppHost in our self hosted windows service I have created a FileSystemVirtualPathProvider

base.VirtualFileSources = new FileSystemVirtualPathProvider(this,"~/…/FileSystem".MapHostAbsolutePath());

There is a file service , exactly as your new example with [Route("/GetFile/Locations/{LotNumber}/{ImagesPath}/{FileName}")]

and a GlobalRequestFilter to check the session, using the X-ss-id header.

The problem is that when I call the service URL the request is checked very fast from the GlobalRequestFilter (only the X-ss-Id ), but the performance of download through the service is slow.

In the opposite , if I call directly only the relative path
serviceURL + /Locations/{LotNumber}/{ImagesPath}/{FileName}
the FileSystemVirtualPathProvider downloads the image very fast.
As we use it , in this case, for the rendering of Razor pages , this performance is acceptable and fast.
but I don’t have security.
My question is the following
Is there a way in the FileSystemVirtualPathProvider to check the request’s header
if it is a valid session, as I do in the GlobalRequestFilter, but without to call actually the file service,
in order to get the superior performance?

thanks

#2

No the FileSystemVirtualPathProvider has nothing to do with the HTTP Request, it just abstracts the FileSystem access behind a common IVirtualFiles API. You could create a custom VirtualPathProvider and fetch the current request from the HttpContext.Current.ToRequest() but the singleton is only available from ASP.NET hosts.

But I can’t tell what the issue is from just your description, why 1 request is faster than the other without seeing the implementation of both, or what you mean by “slow” without any performance numbers comparing the two. Have you identified why it’s slower? I’d start by looking how the fast path works and see what you’re doing different in the slower service - the performance of FileSystemVirtualPathProvider shouldn’t change depending on who’s calling it, I’m assuming there’s an issue with the implementation.

#4

I’m having difficulty understanding what you mean, are you saying when you call the file directly, so the request isn’t actually going through FileSystemVirtualPathProvider or ServiceStack? If IIS serves the file directly that would definitely be much faster, in which case it’s not going to matter if you can check the request in FileSystemVirtualPathProvider because that’s not providing the performance benefits.

#5

Yes , IIS serves the file very fast.
But we want to have a separate file storage , services to update the virtual files system.
and some security because we are a self-hosted windows service API.
Your FileSystemVirtualPathProvider is almost fast as IIS if we don’t call the file service.
we need only the security

#6

I’m still no closer to understanding what’s actually happening. If they’re both using FileSystemVirtualPathProvider than why would the performance be different? How are you testing the performance of FileSystemVirtualPathProvider without going through a Service?

#7

I am testing the performance of FileSystemVirtualPathProvider from the Razor Pages we are rendering. and simply from the browser’s address line.
Before using directly the FileSystemVirtualPathProvider, we had the FileService for the upload + Directories, etc.
But for the rendering of images in the browser, I saved them in an IIS virtual directory and we call them via an ASP.Net web site. The reason for that was that the file service is slow in performance when we render in browser Razor pages (with images) from a self hosted API .
After I found that using the FileSystemVirtualPathProvider directly (like the IIS) is almost the same fast. of course I prefer this solution (instead to use the IIS) , but I need some security.

#8

Can you show me what that looks like? as I still don’t know what you mean by this. If you’re able to access the FileSystemVirtualPathProvider directly then I’m assuming you can check the Request when you’re doing this?

#9 
 base.VirtualFileSources = new FileSystemVirtualPathProvider(this, "~/../FileSystem".MapHostAbsolutePath());

           GlobalRequestFilters.Add(HandleRequestFilter);

               private void HandleRequestFilter(IRequest request, IResponse response, object requestDto)
              {                           
                       if (request.Headers["X-ss-id"] == null)  .//...............etc................
               }

Service Call Route [Route("/GetFile/Locations/{LotNumber}/{DocumentPath}/{FileName}")]

IF I call the the GetFile , the HandleRequestFilter checks the Request and the Service takes the request accordingly. This is slow. We went to IIS.

IF I call with GET only the serviceUrl + only the relative path Locations/{LotNumber}/{DocumentPath}/{FileName}
then 1. the FileSystemVirtualPathProvider serves directly the file ,
2. the HandleRequestFilter does not get the request to check the header
and I am quite happy with the performance but I missed the security check.
In this case there is not IIS at all , in the equation.

#10

When you keep repeating this, who’s actually calling FileSystemVirtualPathProvider? is it your code (if it is please show me the code), or are you talking about when ServiceStack serves static files? (only place that happens is in Static File Handler).

#11

It is the behavior of your system.
There is not a special call to FileSystemVirtualPathProvider. All the code is what I show you, plus the File service example from your test ( I will put below the code) .
You can test it , either from the browser address line or if you put the relative path in an image src of a Razor page.

the service’s code is below. There is not other code , except the appHost’s configuration (derived from AppSelfHostBase) that I show you in my previous comments.

               [Route("/GetFile/Locations/{LotNumber}/{DocumentPath}/{FileName}")]
              public class GetLotFile
             { 
                 public string LotNumber { get; set; }
                  public string DocumentPath { get; set; }
                 public string FileName { get; set; }
           }

public class LocationFileService : Service
{
   
    public object Any(GetLotFile request)
    {
        
              string path = MakeThePath("/Locations/{request.LotNumber}/{request.DocumentPath}/{request.FileName}";               //actually  it is  C6  $ and  the editor here destroyes  the text.

        var file = HostContext.VirtualFileSources.GetFile(path);
        if (file == null)
            throw HttpError.NotFound( "File   does not exist");

        return new HttpResult(file)
        {
            ContentType = MimeTypes.GetMimeType(file.Extension)
        };
    }
}
#12

Ok so if your talking about the static file feature built into ServiceStack and you’re Self Hosting then you just mean ServiceStack’s built-in Static File Handler. One thing Static File Handler does for you automatically is that it adds Last Modified Headers and returns a 304 NotModified HTTP Response if the File hasn’t been modified since the last request. This can make it appear instant because it doesn’t need to send the file over and the browser can just use its locally cached version. You’ve not provided any info showing it’s slow so I don’t know if this is what you’re comparing.

Now you can also make use of the same StaticFileHandler class ServiceStack uses by registering a RawHttpHandler which are the first thing executed in ServiceStack’s Request Pipeline. Here you can add custom logic to inspect the request before either returning a static file or a 403 Forbidden response if they’re not allowed access with something like:

RawHttpHandlers.Add(req =>
{
    if (!req.PathInfo.ToLower().StartsWith("/locations/"))
        return null;

    var sessionId = req.Headers["X-ss-id"];
    if (!MyCustomValidate(sessionId))
        return  new ForbiddenHttpHandler();

    return new StaticFileHandler {
        VirtualNode = VirtualFileSources.GetFile(req.PathInfo)
    };
});
#13

i have cleared all mybrowsers’ cache before any call and I am using 4 different browsers in my laptop
and other 3 in the Remote Desktop
Also our client application in the Call Centers uses CefSharp embedded browsers with its cache every one.
cleared all time , so the agents to get the newest data

thank you

#14

Only way to make sure what’s happening is if we can actually see the HTTP Headers / request logs / source code / etc - we need to know what’s actually happening in order to help. I’ve really struggled trying to guess what the root cause of the issue was or even what’s actually happening from the vague description and observations provided, e.g. the route definitions and paths provided aren’t useful without knowing the implementation. In future questions please provide as much relevant context as you can upfront so that someone without knowledge of your system can understand what’s going on and be able to help.

#15

I don’t think that my description was vague.
In any case if you accepted private email , I could send you the 2 different links to compare performance in your browsers

#16

I’ve had to ask essentially the same question 5 times to find out exactly what implementation you’re referring to - i.e. the most important information needed to figure out what’s actually happening.

#17

I wrote you all the code , there is nothing more.
We build products with a 10s of libraries and we are not so ingenious like you.
So what other you ask ? I am happy when our customers offer any feedback to our products and I praised them.
You had a conclusion that I forgot the browser’s cache.
when I wrote you before that we are moving from IIS to VFS , so it something that we care a lot.

#18

It’s far from all the code, e.g. the small fragments provided can’t be run locally. But regardless I just asked to know the implementation being executed.

Please refrain from being personal in these forums.

I just wanted clarification on exactly what implementations you’re referring to, any context/info showing why its slow or how much slower the different implementations were also would’ve helped, as would’ve the HTTP Request / Response Headers.

All I’m asking in future is to please provide as much relevant information about the issue upfront so someone who doesn’t know your system has a chance to find out what’s actually happening.

#19

At least as the final result of our conversation. I should tell you that

I am very happy that your snippet, the RawHttpHandlers did the job.
With All my browsers cleared for n+1 time !!! the performance is fantastic , vs the file service,
So fast as the IIS and I have also my security check. We are moving on.

You have a killer product, it should be much more used from the .Net companies.
But please, give more patience. What I wrote you it was a real issue an I have thousand other things
in my mind like you have also.

#20

ok great, I’m glad it helped.

#21

One more thing since this is a security check it’s better to make the comparison case-insensitive so they can’t get around it by changing the case, e.g:

if (!req.PathInfo.ToLower().StartsWith("/locations/"))
    return null;