Thoughts on adding IStartup extensibility in Template WebApps

I love the idea of templates the way it’s laid out in https://github.com/NetCoreWebApps/Web. Just curl the web dir, use a plugins directory in each app (or share a plugins directory across apps) and drop your html files in your app directories with templated filters and actions. It’s elegantly simple and may best be that it shouldn’t be tampered with, that it should just remain as is.

While I can create a separate AppHost in each app via plugins, what NetCoreWebApps/Web doesn’t do right now from I can tell, which keeps me from being able to use it’s simplicity on many of my projects, is allow further configuring aspnet core services by taking advantage of the ConfigureServices and Configure methods at startup (I’m referring to https://github.com/NetCoreWebApps/WebApp/blob/master/src/WebApp/Program.cs). Of course I can create my own version of NetcoreWebApps/Web, but why do that if you felt this could be in your implementation. But you may not like this, and that’s fine.

What I’ve been able to do successfully, prior to this cool implementation you provided, was create a plugin architecture where IPlugin implementations (or the AppHostBase implementation) could also extend the IStartup interface. The plugin discovery process then kicks off early on in the ConfigureServices portion, and for every plugin that implements IStartup, or if the AppHost implements IStartup, you can call the ConfigureServices and Configure methods on those, all this prior to doing app.UseServiceStack() in the Configure method. I also typically add a services.AddSingleton(appHost) inside ConfigureServices, so that the plugins that implement IStartup can access the appHost.

Notionally, it would be something like:

public void ConfigureServices(IServiceCollection services) {
    var plugins = MethodToGetPlugins(services);
    plugins.Each(p => services.AddSingleton(p.GetType(), p));

    var appHost = MethodToGetAppHost(services, plugins);
    services.AddSingleton<AppHostBase>(appHost);

    plugins.Each(p => (p as IStartup)?.ConfigureServices(services));
    (appHost as IStartup)?.ConfigureServices(services);
}

public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
    var appHost = app.ApplicationServices.GetRequiredService<AppHostBase>();

    var plugins = MethodToGetPlugins(app, appHost);
    plugins.Each(p => (p as IStartup)?.Configure(app));

    appHost.BeforeConfigure.Add(ConfigureAppHost);
    app.UseServiceStack(appHost);
}

The benefit of this is we can still use the https://github.com/NetCoreWebApps/Web methodology and just drop our plugins in a directory where are plugins can also configure additional 3rd party services to include in our ServiceStack app (for example: services.AddNodeServices, there are dozens/hundreds of useful libs out there we are able to leverage then with the simplicity you’ve provided with NetCoreWebApps/Web). This does also bring out some additional questions too, like how to deploy plugin related assemblies (for example if some plugins were referencing Handlebars.Net or EPPlus.Core do these just need to be part of the /web assemblies directory ahead of time or can they also be dropped in the plugins directory as 3rd party assemblies). A lot of this is what the guys at http://extcore.net/ are doing for their .net core extensible plugin framework.

I’d really love also where you can just put features *, OpenApiFeature, PostmanFeature, CorsFeature, ValidationFeature or something like it, where * means automatically include all plugins found from the /plugins directory assemblies so that I can make my apps more flexible in their implementation.

Interested in your thoughts about this for WebApps, and if nothing here fits within what you feel is the scope for ServiceStack WebApps & Templates, that’s perfectly ok. I can proceed with my own custom implementation of this, no problem.

1 Like

Hi Matt,

I think these are great extensibility options to add to Web Apps which I believe I’ve implemented all suggestions above in this commit.

I’ve create a /plugins2 to test this feature which uses the new plugins/* option to add any plugins in the /plugins folder that’s not specified in the list. You’d list the plugins explicitly when you want them registered in a pre-defined order, then you can use plugins/* at the end to register all remaining plugins it can find.

The new StartupPlugin.cs plugin implements IStartup and shows an example of a Service that accesses a dependency it registered in its ConfigureServices().

For any additional .dll dependencies they’d still need to be placed in the /web directory so the .NET Core runtime can resolve them.

Can you test the latest version in the WebApp/web folder as I’d prefer to only deploy new /web versions to coincide with new SS releases. Feel free to send PR’s for any other customizations you need. I’d also be interested in hearing any feedback or demos/examples you create with it.

1 Like

Awesome, thank you! I’ll test it out a bit over the next week or so, but at first look, the commit looks good … I’ll follow up with a PR if I need anything else and post links to any demos/examples I create

1 Like

Sorry for the long absence. I continue to experiment with this feature, hopefully working toward a good demo still. Would there be a way to make the folders become dynamic parameters as well into a template page? I’m wondering how to do this easily.

For example:

One could have a physical folder:

  • /products/hardware/index.html
  • /products/_category/index.html
  • /products/_category/_subcategory/index.html

And the request to /products/hardware would render the 1st item above, but any other /products/software, /products/books would get routed to the 2nd item above and the [_category] would get passed into the page as some kind of argument … That would make this behave a lot like how Nuxt does dynamic parameters… This would help with having one page that can use the argument(s) to query a url api, or other …

Just wondering what a good way to accomplish this might be, if you have any ideas.

You would need to create a Custom Model View Controller Service that you would make available as a plugin, e.g:

[Route("/products/{Category}")]
public class ViewCategory {
    public string Category { get; set; }
}

public class ViewCategoryService : Service
{
    public object Any(ViewCategory request) =>
        new PageResult(Request.GetPage("products/_category")) {
            Model = request
        };
}
2 Likes

ok, maybe i’ll put it then into the fallback route and do a little bit of magic to figure out if a page is available at runtime since I’m trying to make it generic where the parameters would not be in the compiled code. thanks!

FYI if you’ve just started looking at this again I’d recommend getting a fresh copy of the .dlls from the apps/web folder a clone of the WebApp repo.

The latest version is a lot more sophisticated then the current version with support for Blocks, full JavaScript Expressions and all internal parsing being rewritten to use .NET’s new efficient Span<T> types.

1 Like

As I like this feature, I’ve just added support for Nuxt-like page-based routing with ServiceStack templates in this commit.

I’ve updated the .dlls in app/web to the latest v5.1.1 which has this feature. Let me know if you find any issues, I basically just used the TechStacks page conventions as a test case which are all passing.

1 Like

That’s very cool, I’ll try it out. I actually did implement a version of my own (in a fallback route), but mine only works for directories and is hooked up in a FallbackRoute, so yours is much more effective. I prefixed my partials with “_” in a demo I did, so that the files couldn’t be called directly, I wonder if that partial naming strategy should change now, or maybe serve partials from a different directory like /partials, what is your recommendation for naming partials?

Is FallbackHandlers something new that you’ll eventually add and implement throughout the framework?

– a few comments on some testing i’ve done, none of this is me asking for changes :slight_smile:

When I was testing things out with a CI/CD, I ended up changing the Program.cs file in a few ways (with my project file loading the *.dlls directly as references), possibly not ways desired by all:

  • allow the web.settings file to use ‘full’ paths in the contentRoot and webRoot parameters (instead of relative paths which were getting a little gnarly)
  • not care about configurations in the web.settings for non-existent plugins, basically ignores them instead of throwing an exception
  • not care about missing plugins, and allow for an empty plugins/* directory even if that’s in the web.settings (allows for dynamic runtime configuration and upload of plugins with a quick app restart). alternatively start with a dummy plugin file in there
  • allow using a different folder than plugins/* as that can have conflicts with routes since /plugins is a pretty common path in lots of apps, only a problem when contentRoot and webRoot are the same folder

Here are a few great features that demo’d well for me last week with a client:

  • Ability to quickly convert an HTML template (like those on ThemeForest) into Templates enabled HTML, and breakdown the HTML5 template sections/blocks into partials
  • Ability to create an api and or use json files directly to populate the HTML files and dynamically create a blog, and documentation as an example
  • Showed how easy it was to implement the equivalent of vue-meta (similar to react-helmet)

Couple things I’m still trying to figure out for a full, even more complete demo (kinda pushing the limits):

  • npm build/watch task for compiling es6 JS and sass/css for development (should be easy with rollup or webpack)
  • an equivalent to nuxt generate, which I think can just be done by crawling the site and outputting the HTML to files on disk (maybe with a config file), then just merging the output of the JS/CSS compilation and static files up to an S3 static site or Netlify --> which basically makes Templates a true static site generator (Wyam is the only .NET static generator right now and it’s not .NET core compatible)
  • create some NodeServices filters that do SSR of Riot and Vue library or single-file-components passing in data (something a bit more elegant than this simple demo)
  • sourcing data from a headless CMS like self-hosted https://getdirectus.com/, and/or https://www.datocms.com/ (the simplest and most small-business friendly headless CMS i’ve found so far) - ultimately a simple ServiceStack powered CMS with an EAV OrmLite db model, or dynamically generated entity models, and media automatically uploaded to an S3 bucket is where I’d really like to go with this (s3 bucket of media served by imgix and/or cloudinary).

More to come!

I prefer the existing cascading resolution being able to group layouts and partials together with the pages that use them which makes it easier to encapsulate and copy/move features around.

To get around the naming convention of hidden partials/layouts essentially any “wildcard page” i.e. starting with _ that contains the word layout or partial are ignored, so they should be included as part of the naming conventions for layouts/partials.

Yes they’re the last filters that gets executed for unhandled requests directly after any [FallbackRoute]. I’ve added them here to avoid any Disk I/O for normal requests as the new IAppHost.FallbackHandlers are only executed for what would otherwise return a 404 NotFound response.

Ok that’s unfortunate as it would be nice if all Web Apps could use the same unmodified binaries with any customizations available by either config settings or environment variables.

This should already be supported, i.e. the paths are only resolved if they start with ~/ otherwise the full paths are used as-is.

We could have a strict false web.setting to control this behavior.

We could have a pluginsDir webappPlugins setting to control this.

Sweet they look like great use-cases for this.

Sounds like PacelJS would be ideal at this as it optimizes a normal website OOB with zero configuration and includes built-in support for ES6 / scss if you need it. It appears everyone is trying hard to hide Webpack these days with all JS frameworks we support in the latest NetCoreTemplates (i.e. angular, aurelia, react, vue + nuxt) now use their own “CLI” solution in which they hide Webpack configuration from devs (Aurelia has a CLI but still includes a webpack config).

But ideally I’d like to get by just using a single TypeScript dependency. v3 adds some useful features like project references and --build mode which may help with this. Transpiling a high-level language like ES6/TypeScript is the one must have feature for modern websites as I don’t think scss offers that much given most websites start off with a css framework like Bootstrap which is already compiled/minified.

Yeah I think it needs to work as transparent as possible so something like calling a list of urls and saving static HTML files would be the easiest solution. The issue is for dynamic sites that make use of _wildcard pages can’t be predicted statically and would need to be supplied a list of URLs/paths. This is a similar to what the Sitemap feature needs where it generates a dynamic list of routes to tell search engines which pages to index. So maybe the same solution for generating Sitemap routes can be reused for the dynamic list of routes needed to generate a static site.

I’m of the opinion that SSR only makes sense for Node Apps where the overhead and complexity to dynamically evaluate JS in a different language isn’t worth it. So I think a proxy solution to a node.exe App deployed together would be the path of least resistance.

FYI I’ve created a new parcel-webapp template which integrates the zero configuration simplicity of Parcel JS in a npm-powered Web App template.

It also includes a server plugin which can be built and copied to /app/plugin before starting the server with:

$ npm run server

The parcel-web template contains docs on the other npm scripts.

Hi Matt,

I’ve just finished creating a new Blog Web App you might be interested in as it’s a multi-user, Twitter Auth-enabled minimal blogging solution that uses ServiceStack Templates for all its functionality and showcases a lot of new capabilities recently added to Web Apps.

The App is self documenting :slight_smile: where the features of the Blog App and new Template features it uses are written as several blog posts.

If you clone the NetCoreWebApps/Blog repo and run it the first time it will create a new blog.sqlite database populated with new blog posts, or a quicker way would be to visit the website below where it’s also published to:

1 Like

This looks really good! This example is very helpful too, the parcel one as well. I’m glad you just create the sqlite db now too :slight_smile: instead of the exception.

1 Like

Is there a way to rsync deploy html files and force the server to pick up the new html files without restarting the app? Invalidate the cache with an api call, or something like that would be fine (for production, where debug is false)

I’ve added a new invalidateAllCaches filter that clears all caches and invalidates pages which should force a check to see if their backing file has been modified.

So you could call this filter via an API Page:

/postdeploy.html

  {{ invalidateAllCaches | return }}

But I’ve also added a new TemplatesAdminService which will let you call special filters like this with:

/templates/admin/invalidateAllCaches

By default it requires the Admin role, but you can change it to allow any authenticated user to call it with:

Plugins.Add(new TemplatePagesFeature {
    TemplatesAdminRole = RoleNames.AllowAnyUser,
});

Or open it up and allow access to anyone with:

Plugins.Add(new TemplatePagesFeature {
    TemplatesAdminRole = RoleNames.AllowAnon,
});

This can also be configured in Web Apps web.settings with:

TemplatePagesFeature { TemplatesAdminRole: 'AllowAnon' }

Although you’d avoid doing this for publicly accessible apps as it clears all caches and invalidates all pages which would degrade performance if called repeatedly.

This change is available from v5.1.1 that’s now available on MyGet and available in the latest Web App .dll’s in:

https://github.com/NetCoreWebApps/WebApp/tree/master/src/apps/web

1 Like

I get errors when using htmlClass and htmlAttrs. Any idea what I’m doing wrong? For example:

<body {{ {active:true} | htmlClass }}>

produces the error:

SyntaxErrorException: Expected start of identifier but was '=' near: '="" {active:true}="" |="" htmlclass="" }}="">

   ...'
   at ServiceStack.Templates.JsTokenUtils.ParseVarName(ReadOnlySpan`1 literal, ReadOnlySpan`1& varName)
   at ServiceStack.Templates.JsTokenUtils.ParseIdentifier(ReadOnlySpan`1 literal, JsToken& token)
   at ServiceStack.Templates.JsTokenUtils.ParseJsToken(ReadOnlySpan`1 literal, JsToken& token, Boolean filterExpression)
   at ServiceStack.Templates.JsExpressionUtils.ParseJsExpression(ReadOnlySpan`1 literal, JsToken& token, Boolean filterExpression)
   at ServiceStack.Templates.TemplatePageUtils.ParseTemplatePage(ReadOnlyMemory`1 text)
   at ServiceStack.Templates.TemplatePage.Load()
   at ServiceStack.Templates.TemplatePage.Init()
   at ServiceStack.Templates.PageResult.WriteToAsyncInternal(Stream outputStream, CancellationToken token)
   at ServiceStack.Templates.PageResult.WriteToAsync(Stream responseStream, CancellationToken token)
   at ServiceStack.TemplatePageHandler.ProcessRequestAsync(IRequest httpReq, IResponse httpRes, String operationName)

Maybe you have an old version? It works as expected on http://templates.servicestack.net/

Or if you’re using a Parcel template its html minification changes/breaks the syntax as it doesn’t recognize it as a valid html attribute, so you need to use htmlClassList instead, e.g:

1 Like

Looks like parceljs was breaking it. Thx.

@mattjcowan Hey Matt I’ve added significant enhancements in the latest v5.4 Release that simplifies creating Web Apps which I’ll think you’ll like :slight_smile:

https://docs.servicestack.net/releases/v5.4

1 Like