SSR for FAST Elements

[EisenbergEffect]

Overview

Server-side rendering (SSR) is the process of generating HTML markup for an arbitrary experience on the server and sending the result to the client. The primary benefit of SSR is that it can increase real and perceived first-load performance. It does this by reducing the amount of JavaScript that needs to be parsed and executed to render a view, instead delegating that rendering code to the server. This means the first bits a client receives can be the HTML representing the view, not the JavaScript required to create the markup.

This document outlines the core features required to enable SSR for the FAST library. It describes the features in mid-level detail to aid feature and task planning.

The key browser feature added to enable SSR for Web Components is called declarative shadow DOM. Simply put, this feature enables writing HTML in such a way that the browser itself will attach the shadow DOM of a custom element automatically without JavaScript. This allows pre-generating web-component shadow DOMs on the server and automatic upgrades on the client, meaning full web-component experience generation outside the browser is possible.

FAST’s approach to SSR will be to conform to the interfaces proposed by Lit (details below) that will allow SSR interoperability with other Web Component libraries.

Challenges / Risks

Hydrating Components

In most cases, components will need to be hydrated on the client, with hydration meaning that control of the component is give back to the JavaScript constructor for that element and it generally functions as if it had not been initially created using declarative shadow DOM. This JavaScript execution has a cost, and that cost eats into any potential performance improvements gained by SSR, though it is likely not substantially different than client-side rendering in the first place.

State Changes

There are times when the state on the client may be different than the state used during HTML generation on the server. This can mean the DOM must update immediately upon hydration, causing jarring visual changes, such as UI components moving, text content changing, themes changing, etc.

Additional Data Transfer

In most (but not all) cases the Web Component and application logic JavaScript will still need to be sent to the client. This can mean that more data must be sent to the client in total to support SSR. This is due to the need to send a larger HTML file and any additional JavaScript to facilitate client-side pre-hydration (such as event capturing) and styling.

Code Compression

The template renderer implements a streaming model where template results can begin to be sent to the client prior to the entire template being processed. Leveraging this feature likely means that code compression techniques like gzipping cannot be leveraged.

Considerations / Features

Community Interface Conformance

Lit has unofficially proposed a interface to facilitate SSR interop of Custom Elements from different Custom Element templating libraries. The general problem this solves is that because each Custom Element library (FAST, Lit, etc) will contain distinct implementation details for component creation, lifecycles, and templating features and syntaxes, the processes for creating a SSR string representation of that component will also be specific to the library. The common interface serves as an interop mechanism for delegating rendering to renderers written by library authors for their library’s components. This manifests as two distinct categories of code:

Template Renderer

The template renderer is the piece of code that converts a string representing a custom element with attributes () into the SSR result of that element. It does this by simulating DOM connection for the element instantiation, updating any necessary properties from attributes, and yielding out any shadow-DOM the element has as declarative shadow DOM. This will be the bulk of the SSR implementation.
The template renderer is responsible for expanding arbitrary HTML templates into a stream of HTML string data expanded with declarative shadow DOM and any HTML attributes dynamically added by custom element construction and connection lifecycles.

The template renderer will look something like:

/**
 * @param template - The product of fast-elements html tagged template literal
 * @param source - Any source data the template should be bound with
 * @param renderInfo - Data about the current render context that facilitates interop with the community interfaces
 */
render(template: ViewTemplate, source: unknown, renderInfo: RenderInfo)

Whole Page Rendering

The renderer will function as the mechanism to render both custom element shadow DOMs and arbitrary HTML fragments, including entry point experiences that include HTML document information such as doctype, html, head, and body elements, meta tags, scripts, styles, etc. This will facilitate simple usage with fast-element’s html tagged-template literal.

SSR Directive Implementations

FAST’s directive architecture is closely tied to a DOM implementation, so the renderer will need to patch implementations of directives so that they execute in a NodeJS environment with a minimal DOM shim. This is an opportunity for FAST to implement a directive plugin model for all first-party directives, including:

1. When
2. Repeat
3. Slotted
4. Ref
5. Children
   Architecting this feature using a plugin model will allow any author to create their own directives and SSR implementations (if necessary) and additionally override any first-party directive SSR implementations.

Open Questions:

1. How does an author configure directive implementations? Where are they provided as a configuration?

Light DOM Emission

Part of the community interfaces establish a mechanism for a custom element to emit to light DOM instead of shadow DOM. This is to support browsers that do not natively support shadow DOM. Emission to light DOM will need to change the behavior of the renderer to skip the declarative shadow DOM template element, instead simply emitting the shadow DOM as children of the custom element.

Preventing Rendering

There may be cases where an author may wish to prevent emission of all custom element constructors or custom element instances. We can add a mechanism to opt constructors out by configuring the ElementRenderer, and there is potential to upstream that mechanism into the community protocol.

Opting individual component instances is trickier because that information will need to be embedded into the component implementation itself. I think we’ll need some method that can be implemented into a component constructor that can be executed by the ElementRenderer or the renderer.

Open Questions:

1. How does an author configure the renderer to opt specific constructors out of SSR emission?

ElementRenderer

The ElementRenderer facilitates interop between other renderers and orchestrates custom element creation and rendering. It is responsible for recognizing custom element constructors it should be responsible for, creating custom element instances, dispatching attributes, and invoking the renderer for a custom element. This is a small and somewhat trivial piece of code.

Stylesheet Emission

Custom Element stylesheets are a potential source for significant redundant shadow DOM content that must be sent as part of the SSR string. If the SSR result is streamed and not pre-compiled, these redundancies will not be able to benefit from compression processes that otherwise would help negate the negative impacts of redundant string data. It is therefore important to devise a technique to memoize stylesheets on the server and re-hydrate them on the client to avoid a significant increase in redundant data transfer.
This is not a problem that is specific to FAST, so there is a potential opportunity here to either expand the community protocol proposed by Lit to help address this problem, or to propose a new community protocol.

FASTStyle Custom Element

We will address the above by injecting a small bit of JavaScript into the yielded SSR string. This JavaScript will define a custom element that manages a cache of stylesheets. The cache is populated by assigning the css attribute of an element instance with the raw CSS string and assigning the ‘key’ attribute to be a hash of the sheet.
During rendering, the renderer will emit the above element for all stylesheets in the component. If the stylesheet is being reflected to this element for the first time, the css attribute will be populated with the stylesheet content. The id of the stylesheet, a hash of the sheet content, will always be assigned.
When the FASTStyle element constructs on the client, the first construction will always be the instance of the element with stylesheet content (because content is only emitted to the first instance). It will populate its cache with a CSSStyleSheet instance with the stylesheet content from the server. It will then append the sheet to its root element (the custom element’s shadow root). Any subsequent constructions of the FASTStyle element for that stylesheet will skip CSSStyleSheet construction and append the sheet immediately to the root element.

In cases where CSSStyleSheet isn’t constructable, the FASTStyle will create style elements instead, and append those to the root element.

Open Questions:

1. How do we opt into this behavior? Where is this configured?
2. How do we configure the custom element name so that it doesn’t conflict with any other custom elements in the experience?

Custom Element Client-Side Hydration

Phase 1 - Naïve implementation

This initial implementation does not “hydrate” the server side rendered HTML but allows the component to re-create HTML during its usual initialization phase.

Phase 2 – Comprehensive DOM hydration

Hydration involves a different process from custom-element upgrading and connection because the shadow DOM for the element will have already been created. In hydration cases, the DOM should be re-used to preserve the performance benefits of rendering on the server.
To do this, several aspects of fast-element will need to change:

1. Constructing the element will need to adopt the existing shadowRoot instead of creating its own. Because of this, closed shadow roots will not be supported from SSR (if shadow roots are closed, fast-element cannot gain a reference to it reliably).
2. Constructing the element should not emit elements to the shadow root initially.
3. Directives will need to be hydrated with the appropriate source elements
4. Stylesheets will need to be converted to ElementStyles as appropriate from any FASTStyle custom elements. Alternatively, all FASTStyle CSSStyleSheets could be removed, leaving only the sheets created by FAST.
5. Descendent elements will need to be instantiated so that they can hydrate (hydrate should happen from the top down so that child elements can reference parent elements as necessary).

Command Buffering

Command buffering involves catching any user interaction that happens between page load and hydration of custom elements.
Typically command buffering can be complicated since in general (non-web component) scenarios it is not always known in advance what user events may occur or need to be captured. This results in large, complicated logic intended to handle any generic web page code. However, in the case of web components and FAST specifically the user events that any component needs to respond to are declared in the template and therefore the SSR Renderer can know exactly which events on each component it needs to care about.
FAST SSR Command Buffering can therefore be accomplished by emitting a small amount of code into the tag which will define a global scope method for recording user events. This could be an invisible web component or simply a vanilla javascript object. This method needs only to take in some form of identifier (hash or otherwise) for the exact element within a specific component instance that is handling the event along with the event object and place them into a queue. Then as the Element Renderer parses the template on the server, whenever it encounters an event directive (@click, @input, etc.) it can emit an appropriate event attribute into the tag (click, input, etc.) with code that will call the global event recording method if that event occurs.
On the client side as FAST Element hydrates the components whenever it needs to bind an event it can check with the Command Buffer to see if it has recorded that event and if so match the recorded event to the newly bound expression. FAST can then clean up the original event attribute so that it will no longer record events after upgrading is complete. Then once all components are upgraded FAST can trigger the Command Buffer to replay the events in the order that they were received.
The Command Buffering process will then be:

1. Emit the Command Buffer object into the document head with methods for recording an event, mapping a recorded event to a component event expression, and replaying all mapped events in order.
2. As the Element Renderer encounters event binding expressions in a component template it instead emits the matching HTML event attribute that simply calls the Command Buffer record method.
3. If the user triggers one of these events before hydration occurs, then the event is recorded. Since the components do not respond to any other events except those that are declaratively bound there is no need to record anything else the user does.
4. As FAST Element upgrades the components it can check to see if the event on each specific instance has been triggered and match up the recorded event with the correct expression bound to that event and remove the original HTML event attribute code so it will no longer record events.
5. Once all components are upgraded, FAST can call the “playAll” method on the Command Buffer object.

Phase 2 - CLI / Tooling

A command line interface would be beneficial for facilitating SSR as part of the project creation and a continuing use as part of a pipeline.

• Project setup – This may provide initial configuration and setup of a project that includes SSR
• File generation – This may include some commands to generate HTML files on the server

Testing Plan

Renderer

The renderer will largely leverage playwright to test the product of the render process. This will allow navigating to a real browser page containing the SSR result, testing real elements. We should capture tests for all the following:

1. DOCTYPE, html, head, and body element emission (this could be challenging to test w/ playwright because the browser will auto create them. It might be prudent to just check the string output.
2. Defer-hydration behavior (all custom elements within a shadow DOM should have the defer-hydration attribute)
3. Native element emission
4. Text emission
5. Nested element emission
6. Slotted element emission
7. Bound string content emission
8. When and repeat directive emission
9. Declarative shadow DOM emission
10. Attribute emission
    a. Unbound and bound
11. Boolean attribute emission
    a. Unbound and bound
12. Property binding
13. Attribute binding
14. Style emission (See below)
15. Custom SSR directive plugin
16. SSR opt-out for components and instances

Hydration

Hydration will be a feature of fast-element, so tests for hydration should exist in that package. Test cases will be added for:

1. Directive hydration w/ SSR generated source elements
2. Directive updates after source element adoption
3. Detecting and adopting existing shadow roots for declarative shadow DOM elements
4. Hydrating controller with stylesheets from fast-style elements
5. Phase 1 – in the case that content is not hydrated but blown away instead, does this result in content jumping in the UI

FASTStyle Element

Tests for the FASTStyle element will be relatively trivial because we’ll just be testing isolated custom element behavior. We should use playwright to avoid requiring multiple unit testing architectures in the SSR package. Tests should include:

1. Testing cache hydration with style strings
2. Test and catch duplicate key assignment (how are hash conflicts resolved?)
3. Style retrieval from cache, what should happen in failure case
4. CSSStyleSheet creation and append for browsers that support CSSStyleSheet constructor
5. Style element creation and append for browsers that don’t support CSSStyleSheet constructor

[nicholasrice]

Some investigation work for this going on here: https://github.com/microsoft/fast/tree/users/nirice/ssr-invesigation

[yinonov]

another challenge / risk is any code dependent on browsers' API which isn't supported by the SSR run-time engine.
might not be the optimal use-case here but in a practice where a library of component has a common core dependency which contains a style mounting practice of tokens. A code that appends a stylesheet to a document will fail.
This is just a close issue I experienced myself but there are probably more cases where some communication between components is required and rely on browsers.

(it's kind of mentioned above but in a relation to a Fast API)