libresilient/docs/ARCHITECTURE.md

Architecture

Eventually this will document the architecture of LibResilient.

Plugins

There are three kinds of plugins:

• Transport plugins
  Plugins that retrieve remote content by any means, for example by using regular HTTPS fetch(), or by going through IPFS. They should also offer a way to publish content by website admins (if relevant credentials or encryption keys are provided, depending on the method).
  Methods these plugins implement:

  • fetch - fetch content from an external source (e.g., from IPFS)
  • publish - publish the content to the external source (e.g., to IPFS)

• Stashing plugins
  Plugins that stash content locally (for example, in the browser cache). This is useful when no transport plugin works, or before remote content is received.
  Methods these plugins implement:

  • fetch - fetch the locally stored content (e.g., from cache)
  • stash - stash the content locally (e.g., in cache)
  • unstash - clear the content from the local store (e.g., clear the cache)

• Composing plugins and Wrapping plugins
  Plugins that compose multiple other plugins (for example, by running them simultaneously to retrieve content from whichever succeeds first); or that wrap other plugins, applying some action to the results returned by the wrapped plugin (for instance, checking known resource integrity hashes on returned content).
  Methods these plugins implement depend on which plugins they compose. Additionally, plugins being composed the uses key, providing the configuration for them the same way configuration is provided for plugins in the plugins key of LibResilientConfig (which is configurable via config.json).

Every plugin needs to be implemented as a constructor function that is added to the LibResilientPluginConstructors Map() object for later instantiation.

The constructor function should return a structure as follows (fields depending on the plugin type):

{
    name: 'plugin-name',
    description: 'Plugin description. Just a few words, ideally.',
    version: 'any relevant plugin version information',
    fetch: functionImplementingFetch,
    publish|stash|unstash: functionsImplementingRelevantFunctionality,
    uses: []
}

Transport plugins

Transport plugins must add X-LibResilient-Method and X-LibResilient-ETag headers to the response they return, so as to facilitate informing the user about new content after content was displayed using a stashing plugin.

• X-LibResilient-Method:
  contains the name of the plugin used to fetch the content.

• X-LibResilient-ETag:
  contains the ETag for the content; this can be an actual ETag header for HTTPS-based plugins, or some arbitrary string identifying a particular version of the resource (e.g., for IPFS-based plugins this can be the IPFS address, since that is based on content and different content results in a different IPFS address).

Stashing plugins

Stashing plugins must stash the request along with the X-LibResilient-Method and X-LibResilient-ETag headers.

Composing plugins

Composing plugins work by composing other plugins, for example to: run them simultaneously and retrieve content from the first one that succeeds; or to run them in a particular order. A composing plugin needs to set the uses key in the object returned by it's constructor. The key should contain mappings from plugin names to configuration:

uses: [{
          name: "composed-plugin-1",
          configKey1: "whatever-data-here"
      },{
          name: "composed-plugin-2",
          configKey2: "whatever-data-here"
      },
      {...}
}]

If these mappings are to be configured via the global configuration file (which is most often the case), the uses key should instead point to config.uses:

uses: config.uses

Wrapping plugins

Wrapping plugins wrap other plugins, in order to performing some actions on request data sent to them, or on response data received from them. A wrapping plugin needs to set the uses key in the object returned by it's constructor. The key should contain a mapping from wrapped plugin name to configuration:

uses: [{
          name: "composed-plugin-1",
          configKey1: "whatever-data-here"
      }
}]

If this mapping is to be configured via the global configuration file (which is most often the case), the uses key should instead point to config.uses:

uses: config.uses

Fetching a resource via LibResilient

Whenever a resource is being fetched on a LibResilient-enabled site, the service-worker.js script dispatches plugins in the set order. This order is configured via the plugins key of the LibResilientConfig variable, usually set via the config.json config file.

A minimal default configuration is hard-coded in case no site-specific configuration is provided. This default configuration runs these plugins:

1. fetch, to use the upstream site directly if it is available,
2. cache, to display the site immediately from the cache in case regular fetch fails (if content is already cached from previous visit).

A more robust configuration could look like this:

{
    "plugins": [{
            "name": "fetch"
        },{
            "name": "cache"
        },{
            "name": "alt-fetch",
            "endpoints": [
                "https://fallback-endpoint.example.com"
            ]
        }]
}

For each resource, such a config would:

1. Perform a regular fetch() to the main site domain first; if that succeeds, content is added to cache and displayed to the user.
2. If the fetch() failed, the cache would be checked.
   1. If the resource was cached, it would be displayed; at the same time, a background request for that resource would be made to fallback-endpoint.example.com instead of the (failing) main domain; if that succeeded, the new version of the resource would be cached.
   2. If the resource was not cached, a request for that resource would be made to fallback-endpoint.example.com; if that succeded, the resource would be displayed and cached.

Stashed versions invalidation

Invalidation heuristic is rather naïve, and boils down to checking if either of X-LibResilient-Method or X-LibResilient-ETag differs between the response from a transport plugin and whatever has already been stashed by a stashing plugin. If either differs, the transport plugin response is considered "fresher".

This is far from ideal and will need improvements in the long-term. The difficulty is that different transport plugins can provide different ways of determining the "freshness" of fetched content -- HTTPS-based requests offer ETag, Date, Last-Modified, and other headers that can help with that; whereas IPFS can't really offer much apart from the address which itself is a hash of the content, so at least we know the content is different (but is it fresher though?).

Messaging

The ServiceWorker can communicate with the browser window using the Client.postMessage() to post messages to the browser window context using the relevant Client ID, retrieved from the fetch event object.

When the browser window context wants to message the service worker, it uses the Worker.postMessage() call, with clientId field set to the relevant client ID if a response is expected. ServiceWorker then again responds using Client.postMessage() using the clientId field as source of the Client ID.

Messages

This section is a work in progress.