libresilient/docs/ARCHITECTURE.md

# Architecture

Eventually this will document the architecture of LibResilient.

## Plugins

There are three kinds of plugins:

- **Transport plugins**  
  Plugins that *retrieve* remote content by any means, for example by using regular HTTPS [`fetch()`](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API), or by going through [IPFS](https://js.ipfs.io/). They *should* also offer a way to *publish* content by website admins (if relevant credentials or encryption keys are provided, depending on the method).  
Methods these plugins implement:
  - `fetch` - fetch content from an external source (e.g., from IPFS)
  - `publish` - publish the content to the external source (e.g., to IPFS)

- **Stashing plugins**  
  Plugins that *stash* content locally (for example, in the [browser cache](https://developer.mozilla.org/en-US/docs/Web/API/Cache)). This is useful when no *transport plugin* works, or before remote content is received.  
Methods these plugins implement:
  - `fetch` - fetch the locally stored content (e.g., from cache)
  - `stash` - stash the content locally (e.g., in cache)
  - `unstash` - clear the content from the local store (e.g., clear the cache)

- **Composing plugins** and **Wrapping plugins**  
  Plugins that *compose* multiple other plugins (for example, by running them simultaneously to retrieve content from whichever succeeds first); or that *wrap* other plugins, applying some action to the results returned by the wrapped plugin (for instance, checking known resource integrity hashes on returned content).  
Methods these plugins implement depend on which plugins they compose. Additionally, plugins being composed the `uses` key, providing the configuration for them the same way configuration is provided for plugins in the `plugins` key of `LibResilientConfig` (which is configurable via `config.json`).

Every plugin needs to be implemented as a constructor function that is added to the `LibResilientPluginConstructors` [Map()](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map) object for later instantiation.

The constructor function should return a structure as follows (fields depending on the plugin type):

```javascript
{
    name: 'plugin-name',
    description: 'Plugin description. Just a few words, ideally.',
    version: 'any relevant plugin version information',
    fetch: functionImplementingFetch,
    publish|stash|unstash: functionsImplementingRelevantFunctionality,
    uses: []
}
```

### Transport plugins

Transport plugins *must* add `X-LibResilient-Method` and `X-LibResilient-ETag` headers to the response they return, so as to facilitate informing the user about new content after content was displayed using a stashing plugin.

 - **`X-LibResilient-Method`**:  
   contains the name of the plugin used to fetch the content.

 - **`X-LibResilient-ETag`**:  
   contains the [ETag](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag) for the content; this can be an actual `ETag` header for HTTPS-based plugins, or some arbitrary string identifying a particular version of the resource (e.g., for IPFS-based plugins this can be the IPFS address, since that is based on content and different content results in a different IPFS address).

### Stashing plugins

Stashing plugins *must* stash the request along with the `X-LibResilient-Method` and `X-LibResilient-ETag` headers.

### Composing plugins

Composing plugins work by composing other plugins, for example to: run them simultaneously and retrieve content from the first one that succeeds; or to run them in a particular order. A composing plugin needs to set the `uses` key in the object returned by it's constructor. The key should contain mappings from plugin names to configuration:

```javascript
uses: [{
          name: "composed-plugin-1",
          configKey1: "whatever-data-here"
      },{
          name: "composed-plugin-2",
          configKey2: "whatever-data-here"
      },
      {...}
}]
```

If these mappings are to be configured via the global configuration file (which is most often the case), the `uses` key should instead point to `config.uses`:

```javascript
uses: config.uses
```

### Wrapping plugins

Wrapping plugins wrap other plugins, in order to performing some actions on request data sent to them, or on response data received from them.
A wrapping plugin needs to set the `uses` key in the object returned by it's constructor. The key should contain a mapping from wrapped plugin name to configuration:

```javascript
uses: [{
          name: "composed-plugin-1",
          configKey1: "whatever-data-here"
      }
}]
```

If this mapping is to be configured via the global configuration file (which is most often the case), the `uses` key should instead point to `config.uses`:

```javascript
uses: config.uses
```

## Fetching a resource via LibResilient

Whenever a resource is being fetched on a LibResilient-enabled site, the `service-worker.js` script dispatches plugins in the set order. This order is configured via the `plugins` key of the `LibResilientConfig` variable, usually set via the `config.json` config file.

A minimal default configuration is hard-coded in case no site-specific configuration is provided. This default configuration runs these plugins:

1. `fetch`, to use the upstream site directly if it is available,
1. `cache`, to display the site immediately from the cache in case regular `fetch` fails (if content is already cached from previous visit).

A more robust configuration could look like this:

```json
{
    "plugins": [{
            "name": "fetch"
        },{
            "name": "cache"
        },{
            "name": "alt-fetch",
            "endpoints": [
                "https://fallback-endpoint.example.com"
            ]}
        }]
}
```

For each resource, such a config would:

1. Perform a regular `fetch()` to the main site domain first; if that succeeds, content is added to cache and displayed to the user.
1. If the `fetch()` failed, the cache would be checked.
   1. If the resource was cached, it would be displayed; at the same time, a background request for that resource would be made to `fallback-endpoint.example.com` instead of the (failing) main domain; if that succeeded, the new version of the resource would be cached.
   1. If the resource was not cached, a request for that resource would be made to `fallback-endpoint.example.com`; if that succeded, the resource would be displayed and cached.

## Stashed versions invalidation

Invalidation heuristic is rather naïve, and boils down to checking if either of `X-LibResilient-Method` or `X-LibResilient-ETag` differs between the response from a transport plugin and whatever has already been stashed by a stashing plugin. If either differs, the transport plugin response is considered "*fresher*".

This is far from ideal and will need improvements in the long-term. The difficulty is that different transport plugins can provide different ways of determining the "*freshness*" of fetched content -- HTTPS-based requests offer `ETag`, `Date`, `Last-Modified`, and other headers that can help with that; whereas IPFS can't really offer much apart from the address which itself is a hash of the content, so at least we know the content is *different* (but is it *fresher* though?).

## Messaging

The ServiceWorker can communicate with the browser window using the [`Client.postMessage()`](https://developer.mozilla.org/en-US/docs/Web/API/Client/postMessage) to post messages to the browser window context using the relevant [`Client ID`](https://developer.mozilla.org/en-US/docs/Web/API/Client/id), retrieved from the fetch event object.

When the browser window context wants to message the service worker, it uses the [`Worker.postMessage()`](https://developer.mozilla.org/en-US/docs/Web/API/Worker/postMessage) call, with `clientId` field set to the relevant client ID if a response is expected. ServiceWorker then again responds using `Client.postMessage()` using the `clientId` field as source of the `Client ID`.

### Messages

This section is a work in progress.