archive.social/README.md

thread-keeper 📚

High-fidelity capture of Twitter threads as sealed PDFs @ social.perma.cc.

An experiment of the Harvard Library Innovation Lab.

🚧 Experimental - Prototype. Early release to be consolidated.

---

Summary

• Dependencies
• Local development
• Dev CLI
• Code docs

---

Dependencies

Runtimes

• Node.js 18+
• Python 3.9+

Browsers

• Google Chrome (npx playwright install --force chrome may be used).

Python dependencies

• ⚠️ For now: Python dependencies are installed at machine level, as a post-install step of npm install.

Known Ubuntu packages

curl bash gcc g++ python3 python3-pip python3-dev zlib1g zlib1g-dev libjpeg-dev libssl-dev libffi-dev ghostscript poppler-utils

• ⚠️ On Linux, this project is only compatible with Ubuntu at the time, because it uses Playwright + Chrome.
• Node may be sourced from Nodesource.

For development on Mac OS

A brewfile is available. Run brew bundle to install machine-level dependencies that can be provided by homebrew.

☝️ Back to summary

---

Local development

Getting started

Run the following commands to initialize the project and start the development server.

brew bundle # (Mac OS only) - See Linux dependencies above.
npm install # To install npm packages
npx playwright install chrome # To ensure Playwright has a version of Chrome to talk to
npm run generate-dev-cert # Will generate a certificate for self-signing PDFs. For testing purposes only.
npm run dev # Starts the development server on port 3000

Access keys

Create an access key to test with:

$ uuidgen
BB67BBC4-1F4B-4353-8E6D-9927A10F4509

And then add the key to app/data/access-keys.json:

{
  "BB67BBC4-1F4B-4353-8E6D-9927A10F4509": true,
}

Certificates history

The "Signatures Verification Page" page lists the certificates that were used for signing PDFs with the app. You may provide that history by creating two files under /data:

• signing-certs-history.json
• timestamping-certs-history.json

Expected format:

[
  {
    "from": "2022-11-18 13:07:56 UTC",
    "to": "present",
    "domain": "domain.ext",
    "info": "https://...",
    "cert": "https://..."
  },
  ...
]

☝️ Back to summary

---

Dev CLI

start

npm run start

Starts the app's server on port 3000 with warning-level logs.

dev

npm run dev

Starts the app's server on port 3000 with info-level logs. Watches for file changes.

generate-dev-cert

npm run generate-dev-cert

Generate a certs/cert.pem and certs/key.pem for local development purposes.

docgen

npm run docgen

Generates JSDoc-based code documentation under /docs.

test

npm run test

Runs the test suite. Requires test fixtures (see fixtures folder).

⚠️ At the moment, this codebase only features a very limited set of high-level integration tests.

☝️ Back to summary