Got an IndieWeb site? Want to interact with federated social networks like Mastodon, Hubzilla, and more? Bridgy Fed is for you.
 
 
 
 
Go to file
Ryan Barrett 7c7e2b1153
upgrade google-cloud-logging to 2.0
2020-12-30 10:39:07 -08:00
.circleci circle: clear cache to work around grpcio build failure 2020-11-14 12:46:42 -08:00
static [ImgBot] Optimize images 2019-10-16 11:48:05 -07:00
templates webmention: refactor storing Responses so we can do it earlier 2017-10-26 12:31:04 -07:00
tests webfinger: handle acct:[domain]@fed.brid.gy requests 2020-12-30 10:26:48 -08:00
.gcloudignore webmention => AP: handle multiple in-reply-to links 2020-06-06 08:40:07 -07:00
.gitignore temporarily disable actually deleting Followers on AP Delete requests 2020-10-11 08:35:14 -07:00
README.md temporarily disable actually deleting Followers on AP Delete requests 2020-10-11 08:35:14 -07:00
activitypub.py fix outbound AP headers and signature for Mastodon 3.2.1 2020-11-05 12:43:27 -08:00
add_webmention.py unify request handler classes and handle_exception; move away from HOST[_URL] 2020-01-31 14:40:11 -08:00
app.py migrate to the app engine standard python 3 runtime! 2019-12-25 22:26:26 -08:00
app.yaml temporarily disable actually deleting Followers on AP Delete requests 2020-10-11 08:35:14 -07:00
appengine_config.py unify request handler classes and handle_exception; move away from HOST[_URL] 2020-01-31 14:40:11 -08:00
common.py webfinger: handle acct:[domain]@fed.brid.gy requests 2020-12-30 10:26:48 -08:00
index.yaml migrate to the app engine standard python 3 runtime! 2019-12-25 22:26:26 -08:00
logs.py migrate to the app engine standard python 3 runtime! 2019-12-25 22:26:26 -08:00
models.py unify request handler classes and handle_exception; move away from HOST[_URL] 2020-01-31 14:40:11 -08:00
oauth_dropins webmention: handle missing Content-Type header 2020-05-20 23:36:11 -07:00
redirect.py unify request handler classes and handle_exception; move away from HOST[_URL] 2020-01-31 14:40:11 -08:00
render.py unify request handler classes and handle_exception; move away from HOST[_URL] 2020-01-31 14:40:11 -08:00
requirements.txt upgrade google-cloud-logging to 2.0 2020-12-30 10:39:07 -08:00
salmon.py handle django-salmon port to pycryptodome 2020-10-31 13:23:36 -07:00
superfeedr.py unify request handler classes and handle_exception; move away from HOST[_URL] 2020-01-31 14:40:11 -08:00
webfinger.py webfinger: handle acct:[domain]@fed.brid.gy requests 2020-12-30 10:26:48 -08:00
webmention.py if ActivityPub inbox delivery fails, don't fall back to Salmon 2020-11-13 09:50:14 -08:00

README.md

Bridgy Fed Circle CI Coverage Status

Got an IndieWeb site? Want to interact with federated social networks like Mastodon, Hubzilla, and more? Bridgy Fed is for you.

https://fed.brid.gy/

Bridgy Fed connects the IndieWeb with federated social networks using ActivityPub and OStatus. Original design docs:

License: This project is placed in the public domain.

Development

You'll need Python 3. Install the Google Cloud SDK (aka gcloud) with the gcloud-appengine-python and gcloud-appengine-python-extras components. Then, run:

python3 -m venv local
source local/bin/activate
pip install -r requirements.txt

Now, run the tests to check that everything is set up ok:

gcloud beta emulators datastore start --no-store-on-disk --consistency=1.0 --host-port=localhost:8089 < /dev/null >& /dev/null &
python3 -m unittest discover
kill %1

Finally, run this in the repo root directory to start the web app locally:

dev_appserver.py --log_level debug --enable_host_checking false --support_datastore_emulator --datastore_emulator_port=8089 --application=bridgy-federated app.yaml

If you send a pull request, please include (or update) a test for the new functionality!

If you hit an error during setup, check out the oauth-dropins Troubleshooting/FAQ section. For searchability, here are a handful of error messages that have solutions there:

bash: ./bin/easy_install: ...bad interpreter: No such file or directory

ImportError: cannot import name certs

ImportError: No module named dev_appserver

ImportError: cannot import name tweepy

File ".../site-packages/tweepy/auth.py", line 68, in _get_request_token
  raise TweepError(e)
TweepError: must be _socket.socket, not socket

error: option --home not recognized

You may need to change granary, oauth-dropins, webmention-tools, mf2util, or other dependencies as well as as Bridgy Fed. To do that, clone their repo locally, then install them in "source" mode with e.g.:

pip uninstall -y granary
pip install -e <path to granary>

To deploy to the production instance on App Engine - if @snarfed has added you as an owner - run:

gcloud -q beta app deploy --no-cache --project bridgy-federated *.yaml

Compatibility

Here are in progress notes on how I'm testing interoperability with various federated social networks.

ActivityPub

Salmon

Stats

I occasionally generate stats and graphs of usage and growth via BigQuery, like I do with Bridgy. Here's how.

  1. Export the full datastore to Google Cloud Storage. Include all entities except MagicKey. Check to see if any new kinds have been added since the last time this command was run.

    gcloud datastore export --async gs://bridgy-federated.appspot.com/stats/ --kinds Follower,Response
    

    Note that --kinds is required. From the export docs:

    Data exported without specifying an entity filter cannot be loaded into BigQuery.

  2. Wait for it to be done with gcloud datastore operations list | grep done.

  3. Import it into BigQuery:

    for kind in Follower Response; do
      bq load --replace --nosync --source_format=DATASTORE_BACKUP datastore.$kind gs://bridgy-federated.appspot.com/stats/all_namespaces/kind_$kind/all_namespaces_kind_$kind.export_metadata
    done
    
  4. Check the jobs with bq ls -j, then wait for them with bq wait.

  5. Run the full stats BigQuery query. Download the results as CSV.

  6. Open the stats spreadsheet. Import the CSV, replacing the data sheet.

  7. Check out the graphs! Save full size images with OS or browser screenshots, thumbnails with the Download Chart button.