Wykres commitów

249 Commity (main)

Autor SHA1 Wiadomość Data
Ryan Barrett e18dabf510
implement ATProto.target_for, .fetch for at:// URIs 2023-08-31 10:48:28 -07:00
Ryan Barrett e194ccb3ad
Protocol.load/fetch returns True/False (or None) to mean "not this protocol"
ie, nothing failed, but the given id eitehr doesn't below to the given protocol, or fetched successfully but its data belongs to a different protocol.

vs raising exceptions for fetches that fail or error, eg network connection problems, Web got HTML with microformats2 but no h-entry, ActivityPub got non-AS2, etc.

used in Protocol.for_id: when a given Protocol's fetch doesn't work, we move on and try the next protocol _if_ it returned False (ie wasn't fetchable with that protocol, but we fail fast if it raises an exception (ie fetch failed).
2023-07-14 12:58:47 -07:00
Ryan Barrett b3a3de73f2
extend Web/ActivityPub.owns_id() to return False if blocklisted 2023-07-11 13:25:29 -07:00
Ryan Barrett 0caca9243e
add common.add to add to repeated properties only if not already there
tried a bunch of other more sophisticated ways to do this in snarfed/webutil@280a93e092 (plus snarfed/webutil@6cb9c1f719), tried a custom repeated property class that acted like a set instead of a list, but couldn't get it to work reliably or compatibly enough.
2023-06-29 22:15:07 -07:00
Ryan Barrett ab1c28ee4d
noop, lint fixes from flake8
remaining:

$ flake8 --extend-ignore=E501 *.py tests/*.py
"pyflakes" failed during execution due to "'FlakesChecker' object has no attribute 'NAMEDEXPR'"
Run flake8 with greater verbosity to see more details
activitypub.py:15:1: F401 'oauth_dropins.webutil.util.json_loads' imported but unused
activitypub.py:36:1: F401 'web' imported but unused
activitypub.py:48:1: E302 expected 2 blank lines, found 1
activitypub.py:51:9: F811 redefinition of unused 'web' from line 36
app.py:6:1: F401 'flask_app.app' imported but unused
app.py:9:1: F401 'activitypub' imported but unused
app.py:9:1: F401 'convert' imported but unused
app.py:9:1: F401 'follow' imported but unused
app.py:9:1: F401 'pages' imported but unused
app.py:9:1: F401 'redirect' imported but unused
app.py:9:1: F401 'superfeedr' imported but unused
app.py:9:1: F401 'ui' imported but unused
app.py:9:1: F401 'webfinger' imported but unused
app.py:9:1: F401 'web' imported but unused
app.py:9:1: F401 'xrpc_actor' imported but unused
app.py:9:1: F401 'xrpc_feed' imported but unused
app.py:9:1: F401 'xrpc_graph' imported but unused
app.py:9:19: E401 multiple imports on one line
models.py:19:1: F401 'oauth_dropins.webutil.util.json_loads' imported but unused
models.py:364:31: E114 indentation is not a multiple of four (comment)
models.py:364:31: E116 unexpected indentation (comment)
protocol.py:17:1: F401 'oauth_dropins.webutil.util.json_loads' imported but unused
redirect.py:26:1: F401 'oauth_dropins.webutil.util.json_loads' imported but unused
web.py:18:1: F401 'oauth_dropins.webutil.util.json_loads' imported but unused
webfinger.py:13:1: F401 'oauth_dropins.webutil.util.json_loads' imported but unused
webfinger.py:110:13: E122 continuation line missing indentation or outdented
webfinger.py:111:13: E122 continuation line missing indentation or outdented
webfinger.py:131:13: E122 continuation line missing indentation or outdented
webfinger.py:132:13: E122 continuation line missing indentation or outdented
webfinger.py:133:13: E122 continuation line missing indentation or outdented
webfinger.py:134:13: E122 continuation line missing indentation or outdented
tests/__init__.py:2:1: F401 'oauth_dropins.webutil.tests' imported but unused
tests/test_follow.py:11:1: F401 'oauth_dropins.webutil.util.json_dumps' imported but unused
tests/test_follow.py:14:1: F401 '.testutil.Fake' imported but unused
tests/test_models.py:156:15: E122 continuation line missing indentation or outdented
tests/test_models.py:157:15: E122 continuation line missing indentation or outdented
tests/test_models.py:158:11: E122 continuation line missing indentation or outdented
tests/test_web.py:12:1: F401 'oauth_dropins.webutil.util.json_dumps' imported but unused
tests/test_web.py:17:1: F401 '.testutil' imported but unused
tests/test_web.py:1513:13: E128 continuation line under-indented for visual indent
tests/test_web.py:1514:9: E124 closing bracket does not match visual indentation
tests/testutil.py:106:1: E402 module level import not at top of file
tests/testutil.py:107:1: E402 module level import not at top of file
tests/testutil.py:108:1: E402 module level import not at top of file
tests/testutil.py:109:1: E402 module level import not at top of file
tests/testutil.py:110:1: E402 module level import not at top of file
tests/testutil.py:301:24: E203 whitespace before ':'
tests/testutil.py:301:25: E701 multiple statements on one line (colon)
tests/testutil.py:301:25: E231 missing whitespace after ':'
2023-06-20 11:29:31 -07:00
Ryan Barrett 8e14ce003f
Web: relax domain validation, allow underscores in all segments except TLD
we already have one Web user with an underscore, brandon_swoop.ngrok.io
2023-06-16 12:05:55 -07:00
Ryan Barrett d3c4415cd3
migrate User.actor_as2 to obj key pointing to Object
for #547
2023-06-15 21:23:31 -07:00
Ryan Barrett 0902994af0
webfinger: infer protocol from brid.gy subdomain in request or resource
first commit for inferring protocol. exciting! for #512
2023-06-12 15:50:00 -07:00
Ryan Barrett 510cd036bf
AP users: add Protocol.for_domain and for_request
getting ready to serve per-protocol subdomains, eg nostr.brid.gy, ap.brid.gy, etc

for #512
2023-06-10 19:55:08 -07:00
Ryan Barrett 4909046506
add new brid.gy subdomains and todo to canonicalize them 2023-06-10 16:29:58 -07:00
Ryan Barrett 6796b47a9f
noop: move web.NoMicroformats to common, misc other minor cleanup 2023-06-10 07:53:07 -07:00
Ryan Barrett 7f6cc61683
improve domain validation for Web key ids, normalize to lower case 2023-06-09 10:58:28 -07:00
Ryan Barrett 958f81ddd1
AP users: rename [is_]homepage => [is_]web_url, move into Protocol subclasses
for #512
2023-05-31 18:47:40 -07:00
Ryan Barrett b7bebb4075
move base64_to_long, long_to_base64 to common 2023-05-27 14:58:22 -07:00
Ryan Barrett c8b2103e13
rename Webmention class => Web, webmention.py => web.py 2023-05-26 17:40:29 -07:00
Ryan Barrett 0dbe18aae4
remove common.actor() 2023-04-04 22:19:35 -07:00
Ryan Barrett 4c9744cd65
common.error bug fix, accept exc_info kwarg 2023-04-04 07:14:31 -07:00
Ryan Barrett 88dd51b05b
continue abstracting logic in Protocol class
lots more to do.

also convert Object.as1 to AS2 in ActivityPub.send if necessary.
2023-03-20 19:17:55 -07:00
Ryan Barrett fb5f7b3fb0
move current user into Flask g request-global 2023-03-20 14:28:14 -07:00
Ryan Barrett 5e3c623552
add User.actor_id() 2023-03-19 09:34:42 -07:00
Ryan Barrett d8932d4e76
add mf2 to user page dashboard to make a notifications feed
for #442
2023-03-14 11:27:10 -07:00
Ryan Barrett 9cc8451182
fix webmention endpoint discovery cache key
#423
2023-03-11 12:14:48 -08:00
Ryan Barrett 76e26a7c2c
cache webmention endpoint discovery for 2h 2023-03-10 22:24:58 -08:00
Ryan Barrett ffd8810b44
start to separate logic from protocols with new Protocol/ActivityPub classes
for #388
2023-03-10 07:36:50 -08:00
Ryan Barrett 85b8d86b17
common.get_as2: handle empty and non/bad JSON responses
fixes https://console.cloud.google.com/errors/detail/CKH_9oGur7T1ag;time=P30D?project=bridgy-federated
2023-02-28 16:21:33 -08:00
Ryan Barrett fd27dabe61
switch JSON properties to custom JSONProperty that works in web console UI
https://github.com/googleapis/python-ndb/issues/874#issuecomment-1442753255
2023-02-24 07:25:29 -06:00
Ryan Barrett 91a60c7e67
switch Object.as1 to be a ComputedProperty 2023-02-23 21:17:26 -06:00
Ryan Barrett 1f3bd41dc5
common.redirect_unwrap: stop following redirects when unwrapping actor URLs 2023-02-23 08:35:06 -06:00
Ryan Barrett 74b3b3b689
refactoring, move Object creation out of common.send_webmentions
ugh this was painful
2023-02-23 07:44:54 -06:00
Ryan Barrett 3d3335e860
AP => webmention repost bug fix, merge follower and wm domains 2023-02-23 07:39:26 -06:00
Ryan Barrett 65bbc6751e
HTTP Signature verification: bug fix, use actual HTTP method 2023-02-16 07:34:12 -08:00
Ryan Barrett 48a7720f88
common.get_object: normalize ids by removing fragments
this is still underspecified and a bit inconsistent across fediverse implementations:

https://socialhub.activitypub.rocks/t/problems-posting-to-mastodon-inbox/801/11

> The reason might also be that your IDs aren’t permanent, as in, they contain a #fragment. Posts and their corresponding Create activities are supposed to be resolvable — which means one should be able to send a GET request to the ID URL and get the object back. This can’t be done with an URL that contains a fragment as the fragment is not a part of the HTTP exchange, it’s processed on the client.

https://socialhub.activitypub.rocks/t/problems-posting-to-mastodon-inbox/801/23

> I ran into this object id #fragment problem as well. It seems because of some URL normalization, Mastodon will remove the fragment, and drop any additional posts with different fragments (because they become the same url).

https://socialhub.activitypub.rocks/t/s2s-create-activity/1647/5
https://github.com/mastodon/mastodon/issues/13879 (open!)
https://github.com/w3c/activitypub/issues/224

nothing in the http sig spec, example key ids aren't even URLs there:
https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-message-signatures-16
2023-02-15 20:00:12 -08:00
Ryan Barrett 63d0e59c2c
AP: verify incoming signatures!
for #315
2023-02-15 14:10:25 -08:00
Ryan Barrett e423bde6d0
unify logging for writing datastore entities into post put hook
and other logging tweaks
2023-02-14 15:38:42 -08:00
Ryan Barrett 47b83c4e51
common.get_object: update existing object if available 2023-02-14 15:26:25 -08:00
Ryan Barrett f44aa3b39e
common.get_object: fall back to HTTP fetch if stored entity has no as2
also add more logging
2023-02-14 15:17:03 -08:00
Ryan Barrett 588598c5ff
switch most uses of common.get_as2 to common.get_object 2023-02-14 14:56:27 -08:00
Ryan Barrett c2e6174330
add common.get_object() 2023-02-14 14:30:00 -08:00
Ryan Barrett c12bb6db6d
serve AS2 /[domain] actors from datastore instead of refetching h-card
for #392. not pretty, but gets the job done. more code cleanup needed eventually.
2023-02-14 08:25:41 -08:00
Ryan Barrett 6f91a2538d
AP inbox delivery: drop same-domain targets from Objects 2023-02-13 11:28:02 -08:00
Ryan Barrett f39c532ad3
add User.homepage, is_homepage() 2023-02-12 12:03:27 -08:00
Ryan Barrett b739bd4cee
return 404 on requests for unknown users instead of automatically creating them 2023-02-07 18:28:52 -08:00
Ryan Barrett e1856c21a1
add more logging for when we write Objects 2023-02-07 08:24:36 -08:00
Ryan Barrett cb605e96c6
user page activities: if object is user, render as pretty user link
for #406
2023-02-06 21:28:40 -08:00
Ryan Barrett f922ce89bb
user page: activities: render fediverse actor links as @-@
for #406
2023-02-06 21:08:52 -08:00
Ryan Barrett 3623df8e20
bring back default signing user for shared inbox
for #403, fixes https://console.cloud.google.com/errors/detail/CIfnyvKDx8mDTA;time=P30D?project=bridgy-federated
2023-02-06 20:08:32 -08:00
Ryan Barrett 4b37674624
always sign requests with current user's key, stop defaulting to snarfed.org
for #403
2023-02-06 19:23:25 -08:00
Ryan Barrett 717b068193
signed_post: don't follow redirects
fixes https://console.cloud.google.com/errors/detail/CLuyuajU2sqLtAE;time=P30D;refresh=true?project=bridgy-federated
2023-02-06 14:44:17 -08:00
Ryan Barrett 08265123f6
Activity => Object: use 'in progress' status
#286
2023-02-04 20:46:28 -08:00
Ryan Barrett 532ccb8ac1
Activity => Object: use StructuredProperty for Object.delivered etc, add protocol
#286

ndb implements StructuredProperty by hoisting each nested property into a
corresponding property on the parent entity, prefixed by the StructuredProperty's
name, eg delivered.uri, delivered.protocol, etc.

For repeated StructuredPropertys, the hoisted properties are all repeated on the
parent entity, and reconstructed into StructuredPropertys based on their order.

https://googleapis.dev/python/python-ndb/latest/model.html#google.cloud.ndb.model.StructuredProperty
2023-02-04 20:46:27 -08:00
Ryan Barrett f9891b6ef7
Activity => Object: add activity label, unify Objects for AP => wm
#286
2023-02-04 20:46:27 -08:00
Ryan Barrett 4d2fcdd76f
Activity => Object: fully populate labels, source_protocol webmention
#286
2023-02-04 20:46:27 -08:00
Ryan Barrett eafba6d7cd
Activity => Object: populate Object.type
#286
2023-02-04 20:46:27 -08:00
Ryan Barrett 060ad96752
Activity => Object: update activitypub.py
#286
2023-02-04 20:46:27 -08:00
Ryan Barrett 789da26362
AP: return 501 for Undo not supported, not 400 2023-01-26 12:03:13 -08:00
Ryan Barrett 05c8dc83c4
override link label for home page to be "Web site"
for #331
2023-01-25 12:33:53 -08:00
Ryan Barrett eca2f6f1ca
outbound AP: only log pretty-printed AS2 object on first request, not all 2023-01-24 16:49:48 -08:00
Ryan Barrett a2d9dcb4a9
cache more aggressively: bump expiration up to 60s, threshold to 3k
for #378
2023-01-24 16:13:22 -08:00
Ryan Barrett bb2d3e03de
noop: convert a few %s'es to f-strings
thanks flynt! https://github.com/ikamensh/flynt
2023-01-24 12:17:24 -08:00
Ryan Barrett 46736df21b
common.postprocess_as2 bug fix, handle multiply valued image field 2023-01-23 21:13:05 -08:00
Ryan Barrett 1627ff667a
translate webmention from home page into profile (actor) Update
for #376, #331
2023-01-23 16:09:25 -08:00
Ryan Barrett 21c79a304c
when converting h-card to actor, use link text or title for profile links
for #331
2023-01-22 07:47:27 -08:00
Ryan Barrett af769de99e
implement rest of followers/following collections
for #264
2023-01-20 07:02:55 -08:00
Ryan Barrett 74ebbc428b
add new common.fetch_followers() fn, use in UI, AP collections, and XRPCs
for #264
2023-01-19 15:29:52 -08:00
Ryan Barrett c3edf3a68e
make Follower.to_as1() handle both inbound and outbound last_follow
...and use it in followers and following UI pages
2023-01-18 22:31:13 -08:00
Ryan Barrett 02aa3e9801
refactoring: move fetch_page from pages to common 2023-01-18 22:19:51 -08:00
Ryan Barrett 84606500fa
refactoring: unify followers and following AP collection endpoints 2023-01-18 20:32:23 -08:00
Ryan Barrett 2c9850c576
HTTP Signatures: add special (request-target) header
for #40, special synthetic HTTP header for HTTP Signatures, required by Peertube.
https://datatracker.ietf.org/doc/html/draft-cavage-http-signatures-12#section-2.3
2023-01-17 19:50:40 -08:00
Ryan Barrett e6d4441292
handle redirects of signed requests manually, generate a new HTTP sig each time
fixes #326
2023-01-16 13:10:33 -08:00
Ryan Barrett 17c5c83105
common.host_url: use fed.brid.gy when running locally against prod datastore 2023-01-08 12:01:31 -08:00
Ryan Barrett aa0dcbb222
noop: drop content type constants in common, use granary's instead 2023-01-06 21:01:33 -08:00
Ryan Barrett 7ac1006021
background task bug fix: override task runner host
we were using request.host_url blindly, which was fed.brid.gy for user-facing requests, but bridgy-federated.uc.r.appspot.com for tasks. #335
2023-01-05 15:05:35 -08:00
Ryan Barrett fe5c3947a8
drop OStatus! it's been unused for years
fixes #282
2023-01-04 19:22:11 -08:00
Ryan Barrett 45f4bd71a2
switch common.utcnow to util.now 2023-01-03 15:41:43 -08:00
Ryan Barrett 3fedc05222
add aaronparecki.com to blocklist
for #348
2022-12-28 06:39:16 -08:00
Ryan Barrett 5b6b990efb
outbound conneg bug fix 2022-12-27 09:21:18 -08:00
Ryan Barrett 4c383a3037
request caching tweaks, webfinger, use new flask_util.cached headers kwarg 2022-12-26 13:34:50 -08:00
Ryan Barrett 2150693893
handle full conneg Accept header parsing in /r/ handler
for #352
2022-12-25 22:01:29 -08:00
Ryan Barrett f0eb705841
log all outgoing AS2 objects 2022-12-14 10:11:19 -08:00
Ryan Barrett d4741310e7
common.postprocess_as2 bug fix for multiply valued AS2 url field
fixes https://console.cloud.google.com/errors/detail/CMOZzPLojuud_QE;time=P30D?project=bridgy-federated
2022-12-11 07:28:30 -08:00
Ryan Barrett 8751fea257
harden URL unwrapping a bit, require http(s) URL or domain
fixes https://console.cloud.google.com/errors/detail/CMKn7tqbq-GIRA;time=P30D?project=bridgy-federated
2022-12-10 09:09:44 -08:00
Ryan Barrett 222be38a98
user page: redirect www-stripped users to non-www user pages
for #314
2022-12-02 11:08:24 -08:00
Ryan Barrett 0363db65fa
handle new multi-valued url field in AS2 objects
snarfed/granary@79c512402b
2022-11-30 21:04:41 -08:00
Ryan Barrett 559f02ad1d
AP actor handler: handle bad domain
https://console.cloud.google.com/errors/detail/CKGv-b6impW3Jg;time=P30D?project=bridgy-federated
2022-11-28 17:27:55 -08:00
Ryan Barrett 7d902ec616
bug fix for custom username in fediverse address on user page UI
should come from acct: actor URL, not preferredUsername field, which always has to be user's domain for AP interop.

in the process, refactored common.get_username() into User.username().

for #281
2022-11-26 16:05:02 -08:00
Ryan Barrett 2fdc6f29a9
sign HTTP GET requests for AS2 objects
to support Mastodon's AUTHORIZED_FETCH aka secure mode: https://docs.joinmastodon.org/admin/config/#authorized_fetch

fixes #291
2022-11-24 09:41:14 -08:00
Ryan Barrett 660682c93e
fix preferredUsername regression, it needs to be domain for Webfinger
originally added in 65f3ef3cc7. fixes #302. props to @fancypantalons for debugging!
2022-11-22 14:47:15 -08:00
Ryan Barrett 65f3ef3cc7
show fediverse address on user page UI, including custom username
for #281
2022-11-20 11:46:05 -08:00
Ryan Barrett b854981497
UI edits: enter web site page, /docs, flashed messages, copy Bridgy's CSS 2022-11-18 22:30:07 -08:00
Ryan Barrett 2d07ac358d
move activitypub.actor() to common so I can reuse it 2022-11-18 18:49:34 -08:00
Ryan Barrett 8fd49dc882
drop non-public Create activities
fixes #289
2022-11-16 16:37:50 -08:00
Ryan Barrett 2bb76a90e7
store Activity entities for AP activities delivered to followers
for #265
2022-11-16 15:54:02 -08:00
Ryan Barrett 50956c02b0
noop, rename Domain => User 2022-11-15 22:00:28 -08:00
Ryan Barrett 46f039af63
AP: always generate an actor for every outbound activity
for #279
2022-11-15 21:43:24 -08:00
Ryan Barrett e3f1431018
noop identifier renames, key => domain or entity 2022-11-15 19:12:20 -08:00
Ryan Barrett fd5c88c713
add domain blocklist, right now primarily for Twitter 2022-11-14 22:31:45 -08:00
Ryan Barrett 7966650178
AP: put Public in to instead of cc to make Mastodon posts public, not unlisted
fixes #277
2022-11-13 21:46:26 -08:00
Ryan Barrett 4f3dc03a3e
rename MagicKey => Domain, Response => Activity
the Python classes, but not (yet) the kinds in the datastore. maybe eventually.
2022-11-12 15:27:59 -08:00
Ryan Barrett b24a22e817
add /responses/[DOMAIN] user pages and Response.domain property 2022-11-08 06:56:19 -08:00
Ryan Barrett 8e60346dc4
don't strip www, m, and mobile subdomains from user domains
fixes #267
2022-11-07 16:28:46 -08:00