Ryan Barrett
510cd036bf
AP users: add Protocol.for_domain and for_request
...
getting ready to serve per-protocol subdomains, eg nostr.brid.gy, ap.brid.gy, etc
for #512
2023-06-10 19:55:08 -07:00
Ryan Barrett
6796b47a9f
noop: move web.NoMicroformats to common, misc other minor cleanup
2023-06-10 07:53:07 -07:00
Ryan Barrett
c98ab3f2d5
AP users: Object schema change, domains => users
...
also for AP => wm, only try to send webmentions to domains that we already have an existing Web user for. the vast majority of targets are fediverse URLs, and we were trying to send them all wms, ie at least running wm discovery and finding nothing. harmless, but a waste.
2023-06-09 12:56:45 -07:00
Ryan Barrett
7c82bf7ec5
AP users: finish migrating Follower from string domains/ids to User keys
...
for #512
2023-06-07 19:06:44 -07:00
Ryan Barrett
9cb8c1f9f2
AP users: start migrating Follower from string domains/ids to User keys
...
for #512
2023-06-07 12:43:48 -07:00
Ryan Barrett
51c2773594
AP users: /r/: switch external user to indirect user
...
for #512
2023-06-04 20:58:21 -07:00
Ryan Barrett
47b3dd29d0
AP users: minor tweak, include full @context in indirect users' actors
...
for #512
2023-06-04 07:33:30 -07:00
Ryan Barrett
fe27742b80
AP users: promote ActivityPub.label_id computed prop to User.readable_id
...
and drop ActivityPub.get_by_id() override, move logic to user() page handler. fixes 'Only ancestor queries are allowed inside transactions.'
for #512
2023-06-03 21:48:15 -07:00
Ryan Barrett
cf68c1ebc7
AP users: fill in more fields in base actor objects
...
working on Mastodon account discovery interop. for #512
2023-06-03 08:36:55 -07:00
Ryan Barrett
c41b55a199
Web.fetch: switch back to raising instead of returning empty Object
...
raise new custom NoMicroformats exception
2023-06-03 07:28:01 -07:00
Ryan Barrett
d9cd5d14b9
AP users: fetch user on demand on /[actor] if we don't already have it
...
also change Web.fetch() to allow missing mf2 instead of raising exception. we now create Web users with None for mf2 property if the site has no mf2.
for #512
2023-06-02 12:55:07 -07:00
Ryan Barrett
e05ddb0a45
AP users: serve ActivityPub user page with address (handle) in URL
...
eg /activitypub/@me@instance.com
for #512
2023-06-01 22:00:47 -07:00
Ryan Barrett
ca64793fff
AP users: add User.name() and label_id(), ActivityPub.address computed property
...
for #512
2023-06-01 21:37:58 -07:00
Ryan Barrett
17ed24b6f5
AP users: switch ActivityPub.ap_address() to use as2.address()
...
for #512
2023-06-01 20:58:42 -07:00
Ryan Barrett
958f81ddd1
AP users: rename [is_]homepage => [is_]web_url, move into Protocol subclasses
...
for #512
2023-05-31 18:47:40 -07:00
Ryan Barrett
28eabd07a3
create AP users on inbox delivery for an indirect or nonexistent Web user
...
for #512
2023-05-31 13:17:17 -07:00
Ryan Barrett
35060c172a
switch actor_id() and address90 back to User methods, to be implemented by subclasses
...
partially reverts 9e906f18e4
2023-05-31 10:10:14 -07:00
Ryan Barrett
9e906f18e4
move address, actor_id from User to activitypub.py, /web-site to web.py
...
for #512
2023-05-30 17:24:49 -07:00
Ryan Barrett
29f1a1d4ac
AP users: start parameterizing URLs, including UI pages and AP endpoints
...
user pages now have /[protocol]/ prefix, AP endpoints have /ap/ prefix. for #512
2023-05-30 16:38:58 -07:00
Ryan Barrett
72024f90b1
bug fix for application/ld+json content type
...
fixes #522
2023-05-28 08:06:36 -07:00
Ryan Barrett
c8b2103e13
rename Webmention class => Web, webmention.py => web.py
2023-05-26 17:40:29 -07:00
Ryan Barrett
eaa4e5333a
start to make User subclasses for each protocol
...
#512
2023-05-26 16:07:36 -07:00
Ryan Barrett
cf86f4d808
AP users: extract out Protocol.serve() method
...
#512
2023-05-23 21:30:57 -07:00
Ryan Barrett
f933c7e237
AP users: serve AS2 for external homepage URLs
...
#512
2023-05-22 23:17:42 -07:00
Ryan Barrett
5aba64004f
wm => AP: inline most objects as bare string ids, not composite objects
...
for interop
2023-04-19 16:21:21 -07:00
Ryan Barrett
d9b1223392
wm => AP: fix reposts from h-cites that turn into composite objects
...
instead of bare string id objects. thanks @gRegorLove for reporting!
2023-04-19 14:37:42 -07:00
Ryan Barrett
d2ab48b23e
switch circular imports to runtime imports; split out flask_app.py from app.py
...
runtime imports are just as bad, but...meh. eventually I'll untangle them for real. #486
2023-04-18 17:18:17 -07:00
Ryan Barrett
212f2b11ec
wm => AP: add delete support
...
fixes #30
2023-04-17 15:36:29 -07:00
Ryan Barrett
ba38d6853b
bug fixes for hs2019 HTTP Signatures algorithm, hard coding to rsa-sha256
2023-04-17 11:15:05 -07:00
Ryan Barrett
842f8ac964
temporary: assume HTTP Signature algorithm=hs2019 means rsa-sha256
...
still trying to figure this out. see https://github.com/snarfed/bridgy-fed/issues/430#issuecomment-1510462267 , https://arewehs2019yet.vpzom.click/ , etc.
2023-04-16 16:35:31 -07:00
Ryan Barrett
629c1a2bd4
unify Object.new/changed generation into Protocol.load
...
also:
* switch back to preserving fragments in URL ids
* webmention.fetch: if URL id redirects, preserve original id in Object
2023-04-03 07:53:15 -07:00
Ryan Barrett
f836762b0f
big webmention.py refactoring
...
* use Objects properly
* use Object's mf2/AS2/AS1 conversion
* drop resuming partially completed deliveries
* use load, fetch, send
* get closer to unifying all activity handling into Protocol.receive
2023-04-01 19:13:51 -07:00
Ryan Barrett
e41ce9216c
noop, rename Protocol.get_object => Protocol.load
2023-03-29 13:13:32 -07:00
Ryan Barrett
e6be9cabb8
incoming AP: when we can't load the keyId, store a deleted Object
...
prevents re-fetching the same failing keyId actor over and over when we get the same Delete actor activity sent to all of our inboxes
2023-03-29 12:51:37 -07:00
Ryan Barrett
754a6eb651
minor noop tweaks: dupe test, test name, docstring
2023-03-27 14:18:24 -07:00
Ryan Barrett
28e8baff6c
change Protocol.fetch() to return an Object instead of populating one
2023-03-22 20:49:28 -07:00
Ryan Barrett
88dd51b05b
continue abstracting logic in Protocol class
...
lots more to do.
also convert Object.as1 to AS2 in ActivityPub.send if necessary.
2023-03-20 19:17:55 -07:00
Ryan Barrett
fb5f7b3fb0
move current user into Flask g request-global
2023-03-20 14:28:14 -07:00
Ryan Barrett
b2c60226b7
add Webmention.send, tweak method signature
2023-03-20 11:26:43 -07:00
Ryan Barrett
d433dd3abe
wm => AP: switch actors in outgoing activities from objects to string ids
...
mostly. also inlines single-element attributedTo lists to just the element, without the list.
2023-03-19 09:34:42 -07:00
Ryan Barrett
5169c2e44e
misc noop tweaks, mostly tests
2023-03-19 09:34:42 -07:00
Ryan Barrett
5e3c623552
add User.actor_id()
2023-03-19 09:34:42 -07:00
Ryan Barrett
be54842ce6
docs for hashtags
...
#45
2023-03-14 15:36:18 -07:00
Ryan Barrett
ce37dd8637
process hashtags in tab objects with Mastodon-specific logic
...
lots of background in #45 . also https://docs.joinmastodon.org/spec/activitypub/#properties-used
2023-03-14 14:59:28 -07:00
Ryan Barrett
01768fd58e
move accept_follow from activitypub to protocol
2023-03-11 13:01:23 -08:00
Ryan Barrett
bd748f5db1
bug fix for deb4b12659
...
fixes https://console.cloud.google.com/errors/detail/CNrPlPDIlZ-CUA;time=P30D?project=bridgy-federated
2023-03-10 07:49:41 -08:00
Ryan Barrett
abf7a9adfe
activitypub.py: minor noop tweaks, add log message
2023-03-10 07:36:50 -08:00
Ryan Barrett
deb4b12659
AS2: short circuit out on Delete actor that we don't have stored
...
...since when we try to fetch the actor to get their key to verify the signature, we get an HTTP 410 response, at least from Mastodon.
2023-03-10 07:36:50 -08:00
Ryan Barrett
31645618dc
abstracting protocols: start to use send()
...
#388
2023-03-10 07:36:50 -08:00
Ryan Barrett
ffd8810b44
start to separate logic from protocols with new Protocol/ActivityPub classes
...
for #388
2023-03-10 07:36:50 -08:00
Ryan Barrett
bc15902bed
bug fix, don't deliver activities to users if their Follower is inactive
2023-03-04 12:12:45 -08:00
Ryan Barrett
fa2a9a1afe
activitypub: store Create inner objects in their own Objects
2023-03-03 15:12:51 -08:00
Ryan Barrett
2dfddec2ef
convert activitypub.py logic to use all AS1
2023-03-03 09:28:40 -08:00
Ryan Barrett
0235d31a1c
noop: activitypub.py: extract sig verif out into a new verify_signature() fn
2023-02-26 20:41:57 -06:00
Ryan Barrett
116cd55ddb
finally start actually requiring HTTP Signatures
...
fixes #315
2023-02-26 07:34:15 -06:00
Ryan Barrett
fd27dabe61
switch JSON properties to custom JSONProperty that works in web console UI
...
https://github.com/googleapis/python-ndb/issues/874#issuecomment-1442753255
2023-02-24 07:25:29 -06:00
Ryan Barrett
91a60c7e67
switch Object.as1 to be a ComputedProperty
2023-02-23 21:17:26 -06:00
Ryan Barrett
1f3bd41dc5
common.redirect_unwrap: stop following redirects when unwrapping actor URLs
2023-02-23 08:35:06 -06:00
Ryan Barrett
74b3b3b689
refactoring, move Object creation out of common.send_webmentions
...
ugh this was painful
2023-02-23 07:44:54 -06:00
Ryan Barrett
3d3335e860
AP => webmention repost bug fix, merge follower and wm domains
2023-02-23 07:39:26 -06:00
Ryan Barrett
4982906ed8
AP => webmention, reposts: handle loading original federated posts
...
bug fix for 72eb26e248
2023-02-21 21:57:38 -06:00
Ryan Barrett
72eb26e248
AP => webmention, bug fix for sending webmentions for reposts
...
thanks to @tantek for reporting!
2023-02-21 21:19:56 -06:00
Ryan Barrett
65bbc6751e
HTTP Signature verification: bug fix, use actual HTTP method
2023-02-16 07:34:12 -08:00
Ryan Barrett
d505b3859a
HTTP Signature verification: fetch and use keyId from signature
...
#315
2023-02-15 20:10:17 -08:00
Ryan Barrett
e1560587bf
AP: rearrange HTTP Signature logging
...
#315
2023-02-15 15:48:12 -08:00
Ryan Barrett
e5d3f94b3e
AP signature verification: just log for now, don't fail
...
for #315
2023-02-15 14:10:25 -08:00
Ryan Barrett
63d0e59c2c
AP: verify incoming signatures!
...
for #315
2023-02-15 14:10:25 -08:00
Ryan Barrett
e423bde6d0
unify logging for writing datastore entities into post put hook
...
and other logging tweaks
2023-02-14 15:38:42 -08:00
Ryan Barrett
588598c5ff
switch most uses of common.get_as2 to common.get_object
2023-02-14 14:56:27 -08:00
Ryan Barrett
c2e6174330
add common.get_object()
2023-02-14 14:30:00 -08:00
Ryan Barrett
c12bb6db6d
serve AS2 /[domain] actors from datastore instead of refetching h-card
...
for #392 . not pretty, but gets the job done. more code cleanup needed eventually.
2023-02-14 08:25:41 -08:00
Ryan Barrett
6446c8c421
convert Object.type and .object_ids to ComputedPropertys
...
https://googleapis.dev/python/python-ndb/latest/model.html#google.cloud.ndb.model.ComputedProperty
2023-02-13 21:43:49 -08:00
Ryan Barrett
46ab1cfdf7
AP inbox: simplify already-seen-id handling
2023-02-13 19:10:01 -08:00
Ryan Barrett
e39a92cee2
optimize AP inbox delivery by moving user load back further down
2023-02-13 11:25:33 -08:00
Ryan Barrett
c305dcc8d5
AP inbox delivery: cache activity ids that we've already seen in memory
...
for #411
2023-02-12 22:17:04 -08:00
Ryan Barrett
7a625c5a02
AP inbox delivery: short circuit out if we've already seen the activity id
...
for #411 . next is caching in memory.
2023-02-12 21:58:59 -08:00
Ryan Barrett
bf97c1af4f
store all incoming AP activities in Objects
2023-02-11 22:53:50 -08:00
Ryan Barrett
3feb44e414
apply AP Updates to stored Objects in datastore
...
for #409
2023-02-11 22:23:01 -08:00
Ryan Barrett
7e2fbd1ed0
make AP Deletes mark stored Objects deleted
...
for #409
2023-02-11 21:47:55 -08:00
Ryan Barrett
45fc9786a2
activitypub: fetch full objects of reposts so we can render them in feeds
...
for #419
2023-02-11 19:58:07 -08:00
Ryan Barrett
cb66a2f33b
fix delivering reposts to followers
...
evidently broken by #286 . fixes #418
2023-02-08 11:03:54 -08:00
Ryan Barrett
b739bd4cee
return 404 on requests for unknown users instead of automatically creating them
2023-02-07 18:28:52 -08:00
Ryan Barrett
532ccb8ac1
Activity => Object: use StructuredProperty for Object.delivered etc, add protocol
...
#286
ndb implements StructuredProperty by hoisting each nested property into a
corresponding property on the parent entity, prefixed by the StructuredProperty's
name, eg delivered.uri, delivered.protocol, etc.
For repeated StructuredPropertys, the hoisted properties are all repeated on the
parent entity, and reconstructed into StructuredPropertys based on their order.
https://googleapis.dev/python/python-ndb/latest/model.html#google.cloud.ndb.model.StructuredProperty
2023-02-04 20:46:27 -08:00
Ryan Barrett
f9891b6ef7
Activity => Object: add activity label, unify Objects for AP => wm
...
#286
2023-02-04 20:46:27 -08:00
Ryan Barrett
4d2fcdd76f
Activity => Object: fully populate labels, source_protocol webmention
...
#286
2023-02-04 20:46:27 -08:00
Ryan Barrett
97ffc10f72
Activity => Object: populate Object.object_ids property
...
#286
2023-02-04 20:46:27 -08:00
Ryan Barrett
eafba6d7cd
Activity => Object: populate Object.type
...
#286
2023-02-04 20:46:27 -08:00
Ryan Barrett
060ad96752
Activity => Object: update activitypub.py
...
#286
2023-02-04 20:46:27 -08:00
Ryan Barrett
d724ae8cba
accept Updates to Person objects, do nothing with them
...
fixes #387
2023-01-25 19:44:48 -08:00
Ryan Barrett
d85ffaec04
implement stub empty outbox
...
for #383
2023-01-25 13:12:24 -08:00
Ryan Barrett
55d5206188
cache 502s and 504s for requests that fetch users' sites
...
for #378
2023-01-24 16:29:19 -08:00
Ryan Barrett
bb2d3e03de
noop: convert a few %s'es to f-strings
...
thanks flynt! https://github.com/ikamensh/flynt
2023-01-24 12:17:24 -08:00
Ryan Barrett
1627ff667a
translate webmention from home page into profile (actor) Update
...
for #376 , #331
2023-01-23 16:09:25 -08:00
Ryan Barrett
af769de99e
implement rest of followers/following collections
...
for #264
2023-01-20 07:02:55 -08:00
Ryan Barrett
74ebbc428b
add new common.fetch_followers() fn, use in UI, AP collections, and XRPCs
...
for #264
2023-01-19 15:29:52 -08:00
Ryan Barrett
fc4e8b11d2
return actors in AP Followers/Following collections
...
for #264 . next: paging
2023-01-19 07:00:35 -08:00
Ryan Barrett
84606500fa
refactoring: unify followers and following AP collection endpoints
2023-01-18 20:32:23 -08:00
Ryan Barrett
6520331627
incoming Follows: synthesize url into follow object stored in Activity
...
rendered mf2 HTML proxy pages (in render.py) fall back to redirecting to the follow's AS2 id field, but Mastodon's ids are URLs that don't load in browsers, eg https://jawns.club/ac33c547-ca6b-4351-80d5-d11a6879a7b0 . so, set a synthetic URL based on the follower's profile. fixes #336
2023-01-12 12:28:34 -08:00
Ryan Barrett
4e539488a7
inbound AP: handle Undo Follow with composite inner object
2023-01-09 22:58:35 -08:00
Ryan Barrett
8cdc5ecc4c
inbound AP: handle Follow with composite object
2023-01-08 17:41:15 -08:00
Ryan Barrett
9bb82e6db3
follow UI: store last_follow, misc bug fixes
2023-01-08 12:03:56 -08:00
Ryan Barrett
aa0dcbb222
noop: drop content type constants in common, use granary's instead
2023-01-06 21:01:33 -08:00
Ryan Barrett
7ac1006021
background task bug fix: override task runner host
...
we were using request.host_url blindly, which was fed.brid.gy for user-facing requests, but bridgy-federated.uc.r.appspot.com for tasks. #335
2023-01-05 15:05:35 -08:00
Ryan Barrett
4c383a3037
request caching tweaks, webfinger, use new flask_util.cached headers kwarg
2022-12-26 13:34:50 -08:00
Ryan Barrett
dc9c699d4f
/inbox: wait until later to load user from datastore
...
optimizes Accepts and Deletes sent to individual inbox that don't need the user
2022-12-08 09:00:53 -08:00
Ryan Barrett
a95c2c4b55
incoming follow bug fix, handle www and use_instead
...
fixes #314
2022-12-06 14:09:44 -08:00
Ryan Barrett
0363db65fa
handle new multi-valued url field in AS2 objects
...
snarfed/granary@79c512402b
2022-11-30 21:04:41 -08:00
Ryan Barrett
ab834f586a
incoming AP Follow bug fix, reactivate if Follower exists and is inactive
...
thanks again @gregorLove!
2022-11-29 22:43:11 -08:00
Ryan Barrett
559f02ad1d
AP actor handler: handle bad domain
...
https://console.cloud.google.com/errors/detail/CKGv-b6impW3Jg;time=P30D?project=bridgy-federated
2022-11-28 17:27:55 -08:00
Ryan Barrett
2fdc6f29a9
sign HTTP GET requests for AS2 objects
...
to support Mastodon's AUTHORIZED_FETCH aka secure mode: https://docs.joinmastodon.org/admin/config/#authorized_fetch
fixes #291
2022-11-24 09:41:14 -08:00
Ryan Barrett
2279db46fc
start on AP followers/following collections
...
for #264
2022-11-21 18:47:08 -08:00
Ryan Barrett
01cb91d542
handle actor Deletes that get sent to shared inbox
...
...by actually deleting the corresponding Follower entities. fixes #284
2022-11-21 12:14:37 -08:00
Ryan Barrett
2d07ac358d
move activitypub.actor() to common so I can reuse it
2022-11-18 18:49:34 -08:00
Ryan Barrett
d0c16d8849
bug fix, use https://fed.brid.gy/[domain] as actor id, not bare domain
...
fixes #293
2022-11-17 16:24:53 -08:00
Ryan Barrett
dde86008e2
shared inbox bug fix
...
fixes https://console.cloud.google.com/errors/detail/CK7Cx8KS7qyNHQ;time=P30D?project=bridgy-federated
2022-11-17 14:55:54 -08:00
Ryan Barrett
023f2aa536
sharedInbox bug fix, needs to be inside endpoints mapping
...
for #285 , https://www.w3.org/TR/activitypub/#endpoints
2022-11-17 12:59:09 -08:00
Ryan Barrett
8fd49dc882
drop non-public Create activities
...
fixes #289
2022-11-16 16:37:50 -08:00
Ryan Barrett
2bb76a90e7
store Activity entities for AP activities delivered to followers
...
for #265
2022-11-16 15:54:02 -08:00
Ryan Barrett
2d9cc31ff9
switch default home page scheme from http to https
...
fixes #269
2022-11-16 10:43:34 -08:00
Ryan Barrett
320b117470
add AP sharedInbox
...
#285 , https://www.w3.org/TR/activitypub/#sharedInbox
2022-11-16 10:09:24 -08:00
Ryan Barrett
50956c02b0
noop, rename Domain => User
2022-11-15 22:00:28 -08:00
Ryan Barrett
e3f1431018
noop identifier renames, key => domain or entity
2022-11-15 19:12:20 -08:00
Ryan Barrett
4f3dc03a3e
rename MagicKey => Domain, Response => Activity
...
the Python classes, but not (yet) the kinds in the datastore. maybe eventually.
2022-11-12 15:27:59 -08:00
Ryan Barrett
8aa11f0662
webmention => AP: store outbound followees
...
for #264
2022-11-11 22:50:05 -08:00
Ryan Barrett
8f1d716843
remove debug comment
...
for #268
2022-11-07 19:53:43 -08:00
Ryan Barrett
8e60346dc4
don't strip www, m, and mobile subdomains from user domains
...
fixes #267
2022-11-07 16:28:46 -08:00
Ryan Barrett
001cd1fe11
tried switching HTTP Sig keyId to actor URL for aaronpk, but Mastodon wouldn't verify
...
https://chat.indieweb.org/dev/2022-11-05#t1667687865621700
2022-11-05 15:39:05 -07:00
Ryan Barrett
db1441c34c
activitypub: log pretty-printed incoming AP requests
2022-11-05 13:34:58 -07:00
Ryan Barrett
80409af1fe
add missing `f` to log format string
2022-04-06 14:31:58 -07:00
Ryan Barrett
1d36d31064
set User-Agent header globally
2022-03-24 11:18:33 -07:00
Ryan Barrett
96dba41c9b
logging: use separate loggers for each module with their names
2022-02-11 22:38:56 -08:00
Ryan Barrett
535cf7ba70
add response body for inbox Accept activities
...
fixes https://console.cloud.google.com/errors/COew6ZH4_eyy2wE
2021-12-29 19:34:46 -08:00
Ryan Barrett
2f28be09be
convert %-formatted strings with args in logging.* calls to f-strings
2021-12-27 22:45:57 -08:00
Ryan Barrett
fdd42ad1d4
override preferredUsername to always be the user's domain
...
...since Mastodon looks up [preferredUsername]@fed.brid.gy. see #77
2021-10-22 21:17:45 -07:00
Ryan Barrett
17d2066008
use @flask_util.cached
2021-08-28 07:18:46 -07:00
Ryan Barrett
cf12817212
Flask port: error handling bug fixes, other misc tweaks
2021-08-16 11:47:31 -07:00
Ryan Barrett
32d9e2bf6c
move common.error() to webutil.flask_util
...
corresponds to snarfed/webutil@10c088cebd
2021-08-06 11:01:58 -07:00
Ryan Barrett
e1aace0655
flask: use new regex URL route converter
2021-07-12 14:25:00 -07:00
Ryan Barrett
87e0cb8be7
flask: minor tweaks
2021-07-11 23:04:25 -07:00
Ryan Barrett
371a92a5db
flask: port XrdOrJrdHandler, finish porting webfinger
2021-07-11 16:30:14 -07:00
Ryan Barrett
b8be570d66
flask: port activitypub
2021-07-10 08:07:40 -07:00
Ryan Barrett
f26aed5cbb
flask: port salmon
2021-07-08 22:50:33 -07:00
Ryan Barrett
155be76bca
fix outbound AP headers and signature for Mastodon 3.2.1
...
https://github.com/snarfed/bridgy-fed/issues/68#issuecomment-720059695
2020-11-05 12:43:27 -08:00
Ryan Barrett
d22903c3a7
update AP signature generation for Mastodon 3.2.1 changes
...
not working yet. :( details in https://github.com/snarfed/bridgy-fed/issues/68#issuecomment-714752553
2020-10-29 16:08:48 -07:00
Ryan Barrett
dba533a14b
temporarily disable actually deleting Followers on AP Delete requests
...
mastodon.social sends Deletes for every Bridgy Fed account, all at basically the same time, and we have many Follower objects, so we have to do this table scan for each one, so the requests take a long time and end up spawning extra App Engine instances that we get billed for. and the Delete requests are almost never for followers we have. TODO: revisit this and do it right.
2020-10-11 08:35:14 -07:00
Ryan Barrett
9eae3087fd
404 requests for root paths that aren't user domains
...
lots of these are fediverse tools sniffing for specific implentations, eg mastodon etc. /siteinfo.json, /status.php, etc.
2020-03-02 10:50:43 -08:00
Ryan Barrett
9784686b1c
implement AP Delete of actors when their accounts are deleted
...
fixes #63
2020-02-29 20:28:53 -08:00
Ryan Barrett
df6b0b58ba
unify request handler classes and handle_exception; move away from HOST[_URL]
...
fixes:
* https://console.cloud.google.com/errors/CN68rO-5sOK9cQ
* https://console.cloud.google.com/errors/CJWCu8b5_ureAg
* https://console.cloud.google.com/errors/CN6W4Zy7irzgOA
* https://console.cloud.google.com/errors/CN278MyjhZbtOQ
2020-01-31 14:40:11 -08:00
Ryan Barrett
ee7b324525
bring back HOST_URL override to fed.brid.gy, for URLs emitted into AP
2019-12-26 18:18:44 -08:00
Ryan Barrett
02d36b3b1f
migrate to the app engine standard python 3 runtime!
...
https://cloud.google.com/appengine/docs/standard/python3/python-differences
https://cloud.google.com/appengine/docs/standard/python/migrate-to-python3/
2019-12-25 22:26:26 -08:00
Ryan Barrett
ab55c9d62c
ujson.dumps/loads() => util.json_dumps/loads()
2019-12-24 23:26:58 -08:00
Ryan Barrett
99a5c35ae3
switch from webutil.util.memcache_response to cache_response
2019-11-12 21:28:44 -08:00
Ryan Barrett
faf676e8bb
switch from json to ultrajson for performance
...
https://github.com/esnme/ultrajson/
2019-10-29 07:06:15 -07:00
Ryan Barrett
7c9a03c827
unify HTTP fetching, HTML and mf2 parsing, and error handling into webutil
...
see:
* snarfed/granary#171
* snarfed/webutil@f994884b2b
* snarfed/oauth-dropins@f5b6e73530
* snarfed/granary@284eb11508
2019-10-03 21:08:26 -07:00
Ryan Barrett
c623a295c6
misc noop cleanups based on LGTM alerts
...
https://lgtm.com/projects/g/snarfed/bridgy/?mode=list
2019-09-19 11:57:16 -07:00
Ryan Barrett
9ff0a28dd3
minor bug fix in AP Undo Follow error handling
2019-08-14 11:16:37 -07:00
Ryan Barrett
65044b081c
accept Undo of Follow even if Follower doesn't exist
2019-08-01 10:39:15 -07:00
Ryan Barrett
4b70a89046
handle AP Undo of Follow
...
needed for eg mastodon when you unfollow and then try to refollow someone. on unfollow, mastodon sends an Undo, and waits for a 200 before it shows success and will let you refollow.
2019-08-01 10:32:50 -07:00
Ryan Barrett
c055f3aeba
drop HTML tags from webfinger error messages
...
fixes #53
2019-06-24 14:46:45 -07:00
Ryan Barrett
76686af538
cache lots of GET requests
...
this helps with requests that make external HTTP fetches, eg indieweb home pages/posts and federated profiles/posts, which change rarely.
in particular, outbound AP requests that get federated widely (eg Mastodon) result in all of those federated instances making the same bridgy fed requests at the same time, often within just a few seconds. more background: tootsuite/mastodon#4486
2019-04-16 11:05:18 -07:00
Ryan Barrett
aa45f0c526
convert AP mentions to home page webmentions
...
for #46
2019-04-16 10:46:40 -07:00
Ryan Barrett
2e33354c27
add following and followers fields to AS2 actors
...
for #40 . they currently both point to endpoint URLs that don't actually work yet, just like the recently added outbox field.
2019-01-14 11:08:08 -08:00
Ryan Barrett
b237e52d6a
add outbox, summary, and publicKey.id properties to actors
...
for #39 , for pixelfed
2019-01-04 08:05:07 -08:00
Ryan Barrett
a88165c4e3
webmention => AP: big refactoring to support Create
...
#33 . still in progress!
2018-11-16 07:09:55 -08:00
Ryan Barrett
1c728f6590
AP => webmention: fetch actor for all activity types, including replies
...
fixes #41
2018-11-15 13:03:32 -08:00
Ryan Barrett
1c41aa4fd0
AP: blindly accept all Accepts
...
#21
2018-10-24 21:44:21 -07:00
Ryan Barrett
42ad9d1998
include full AS2 actor object in Response activities
...
for #21
2018-10-23 13:09:49 -07:00
Ryan Barrett
98a19a994d
convert AP Follow to indieweb webmention with u-follow-of
...
#21 , https://indieweb.org/follow
2018-10-23 11:11:50 -07:00
Ryan Barrett
abc5ecd919
AP Follow: store users' followers in the datastore
...
#21
2018-10-21 17:37:33 -07:00
Ryan Barrett
a4b309afef
activitypub: refactor Accept response to Follow activity
...
...and get tests working. #21
2018-10-21 15:28:42 -07:00
Ryan Barrett
ed3a685831
accept AP follow requests: original patch from @swentel
...
...from https://github.com/snarfed/bridgy-fed/issues/21#issuecomment-430403595
2018-10-19 06:50:00 -07:00
Ryan Barrett
9fc47c8ba9
AP => wm: switch to proxying reply wm source, unwrap URLs
...
for #35
example: https://webmention.io/realize.be/webmention/hG654HOgobymY54pMaNS
2018-10-17 17:49:56 -07:00
Ryan Barrett
2ffd10306f
AP => webmention: handle full AS2 activities
...
mastodon used to send bare objects for lots of things, e.g. replies, but now sends ful AS2 Create activities with embedded objects much more often. handle those too.
thanks for the nudge @swentel!
2018-10-15 11:52:11 -07:00
Ryan Barrett
5047337738
upgrade BS4 to 4.6.3, pin lxml to app engine prod's version
2018-10-11 17:03:26 -07:00
Ryan Barrett
a1a66c2f24
fix logging for failed outbound HTTP requests
...
also disable logging full parsed mf2 in activitypub actor handler.
2017-10-24 18:27:07 -07:00
Ryan Barrett
c81a6b8ed2
activitypub: follow HTML link rels that point to AS2 objects
...
...by reusing common.get_as2(). also lots of misc refactoring.
2017-10-20 12:49:48 -07:00
Ryan Barrett
e985516dc3
return parsed JSON from common.get_as2(); use it in activitypub
2017-10-20 07:13:04 -07:00
Ryan Barrett
bb9294ea51
tell common.error() explicitly when to include exc_info
2017-10-17 11:47:01 -07:00
Ryan Barrett
bf0897ba81
activitypub and salmon: return 501 on unsupported activity type
2017-10-16 22:21:13 -07:00
Ryan Barrett
73ac7ba1e7
unify webmention sending from salmon and activitypub into common
2017-10-16 09:13:55 -07:00
Ryan Barrett
6190f3bc0c
refactor abort() calls to use common.error()
2017-10-15 16:57:33 -07:00
Ryan Barrett
f940718790
salmon: support incoming likes and shares (reposts)
2017-10-15 16:57:05 -07:00
Ryan Barrett
f2fa44bfff
activitypub: fetch and include source actor if necessary
...
e.g. for likes and reposts, so that we can render them in our proxy page.
2017-10-12 23:14:46 -07:00
Ryan Barrett
ef89af7372
switch to using Response.get_or_create()
...
also test an incoming activitypub like with an id with a fragment
2017-10-11 18:13:06 -07:00
Ryan Barrett
9a7975bf5d
activitypub: use proxy mf2 URLs for likes and reposts/boosts
...
for #4
2017-10-10 22:42:19 -07:00
Ryan Barrett
6a65e2e74a
store source post in Response
...
...so we can use it to render proxy ActivityPub mf2 pages for webmentions
2017-10-09 21:11:40 -05:00
Ryan Barrett
19a686edc3
store responses in new Response datastore model
2017-10-09 19:29:50 -05:00
Ryan Barrett
41cf159595
activitypub: minor log message bug fix
2017-10-04 16:59:23 -07:00
Ryan Barrett
4f79e6ebd5
incoming activitypub: return 400 for non-replies (likes etc)
...
for #4
2017-10-04 09:07:29 -07:00
Ryan Barrett
71f0a4c8d4
AP: add publicKey to actor object
...
needed for mastodon interop
2017-10-01 07:01:35 -07:00
Ryan Barrett
fcef1e84ec
switch from request.host_url to appengine_config.HOST_URL
...
...because for requests that were redirected, request.host_url would sometimes lose port. no clue how or why.
e.g. testing mastodon interop locally with me@www.brid.gy, mastodon would fetch
https://www.brid.gy/.well-known/webfinger?resource=acct:me@www.brid.gy
which would 302 redirect to
http://localhost:8080/.well-known/webfinger?resource=acct:me@www.brid.gy
and in that request, request.host_url would be http://localhost . but if i curled that second webfinger URL directly, request.host_url would be http://localhost:8080 . weird.
2017-09-30 07:56:40 -07:00
Ryan Barrett
c998901108
update webfinger, AS2, etc for receiving AP
2017-09-28 07:25:21 -07:00
Ryan Barrett
4dd8f7e1a7
mastodon interop: salmon + activitypub + webfinger cleanup, tests
2017-09-03 13:03:31 -07:00
Ryan Barrett
b975d68a72
mastodon interop: webfinger cleanup, tests
2017-09-03 12:35:18 -07:00
Ryan Barrett
c9090401b0
misc tweaks after testing against distbin, mastodon, etc
...
- salmon: use top-level <entry>, not <feed>
- activitypub: actor and attributedTo are objects, not string URLs
- activitypub: cc public audience
- activitypub: use inbox in target post object if available
etc...
2017-08-26 15:20:54 -07:00
Ryan Barrett
b7daa0e4ee
400 when we can't find a representative h-card
2017-08-19 13:31:06 -07:00
Ryan Barrett
c1c99c6ca5
minor activitypub bug fix (default http, not https) and test improvements
2017-08-19 12:29:10 -07:00
Ryan Barrett
6ddd1d66d9
start on webfinger
2017-08-19 08:21:05 -07:00
Ryan Barrett
04e65d612d
add webmention endpoint and conversion to AP inbox delivery
2017-08-15 08:39:32 -07:00
Ryan Barrett
576410005b
extract out requests.get() and user agent header into new common.py file
2017-08-14 23:07:24 -07:00