4ff28ca377
minor noop refactorings
2024-07-15 20:09:33 -07:00
1a23d8eba3
Protocol.targets: when adding in-reply-to, check from user's enabled protocols
...
fixes https://console.cloud.google.com/errors/detail/CMHloZaRmPXQZQ;time=P7D?project=bridgy-federated
2024-07-14 13:42:38 -07:00
2a83cbb0d8
Protocol.targets: bug fix for self replies
...
this bug snuck in because of our special case handling of Fake and OtherFake in Protocol.targets. ugh. need to get rid of that, and elsewhere, eg ids.*.
fixes #1179
2024-07-13 12:58:06 -07:00
1f73058ef6
Protocol.receive, ActivityPub.inbox: consolidate memcache key for seen activities
2024-07-12 08:21:30 -07:00
7b0151b0cf
memcache: truncate keys at 250 chars
...
max memcache key len:
https://github.com/memcached/memcached/wiki/Commands#standard-protocol
https://pymemcache.readthedocs.io/en/latest/apidoc/pymemcache.client.base.html
fixes https://console.cloud.google.com/errors/detail/CNPzqYzWg4_TyQE;time=P30D?project=bridgy-federated
2024-07-12 07:50:54 -07:00
06f3694c9d
Protocol.receive: return HTTP 299 for deterministic errors to prevent retries
...
for https://github.com/snarfed/bridgy-fed/issues/1063
2024-07-10 21:04:04 -07:00
89e2372922
Protocol.receive: detect and prevent duplicate calls for the same object
...
for https://github.com/snarfed/bridgy-fed/issues/1063 , should fix it. coming up with the test for this was fun!
2024-07-10 20:59:03 -07:00
6879aa93b0
noop: Protocol.targets: revise a misleading log message
2024-07-09 19:17:04 -07:00
714c317dd4
Protocol.targets: only deliver reposts, replies etc if the original post was bridged
...
for #1166 , #1151 , maybe others. #1151 is the big motivation here: that we're currently generating and sending Bluesky a ton of reposts with blank subject cid/URL, which afaict is the main reason bridging is getting stuck for some accounts and the relay is constantly calling `getRepo`. I'm hoping this will reduce or eliminate that.
this isn't a big behavior change, but it is a kind of big change to routing logic. it's tested pretty well, but I still expect something to break. we'll see.
2024-07-09 17:29:02 -07:00
893176fb56
Protocol.targets bug fix for undo share
...
fixes https://console.cloud.google.com/errors/detail/CJK54eaoneesMg;time=P30D?project=bridgy-federated
2024-07-09 07:41:24 -07:00
23e1a90c88
Protocol.targets bug fix for 71ebc99: don't add object to feed if internal call
...
for https://console.cloud.google.com/errors/detail/CMzg3eXbyOi1ygE;time=P1D?project=bridgy-federated
2024-07-09 07:41:24 -07:00
4b0146c590
Protocol.targets: skip copy protocol if inReplyTo or repost original wasn't copied there
...
once more with feeling!
for #1166
2024-07-09 07:41:24 -07:00
71ebc99a9f
Protocol.targets: give undo same targets as the activity it's undoing
...
for #1162
2024-07-07 17:21:37 -07:00
2189652a1d
Protocol.maybe_accept_follow: switch send from inline to task
2024-07-05 23:20:34 -07:00
e5a6ebf228
drop Protocol.HAS_FOLLOW_ACCEPTS in favor of SUPPORTED_AS1_TYPES
2024-07-05 23:12:35 -07:00
17ced85c6f
Protocol.receive: support blocks
...
for #1162
2024-07-05 22:46:52 -07:00
2d1cb80e80
Protocol.check_supported bug fix: handle undo of activity
2024-07-05 12:26:59 -07:00
fa1ae889d8
Protocol.check_supported: don't report error, just log
2024-07-04 22:13:28 -07:00
b606924d63
refactor SUPPORTED_AS1_TYPES usage into new Protocol.check_supported method
2024-07-04 21:11:38 -07:00
850d681f83
add Protocol.SUPPORTED_AS1_TYPES
2024-07-04 16:58:06 -07:00
036ccb8d16
fix a few errant AS1 'create' verbs to 'post'
2024-06-30 21:29:06 -07:00
b87b4aa5f8
Protocol.receive: stop special casing replies, bridge them everywhere more often
...
For https://github.com/snarfed/bridgy-fed/issues/1047#issuecomment-2195298955 . This is a bit of a YOLO, I'm a bit apprehensive about it: I'm jettisoning much of the reply-specific logic in Protocol.receive and instead delivering replies across protocols more often. Still only delivering self-replies to followers though, not other replies.
🤞 🤞 🤞
2024-06-27 13:20:07 -07:00
45b48c6efb
Protocol.translate_ids: bug fix for ac2df74 for attachments with blocklisted url
...
fixes https://console.cloud.google.com/errors/CO25nvfq2Ojq0gE?project=bridgy-federated&time=P30D& , https://console.cloud.google.com/errors/CO2h-5yh276BQw?project=bridgy-federated&time=P30D& , etc
for #1154
2024-06-26 11:07:18 -07:00
ac2df747a6
Protocol.translate_ids: fix user-visible quote post URL
...
broken by a735396#1154 and #461
2024-06-25 21:10:33 -07:00
308501236e
Protocol.receive bug fix for 350e234b9c
...
for #1047
2024-06-25 15:57:55 -07:00
350e234b9c
Protocol.receive: bridge native replies if the in-reply-to was bridged
...
for https://github.com/snarfed/bridgy-fed/issues/1047
2024-06-25 13:37:14 -07:00
a735396864
Protocol.translate_ids: add attachment.url
...
for https://github.com/snarfed/bridgy-fed/issues/461 , fixes AP => ATProto quote post of AP original post that was bridged
2024-06-25 12:27:43 -07:00
6e7667c266
ATProto.send and nearby: don't assume that ids are already translated
...
for refactoring opt_out.py to be more generic
2024-06-22 16:41:23 -07:00
81bb033e68
Protocol.receive: on bot user follow, refresh profile id, not just actor id
...
fixes https://console.cloud.google.com/errors/detail/CIvu-KTludO5LA;time=P7D?project=bridgy-federated
2024-06-20 14:19:17 -07:00
a59eb8c466
Protocol.maybe_accept_follow: pass authed_as to Object.get_or_create
...
fixes https://console.cloud.google.com/errors/detail/CM7b86Se9r_qfQ;time=P7D?project=bridgy-federated
2024-06-20 13:08:35 -07:00
3d3ac1e9f4
ATProto.send: implement stop-following
...
hopefullly fixes https://console.cloud.google.com/errors/detail/CPvA_dLO4r_hIA;time=P7D?project=bridgy-federated
2024-06-19 19:59:55 -07:00
679e89c175
Protocol.translate_ids: add AS1 attachments[].id
...
needed for quote posts, #461
2024-06-17 16:36:44 -07:00
da5d434595
bug fix for deleting bridged fediverse actors: translate object as actor id
...
...not as object id. for #783 , 81319a7628
2024-06-14 12:54:55 -07:00
81319a7628
when a Bluesky user blocks the ap.brid.gy bot, delete their bridged fediverse profile
...
for #783
2024-06-13 21:47:19 -07:00
fdca3b6d55
Protocol.receive: don't attempt to deliver to followers on non-enabled protocols
...
mostly just handles this more gracefully. these deliveries were failing anyway, just not cleanly. fixes eg https://console.cloud.google.com/errors/detail/CMHloZaRmPXQZQ;time=P30D?project=bridgy-federated , probably others too
2024-06-13 12:02:54 -07:00
796edf2033
Protocol.receive: drop direct targets if user isn't bridged into their protocol
...
fixes https://console.cloud.google.com/errors/detail/CMHloZaRmPXQZQ;time=P30D?project=bridgy-federated
2024-06-08 15:08:12 -07:00
aca0a060a3
Protocol.send_task: only interpret HTTP exceptions as failures; re-raise others
...
this contributed to hiding the root cause of #1121 . log:
```
Running send task 6251425858687873358
Params: [('obj', 'ahBicmlkZ3ktZmVkZXJhdGVkclYLEgZPYmplY3QiSmh0dHBzOi8vaW5kaWV3ZWIuc29jaWFsL3VzZXJzL3NuYXJmZWQvc3RhdHVzZXMvMTEyNTgyNzgwNzcwODQzNjQ1L2FjdGl2aXR5DA'), ('orig_obj', ''), ('protocol', 'atproto'), ('url', 'https://atproto.brid.gy '), ('user', 'ahBicmlkZ3ktZmVkZXJhdGVkcjYLEgtBY3Rpdml0eVB1YiIlaHR0cHM6Ly9pbmRpZXdlYi5zb2NpYWwvdXNlcnMvc25hcmZlZAw')]
Sending https://indieweb.social/users/snarfed/statuses/112582780770843645/activity AS1: ...
Using server at https://api.bsky.app
Determining protocol for id https://indieweb.social/users/snarfed/statuses/112582780770843645
Key('Object', 'https://indieweb.social/users/snarfed/statuses/112582780770843645 ') owned by source_protocol activitypub
Can't translate object id https://indieweb.social/users/snarfed/statuses/112582780770843645 to atproto , haven't copied it there yet!
Determining protocol for id https://indieweb.social/users/snarfed
Key('Object', 'https://indieweb.social/users/snarfed ') owned by source_protocol activitypub
Using server at https://bsky.social/
"name 'link_length' is not defined
Traceback (most recent call last):
File "/workspace/protocol.py", line 1611, in send_task
sent = PROTOCOLS[protocol].send(obj, url, from_user=user, orig_obj=orig_obj)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/atproto.py", line 423, in send
record = to_cls.convert(base_obj, fetch_blobs=True, from_user=from_user)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/protocol.py", line 540, in convert
converted = cls._convert(obj, from_user=from_user, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/atproto.py", line 610, in _convert
ret = bluesky.from_as1(cls.translate_ids(obj.as1), blobs=blobs,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "granary/bluesky.py", line 577, in from_as1
text = src.truncate(full_text, None, OMIT_LINK)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "granary/source.py", line 908, in truncate
truncated = brevity.shorten(content, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "brevity.py", line 247, in shorten
base_length = total_length(tokens)
^^^^^^^^^^^^^^^^^^^^
File "brevity.py", line 188, in total_length
return sum(token_length(t) for t in tokens)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "brevity.py", line 188, in <genexpr>
return sum(token_length(t) for t in tokens)
^^^^^^^^^^^^^^^
File "brevity.py", line 183, in token_length
return link_length
^^^^^^^^^^^
NameError: name 'link_length' is not defined"
```
2024-06-08 13:20:34 -07:00
e761f23324
Protocol: noop, comment tweak and TODO
2024-06-08 13:14:56 -07:00
026d99edc0
pass ignore_links=True to html_to_text everywhere
...
for #976
2024-06-06 18:14:35 -07:00
27fc843825
Protocol.add_source_links bug fixes, handle actors inside create/update activities
...
for #976
2024-06-06 11:40:25 -07:00
be30afd55a
ATProto: use plain text URLs domains in "bridged by Bridgy Fed" profile text
...
for #976
2024-06-06 08:58:09 -07:00
9e18b7f110
tweak "bridged by Bridgy Fed" links in profile: use handle, add source protocol
...
for #976
2024-06-06 08:58:09 -07:00
ad81b1883e
refactor Protocol.convert, extract out new add_source_links method
...
for #976
2024-06-06 08:58:08 -07:00
3fe402e54c
drop Object cache in Protocol.load
2024-06-04 14:27:04 -07:00
5e2dacc21f
Protocol.send_task: if send returns False, return 204, not 304
...
send returns False when the object or target URL are undeliverable. retries shouldn't fix this, so we don't want to retry.
2024-06-04 12:30:50 -07:00
712da61181
auth error reporting: standardize messages, include HTTP context
...
trying to collect them into two top-level Error Reporting errors, with details for each one in the user field
for #566
2024-06-04 11:27:17 -07:00
cc564a2b19
turn on authorization checks! HTTP sig matches actor, actor is allowed to modify object
...
for #566
2024-06-03 14:12:08 -07:00
24559c8f17
Protocol.receive: normalize ids before resolving
...
for #959 , and possibly others
2024-06-02 08:41:52 -07:00
5266974731
Protocol.for_id: make Web own bot actor AP ids, not bot protocol
2024-06-02 08:21:18 -07:00
a2376f8f14
prune more logging messages
2024-06-01 07:17:44 -07:00
7e594e3227
authz: move NO_AUTH_DOMAINS (a.gup.pe) from Protocol.receive to activitypub.inbox
...
for #566
2024-05-30 13:34:14 -07:00
9f33767f33
authz: move LD Sig check from Protocol.receive to activitypub.inbox
...
for #566
2024-05-29 21:17:38 -07:00
8f5d8221a9
add ids.profile_id, use everywhere
...
for #566
2024-05-29 16:18:15 -07:00
bfbae4b280
authz: normalize user ids before comparing, in both receive and Object.get_or_create
...
for #566
2024-05-29 12:29:38 -07:00
d8910d52cd
LIMITED_DOMAINS bug fix, read lines from file separately
2024-05-28 20:40:48 -07:00
ea0f61f446
authz: when comparing domains, handle common subdomains like www
...
for #566
2024-05-28 20:36:18 -07:00
f2a64c8b5c
Object.get_or_create: rename actor => authed_as, add to all calls
...
for #566
2024-05-28 16:07:32 -07:00
820f7fa4f4
authz: add protocol.NO_AUTH_DOMAINS for domains we can't authorize yet
...
...eg a.gup.pe. https://github.com/snarfed/bridgy-fed/issues/566#issuecomment-2130714037
2024-05-28 15:23:22 -07:00
fc14298f0d
authz: expand LD Sigs check to any signature field, regardless of creator
...
for #566
2024-05-28 15:15:23 -07:00
03672679b9
activitypub, protocol: misc noop logging and code style tweaks
2024-05-27 20:38:49 -07:00
6f84a98a5d
Protocol.receive: more authz log message tweaks
...
for #566
2024-05-24 20:00:23 -07:00
4ca02824c9
Protocol.receive: tweak authz logging messages
...
for #566
2024-05-24 19:30:31 -07:00
6ffc4a3c3b
infra: add new router flex service, switch tasks to run on it
...
receive, send, webmention, and poll-feed
2024-05-23 20:40:54 -07:00
aa5ec4cccd
Protocol.maybe_delete_copy: minor tweaks
...
for #783
2024-05-23 11:43:02 -07:00
9de5eb400c
Protocol.targets: don't add ATProto for reposts of/replies to non-bridged posts
2024-05-22 23:28:53 -07:00
15af8b2585
limited domains bug fix: don't count protocol bot user follows
2024-05-22 22:33:57 -07:00
f39392e225
Protocol: when disabling a protocol, also delete that protocol's copy user
...
for #783
2024-05-22 16:33:13 -07:00
45ba0fb69f
Protocol.send_task: handle missing url or protocol param
2024-05-22 14:28:17 -07:00
1778a1145e
Protocol.bot_follow: send in separate task instead of inline
...
fixes #1041
2024-05-22 14:28:17 -07:00
06e2e61b1e
ATProto: switch to granary's logic to populate strongRefs, via DatastoreClient
...
for #1007
2024-05-16 13:11:29 -07:00
e4f9966686
Protocol.receive: second part of making explicit opt in override implicit opt out
...
for #1052 , #974
2024-05-15 11:41:20 -07:00
03c5638bc1
Protocol.receive bug fix: resolve ids before checking for bot user follow, not after
...
for #1017
2024-05-14 17:01:20 -07:00
f6798f2725
abstract "bridged from Bridgy Fed" profile label across protocols
...
for #976
2024-05-14 16:03:42 -07:00
79e5d2be07
bug fix for User.status handling: check for false-ness, not is None
...
we have some users in the datastore with status None, some with False. 😕 fixes #1048
2024-05-12 19:02:07 -07:00
348987f0b4
add Protocol.REQUIRES_OLD_ACCOUNT, enable for ActivityPub
2024-05-11 19:20:33 -07:00
59b9e9c904
add protocol.LIMITED_DOMAINS
2024-05-11 17:39:04 -07:00
de0af66979
basic anti-spam: add new Protocol.REQUIRES_AVATAR/NAME constants
...
...and start returning `blocked` from User.status for them
2024-05-11 16:03:07 -07:00
389e2d9205
Protocol.targets: don't add trailing slash to PDS URL https://atproto.brid.gy
...
fixes #1032
2024-05-08 09:37:47 -07:00
f533479e0b
noop: drop unused flask.g imports
2024-05-07 17:14:59 -07:00
4da00192b0
when someone follows a bot user to opt in, have the bot user follow them back
...
fixes #999 . this makes sure we start receiving posts from AP users immediately, instead of waiting until a Bluesky opted in user follows them.
2024-05-07 10:54:18 -07:00
2fa1ae73eb
when someone follows a bot user, refresh their profile first
...
...so that we're up to date on whether they're opted out or not
fixes #1017
2024-05-06 11:30:33 -07:00
515be28cdd
Protocol.receive: automatically send to ATProto PDS if user has ATProto enabled
...
for #999
2024-05-05 17:21:24 -07:00
2bf526ab7c
refactor validating handles in ATProto and elsewhere
...
for https://github.com/snarfed/bridgy-fed/issues/982
2024-05-03 15:18:16 -07:00
91638ef8bb
bring back self replies check, only send self replies to followers
...
revises 073ce475e5#981 , #950 , #949
2024-05-02 13:39:10 -07:00
0cfd188fd4
Protocol.targets fix for bug introduced in 073ce475e5
...
fixes https://console.cloud.google.com/errors/detail/CIzN5p2u2IPGpQE;time=P30D?project=bridgy-federated
2024-05-01 18:53:09 -07:00
073ce475e5
Protocol.receive: bridge replies to followers where in-reply-to already is
...
...ie if it's on the follower's protocol, either native or bridged.
for #981 , #950 , #949
not confident in this, despite the tests. worried it will either over- or under-deliver. guess we'll see.
2024-05-01 17:45:48 -07:00
ee9bb53745
ATProto: drop trailing slash from PDS URL
...
the ATProto DID spec says it shouldn't be there: https://atproto.com/specs/did#did-documents
> The serviceEndpoint field must contain an HTTPS URL of server. It should contain only the URI scheme (http or https), hostname, and optional port number, not any "userinfo", path prefix, or other components.
thanks for the nudge @mackuba!
2024-05-01 17:45:48 -07:00
9617d8ab71
move Protocol.is_enabled to User.is_enabled
2024-04-27 20:34:37 -07:00
cbe4b59206
convert Protocol.is_enabled_to to an instance method
...
next commit will move it to User
2024-04-27 20:27:33 -07:00
c065c712f1
more id/handle translation tweaks for protocol bot users
...
hopefully fixes Accept actor for responding to bot user follows in AP
2024-04-25 14:29:48 -07:00
1d20befe7e
Protocol.receive: always wrap actor objects in update, not post/create
2024-04-25 13:45:27 -07:00
dd1d53c28b
allow receive tasks for protocol bot domains, for update profile button
2024-04-25 12:39:04 -07:00
3d84dc4d36
don't show protocol bot users in their own protocols
2024-04-25 11:21:33 -07:00
d6a10b4be0
update profile button: make receive task instead of running it inline
2024-04-24 17:06:25 -07:00
41b2aaa1a8
incoming DMs to protocol bot users: filter out @-mentions
...
for #880
2024-04-24 16:45:43 -07:00
06bf3bf534
receive: drop activity if protocol doesn't own actor
...
it's probably from a bridged user, and we only want to handle original activities, not bridged ones.
fixes https://console.cloud.google.com/errors/detail/CM6i4sH4176iaQ;time=P30D?project=bridgy-federated
2024-04-24 15:57:00 -07:00
b543fdb1d5
switch to Gmail SMTP, other minor tweaks
2024-04-24 14:26:20 -07:00
0238b46e36
add Protocol.HAS_COPIES, use to generate ids.COPIES_PROTOCOLS
2024-04-23 16:52:53 -07:00
304994e3b7
Protocol.for_id: return Web for protocol bot users
2024-04-23 09:39:30 -07:00
d4a56127d9
incoming webmentions: stop converting home pages to update activities
...
pass Protocol.receive the bare actor object, let it convert to update activity, like with posts etc
2024-04-23 06:50:55 -07:00
Ryan Barrett