05b86df9d2
Web.fetch: if we're fetching a homepage, set its url to the initial url
...
not the final response URL. handles when root domain redirects to www. for #1490
2024-11-15 21:31:22 -08:00
05b6e3352b
Web.fetch: handle representative h-card URL with and without trailing slash
2024-10-24 23:06:25 -07:00
fa8149165b
receive task: fix delay logging
...
for #1378
2024-10-23 21:11:21 -07:00
bba9cb2d16
send and send task: switch orig_obj param from urlsafe-key to string id
...
...so that we can defer loading it to each protocol's send(). only ActivityPub actually needs it right now. should get rid of it eventually! see #1257
2024-10-11 11:34:31 -07:00
20477db813
noop: suppress logging in web.is_valid_domain
2024-10-08 15:03:43 -07:00
5b75fa43d5
noop: tests: rename ExplicitEnableFake => ExplicitFake, eefake => efake
2024-10-08 15:03:43 -07:00
d37f83b11b
use raise_=False in more Protocol.load calls
2024-10-04 12:59:57 -07:00
035ed3fdd3
Web.poll_feed: handle when we fetch a post and find no mf2 or metaformats
2024-10-03 13:35:25 -07:00
4255719939
activitypub.actor: put native user id in alsoKnownAs
...
for https://github.com/snarfed/bridgy-fed/issues/1331
2024-10-02 15:01:07 -07:00
788b37279a
pass Object data in receive task HTTP payload instead of through datastore
...
trying to cut down datastore load and costs. this switches receive tasks from storing and fetching Objects from the datastore to passing their properties - notably as2, our_as1, bsky, etc - in the HTTP request body, as serialized JSON inside form-encoded params.
for #1354 , #1149
2024-10-01 21:44:12 -07:00
af3a7e68d7
Web.fetch: update to handle util.fetch_mf2 now checking Content-Type
...
see snarfed/webutil@da13dbf9c8
2024-09-27 15:02:48 -07:00
75ff3f662e
incoming webmentions: if source is homepage and it 404/410s, don't try to delete user
...
we wouldn't anyway, since we wouldn't have a #bridgy-fed-create Object for their homepage, but still
2024-09-23 15:33:21 -07:00
3ebcaac7bc
Web.webmention_task: expand error handling to non-HTTP connection errors
...
fixes https://console.cloud.google.com/errors/detail/CPGR79OCxezpuwE;locations=global;time=P30D?project=bridgy-federated
2024-09-22 15:24:49 -07:00
ce0dc1da6d
another Web.poll_feed_task bug fix, update user even on error
...
specifically last_polled_feed
2024-09-21 09:13:51 -07:00
a64316df74
Web.poll_feed_task: add last_polled param to detect and suppress dupes
...
re 0f805e08856cdbefa6de
2024-09-21 08:22:21 -07:00
6cdbefa6de
Web.poll_feed_task: don't return 304 because it incurs retries
...
0f805e0885
2024-09-21 07:48:08 -07:00
0f805e0885
refactor Web.poll_feed_task, add better error handling
...
pull out an internal poll method, wrap it in a mostly-catch-all exception handler, make sure we more reliably create the next task even if fetching or parsing fails
fixes:
https://console.cloud.google.com/errors/detail/CICLhIKMm6nSHw;locations=global;time=P30D?project=bridgy-federated
https://console.cloud.google.com/errors/detail/CJCrlNfNn6n3FQ;locations=global;time=P30D?project=bridgy-federated
https://console.cloud.google.com/errors/detail/CPnSx-yemJ-hlAE;locations=global;time=P30D?project=bridgy-federated
etc
2024-09-20 12:12:28 -07:00
26c7796bba
Web.verify host-meta bug fix, only interpret 200 as owning webfinger
...
for https://console.cloud.google.com/errors/detail/CPWdpJyQ4I-GVg;time=PT6H;locations=global?project=bridgy-federated
2024-09-19 22:18:16 -07:00
0792ac1ae1
Web.owns_id: stop loading users from datastore
...
I *think* this is unnecessary now. guess we'll see!
2024-09-19 19:41:38 -07:00
dda4416132
Web.poll_feed_task: handle connection errors when reading response
...
fixes https://console.cloud.google.com/errors/detail/CPXxtLHG78r88gE;time=P1D;locations=global?project=bridgy-federated
2024-09-19 11:49:55 -07:00
48b4a52a63
Web.get_or_create bug fix for opted out/blocked users
...
fixes https://console.cloud.google.com/errors/detail/CO7DoqKtjOWQ5QE;time=P1D;locations=global?project=bridgy-federated
2024-09-19 11:42:27 -07:00
1b19fde4b3
/web-site: show nice error for fediverse servers
...
for #911
2024-09-18 21:05:50 -07:00
e4db9d62cd
Web.verify: check host-meta, and if site serves it, interpret as a fediverse server
...
for #911
2024-09-18 20:46:21 -07:00
c5809e4de9
atproto_firehose.subscribe: skip record types we don't support
...
for #1329
2024-09-13 11:10:51 -07:00
832b3e3914
ATProto chat log polling: update to handle getLog semantics
...
re https://github.com/bluesky-social/atproto/issues/2760 , for #966
2024-08-29 12:52:27 -07:00
9c0ad8c00c
ATProto: poll for new chat messages to protocol bot users
...
for #966
2024-08-27 22:44:37 -07:00
0cc576e091
noop: Web.poll_feed_task: log when we look for an image in metaformats
...
for #1264
2024-08-19 17:18:45 -07:00
9265c40195
/webmention: reject fed.brid.gy/r/ URLs
...
fixes https://console.cloud.google.com/errors/detail/CLSnttKfy4v90wE;locations=global;time=P30D?project=bridgy-federated
2024-08-17 22:32:51 -07:00
34cb4fbe59
AP: fix followers and outbox for protocol bot users
...
eg https://bsky.brid.gy/bsky.brid.gy/followers , wasn't working before
2024-08-17 12:01:58 -07:00
bfbd7494bb
web: drop feed polling max delay from 1w down to 1d
2024-08-13 09:59:37 -07:00
21fb033ac7
Web.verify: if redirect destination is right, ignore HTTP status code
...
we now return 404 if the user doesn't have `has_redirects` set yet, so we need to break that loop 😀
fixes #1247
2024-08-09 12:12:27 -07:00
479acea657
move Web.verify from /web-site handler to Web.get_or_create
...
for #1232
2024-08-06 11:07:13 -07:00
70dd6d3ceb
minor logging optimizations
...
for https://github.com/snarfed/bridgy-fed/issues/1149
2024-08-02 08:03:44 -07:00
100a208e5e
add allow_opt_out to Protocol.key_for, User.get_or_create, Protocol.receive internal
...
for #1204
2024-07-23 19:26:30 -07:00
4e958f68e2
Web feed polling: don't use user profile images as post metaformats images
...
for #1191
2024-07-23 13:05:27 -07:00
b90cdefacf
Web.fetch bug fix: only default metaformats on for home pages
...
fixes 45df0ea
2024-07-23 11:54:56 -07:00
45df0ea410
Web feed polling: if an entry doesn't have an image, fetch it from metaformats
2024-07-22 18:11:49 -07:00
3b37890635
web feed poll: accept text/xml as well as application/xml
...
https://bsky.app/profile/kerim.oxus.net/post/3kxtd6hvpt62s
https://stackoverflow.com/questions/4832357/whats-the-difference-between-text-xml-vs-application-xml-for-webservice-respons
2024-07-21 20:08:14 -07:00
f1713e68d9
drop a bunch of log messages from info to debug
...
...and other logging tweaks
for https://github.com/snarfed/bridgy-fed/issues/1060
2024-07-18 14:15:12 -07:00
6b1290eb53
whoops, revert the Web.get_or_create change in 4ff28ca377
2024-07-15 20:22:47 -07:00
4ff28ca377
minor noop refactorings
2024-07-15 20:09:33 -07:00
ee62334a93
turn on web <=> Bluesky!
...
for #1034
2024-07-15 19:40:10 -07:00
a2c382651a
Web.get_or_create: noop: refactor to use ids.normalize_user_id
...
for #1034
2024-07-15 18:08:10 -07:00
b606924d63
refactor SUPPORTED_AS1_TYPES usage into new Protocol.check_supported method
2024-07-04 21:11:38 -07:00
850d681f83
add Protocol.SUPPORTED_AS1_TYPES
2024-07-04 16:58:06 -07:00
84949f265b
Web.owns_id: check TLD against brevity's allowlist
...
fixes #1115
2024-06-20 23:36:01 -07:00
7ac501cfff
Web.fetch, get_or_create: only load or create protocol bot users that already exist
...
fixes https://console.cloud.google.com/errors/detail/CM-n8tK-5LXhIw;time=P7D?project=bridgy-federated
2024-06-20 13:41:39 -07:00
441849a22a
Revert "Revert "start using an ndb in memory global cache in the frontends and router""
...
This reverts commit 0cbcf81ecc
2024-06-04 14:27:02 -07:00
83893d0ea7
HTTP caching headers: switch to flask_util.headers
...
for #432 , fixes https://console.cloud.google.com/errors/detail/CMK10__x4ZbgbQ;time=P7D;refresh=true?project=bridgy-federated
2024-06-04 14:19:39 -07:00
3cb9a9366d
Web, ATProto: skip add activities, eg Bluesky list items
...
hopefully fixes https://console.cloud.google.com/errors/detail/CP7I5eaZj-_TxgE;time=P7D;refresh=true?project=bridgy-federated
2024-06-04 14:10:21 -07:00
Ryan Barrett