Ryan Barrett
5b7f8de8d6
ATProto.send: handle delete when we have Object but not contents
...
fixes https://console.cloud.google.com/errors/detail/COjSuYPS0qLvmAE;time=P30D?project=bridgy-federated
2024-06-08 16:40:19 -07:00
Ryan Barrett
4a2abbac5b
ATProto.send, create_report: short circuit if repo is tombstoned
...
fixes https://console.cloud.google.com/errors/detail/CLTU-qezn5vJnQE;time=P30D?project=bridgy-federated
2024-06-08 15:16:39 -07:00
Ryan Barrett
796edf2033
Protocol.receive: drop direct targets if user isn't bridged into their protocol
...
fixes https://console.cloud.google.com/errors/detail/CMHloZaRmPXQZQ;time=P30D?project=bridgy-federated
2024-06-08 15:08:12 -07:00
Ryan Barrett
9ff28ea690
ATProto.send: short circuit out for deletes of objects we don't have
...
fixes https://console.cloud.google.com/errors/detail/COjSuYPS0qLvmAE;time=P30D?project=bridgy-federated , https://console.cloud.google.com/errors/detail/COX1ss-HvOq5Xg;time=P30D?project=bridgy-federated
2024-06-08 14:28:16 -07:00
Ryan Barrett
aca0a060a3
Protocol.send_task: only interpret HTTP exceptions as failures; re-raise others
...
this contributed to hiding the root cause of #1121 . log:
```
Running send task 6251425858687873358
Params: [('obj', 'ahBicmlkZ3ktZmVkZXJhdGVkclYLEgZPYmplY3QiSmh0dHBzOi8vaW5kaWV3ZWIuc29jaWFsL3VzZXJzL3NuYXJmZWQvc3RhdHVzZXMvMTEyNTgyNzgwNzcwODQzNjQ1L2FjdGl2aXR5DA'), ('orig_obj', ''), ('protocol', 'atproto'), ('url', 'https://atproto.brid.gy '), ('user', 'ahBicmlkZ3ktZmVkZXJhdGVkcjYLEgtBY3Rpdml0eVB1YiIlaHR0cHM6Ly9pbmRpZXdlYi5zb2NpYWwvdXNlcnMvc25hcmZlZAw')]
Sending https://indieweb.social/users/snarfed/statuses/112582780770843645/activity AS1: ...
Using server at https://api.bsky.app
Determining protocol for id https://indieweb.social/users/snarfed/statuses/112582780770843645
Key('Object', 'https://indieweb.social/users/snarfed/statuses/112582780770843645 ') owned by source_protocol activitypub
Can't translate object id https://indieweb.social/users/snarfed/statuses/112582780770843645 to atproto , haven't copied it there yet!
Determining protocol for id https://indieweb.social/users/snarfed
Key('Object', 'https://indieweb.social/users/snarfed ') owned by source_protocol activitypub
Using server at https://bsky.social/
"name 'link_length' is not defined
Traceback (most recent call last):
File "/workspace/protocol.py", line 1611, in send_task
sent = PROTOCOLS[protocol].send(obj, url, from_user=user, orig_obj=orig_obj)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/atproto.py", line 423, in send
record = to_cls.convert(base_obj, fetch_blobs=True, from_user=from_user)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/protocol.py", line 540, in convert
converted = cls._convert(obj, from_user=from_user, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/atproto.py", line 610, in _convert
ret = bluesky.from_as1(cls.translate_ids(obj.as1), blobs=blobs,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "granary/bluesky.py", line 577, in from_as1
text = src.truncate(full_text, None, OMIT_LINK)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "granary/source.py", line 908, in truncate
truncated = brevity.shorten(content, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "brevity.py", line 247, in shorten
base_length = total_length(tokens)
^^^^^^^^^^^^^^^^^^^^
File "brevity.py", line 188, in total_length
return sum(token_length(t) for t in tokens)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "brevity.py", line 188, in <genexpr>
return sum(token_length(t) for t in tokens)
^^^^^^^^^^^^^^^
File "brevity.py", line 183, in token_length
return link_length
^^^^^^^^^^^
NameError: name 'link_length' is not defined"
```
2024-06-08 13:20:34 -07:00
Ryan Barrett
e761f23324
Protocol: noop, comment tweak and TODO
2024-06-08 13:14:56 -07:00
Ryan Barrett
1b4a04c70e
hub: bring back ROLLBACK_WINDOW, bumped up to 50k seqs
2024-06-07 16:47:56 -07:00
Ryan Barrett
026d99edc0
pass ignore_links=True to html_to_text everywhere
...
for #976
2024-06-06 18:14:35 -07:00
Ryan Barrett
183c829229
ATProto.add_source_links: truncate before adding source links
...
for #976
2024-06-06 14:16:05 -07:00
Ryan Barrett
27fc843825
Protocol.add_source_links bug fixes, handle actors inside create/update activities
...
for #976
2024-06-06 11:40:25 -07:00
Ryan Barrett
be30afd55a
ATProto: use plain text URLs domains in "bridged by Bridgy Fed" profile text
...
for #976
2024-06-06 08:58:09 -07:00
Ryan Barrett
9e18b7f110
tweak "bridged by Bridgy Fed" links in profile: use handle, add source protocol
...
for #976
2024-06-06 08:58:09 -07:00
Ryan Barrett
ad81b1883e
refactor Protocol.convert, extract out new add_source_links method
...
for #976
2024-06-06 08:58:08 -07:00
dependabot[bot]
64aeae63ad
build(deps): bump redis from 5.0.4 to 5.0.5
...
Bumps [redis](https://github.com/redis/redis-py ) from 5.0.4 to 5.0.5.
- [Release notes](https://github.com/redis/redis-py/releases )
- [Changelog](https://github.com/redis/redis-py/blob/master/CHANGES )
- [Commits](https://github.com/redis/redis-py/compare/v5.0.4...v5.0.5 )
---
updated-dependencies:
- dependency-name: redis
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-06-06 05:10:30 -07:00
dependabot[bot]
0e56a37526
build(deps): bump cbor2 from 5.6.3 to 5.6.4
...
Bumps [cbor2](https://github.com/agronholm/cbor2 ) from 5.6.3 to 5.6.4.
- [Release notes](https://github.com/agronholm/cbor2/releases )
- [Changelog](https://github.com/agronholm/cbor2/blob/master/docs/versionhistory.rst )
- [Commits](https://github.com/agronholm/cbor2/compare/5.6.3...5.6.4 )
---
updated-dependencies:
- dependency-name: cbor2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-06-06 05:10:12 -07:00
Ryan Barrett
025666fcac
docs: mention that webmention trigger requires microformats
...
fixes #1085
2024-06-05 06:36:27 -07:00
dependabot[bot]
b4a3f11f2a
build(deps): bump cryptography from 42.0.7 to 42.0.8
...
Bumps [cryptography](https://github.com/pyca/cryptography ) from 42.0.7 to 42.0.8.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst )
- [Commits](https://github.com/pyca/cryptography/compare/42.0.7...42.0.8 )
---
updated-dependencies:
- dependency-name: cryptography
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-06-05 05:32:44 -07:00
Ryan Barrett
8a5373dcb9
docs: add details to reply controls section
...
fixes #1098
2024-06-04 22:05:23 -07:00
Ryan Barrett
0046f6436e
ATProto: when creating repo, store chat declaration to disable DMs
...
for https://github.com/snarfed/bridgy-fed/issues/1101
2024-06-04 20:22:31 -07:00
Ryan Barrett
ba00850840
turn on memorystore ndb global cache in hub and atproto_firehose
2024-06-04 19:27:52 -07:00
Ryan Barrett
3ccce1bc89
start using memorystore memcached for ndb global cache
2024-06-04 15:40:32 -07:00
Ryan Barrett
3fe402e54c
drop Object cache in Protocol.load
2024-06-04 14:27:04 -07:00
Ryan Barrett
07517f1685
ndb global cache: limit to users and profile objects, not other objects/activities
2024-06-04 14:27:04 -07:00
Ryan Barrett
441849a22a
Revert "Revert "start using an ndb in memory global cache in the frontends and router""
...
This reverts commit 0cbcf81ecc
.
trying again!
2024-06-04 14:27:02 -07:00
Ryan Barrett
83893d0ea7
HTTP caching headers: switch to flask_util.headers
...
for #432 , fixes https://console.cloud.google.com/errors/detail/CMK10__x4ZbgbQ;time=P7D;refresh=true?project=bridgy-federated
2024-06-04 14:19:39 -07:00
Ryan Barrett
3cb9a9366d
Web, ATProto: skip add activities, eg Bluesky list items
...
hopefully fixes https://console.cloud.google.com/errors/detail/CP7I5eaZj-_TxgE;time=P7D;refresh=true?project=bridgy-federated
2024-06-04 14:10:21 -07:00
Ryan Barrett
423b5511dd
ATProto: Web.send: skip polls (objectType question)
...
for https://console.cloud.google.com/errors/detail/CPvA_dLO4r_hIA;time=P30D;refresh=true?project=bridgy-federated
2024-06-04 12:31:38 -07:00
Ryan Barrett
5e2dacc21f
Protocol.send_task: if send returns False, return 204, not 304
...
send returns False when the object or target URL are undeliverable. retries shouldn't fix this, so we don't want to retry.
2024-06-04 12:30:50 -07:00
Ryan Barrett
918b2f04ee
Web.send: skip polls (objectType question)
...
for https://console.cloud.google.com/errors/detail/CPvA_dLO4r_hIA;time=P30D;refresh=true?project=bridgy-federated
2024-06-04 12:03:34 -07:00
Ryan Barrett
712da61181
auth error reporting: standardize messages, include HTTP context
...
trying to collect them into two top-level Error Reporting errors, with details for each one in the user field
for #566
2024-06-04 11:27:17 -07:00
dependabot[bot]
4af44b2553
build(deps): bump googleapis-common-protos from 1.63.0 to 1.63.1
...
Bumps [googleapis-common-protos](https://github.com/googleapis/python-api-common-protos ) from 1.63.0 to 1.63.1.
- [Release notes](https://github.com/googleapis/python-api-common-protos/releases )
- [Changelog](https://github.com/googleapis/python-api-common-protos/blob/main/CHANGELOG.md )
- [Commits](https://github.com/googleapis/python-api-common-protos/compare/v1.63.0...v1.63.1 )
---
updated-dependencies:
- dependency-name: googleapis-common-protos
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-06-04 05:53:30 -07:00
dependabot[bot]
0dabc37239
build(deps): bump grpcio from 1.64.0 to 1.64.1
...
Bumps [grpcio](https://github.com/grpc/grpc ) from 1.64.0 to 1.64.1.
- [Release notes](https://github.com/grpc/grpc/releases )
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md )
- [Commits](https://github.com/grpc/grpc/compare/v1.64.0...v1.64.1 )
---
updated-dependencies:
- dependency-name: grpcio
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-06-04 05:48:34 -07:00
Ryan Barrett
e8e2c9692d
common.report_error bug fix
...
fixes https://console.cloud.google.com/errors/detail/COyxtrLf5r2XowE;time=P30D?project=bridgy-federated
2024-06-03 16:20:53 -07:00
Ryan Barrett
bad49575fb
Web: always use fetched URL as id, not u-url
...
fixes #829 , cc #566
2024-06-03 16:20:46 -07:00
Ryan Barrett
cc564a2b19
turn on authorization checks! HTTP sig matches actor, actor is allowed to modify object
...
for #566
2024-06-03 14:12:08 -07:00
Ryan Barrett
251bd202d7
deprecate Users.atproto_notifs_indexed_at, atproto_feed_indexed_at
2024-06-03 14:12:08 -07:00
dependabot[bot]
4085014055
build(deps): bump certifi from 2024.2.2 to 2024.6.2
...
Bumps [certifi](https://github.com/certifi/python-certifi ) from 2024.2.2 to 2024.6.2.
- [Commits](https://github.com/certifi/python-certifi/compare/2024.02.02...2024.06.02 )
---
updated-dependencies:
- dependency-name: certifi
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-06-03 05:21:57 -07:00
Ryan Barrett
24559c8f17
Protocol.receive: normalize ids before resolving
...
for #959 , and possibly others
2024-06-02 08:41:52 -07:00
Ryan Barrett
5266974731
Protocol.for_id: make Web own bot actor AP ids, not bot protocol
2024-06-02 08:21:18 -07:00
Ryan Barrett
68ef81ccec
webfinger: 404 requests for web users with unknown username
...
ie username doesn't match domain or mf2 custom username
fixes #1025
2024-06-01 17:53:04 -07:00
Ryan Barrett
a2376f8f14
prune more logging messages
2024-06-01 07:17:44 -07:00
Ryan Barrett
0cbcf81ecc
Revert "start using an ndb in memory global cache in the frontends and router"
...
This reverts commit 7e77d7a463
.
This slows down receive task processing, tasks all start taking longer and longer, can't figure out why. some kind of lock starvation in the global cache?
will experiment with this more in the ndb-global-cache branch.
2024-06-01 07:04:38 -07:00
Ryan Barrett
9d135880d7
task queues: receive and send both now 20 concurrent
2024-05-31 13:46:26 -07:00
Ryan Barrett
64324db3da
Web.poll_feed_task: limit to 10 items per poll
2024-05-31 13:46:01 -07:00
Ryan Barrett
3d0aeed2f5
task queues: bump up first retry delay from 2m to 5m
2024-05-31 13:31:45 -07:00
Ryan Barrett
7e77d7a463
start using an ndb in memory global cache in the frontends and router
...
lol yolo 982f8a70df/google/cloud/ndb/global_cache.py (L200-L202)
> Not intended for production use. Uses a single process wide dictionary to
> keep an in memory cache. For use in testing and to have an easily grokkable
> reference implementation. Thread safety is potentially a little sketchy.
2024-05-31 13:08:25 -07:00
Ryan Barrett
d0ca690dc4
HTTP Cache-Control header bug fix, use integer seconds for max-age, not float
...
for #432
2024-05-30 17:12:45 -07:00
Ryan Barrett
11c541b8f0
add HTTP caching to /r/ redirect URLs
...
for #432
2024-05-30 15:12:19 -07:00
Ryan Barrett
9c4888b02e
switch from flask_util.cached to HTTP caching, suppress some verbose logging
2024-05-30 14:55:35 -07:00
Ryan Barrett
93f302d5de
noop: tests: drop unused flask.g imports
2024-05-30 13:50:52 -07:00