fdca3b6d55
Protocol.receive: don't attempt to deliver to followers on non-enabled protocols
...
mostly just handles this more gracefully. these deliveries were failing anyway, just not cleanly. fixes eg https://console.cloud.google.com/errors/detail/CMHloZaRmPXQZQ;time=P30D?project=bridgy-federated , probably others too
2024-06-13 12:02:54 -07:00
616073f788
merge hub and atproto_firehose services into a single service, atproto_hub
...
atproto_firehose is averaging 20-40% CPU right now, hub 5-10%, and both are 500-600M memory steady state, so they can easily fit together.
2024-06-13 12:02:54 -07:00
590021926c
bump up router threads from 40 to 50, task queue rate limits
...
router is hitting 100% CPU sustained during bursts right now, probably need to either rethink its architecture or just add another core soon :/
2024-06-12 15:48:11 -07:00
387d3169a6
ATProto: rename original* custom fields to bridgyOriginal*
...
for #1092
2024-06-12 15:22:45 -07:00
417a88a2a6
ATProto: support new off-lexicon originalDescription and originalUrl fields
...
for snarfed/bridgy-fed#1092
2024-06-12 15:22:45 -07:00
5d42e042d8
build(deps): bump testtools from 2.7.1 to 2.7.2
...
Bumps [testtools](https://github.com/testing-cabal/testtools ) from 2.7.1 to 2.7.2.
- [Release notes](https://github.com/testing-cabal/testtools/releases )
- [Changelog](https://github.com/testing-cabal/testtools/blob/master/NEWS )
- [Commits](https://github.com/testing-cabal/testtools/compare/2.7.1...2.7.2 )
---
updated-dependencies:
- dependency-name: testtools
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-06-11 05:31:30 -07:00
04112c88bf
ATProto.add_source_links: switch to granary's Bluesky.truncate()
...
for #986
2024-06-10 16:15:59 -07:00
5f02f81775
build(deps): bump packaging from 24.0 to 24.1
...
Bumps [packaging](https://github.com/pypa/packaging ) from 24.0 to 24.1.
- [Release notes](https://github.com/pypa/packaging/releases )
- [Changelog](https://github.com/pypa/packaging/blob/main/CHANGELOG.rst )
- [Commits](https://github.com/pypa/packaging/compare/24.0...24.1 )
---
updated-dependencies:
- dependency-name: packaging
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-06-10 05:47:51 -07:00
b2f03414ef
ATProto: deregister XRPC methods we don't support
...
fixes:
* https://console.cloud.google.com/errors/detail/CIGTqdDu7ZPg4QE;time=P30D?project=bridgy-federated
* https://console.cloud.google.com/errors/detail/CPbL1db3m4KzVw;time=P30D?project=bridgy-federated
* https://console.cloud.google.com/errors/detail/CIGTqdDu7ZPg4QE;time=P30D?project=bridgy-federated
2024-06-09 18:46:53 -07:00
551637f6a1
ATProto.send: ignore update/delete of object that we didn't bridge
...
fixes https://console.cloud.google.com/errors/detail/COX1ss-HvOq5Xg;time=P30D?project=bridgy-federated
2024-06-08 16:52:34 -07:00
5b7f8de8d6
ATProto.send: handle delete when we have Object but not contents
...
fixes https://console.cloud.google.com/errors/detail/COjSuYPS0qLvmAE;time=P30D?project=bridgy-federated
2024-06-08 16:40:19 -07:00
4a2abbac5b
ATProto.send, create_report: short circuit if repo is tombstoned
...
fixes https://console.cloud.google.com/errors/detail/CLTU-qezn5vJnQE;time=P30D?project=bridgy-federated
2024-06-08 15:16:39 -07:00
796edf2033
Protocol.receive: drop direct targets if user isn't bridged into their protocol
...
fixes https://console.cloud.google.com/errors/detail/CMHloZaRmPXQZQ;time=P30D?project=bridgy-federated
2024-06-08 15:08:12 -07:00
9ff28ea690
ATProto.send: short circuit out for deletes of objects we don't have
...
fixes https://console.cloud.google.com/errors/detail/COjSuYPS0qLvmAE;time=P30D?project=bridgy-federated , https://console.cloud.google.com/errors/detail/COX1ss-HvOq5Xg;time=P30D?project=bridgy-federated
2024-06-08 14:28:16 -07:00
aca0a060a3
Protocol.send_task: only interpret HTTP exceptions as failures; re-raise others
...
this contributed to hiding the root cause of #1121 . log:
```
Running send task 6251425858687873358
Params: [('obj', 'ahBicmlkZ3ktZmVkZXJhdGVkclYLEgZPYmplY3QiSmh0dHBzOi8vaW5kaWV3ZWIuc29jaWFsL3VzZXJzL3NuYXJmZWQvc3RhdHVzZXMvMTEyNTgyNzgwNzcwODQzNjQ1L2FjdGl2aXR5DA'), ('orig_obj', ''), ('protocol', 'atproto'), ('url', 'https://atproto.brid.gy '), ('user', 'ahBicmlkZ3ktZmVkZXJhdGVkcjYLEgtBY3Rpdml0eVB1YiIlaHR0cHM6Ly9pbmRpZXdlYi5zb2NpYWwvdXNlcnMvc25hcmZlZAw')]
Sending https://indieweb.social/users/snarfed/statuses/112582780770843645/activity AS1: ...
Using server at https://api.bsky.app
Determining protocol for id https://indieweb.social/users/snarfed/statuses/112582780770843645
Key('Object', 'https://indieweb.social/users/snarfed/statuses/112582780770843645 ') owned by source_protocol activitypub
Can't translate object id https://indieweb.social/users/snarfed/statuses/112582780770843645 to atproto , haven't copied it there yet!
Determining protocol for id https://indieweb.social/users/snarfed
Key('Object', 'https://indieweb.social/users/snarfed ') owned by source_protocol activitypub
Using server at https://bsky.social/
"name 'link_length' is not defined
Traceback (most recent call last):
File "/workspace/protocol.py", line 1611, in send_task
sent = PROTOCOLS[protocol].send(obj, url, from_user=user, orig_obj=orig_obj)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/atproto.py", line 423, in send
record = to_cls.convert(base_obj, fetch_blobs=True, from_user=from_user)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/protocol.py", line 540, in convert
converted = cls._convert(obj, from_user=from_user, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/atproto.py", line 610, in _convert
ret = bluesky.from_as1(cls.translate_ids(obj.as1), blobs=blobs,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "granary/bluesky.py", line 577, in from_as1
text = src.truncate(full_text, None, OMIT_LINK)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "granary/source.py", line 908, in truncate
truncated = brevity.shorten(content, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "brevity.py", line 247, in shorten
base_length = total_length(tokens)
^^^^^^^^^^^^^^^^^^^^
File "brevity.py", line 188, in total_length
return sum(token_length(t) for t in tokens)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "brevity.py", line 188, in <genexpr>
return sum(token_length(t) for t in tokens)
^^^^^^^^^^^^^^^
File "brevity.py", line 183, in token_length
return link_length
^^^^^^^^^^^
NameError: name 'link_length' is not defined"
```
2024-06-08 13:20:34 -07:00
e761f23324
Protocol: noop, comment tweak and TODO
2024-06-08 13:14:56 -07:00
1b4a04c70e
hub: bring back ROLLBACK_WINDOW, bumped up to 50k seqs
2024-06-07 16:47:56 -07:00
026d99edc0
pass ignore_links=True to html_to_text everywhere
...
for #976
2024-06-06 18:14:35 -07:00
183c829229
ATProto.add_source_links: truncate before adding source links
...
for #976
2024-06-06 14:16:05 -07:00
27fc843825
Protocol.add_source_links bug fixes, handle actors inside create/update activities
...
for #976
2024-06-06 11:40:25 -07:00
be30afd55a
ATProto: use plain text URLs domains in "bridged by Bridgy Fed" profile text
...
for #976
2024-06-06 08:58:09 -07:00
9e18b7f110
tweak "bridged by Bridgy Fed" links in profile: use handle, add source protocol
...
for #976
2024-06-06 08:58:09 -07:00
ad81b1883e
refactor Protocol.convert, extract out new add_source_links method
...
for #976
2024-06-06 08:58:08 -07:00
64aeae63ad
build(deps): bump redis from 5.0.4 to 5.0.5
...
Bumps [redis](https://github.com/redis/redis-py ) from 5.0.4 to 5.0.5.
- [Release notes](https://github.com/redis/redis-py/releases )
- [Changelog](https://github.com/redis/redis-py/blob/master/CHANGES )
- [Commits](https://github.com/redis/redis-py/compare/v5.0.4...v5.0.5 )
---
updated-dependencies:
- dependency-name: redis
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-06-06 05:10:30 -07:00
0e56a37526
build(deps): bump cbor2 from 5.6.3 to 5.6.4
...
Bumps [cbor2](https://github.com/agronholm/cbor2 ) from 5.6.3 to 5.6.4.
- [Release notes](https://github.com/agronholm/cbor2/releases )
- [Changelog](https://github.com/agronholm/cbor2/blob/master/docs/versionhistory.rst )
- [Commits](https://github.com/agronholm/cbor2/compare/5.6.3...5.6.4 )
---
updated-dependencies:
- dependency-name: cbor2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-06-06 05:10:12 -07:00
025666fcac
docs: mention that webmention trigger requires microformats
...
fixes #1085
2024-06-05 06:36:27 -07:00
b4a3f11f2a
build(deps): bump cryptography from 42.0.7 to 42.0.8
...
Bumps [cryptography](https://github.com/pyca/cryptography ) from 42.0.7 to 42.0.8.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst )
- [Commits](https://github.com/pyca/cryptography/compare/42.0.7...42.0.8 )
---
updated-dependencies:
- dependency-name: cryptography
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-06-05 05:32:44 -07:00
8a5373dcb9
docs: add details to reply controls section
...
fixes #1098
2024-06-04 22:05:23 -07:00
0046f6436e
ATProto: when creating repo, store chat declaration to disable DMs
...
for https://github.com/snarfed/bridgy-fed/issues/1101
2024-06-04 20:22:31 -07:00
ba00850840
turn on memorystore ndb global cache in hub and atproto_firehose
2024-06-04 19:27:52 -07:00
3ccce1bc89
start using memorystore memcached for ndb global cache
2024-06-04 15:40:32 -07:00
3fe402e54c
drop Object cache in Protocol.load
2024-06-04 14:27:04 -07:00
07517f1685
ndb global cache: limit to users and profile objects, not other objects/activities
2024-06-04 14:27:04 -07:00
441849a22a
Revert "Revert "start using an ndb in memory global cache in the frontends and router""
...
This reverts commit 0cbcf81ecc
2024-06-04 14:27:02 -07:00
83893d0ea7
HTTP caching headers: switch to flask_util.headers
...
for #432 , fixes https://console.cloud.google.com/errors/detail/CMK10__x4ZbgbQ;time=P7D;refresh=true?project=bridgy-federated
2024-06-04 14:19:39 -07:00
3cb9a9366d
Web, ATProto: skip add activities, eg Bluesky list items
...
hopefully fixes https://console.cloud.google.com/errors/detail/CP7I5eaZj-_TxgE;time=P7D;refresh=true?project=bridgy-federated
2024-06-04 14:10:21 -07:00
423b5511dd
ATProto: Web.send: skip polls (objectType question)
...
for https://console.cloud.google.com/errors/detail/CPvA_dLO4r_hIA;time=P30D;refresh=true?project=bridgy-federated
2024-06-04 12:31:38 -07:00
5e2dacc21f
Protocol.send_task: if send returns False, return 204, not 304
...
send returns False when the object or target URL are undeliverable. retries shouldn't fix this, so we don't want to retry.
2024-06-04 12:30:50 -07:00
918b2f04ee
Web.send: skip polls (objectType question)
...
for https://console.cloud.google.com/errors/detail/CPvA_dLO4r_hIA;time=P30D;refresh=true?project=bridgy-federated
2024-06-04 12:03:34 -07:00
712da61181
auth error reporting: standardize messages, include HTTP context
...
trying to collect them into two top-level Error Reporting errors, with details for each one in the user field
for #566
2024-06-04 11:27:17 -07:00
4af44b2553
build(deps): bump googleapis-common-protos from 1.63.0 to 1.63.1
...
Bumps [googleapis-common-protos](https://github.com/googleapis/python-api-common-protos ) from 1.63.0 to 1.63.1.
- [Release notes](https://github.com/googleapis/python-api-common-protos/releases )
- [Changelog](https://github.com/googleapis/python-api-common-protos/blob/main/CHANGELOG.md )
- [Commits](https://github.com/googleapis/python-api-common-protos/compare/v1.63.0...v1.63.1 )
---
updated-dependencies:
- dependency-name: googleapis-common-protos
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-06-04 05:53:30 -07:00
0dabc37239
build(deps): bump grpcio from 1.64.0 to 1.64.1
...
Bumps [grpcio](https://github.com/grpc/grpc ) from 1.64.0 to 1.64.1.
- [Release notes](https://github.com/grpc/grpc/releases )
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md )
- [Commits](https://github.com/grpc/grpc/compare/v1.64.0...v1.64.1 )
---
updated-dependencies:
- dependency-name: grpcio
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-06-04 05:48:34 -07:00
e8e2c9692d
common.report_error bug fix
...
fixes https://console.cloud.google.com/errors/detail/COyxtrLf5r2XowE;time=P30D?project=bridgy-federated
2024-06-03 16:20:53 -07:00
bad49575fb
Web: always use fetched URL as id, not u-url
...
fixes #829 , cc #566
2024-06-03 16:20:46 -07:00
cc564a2b19
turn on authorization checks! HTTP sig matches actor, actor is allowed to modify object
...
for #566
2024-06-03 14:12:08 -07:00
251bd202d7
deprecate Users.atproto_notifs_indexed_at, atproto_feed_indexed_at
2024-06-03 14:12:08 -07:00
4085014055
build(deps): bump certifi from 2024.2.2 to 2024.6.2
...
Bumps [certifi](https://github.com/certifi/python-certifi ) from 2024.2.2 to 2024.6.2.
- [Commits](https://github.com/certifi/python-certifi/compare/2024.02.02...2024.06.02 )
---
updated-dependencies:
- dependency-name: certifi
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-06-03 05:21:57 -07:00
24559c8f17
Protocol.receive: normalize ids before resolving
...
for #959 , and possibly others
2024-06-02 08:41:52 -07:00
5266974731
Protocol.for_id: make Web own bot actor AP ids, not bot protocol
2024-06-02 08:21:18 -07:00
68ef81ccec
webfinger: 404 requests for web users with unknown username
...
ie username doesn't match domain or mf2 custom username
fixes #1025
2024-06-01 17:53:04 -07:00
Ryan Barrett