mirror
/
bridgy-fed
kopia lustrzana https://github.com/snarfed/bridgy-fed
1
0
Forkuj 0
Kod Zgłoszenia Packages Projekty Wydania Wiki Aktywność
2 486 Commity
48 Gałęzie
0 Tagi
20 MiB
Wykres commitów

2486 Commity (5b7f8de8d696cf664ccf22078b1101f62365da0e)

Autor SHA1 Wiadomość Data
Ryan Barrett 5b7f8de8d6
ATProto.send: handle delete when we have Object but not contents 
fixes https://console.cloud.google.com/errors/detail/COjSuYPS0qLvmAE;time=P30D?project=bridgy-federated
 2024-06-08 16:40:19 -07:00
Ryan Barrett 4a2abbac5b
ATProto.send, create_report: short circuit if repo is tombstoned 
fixes https://console.cloud.google.com/errors/detail/CLTU-qezn5vJnQE;time=P30D?project=bridgy-federated
 2024-06-08 15:16:39 -07:00
Ryan Barrett 796edf2033
Protocol.receive: drop direct targets if user isn't bridged into their protocol 
fixes https://console.cloud.google.com/errors/detail/CMHloZaRmPXQZQ;time=P30D?project=bridgy-federated
 2024-06-08 15:08:12 -07:00
Ryan Barrett 9ff28ea690
ATProto.send: short circuit out for deletes of objects we don't have 
fixes https://console.cloud.google.com/errors/detail/COjSuYPS0qLvmAE;time=P30D?project=bridgy-federated , https://console.cloud.google.com/errors/detail/COX1ss-HvOq5Xg;time=P30D?project=bridgy-federated
 2024-06-08 14:28:16 -07:00
Ryan Barrett aca0a060a3
Protocol.send_task: only interpret HTTP exceptions as failures; re-raise others 
this contributed to hiding the root cause of #1121. log:

```
Running send task 6251425858687873358
Params: [('obj', 'ahBicmlkZ3ktZmVkZXJhdGVkclYLEgZPYmplY3QiSmh0dHBzOi8vaW5kaWV3ZWIuc29jaWFsL3VzZXJzL3NuYXJmZWQvc3RhdHVzZXMvMTEyNTgyNzgwNzcwODQzNjQ1L2FjdGl2aXR5DA'), ('orig_obj', ''), ('protocol', 'atproto'), ('url', 'https://atproto.brid.gy'), ('user', 'ahBicmlkZ3ktZmVkZXJhdGVkcjYLEgtBY3Rpdml0eVB1YiIlaHR0cHM6Ly9pbmRpZXdlYi5zb2NpYWwvdXNlcnMvc25hcmZlZAw')]
Sending https://indieweb.social/users/snarfed/statuses/112582780770843645/activity AS1: ...
Using server at https://api.bsky.app
Determining protocol for id https://indieweb.social/users/snarfed/statuses/112582780770843645
  Key('Object', 'https://indieweb.social/users/snarfed/statuses/112582780770843645') owned by source_protocol activitypub
Can't translate object id https://indieweb.social/users/snarfed/statuses/112582780770843645 to atproto , haven't copied it there yet!
Determining protocol for id https://indieweb.social/users/snarfed
  Key('Object', 'https://indieweb.social/users/snarfed') owned by source_protocol activitypub
Using server at https://bsky.social/
"name 'link_length' is not defined
Traceback (most recent call last):
  File "/workspace/protocol.py", line 1611, in send_task
    sent = PROTOCOLS[protocol].send(obj, url, from_user=user, orig_obj=orig_obj)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/atproto.py", line 423, in send
    record = to_cls.convert(base_obj, fetch_blobs=True, from_user=from_user)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/protocol.py", line 540, in convert
    converted = cls._convert(obj, from_user=from_user, **kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/atproto.py", line 610, in _convert
    ret = bluesky.from_as1(cls.translate_ids(obj.as1), blobs=blobs,
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "granary/bluesky.py", line 577, in from_as1
    text = src.truncate(full_text, None, OMIT_LINK)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "granary/source.py", line 908, in truncate
    truncated = brevity.shorten(content, **kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "brevity.py", line 247, in shorten
    base_length = total_length(tokens)
                  ^^^^^^^^^^^^^^^^^^^^
  File "brevity.py", line 188, in total_length
    return sum(token_length(t) for t in tokens)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "brevity.py", line 188, in <genexpr>
    return sum(token_length(t) for t in tokens)
               ^^^^^^^^^^^^^^^
  File "brevity.py", line 183, in token_length
    return link_length
           ^^^^^^^^^^^
NameError: name 'link_length' is not defined"
```
 2024-06-08 13:20:34 -07:00
Ryan Barrett e761f23324
Protocol: noop, comment tweak and TODO 2024-06-08 13:14:56 -07:00
Ryan Barrett 1b4a04c70e
hub: bring back ROLLBACK_WINDOW, bumped up to 50k seqs 2024-06-07 16:47:56 -07:00
Ryan Barrett 026d99edc0
pass ignore_links=True to html_to_text everywhere 
for #976
 2024-06-06 18:14:35 -07:00
Ryan Barrett 183c829229
ATProto.add_source_links: truncate before adding source links 
for #976
 2024-06-06 14:16:05 -07:00
Ryan Barrett 27fc843825
Protocol.add_source_links bug fixes, handle actors inside create/update activities 
for #976
 2024-06-06 11:40:25 -07:00
Ryan Barrett be30afd55a
ATProto: use plain text URLs domains in "bridged by Bridgy Fed" profile text 
for #976
 2024-06-06 08:58:09 -07:00
Ryan Barrett 9e18b7f110
tweak "bridged by Bridgy Fed" links in profile: use handle, add source protocol 
for #976
 2024-06-06 08:58:09 -07:00
Ryan Barrett ad81b1883e
refactor Protocol.convert, extract out new add_source_links method 
for #976
 2024-06-06 08:58:08 -07:00
dependabot[bot] 64aeae63ad build(deps): bump redis from 5.0.4 to 5.0.5 
Bumps [redis](https://github.com/redis/redis-py) from 5.0.4 to 5.0.5.
- [Release notes](https://github.com/redis/redis-py/releases)
- [Changelog](https://github.com/redis/redis-py/blob/master/CHANGES)
- [Commits](https://github.com/redis/redis-py/compare/v5.0.4...v5.0.5)

---
updated-dependencies:
- dependency-name: redis
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-06-06 05:10:30 -07:00
dependabot[bot] 0e56a37526 build(deps): bump cbor2 from 5.6.3 to 5.6.4 
Bumps [cbor2](https://github.com/agronholm/cbor2) from 5.6.3 to 5.6.4.
- [Release notes](https://github.com/agronholm/cbor2/releases)
- [Changelog](https://github.com/agronholm/cbor2/blob/master/docs/versionhistory.rst)
- [Commits](https://github.com/agronholm/cbor2/compare/5.6.3...5.6.4)

---
updated-dependencies:
- dependency-name: cbor2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-06-06 05:10:12 -07:00
Ryan Barrett 025666fcac
docs: mention that webmention trigger requires microformats 
fixes #1085
 2024-06-05 06:36:27 -07:00
dependabot[bot] b4a3f11f2a build(deps): bump cryptography from 42.0.7 to 42.0.8 
Bumps [cryptography](https://github.com/pyca/cryptography) from 42.0.7 to 42.0.8.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/42.0.7...42.0.8)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-06-05 05:32:44 -07:00
Ryan Barrett 8a5373dcb9
docs: add details to reply controls section 
fixes #1098
 2024-06-04 22:05:23 -07:00
Ryan Barrett 0046f6436e
ATProto: when creating repo, store chat declaration to disable DMs 
for https://github.com/snarfed/bridgy-fed/issues/1101
 2024-06-04 20:22:31 -07:00
Ryan Barrett ba00850840
turn on memorystore ndb global cache in hub and atproto_firehose 2024-06-04 19:27:52 -07:00
Ryan Barrett 3ccce1bc89
start using memorystore memcached for ndb global cache 2024-06-04 15:40:32 -07:00
Ryan Barrett 3fe402e54c
drop Object cache in Protocol.load 2024-06-04 14:27:04 -07:00
Ryan Barrett 07517f1685
ndb global cache: limit to users and profile objects, not other objects/activities 2024-06-04 14:27:04 -07:00
Ryan Barrett 441849a22a
Revert "Revert "start using an ndb in memory global cache in the frontends and router"" 
This reverts commit 0cbcf81ecc.

trying again!
 2024-06-04 14:27:02 -07:00
Ryan Barrett 83893d0ea7
HTTP caching headers: switch to flask_util.headers 
for #432, fixes https://console.cloud.google.com/errors/detail/CMK10__x4ZbgbQ;time=P7D;refresh=true?project=bridgy-federated
 2024-06-04 14:19:39 -07:00
Ryan Barrett 3cb9a9366d
Web, ATProto: skip add activities, eg Bluesky list items 
hopefully fixes https://console.cloud.google.com/errors/detail/CP7I5eaZj-_TxgE;time=P7D;refresh=true?project=bridgy-federated
 2024-06-04 14:10:21 -07:00
Ryan Barrett 423b5511dd
ATProto: Web.send: skip polls (objectType question) 
for https://console.cloud.google.com/errors/detail/CPvA_dLO4r_hIA;time=P30D;refresh=true?project=bridgy-federated
 2024-06-04 12:31:38 -07:00
Ryan Barrett 5e2dacc21f
Protocol.send_task: if send returns False, return 204, not 304 
send returns False when the object or target URL are undeliverable. retries shouldn't fix this, so we don't want to retry.
 2024-06-04 12:30:50 -07:00
Ryan Barrett 918b2f04ee
Web.send: skip polls (objectType question) 
for https://console.cloud.google.com/errors/detail/CPvA_dLO4r_hIA;time=P30D;refresh=true?project=bridgy-federated
 2024-06-04 12:03:34 -07:00
Ryan Barrett 712da61181
auth error reporting: standardize messages, include HTTP context 
trying to collect them into two top-level Error Reporting errors, with details for each one in the user field

for #566
 2024-06-04 11:27:17 -07:00
dependabot[bot] 4af44b2553 build(deps): bump googleapis-common-protos from 1.63.0 to 1.63.1 
Bumps [googleapis-common-protos](https://github.com/googleapis/python-api-common-protos) from 1.63.0 to 1.63.1.
- [Release notes](https://github.com/googleapis/python-api-common-protos/releases)
- [Changelog](https://github.com/googleapis/python-api-common-protos/blob/main/CHANGELOG.md)
- [Commits](https://github.com/googleapis/python-api-common-protos/compare/v1.63.0...v1.63.1)

---
updated-dependencies:
- dependency-name: googleapis-common-protos
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-06-04 05:53:30 -07:00
dependabot[bot] 0dabc37239 build(deps): bump grpcio from 1.64.0 to 1.64.1 
Bumps [grpcio](https://github.com/grpc/grpc) from 1.64.0 to 1.64.1.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](https://github.com/grpc/grpc/compare/v1.64.0...v1.64.1)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-06-04 05:48:34 -07:00
Ryan Barrett e8e2c9692d
common.report_error bug fix 
fixes https://console.cloud.google.com/errors/detail/COyxtrLf5r2XowE;time=P30D?project=bridgy-federated
 2024-06-03 16:20:53 -07:00
Ryan Barrett bad49575fb
Web: always use fetched URL as id, not u-url 
fixes #829, cc #566
 2024-06-03 16:20:46 -07:00
Ryan Barrett cc564a2b19
turn on authorization checks! HTTP sig matches actor, actor is allowed to modify object 
for #566
 2024-06-03 14:12:08 -07:00
Ryan Barrett 251bd202d7
deprecate Users.atproto_notifs_indexed_at, atproto_feed_indexed_at 2024-06-03 14:12:08 -07:00
dependabot[bot] 4085014055 build(deps): bump certifi from 2024.2.2 to 2024.6.2 
Bumps [certifi](https://github.com/certifi/python-certifi) from 2024.2.2 to 2024.6.2.
- [Commits](https://github.com/certifi/python-certifi/compare/2024.02.02...2024.06.02)

---
updated-dependencies:
- dependency-name: certifi
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-06-03 05:21:57 -07:00
Ryan Barrett 24559c8f17
Protocol.receive: normalize ids before resolving 
for #959, and possibly others
 2024-06-02 08:41:52 -07:00
Ryan Barrett 5266974731
Protocol.for_id: make Web own bot actor AP ids, not bot protocol 2024-06-02 08:21:18 -07:00
Ryan Barrett 68ef81ccec
webfinger: 404 requests for web users with unknown username 
ie username doesn't match domain or mf2 custom username

fixes #1025
 2024-06-01 17:53:04 -07:00
Ryan Barrett a2376f8f14
prune more logging messages 2024-06-01 07:17:44 -07:00
Ryan Barrett 0cbcf81ecc
Revert "start using an ndb in memory global cache in the frontends and router" 
This reverts commit 7e77d7a463.

This slows down receive task processing, tasks all start taking longer and longer, can't figure out why. some kind of lock starvation in the global cache?

will experiment with this more in the ndb-global-cache branch.
 2024-06-01 07:04:38 -07:00
Ryan Barrett 9d135880d7
task queues: receive and send both now 20 concurrent 2024-05-31 13:46:26 -07:00
Ryan Barrett 64324db3da
Web.poll_feed_task: limit to 10 items per poll 2024-05-31 13:46:01 -07:00
Ryan Barrett 3d0aeed2f5
task queues: bump up first retry delay from 2m to 5m 2024-05-31 13:31:45 -07:00
Ryan Barrett 7e77d7a463
start using an ndb in memory global cache in the frontends and router 
lol yolo 982f8a70df/google/cloud/ndb/global_cache.py (L200-L202)

> Not intended for production use. Uses a single process wide dictionary to
> keep an in memory cache. For use in testing and to have an easily grokkable
> reference implementation. Thread safety is potentially a little sketchy.
 2024-05-31 13:08:25 -07:00
Ryan Barrett d0ca690dc4
HTTP Cache-Control header bug fix, use integer seconds for max-age, not float 
for #432
 2024-05-30 17:12:45 -07:00
Ryan Barrett 11c541b8f0
add HTTP caching to /r/ redirect URLs 
for #432
 2024-05-30 15:12:19 -07:00
Ryan Barrett 9c4888b02e
switch from flask_util.cached to HTTP caching, suppress some verbose logging 2024-05-30 14:55:35 -07:00
Ryan Barrett 93f302d5de
noop: tests: drop unused flask.g imports 2024-05-30 13:50:52 -07:00