Ryan Barrett
850d681f83
add Protocol.SUPPORTED_AS1_TYPES
2024-07-04 16:58:06 -07:00
Ryan Barrett
441849a22a
Revert "Revert "start using an ndb in memory global cache in the frontends and router""
...
This reverts commit 0cbcf81ecc
.
trying again!
2024-06-04 14:27:02 -07:00
Ryan Barrett
24559c8f17
Protocol.receive: normalize ids before resolving
...
for #959 , and possibly others
2024-06-02 08:41:52 -07:00
Ryan Barrett
0cbcf81ecc
Revert "start using an ndb in memory global cache in the frontends and router"
...
This reverts commit 7e77d7a463
.
This slows down receive task processing, tasks all start taking longer and longer, can't figure out why. some kind of lock starvation in the global cache?
will experiment with this more in the ndb-global-cache branch.
2024-06-01 07:04:38 -07:00
Ryan Barrett
7e77d7a463
start using an ndb in memory global cache in the frontends and router
...
lol yolo 982f8a70df/google/cloud/ndb/global_cache.py (L200-L202)
> Not intended for production use. Uses a single process wide dictionary to
> keep an in memory cache. For use in testing and to have an easily grokkable
> reference implementation. Thread safety is potentially a little sketchy.
2024-05-31 13:08:25 -07:00
Ryan Barrett
83211cddc2
testutil: noop refactoring, remove path arg in assert_task
2024-05-24 22:16:20 -07:00
Ryan Barrett
f2be280016
atproto_firehose: load ATProto users by updated, not by created
...
fixes #1082
2024-05-24 18:40:22 -07:00
Ryan Barrett
4cf0959080
atproto_firehose: oops, still need to skip commits from our bridged users
...
add a test, should have had one originally!
2024-05-22 22:08:25 -07:00
Ryan Barrett
976653d5bb
atproto_firehose: fix tests for f6575cca4f
2024-05-19 13:57:42 -07:00
Ryan Barrett
f6575cca4f
atproto_firehose.subscribe: skip bad commits
...
fixes #1061
2024-05-19 13:47:13 -07:00
Ryan Barrett
8ee4c3b2ea
atproto_firehose.handle: switch dag_json from compact=True to dialect='atproto'
...
for AT Protocol's DAG-JSON variant with $link and $bytes: https://atproto.com/specs/data-model ; for https://github.com/snarfed/bridgy-fed/issues/1000
2024-05-15 21:00:42 -07:00
Ryan Barrett
8eeb588d11
atproto_firehose: make its own GAE service, lots of other refactoring
...
optimize loading DIDs, move around threads and exception handling, etc.
for #978
2024-05-13 22:02:09 -07:00
Ryan Barrett
6bc9269ad0
ATProto firehose: load bridged dids from AtpRepo instead of users
...
hopefully cuts down on datastore load a bit
for #978
2024-05-10 11:31:47 -07:00
Ryan Barrett
741146eb8d
ATProto firehose: encode Object.bsky as compact DAG-JSON
...
fixes https://console.cloud.google.com/errors/detail/CIGJnbnCq7fm1gE;time=P30D?project=bridgy-federated and similar
2024-05-10 06:51:58 -07:00
Ryan Barrett
989cd29860
ATProto firehose: bug fix for loading DIDs
...
fixes https://console.cloud.google.com/errors/detail/CKKTu6mM0sunXQ;time=P30D?project=bridgy-federated
2024-05-09 21:25:13 -07:00
Ryan Barrett
d56f405c96
ATProto firehose: fix race conditions in loading DIDs
...
for #978
2024-05-09 15:39:18 -07:00
Ryan Barrett
66b8846760
ATProto firehose: refactor and extract out loading our DIDs
2024-05-09 14:05:05 -07:00
Ryan Barrett
eea8779872
ATProto firehose: subscribe at stored cursor + 1
...
for #978
2024-05-09 08:44:30 -07:00
Ryan Barrett
1ecdaf6b34
ATProto firehose: store cursor
2024-05-08 23:11:27 -07:00
Ryan Barrett
b788105a71
ATProto firehose: start implementing cursor
...
for #978
2024-05-08 21:35:03 -07:00
Ryan Barrett
1e5fcfda0b
ATProto firehose: error handling, reconnect in subscribe
2024-05-08 20:43:10 -07:00
Ryan Barrett
b8e0fecc3b
atproto_firehose.handle: implement delete
...
for #978
2024-05-08 20:19:18 -07:00
Ryan Barrett
b4841f3e60
ATProto firehose: noop refactoring, add Op namedtuple
...
for #978
2024-05-08 15:48:39 -07:00
Ryan Barrett
0d24fafefc
ATProto firehose: tests for updates
...
for #978
2024-05-08 13:31:08 -07:00
Ryan Barrett
7867326ebe
ATProto firehose: implement deletes
...
for #978
2024-05-08 13:26:36 -07:00
Ryan Barrett
5c77f02dba
ATProto firehose: bug fix for quote posts with media embeds
...
for #978
2024-05-08 12:13:26 -07:00
Ryan Barrett
20afb59809
ATProto firehose: implement quote posts
...
for #978
2024-05-08 11:39:45 -07:00
Ryan Barrett
72ae63f4a0
ATProto firehose: test replies, mentions
...
#978
2024-05-08 11:31:31 -07:00
Ryan Barrett
8ff5426c49
test_atproto_firehose: noop refactor
2024-05-08 11:25:59 -07:00
Ryan Barrett
44763ca8ab
ATProto firehose: more tests
...
for #978
2024-05-08 11:18:50 -07:00
Ryan Barrett
e214bc0a43
ATProto firehose: more tests
...
for #978
2024-05-08 10:39:03 -07:00
Ryan Barrett
7ecf3dc381
ATProto firehose: start tests
...
for #978
2024-05-07 17:36:34 -07:00