b812dc8ec8
noop: tests: add tz kwarg to mocked util.now
2025-06-17 19:02:01 -07:00
dd9915c388
atproto_firehose.handle: send actor (profile) updates for #identity events
...
for #1074
2025-05-16 13:14:26 -07:00
cf5b62845c
test_atproto_firehose: add back handle stop-following test
...
re 7539fcffce
2025-05-16 12:39:37 -07:00
231c903215
update tests for 7539fcff
2025-05-16 10:38:21 -07:00
f98c82241d
noop: test_atproto_firehose: remove unused import
2025-03-07 08:09:56 -08:00
32a60c980c
updates for ndb context caching everything: Object.new/changed logic, tests
...
for #1149 , 18aa302da
2025-01-13 16:19:32 -08:00
8f087b792b
ATProto: temporarily stop handling stop-following
...
due to follow spammers, #1669
2025-01-07 08:12:58 -08:00
e2b6662776
atproto_firehose.handle: use stop-following AS1 verb for deletes of follows
...
for #1669
2025-01-07 08:02:11 -08:00
57fa6a1294
atproto_firehose load_dids: don't load disabled ATProto users
...
for #1669
2025-01-06 20:45:55 -08:00
d95724ff67
atproto_firehose subscriber: bug fix for #account/identity events for non-bridged repos
2025-01-05 19:54:41 -08:00
aa4f9e5bf8
atproto_firehose: handle #account and #identity events
...
for #1201
2025-01-05 13:39:17 -08:00
ae0a7920a0
atproto_firehose: enqueue #account and #identity events
...
for #1201
2025-01-05 13:39:17 -08:00
20dc814c80
add 2m delay to receive tasks for deletes
...
for #1361
2024-11-28 21:39:30 -08:00
00046eb7c1
update tests for lexrpc Client now sending HTTP headers over websockets
...
re snarfed/lexrpc@4fb5543877
2024-11-03 11:34:50 -08:00
fa8149165b
receive task: fix delay logging
...
for #1378
2024-10-23 21:11:21 -07:00
aaf49e0f37
ATProto: switch from #tombstone to #account active=False for deleting accounts
...
...including reactivating with #account active=True. it works!
for #1130 , #1119
2024-10-22 13:49:18 -07:00
5b75fa43d5
noop: tests: rename ExplicitEnableFake => ExplicitFake, eefake => efake
2024-10-08 15:03:43 -07:00
788b37279a
pass Object data in receive task HTTP payload instead of through datastore
...
trying to cut down datastore load and costs. this switches receive tasks from storing and fetching Objects from the datastore to passing their properties - notably as2, our_as1, bsky, etc - in the HTTP request body, as serialized JSON inside form-encoded params.
for #1354 , #1149
2024-10-01 21:44:12 -07:00
ba9cee0d8c
switch integration tests from atproto_firehose.handle to subscribe
...
...so that they test subscribe's filtering logic
2024-09-23 15:33:21 -07:00
f83e3725ec
atproto_firehose.subscribe bug fix: allow follows of protocol bot users from anyone
...
oops! re https://bsky.app/profile/yamakyu.bsky.social/post/3l4rotguqe42h , #1329
2024-09-22 21:50:25 -07:00
776d2b91e0
ATProto firehose subscriber: narrow filtering
...
this drastically reduces the volume of inbound records that we enqueue receive tasks for, from 3-6/s to .2-.4/s. before, we were enqueueing more or less everything from a bridged ATProto user *and* everything that referenced a bridged non-ATProto user. now, it's more like the intersection.
for #1329 , #1149
2024-09-21 07:27:24 -07:00
ff010b11b9
noop: minor refactoring of test_atproto_firehose
2024-09-20 13:56:11 -07:00
c5809e4de9
atproto_firehose.subscribe: skip record types we don't support
...
for #1329
2024-09-13 11:10:51 -07:00
a8b4333759
atproto_firehose: add missing import
2024-09-10 12:42:49 -07:00
60c92e1465
upgrade App Engine app runtime to python 3.12
...
https://cloud.google.com/appengine/docs/standard/reference/app-yaml?tab=python#runtime
https://cloud.google.com/appengine/docs/flexible/python/runtime#recommended_gunicorn_configuration
2024-09-09 21:58:52 -07:00
9496957819
atproto_firehose: move CPU-bound filtering back onto main subscribe thread
...
effectively reverts 5b280dc235e7c6ced26a#1295
2024-09-06 18:45:40 -07:00
5b280dc235
atproto_firehose: move most logic from subscribe to handle
...
trying to offload more CPU from the firehose client. for #1266
2024-09-03 12:15:25 -07:00
43c94ad2e0
atproto_firehose.subscribe: handle commits on unsupported collections without record
...
fixes https://console.cloud.google.com/errors/detail/CJT6x7XMiNLobg;time=PT1H;locations=global?project=bridgy-federated
2024-08-30 13:29:39 -07:00
5dfb17428e
atproto_firehose: move storing cursor from handle to subscribe
...
...so that I can run multiple handle threads for scaling
2024-08-30 13:16:17 -07:00
c1dcc03ee3
atproto_firehose.handle: handle missing $type
...
fixes https://console.cloud.google.com/errors/detail/CLal7vX3yJ6-1QE;locations=global;time=P30D?project=bridgy-federated
2024-08-29 22:02:02 -07:00
bb9e2d1f95
atproto_firehose.handle: bug fix for encoding record with CID as JSON
2024-08-21 07:55:09 -07:00
755997e5f7
atproto_firehose: don't load DIDs for tombstoned AtpRepos
...
for #1193
2024-07-22 22:24:13 -07:00
13da9becf6
atproto_firehose.handle: map delete of activity to undo, not delete
...
for #1162
2024-07-06 15:26:20 -07:00
df75e97880
atproto_firehose.handle bug fix: deletes don't have records
...
for #1162
2024-07-06 14:47:14 -07:00
850d681f83
add Protocol.SUPPORTED_AS1_TYPES
2024-07-04 16:58:06 -07:00
441849a22a
Revert "Revert "start using an ndb in memory global cache in the frontends and router""
...
This reverts commit 0cbcf81ecc
2024-06-04 14:27:02 -07:00
24559c8f17
Protocol.receive: normalize ids before resolving
...
for #959 , and possibly others
2024-06-02 08:41:52 -07:00
0cbcf81ecc
Revert "start using an ndb in memory global cache in the frontends and router"
...
This reverts commit 7e77d7a463
2024-06-01 07:04:38 -07:00
7e77d7a463
start using an ndb in memory global cache in the frontends and router
...
lol yolo 982f8a70df/google/cloud/ndb/global_cache.py (L200-L202)
2024-05-31 13:08:25 -07:00
83211cddc2
testutil: noop refactoring, remove path arg in assert_task
2024-05-24 22:16:20 -07:00
f2be280016
atproto_firehose: load ATProto users by updated, not by created
...
fixes #1082
2024-05-24 18:40:22 -07:00
4cf0959080
atproto_firehose: oops, still need to skip commits from our bridged users
...
add a test, should have had one originally!
2024-05-22 22:08:25 -07:00
976653d5bb
atproto_firehose: fix tests for f6575cca4f
2024-05-19 13:57:42 -07:00
f6575cca4f
atproto_firehose.subscribe: skip bad commits
...
fixes #1061
2024-05-19 13:47:13 -07:00
8ee4c3b2ea
atproto_firehose.handle: switch dag_json from compact=True to dialect='atproto'
...
for AT Protocol's DAG-JSON variant with $link and $bytes: https://atproto.com/specs/data-model ; for https://github.com/snarfed/bridgy-fed/issues/1000
2024-05-15 21:00:42 -07:00
8eeb588d11
atproto_firehose: make its own GAE service, lots of other refactoring
...
optimize loading DIDs, move around threads and exception handling, etc.
for #978
2024-05-13 22:02:09 -07:00
6bc9269ad0
ATProto firehose: load bridged dids from AtpRepo instead of users
...
hopefully cuts down on datastore load a bit
for #978
2024-05-10 11:31:47 -07:00
741146eb8d
ATProto firehose: encode Object.bsky as compact DAG-JSON
...
fixes https://console.cloud.google.com/errors/detail/CIGJnbnCq7fm1gE;time=P30D?project=bridgy-federated and similar
2024-05-10 06:51:58 -07:00
989cd29860
ATProto firehose: bug fix for loading DIDs
...
fixes https://console.cloud.google.com/errors/detail/CKKTu6mM0sunXQ;time=P30D?project=bridgy-federated
2024-05-09 21:25:13 -07:00
d56f405c96
ATProto firehose: fix race conditions in loading DIDs
...
for #978
2024-05-09 15:39:18 -07:00
Ryan Barrett