Wykres commitów

53 Commity (245ef444f56ae5de7b524715220d20d7a2fe03f6)

Autor SHA1 Wiadomość Data
Ryan Barrett 95e46c5ebb
Revert "cache outbound HTTP request responses, locally to each inbound request"
This reverts commit 30debfc8fa.

seemed promising, but broke in production. Saw a lot of `IncompleteRead`s on both GETs and POSTs. Rolled back for now.

```
('Connection broken: IncompleteRead(9172 bytes read, -4586 more expected)', IncompleteRead(9172 bytes read, -4586 more expected))
...
File "oauth_dropins/webutil/util.py", line 1673, in call
  resp = getattr((session or requests), fn)(url, *args, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "requests_cache/session.py", line 102, in get
  return self.request('GET', url, params=params, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "requests_cache/session.py", line 158, in request
  return super().request(method, url, *args, headers=headers, **kwargs)  # type: ignore
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "requests/sessions.py", line 589, in request
  resp = self.send(prep, **send_kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "requests_cache/session.py", line 205, in send
  response = self._send_and_cache(request, actions, cached_response, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "requests_cache/session.py", line 233, in _send_and_cache
  self.cache.save_response(response, actions.cache_key, actions.expires)
File "requests_cache/backends/base.py", line 89, in save_response
  cached_response = CachedResponse.from_response(response, expires=expires)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "requests_cache/models/response.py", line 102, in from_response
  obj.raw = CachedHTTPResponse.from_response(response)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "requests_cache/models/raw_response.py", line 69, in from_response
  _ = response.content  # This property reads, decodes, and stores response content
      ^^^^^^^^^^^^^^^^
File "requests/models.py", line 899, in content
  self._content = b"".join(self.iter_content(CONTENT_CHUNK_SIZE)) or b""
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "requests/models.py", line 818, in generate
  raise ChunkedEncodingError(e)
```
2024-03-11 15:31:30 -07:00
Ryan Barrett 30debfc8fa
cache outbound HTTP request responses, locally to each inbound request
using requests-cache. pretty straightforward! for #588
2024-03-08 12:34:49 -08:00
Ryan Barrett 9489204d64
AP: add profile to Content-Type: application/ld+json
for #895
2024-02-27 11:38:00 -08:00
Ryan Barrett f9d94522d0
/r/: return 404 for opted out users' URLs 2024-01-28 08:54:35 -08:00
Ryan Barrett fcef6c21ab
Revert "heuristic: assume no AP actor id is the root path on its host"
This reverts commit b7e890b4bb.

it was a nice idea, but we're now hitting AP actor ids that are the root path on their host. this is evidently how microblog.pub does AP; 0x3b0b's site https://bw3.dev/ is an example. general info on https://docs.microblog.pub/user_guide.html#activitypub but not this specifically.

fixes https://console.cloud.google.com/errors/detail/CLSnttKfy4v90wE;time=P30D?project=bridgy-federated
2024-01-26 12:59:54 -08:00
Ryan Barrett 95e699f5b5
/r/: serve Vary: Accept header 2023-12-31 09:21:04 -08:00
Ryan Barrett 4d095fa3d9
misc cleanup
* remove obsolete TODOs, commented out code
* remove obsolete circular imports of per-protocol modules
* minimize Object put in Protocol.load
* remove duplicated Protocol.load tests in test_activitypub
* re-enable rest of ActivityPubUtilsTest.test_postprocess_as2_idempotent
* drop default cls=Web in TestCase.make_user
2023-11-15 14:23:08 -08:00
Ryan Barrett aa5c6a396e
ActivityPub: translate ids in convert 2023-11-03 15:11:21 -07:00
Ryan Barrett 6a867ca2c4
update for granary.as2's new PropertyValue attachment handling
snarfed/granary@20d6c3f065
2023-10-24 16:02:16 -07:00
Ryan Barrett d7a51916ad
/r/ redirect: handle bad URLs
fixes https://console.cloud.google.com/errors/detail/CJ30oPbsoqmI7QE;time=P30D?project=bridgy-federated
2023-10-18 11:03:22 -07:00
Ryan Barrett 15a325c928
AP: compact outgoing actor, author, and attributedTo fields to string ids
hope this improves compatibility with many other AP implementations that choke on objects instead of ids. fixes #658
2023-10-11 16:17:43 -07:00
Ryan Barrett ff2be9e1ea
/r/ redirects: add bsky.app to domain allowlist 2023-09-23 13:50:19 -07:00
Ryan Barrett 6cddd95686
AP: always populate inbox and outbox in actors
required by AP! https://www.w3.org/TR/activitypub/#actor-objects

for #614
2023-08-19 15:12:46 -07:00
Ryan Barrett 42bb4a765d
misc test fixes 2023-07-11 13:25:31 -07:00
Ryan Barrett 8748e9a691
update tests to match new profile link HTML values
corresponds to snarfed/granary@f608f3e63b
2023-06-22 22:32:50 -07:00
Ryan Barrett ab1c28ee4d
noop, lint fixes from flake8
remaining:

$ flake8 --extend-ignore=E501 *.py tests/*.py
"pyflakes" failed during execution due to "'FlakesChecker' object has no attribute 'NAMEDEXPR'"
Run flake8 with greater verbosity to see more details
activitypub.py:15:1: F401 'oauth_dropins.webutil.util.json_loads' imported but unused
activitypub.py:36:1: F401 'web' imported but unused
activitypub.py:48:1: E302 expected 2 blank lines, found 1
activitypub.py:51:9: F811 redefinition of unused 'web' from line 36
app.py:6:1: F401 'flask_app.app' imported but unused
app.py:9:1: F401 'activitypub' imported but unused
app.py:9:1: F401 'convert' imported but unused
app.py:9:1: F401 'follow' imported but unused
app.py:9:1: F401 'pages' imported but unused
app.py:9:1: F401 'redirect' imported but unused
app.py:9:1: F401 'superfeedr' imported but unused
app.py:9:1: F401 'ui' imported but unused
app.py:9:1: F401 'webfinger' imported but unused
app.py:9:1: F401 'web' imported but unused
app.py:9:1: F401 'xrpc_actor' imported but unused
app.py:9:1: F401 'xrpc_feed' imported but unused
app.py:9:1: F401 'xrpc_graph' imported but unused
app.py:9:19: E401 multiple imports on one line
models.py:19:1: F401 'oauth_dropins.webutil.util.json_loads' imported but unused
models.py:364:31: E114 indentation is not a multiple of four (comment)
models.py:364:31: E116 unexpected indentation (comment)
protocol.py:17:1: F401 'oauth_dropins.webutil.util.json_loads' imported but unused
redirect.py:26:1: F401 'oauth_dropins.webutil.util.json_loads' imported but unused
web.py:18:1: F401 'oauth_dropins.webutil.util.json_loads' imported but unused
webfinger.py:13:1: F401 'oauth_dropins.webutil.util.json_loads' imported but unused
webfinger.py:110:13: E122 continuation line missing indentation or outdented
webfinger.py:111:13: E122 continuation line missing indentation or outdented
webfinger.py:131:13: E122 continuation line missing indentation or outdented
webfinger.py:132:13: E122 continuation line missing indentation or outdented
webfinger.py:133:13: E122 continuation line missing indentation or outdented
webfinger.py:134:13: E122 continuation line missing indentation or outdented
tests/__init__.py:2:1: F401 'oauth_dropins.webutil.tests' imported but unused
tests/test_follow.py:11:1: F401 'oauth_dropins.webutil.util.json_dumps' imported but unused
tests/test_follow.py:14:1: F401 '.testutil.Fake' imported but unused
tests/test_models.py:156:15: E122 continuation line missing indentation or outdented
tests/test_models.py:157:15: E122 continuation line missing indentation or outdented
tests/test_models.py:158:11: E122 continuation line missing indentation or outdented
tests/test_web.py:12:1: F401 'oauth_dropins.webutil.util.json_dumps' imported but unused
tests/test_web.py:17:1: F401 '.testutil' imported but unused
tests/test_web.py:1513:13: E128 continuation line under-indented for visual indent
tests/test_web.py:1514:9: E124 closing bracket does not match visual indentation
tests/testutil.py:106:1: E402 module level import not at top of file
tests/testutil.py:107:1: E402 module level import not at top of file
tests/testutil.py:108:1: E402 module level import not at top of file
tests/testutil.py:109:1: E402 module level import not at top of file
tests/testutil.py:110:1: E402 module level import not at top of file
tests/testutil.py:301:24: E203 whitespace before ':'
tests/testutil.py:301:25: E701 multiple statements on one line (colon)
tests/testutil.py:301:25: E231 missing whitespace after ':'
2023-06-20 11:29:31 -07:00
Ryan Barrett d3c4415cd3
migrate User.actor_as2 to obj key pointing to Object
for #547
2023-06-15 21:23:31 -07:00
Ryan Barrett bbcb8de44a
noop test refactoring: finally unify request context push/pop into testutil 2023-06-15 15:09:03 -07:00
Ryan Barrett 51c2773594
AP users: /r/: switch external user to indirect user
for #512
2023-06-04 20:58:21 -07:00
Ryan Barrett 29f1a1d4ac
AP users: start parameterizing URLs, including UI pages and AP endpoints
user pages now have /[protocol]/ prefix, AP endpoints have /ap/ prefix. for #512
2023-05-30 16:38:58 -07:00
Ryan Barrett 624355d85a
AP users: add new User.direct property
direct users signed up or otherwise interacted with BF explicitly, indirect users didn't
2023-05-29 19:37:41 -07:00
Ryan Barrett c8b2103e13
rename Webmention class => Web, webmention.py => web.py 2023-05-26 17:40:29 -07:00
Ryan Barrett 30659a78f7
test_redirect: minor noop test data refactoring 2023-05-24 15:18:16 -07:00
Ryan Barrett e34db7e974
AP users: /r/: don't check for backlink 2023-05-23 09:35:35 -07:00
Ryan Barrett f933c7e237
AP users: serve AS2 for external homepage URLs
#512
2023-05-22 23:17:42 -07:00
Ryan Barrett 4f232ac732
AP users: serve AS2 for external URLs
#512
2023-05-22 15:27:43 -07:00
Ryan Barrett d2ab48b23e
switch circular imports to runtime imports; split out flask_app.py from app.py
runtime imports are just as bad, but...meh. eventually I'll untangle them for real. #486
2023-04-18 17:18:17 -07:00
Ryan Barrett d433dd3abe
wm => AP: switch actors in outgoing activities from objects to string ids
mostly. also inlines single-element attributedTo lists to just the element, without the list.
2023-03-19 09:34:42 -07:00
Ryan Barrett f6e5727602
test_webmention: rename test data hostnames to clarify web vs fediverse 2023-03-19 09:34:42 -07:00
Ryan Barrett 5486d2a2fa
update test_redirect for snarfed/granary@ff1ac56a5b 2023-03-13 17:22:08 -07:00
Ryan Barrett 106bbe623b
tests optimization: reuse same RSA key across Users
speeds up tests from ~22s to ~6s (!) on my laptop. for #398
2023-03-10 15:13:45 -08:00
Ryan Barrett fd27dabe61
switch JSON properties to custom JSONProperty that works in web console UI
https://github.com/googleapis/python-ndb/issues/874#issuecomment-1442753255
2023-02-24 07:25:29 -06:00
Ryan Barrett 91a60c7e67
switch Object.as1 to be a ComputedProperty 2023-02-23 21:17:26 -06:00
Ryan Barrett 7e2fbd1ed0
make AP Deletes mark stored Objects deleted
for #409
2023-02-11 21:47:55 -08:00
Ryan Barrett 6d2668e64d
serve /r/ URLs as AS2 from the datastore, don't fetch and convert on the fly
for #392, #378
2023-02-11 18:35:34 -08:00
Ryan Barrett 1627ff667a
translate webmention from home page into profile (actor) Update
for #376, #331
2023-01-23 16:09:25 -08:00
Ryan Barrett aa0dcbb222
noop: drop content type constants in common, use granary's instead 2023-01-06 21:01:33 -08:00
Ryan Barrett 5f082349d5
/r/ handler: include Accept header in flask cache key
requires pending bug fix https://github.com/pallets-eco/flask-caching/pull/431
2022-12-26 09:52:08 -08:00
Ryan Barrett 8fd49dc882
drop non-public Create activities
fixes #289
2022-11-16 16:37:50 -08:00
Ryan Barrett 50956c02b0
noop, rename Domain => User 2022-11-15 22:00:28 -08:00
Ryan Barrett 7966650178
AP: put Public in to instead of cc to make Mastodon posts public, not unlisted
fixes #277
2022-11-13 21:46:26 -08:00
Ryan Barrett 4f3dc03a3e
rename MagicKey => Domain, Response => Activity
the Python classes, but not (yet) the kinds in the datastore. maybe eventually.
2022-11-12 15:27:59 -08:00
Ryan Barrett c6f0c417a0
/r: handle trailing garbage chars
fixes https://console.cloud.google.com/errors/CO-fpZfMq_6ktgE
2021-09-01 19:48:37 -07:00
Ryan Barrett f4aefff6f4
minor noop test refactoring 2021-08-18 08:04:14 -07:00
Ryan Barrett 5b8699bec1
flask: port /r/, start to port common 2021-07-07 21:02:13 -07:00
Ryan Barrett a4c5a8e4c5
redirect: handle single /s that got collapsed by the browser (etc) 2021-06-28 22:52:04 -07:00
Ryan Barrett 1f977a7d5c
stop /r/ from being an open redirect by checking that we've seen the domain 2021-03-07 07:36:34 -08:00
Ryan Barrett 6b4089efe4
/r/ redirect endpoint: use HTTP 301, not 302 2021-03-06 21:47:08 -08:00
Ryan Barrett 02d36b3b1f
migrate to the app engine standard python 3 runtime!
https://cloud.google.com/appengine/docs/standard/python3/python-differences
https://cloud.google.com/appengine/docs/standard/python/migrate-to-python3/
2019-12-25 22:26:26 -08:00
Ryan Barrett 99a5c35ae3
switch from webutil.util.memcache_response to cache_response 2019-11-12 21:28:44 -08:00