Wykres commitów

54 Commity (main)

Autor SHA1 Wiadomość Data
Ryan Barrett 2ec22de09f
abstract redirect.py to be multi-protocol
...mostly. creating the underlying user opportunistically is still Web-only.
2024-04-14 18:26:34 -07:00
Ryan Barrett 95e46c5ebb
Revert "cache outbound HTTP request responses, locally to each inbound request"
This reverts commit 30debfc8fa.

seemed promising, but broke in production. Saw a lot of `IncompleteRead`s on both GETs and POSTs. Rolled back for now.

```
('Connection broken: IncompleteRead(9172 bytes read, -4586 more expected)', IncompleteRead(9172 bytes read, -4586 more expected))
...
File "oauth_dropins/webutil/util.py", line 1673, in call
  resp = getattr((session or requests), fn)(url, *args, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "requests_cache/session.py", line 102, in get
  return self.request('GET', url, params=params, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "requests_cache/session.py", line 158, in request
  return super().request(method, url, *args, headers=headers, **kwargs)  # type: ignore
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "requests/sessions.py", line 589, in request
  resp = self.send(prep, **send_kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "requests_cache/session.py", line 205, in send
  response = self._send_and_cache(request, actions, cached_response, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "requests_cache/session.py", line 233, in _send_and_cache
  self.cache.save_response(response, actions.cache_key, actions.expires)
File "requests_cache/backends/base.py", line 89, in save_response
  cached_response = CachedResponse.from_response(response, expires=expires)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "requests_cache/models/response.py", line 102, in from_response
  obj.raw = CachedHTTPResponse.from_response(response)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "requests_cache/models/raw_response.py", line 69, in from_response
  _ = response.content  # This property reads, decodes, and stores response content
      ^^^^^^^^^^^^^^^^
File "requests/models.py", line 899, in content
  self._content = b"".join(self.iter_content(CONTENT_CHUNK_SIZE)) or b""
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "requests/models.py", line 818, in generate
  raise ChunkedEncodingError(e)
```
2024-03-11 15:31:30 -07:00
Ryan Barrett 30debfc8fa
cache outbound HTTP request responses, locally to each inbound request
using requests-cache. pretty straightforward! for #588
2024-03-08 12:34:49 -08:00
Ryan Barrett 9489204d64
AP: add profile to Content-Type: application/ld+json
for #895
2024-02-27 11:38:00 -08:00
Ryan Barrett f9d94522d0
/r/: return 404 for opted out users' URLs 2024-01-28 08:54:35 -08:00
Ryan Barrett fcef6c21ab
Revert "heuristic: assume no AP actor id is the root path on its host"
This reverts commit b7e890b4bb.

it was a nice idea, but we're now hitting AP actor ids that are the root path on their host. this is evidently how microblog.pub does AP; 0x3b0b's site https://bw3.dev/ is an example. general info on https://docs.microblog.pub/user_guide.html#activitypub but not this specifically.

fixes https://console.cloud.google.com/errors/detail/CLSnttKfy4v90wE;time=P30D?project=bridgy-federated
2024-01-26 12:59:54 -08:00
Ryan Barrett 95e699f5b5
/r/: serve Vary: Accept header 2023-12-31 09:21:04 -08:00
Ryan Barrett 4d095fa3d9
misc cleanup
* remove obsolete TODOs, commented out code
* remove obsolete circular imports of per-protocol modules
* minimize Object put in Protocol.load
* remove duplicated Protocol.load tests in test_activitypub
* re-enable rest of ActivityPubUtilsTest.test_postprocess_as2_idempotent
* drop default cls=Web in TestCase.make_user
2023-11-15 14:23:08 -08:00
Ryan Barrett aa5c6a396e
ActivityPub: translate ids in convert 2023-11-03 15:11:21 -07:00
Ryan Barrett 6a867ca2c4
update for granary.as2's new PropertyValue attachment handling
snarfed/granary@20d6c3f065
2023-10-24 16:02:16 -07:00
Ryan Barrett d7a51916ad
/r/ redirect: handle bad URLs
fixes https://console.cloud.google.com/errors/detail/CJ30oPbsoqmI7QE;time=P30D?project=bridgy-federated
2023-10-18 11:03:22 -07:00
Ryan Barrett 15a325c928
AP: compact outgoing actor, author, and attributedTo fields to string ids
hope this improves compatibility with many other AP implementations that choke on objects instead of ids. fixes #658
2023-10-11 16:17:43 -07:00
Ryan Barrett ff2be9e1ea
/r/ redirects: add bsky.app to domain allowlist 2023-09-23 13:50:19 -07:00
Ryan Barrett 6cddd95686
AP: always populate inbox and outbox in actors
required by AP! https://www.w3.org/TR/activitypub/#actor-objects

for #614
2023-08-19 15:12:46 -07:00
Ryan Barrett 42bb4a765d
misc test fixes 2023-07-11 13:25:31 -07:00
Ryan Barrett 8748e9a691
update tests to match new profile link HTML values
corresponds to snarfed/granary@f608f3e63b
2023-06-22 22:32:50 -07:00
Ryan Barrett ab1c28ee4d
noop, lint fixes from flake8
remaining:

$ flake8 --extend-ignore=E501 *.py tests/*.py
"pyflakes" failed during execution due to "'FlakesChecker' object has no attribute 'NAMEDEXPR'"
Run flake8 with greater verbosity to see more details
activitypub.py:15:1: F401 'oauth_dropins.webutil.util.json_loads' imported but unused
activitypub.py:36:1: F401 'web' imported but unused
activitypub.py:48:1: E302 expected 2 blank lines, found 1
activitypub.py:51:9: F811 redefinition of unused 'web' from line 36
app.py:6:1: F401 'flask_app.app' imported but unused
app.py:9:1: F401 'activitypub' imported but unused
app.py:9:1: F401 'convert' imported but unused
app.py:9:1: F401 'follow' imported but unused
app.py:9:1: F401 'pages' imported but unused
app.py:9:1: F401 'redirect' imported but unused
app.py:9:1: F401 'superfeedr' imported but unused
app.py:9:1: F401 'ui' imported but unused
app.py:9:1: F401 'webfinger' imported but unused
app.py:9:1: F401 'web' imported but unused
app.py:9:1: F401 'xrpc_actor' imported but unused
app.py:9:1: F401 'xrpc_feed' imported but unused
app.py:9:1: F401 'xrpc_graph' imported but unused
app.py:9:19: E401 multiple imports on one line
models.py:19:1: F401 'oauth_dropins.webutil.util.json_loads' imported but unused
models.py:364:31: E114 indentation is not a multiple of four (comment)
models.py:364:31: E116 unexpected indentation (comment)
protocol.py:17:1: F401 'oauth_dropins.webutil.util.json_loads' imported but unused
redirect.py:26:1: F401 'oauth_dropins.webutil.util.json_loads' imported but unused
web.py:18:1: F401 'oauth_dropins.webutil.util.json_loads' imported but unused
webfinger.py:13:1: F401 'oauth_dropins.webutil.util.json_loads' imported but unused
webfinger.py:110:13: E122 continuation line missing indentation or outdented
webfinger.py:111:13: E122 continuation line missing indentation or outdented
webfinger.py:131:13: E122 continuation line missing indentation or outdented
webfinger.py:132:13: E122 continuation line missing indentation or outdented
webfinger.py:133:13: E122 continuation line missing indentation or outdented
webfinger.py:134:13: E122 continuation line missing indentation or outdented
tests/__init__.py:2:1: F401 'oauth_dropins.webutil.tests' imported but unused
tests/test_follow.py:11:1: F401 'oauth_dropins.webutil.util.json_dumps' imported but unused
tests/test_follow.py:14:1: F401 '.testutil.Fake' imported but unused
tests/test_models.py:156:15: E122 continuation line missing indentation or outdented
tests/test_models.py:157:15: E122 continuation line missing indentation or outdented
tests/test_models.py:158:11: E122 continuation line missing indentation or outdented
tests/test_web.py:12:1: F401 'oauth_dropins.webutil.util.json_dumps' imported but unused
tests/test_web.py:17:1: F401 '.testutil' imported but unused
tests/test_web.py:1513:13: E128 continuation line under-indented for visual indent
tests/test_web.py:1514:9: E124 closing bracket does not match visual indentation
tests/testutil.py:106:1: E402 module level import not at top of file
tests/testutil.py:107:1: E402 module level import not at top of file
tests/testutil.py:108:1: E402 module level import not at top of file
tests/testutil.py:109:1: E402 module level import not at top of file
tests/testutil.py:110:1: E402 module level import not at top of file
tests/testutil.py:301:24: E203 whitespace before ':'
tests/testutil.py:301:25: E701 multiple statements on one line (colon)
tests/testutil.py:301:25: E231 missing whitespace after ':'
2023-06-20 11:29:31 -07:00
Ryan Barrett d3c4415cd3
migrate User.actor_as2 to obj key pointing to Object
for #547
2023-06-15 21:23:31 -07:00
Ryan Barrett bbcb8de44a
noop test refactoring: finally unify request context push/pop into testutil 2023-06-15 15:09:03 -07:00
Ryan Barrett 51c2773594
AP users: /r/: switch external user to indirect user
for #512
2023-06-04 20:58:21 -07:00
Ryan Barrett 29f1a1d4ac
AP users: start parameterizing URLs, including UI pages and AP endpoints
user pages now have /[protocol]/ prefix, AP endpoints have /ap/ prefix. for #512
2023-05-30 16:38:58 -07:00
Ryan Barrett 624355d85a
AP users: add new User.direct property
direct users signed up or otherwise interacted with BF explicitly, indirect users didn't
2023-05-29 19:37:41 -07:00
Ryan Barrett c8b2103e13
rename Webmention class => Web, webmention.py => web.py 2023-05-26 17:40:29 -07:00
Ryan Barrett 30659a78f7
test_redirect: minor noop test data refactoring 2023-05-24 15:18:16 -07:00
Ryan Barrett e34db7e974
AP users: /r/: don't check for backlink 2023-05-23 09:35:35 -07:00
Ryan Barrett f933c7e237
AP users: serve AS2 for external homepage URLs
#512
2023-05-22 23:17:42 -07:00
Ryan Barrett 4f232ac732
AP users: serve AS2 for external URLs
#512
2023-05-22 15:27:43 -07:00
Ryan Barrett d2ab48b23e
switch circular imports to runtime imports; split out flask_app.py from app.py
runtime imports are just as bad, but...meh. eventually I'll untangle them for real. #486
2023-04-18 17:18:17 -07:00
Ryan Barrett d433dd3abe
wm => AP: switch actors in outgoing activities from objects to string ids
mostly. also inlines single-element attributedTo lists to just the element, without the list.
2023-03-19 09:34:42 -07:00
Ryan Barrett f6e5727602
test_webmention: rename test data hostnames to clarify web vs fediverse 2023-03-19 09:34:42 -07:00
Ryan Barrett 5486d2a2fa
update test_redirect for snarfed/granary@ff1ac56a5b 2023-03-13 17:22:08 -07:00
Ryan Barrett 106bbe623b
tests optimization: reuse same RSA key across Users
speeds up tests from ~22s to ~6s (!) on my laptop. for #398
2023-03-10 15:13:45 -08:00
Ryan Barrett fd27dabe61
switch JSON properties to custom JSONProperty that works in web console UI
https://github.com/googleapis/python-ndb/issues/874#issuecomment-1442753255
2023-02-24 07:25:29 -06:00
Ryan Barrett 91a60c7e67
switch Object.as1 to be a ComputedProperty 2023-02-23 21:17:26 -06:00
Ryan Barrett 7e2fbd1ed0
make AP Deletes mark stored Objects deleted
for #409
2023-02-11 21:47:55 -08:00
Ryan Barrett 6d2668e64d
serve /r/ URLs as AS2 from the datastore, don't fetch and convert on the fly
for #392, #378
2023-02-11 18:35:34 -08:00
Ryan Barrett 1627ff667a
translate webmention from home page into profile (actor) Update
for #376, #331
2023-01-23 16:09:25 -08:00
Ryan Barrett aa0dcbb222
noop: drop content type constants in common, use granary's instead 2023-01-06 21:01:33 -08:00
Ryan Barrett 5f082349d5
/r/ handler: include Accept header in flask cache key
requires pending bug fix https://github.com/pallets-eco/flask-caching/pull/431
2022-12-26 09:52:08 -08:00
Ryan Barrett 8fd49dc882
drop non-public Create activities
fixes #289
2022-11-16 16:37:50 -08:00
Ryan Barrett 50956c02b0
noop, rename Domain => User 2022-11-15 22:00:28 -08:00
Ryan Barrett 7966650178
AP: put Public in to instead of cc to make Mastodon posts public, not unlisted
fixes #277
2022-11-13 21:46:26 -08:00
Ryan Barrett 4f3dc03a3e
rename MagicKey => Domain, Response => Activity
the Python classes, but not (yet) the kinds in the datastore. maybe eventually.
2022-11-12 15:27:59 -08:00
Ryan Barrett c6f0c417a0
/r: handle trailing garbage chars
fixes https://console.cloud.google.com/errors/CO-fpZfMq_6ktgE
2021-09-01 19:48:37 -07:00
Ryan Barrett f4aefff6f4
minor noop test refactoring 2021-08-18 08:04:14 -07:00
Ryan Barrett 5b8699bec1
flask: port /r/, start to port common 2021-07-07 21:02:13 -07:00
Ryan Barrett a4c5a8e4c5
redirect: handle single /s that got collapsed by the browser (etc) 2021-06-28 22:52:04 -07:00
Ryan Barrett 1f977a7d5c
stop /r/ from being an open redirect by checking that we've seen the domain 2021-03-07 07:36:34 -08:00
Ryan Barrett 6b4089efe4
/r/ redirect endpoint: use HTTP 301, not 302 2021-03-06 21:47:08 -08:00
Ryan Barrett 02d36b3b1f
migrate to the app engine standard python 3 runtime!
https://cloud.google.com/appengine/docs/standard/python3/python-differences
https://cloud.google.com/appengine/docs/standard/python/migrate-to-python3/
2019-12-25 22:26:26 -08:00