Wykres commitów

33 Commity (2085b131a1147deb3be7c500f2a40c146262bb92)

Autor SHA1 Wiadomość Data
Ryan Barrett 95e46c5ebb
Revert "cache outbound HTTP request responses, locally to each inbound request"
This reverts commit 30debfc8fa.

seemed promising, but broke in production. Saw a lot of `IncompleteRead`s on both GETs and POSTs. Rolled back for now.

```
('Connection broken: IncompleteRead(9172 bytes read, -4586 more expected)', IncompleteRead(9172 bytes read, -4586 more expected))
...
File "oauth_dropins/webutil/util.py", line 1673, in call
  resp = getattr((session or requests), fn)(url, *args, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "requests_cache/session.py", line 102, in get
  return self.request('GET', url, params=params, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "requests_cache/session.py", line 158, in request
  return super().request(method, url, *args, headers=headers, **kwargs)  # type: ignore
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "requests/sessions.py", line 589, in request
  resp = self.send(prep, **send_kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "requests_cache/session.py", line 205, in send
  response = self._send_and_cache(request, actions, cached_response, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "requests_cache/session.py", line 233, in _send_and_cache
  self.cache.save_response(response, actions.cache_key, actions.expires)
File "requests_cache/backends/base.py", line 89, in save_response
  cached_response = CachedResponse.from_response(response, expires=expires)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "requests_cache/models/response.py", line 102, in from_response
  obj.raw = CachedHTTPResponse.from_response(response)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "requests_cache/models/raw_response.py", line 69, in from_response
  _ = response.content  # This property reads, decodes, and stores response content
      ^^^^^^^^^^^^^^^^
File "requests/models.py", line 899, in content
  self._content = b"".join(self.iter_content(CONTENT_CHUNK_SIZE)) or b""
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "requests/models.py", line 818, in generate
  raise ChunkedEncodingError(e)
```
2024-03-11 15:31:30 -07:00
Ryan Barrett 30debfc8fa
cache outbound HTTP request responses, locally to each inbound request
using requests-cache. pretty straightforward! for #588
2024-03-08 12:34:49 -08:00
Ryan Barrett fcef6c21ab
Revert "heuristic: assume no AP actor id is the root path on its host"
This reverts commit b7e890b4bb.

it was a nice idea, but we're now hitting AP actor ids that are the root path on their host. this is evidently how microblog.pub does AP; 0x3b0b's site https://bw3.dev/ is an example. general info on https://docs.microblog.pub/user_guide.html#activitypub but not this specifically.

fixes https://console.cloud.google.com/errors/detail/CLSnttKfy4v90wE;time=P30D?project=bridgy-federated
2024-01-26 12:59:54 -08:00
Ryan Barrett 7941b632ca
Object._pre_put_hook: require that protocol owns id 2024-01-12 19:53:01 -08:00
Ryan Barrett 178b5ccde3
/convert: handle missing protocols
fixes https://console.cloud.google.com/errors/detail/CLHg5u_DuYGWaw;time=P30D?project=bridgy-federated
2024-01-08 07:53:18 -08:00
Ryan Barrett 199fb65dd6
ids.translate_object_id: handle Web.ap_subdomain
for https://github.com/snarfed/bridgy-fed/discussions/768
2023-12-24 10:20:04 -08:00
Ryan Barrett 2fafd83344
ids.translate_user_id: handle Web.ap_subdomain
for https://github.com/snarfed/bridgy-fed/discussions/768
2023-12-24 10:04:01 -08:00
Ryan Barrett d79e2badf4
fix tests for dc3b23f80b 2023-11-26 21:39:31 -08:00
Ryan Barrett dc3b23f80b
Web.fetch: ignore uid if set; we use URL as id
props and thanks to @fluffy-critter for being my unintentional edge case QA department 😆🙏

fixes https://console.cloud.google.com/errors/detail/CKLuk-v4x8X0NQ;time=P30D?project=bridgy-federated
2023-11-26 16:31:29 -08:00
Ryan Barrett 1591dfb641
pass user to Protocol.send/convert instead of using g.user
for #690
2023-11-25 20:07:14 -08:00
Ryan Barrett 4d095fa3d9
misc cleanup
* remove obsolete TODOs, commented out code
* remove obsolete circular imports of per-protocol modules
* minimize Object put in Protocol.load
* remove duplicated Protocol.load tests in test_activitypub
* re-enable rest of ActivityPubUtilsTest.test_postprocess_as2_idempotent
* drop default cls=Web in TestCase.make_user
2023-11-15 14:23:08 -08:00
Ryan Barrett d2dee8752f
/convert/: if dest protocol is AP, don't assume source protocol is Web 2023-11-08 11:28:17 -08:00
Ryan Barrett 9327565095
/convert/: drop req't that ids are web URLs
for abstracting across protocols. also other minor tweaks.
2023-11-08 11:28:17 -08:00
Ryan Barrett 4b7926b3a8
/convert/: drop protocol regexps in URL path routes 2023-11-08 11:28:17 -08:00
Ryan Barrett aa5c6a396e
ActivityPub: translate ids in convert 2023-11-03 15:11:21 -07:00
Ryan Barrett b32045d494
postprocess_as2: duplicate content into contentMap.en
for #681. this suppresses Mastodon's Translate link. longer term "right" fix is to actually detect the source's language, if available, and use it instead.
2023-10-20 13:37:54 -07:00
Ryan Barrett d2e6dd18c2
/convert: on localhost, serve two-protocol paths instead of redirecting 2023-10-12 16:22:56 -07:00
Ryan Barrett a823dd1d65
use protocol subdomains in AP inbox
...and other misc protocol subdomain fixes
2023-09-27 13:55:16 -07:00
Ryan Barrett 2f308b43e2
/convert: use generic Protocol for fed subdomain
not entirely sure about this one, will see

for #600
2023-07-26 09:02:23 -07:00
Ryan Barrett a1d4806e2a
fix /convert/ redirect on fed.brid.gy to preserve URL-encoding
for #600
2023-07-26 09:02:23 -07:00
Ryan Barrett 850e8a1dc8
switch convert URLs from ^^ back to #
for #581. the ^^s didn't help with WordPress, which was the original motivation, and they're confusing everyone else.
2023-07-24 14:31:07 -07:00
Ryan Barrett ab1c28ee4d
noop, lint fixes from flake8
remaining:

$ flake8 --extend-ignore=E501 *.py tests/*.py
"pyflakes" failed during execution due to "'FlakesChecker' object has no attribute 'NAMEDEXPR'"
Run flake8 with greater verbosity to see more details
activitypub.py:15:1: F401 'oauth_dropins.webutil.util.json_loads' imported but unused
activitypub.py:36:1: F401 'web' imported but unused
activitypub.py:48:1: E302 expected 2 blank lines, found 1
activitypub.py:51:9: F811 redefinition of unused 'web' from line 36
app.py:6:1: F401 'flask_app.app' imported but unused
app.py:9:1: F401 'activitypub' imported but unused
app.py:9:1: F401 'convert' imported but unused
app.py:9:1: F401 'follow' imported but unused
app.py:9:1: F401 'pages' imported but unused
app.py:9:1: F401 'redirect' imported but unused
app.py:9:1: F401 'superfeedr' imported but unused
app.py:9:1: F401 'ui' imported but unused
app.py:9:1: F401 'webfinger' imported but unused
app.py:9:1: F401 'web' imported but unused
app.py:9:1: F401 'xrpc_actor' imported but unused
app.py:9:1: F401 'xrpc_feed' imported but unused
app.py:9:1: F401 'xrpc_graph' imported but unused
app.py:9:19: E401 multiple imports on one line
models.py:19:1: F401 'oauth_dropins.webutil.util.json_loads' imported but unused
models.py:364:31: E114 indentation is not a multiple of four (comment)
models.py:364:31: E116 unexpected indentation (comment)
protocol.py:17:1: F401 'oauth_dropins.webutil.util.json_loads' imported but unused
redirect.py:26:1: F401 'oauth_dropins.webutil.util.json_loads' imported but unused
web.py:18:1: F401 'oauth_dropins.webutil.util.json_loads' imported but unused
webfinger.py:13:1: F401 'oauth_dropins.webutil.util.json_loads' imported but unused
webfinger.py:110:13: E122 continuation line missing indentation or outdented
webfinger.py:111:13: E122 continuation line missing indentation or outdented
webfinger.py:131:13: E122 continuation line missing indentation or outdented
webfinger.py:132:13: E122 continuation line missing indentation or outdented
webfinger.py:133:13: E122 continuation line missing indentation or outdented
webfinger.py:134:13: E122 continuation line missing indentation or outdented
tests/__init__.py:2:1: F401 'oauth_dropins.webutil.tests' imported but unused
tests/test_follow.py:11:1: F401 'oauth_dropins.webutil.util.json_dumps' imported but unused
tests/test_follow.py:14:1: F401 '.testutil.Fake' imported but unused
tests/test_models.py:156:15: E122 continuation line missing indentation or outdented
tests/test_models.py:157:15: E122 continuation line missing indentation or outdented
tests/test_models.py:158:11: E122 continuation line missing indentation or outdented
tests/test_web.py:12:1: F401 'oauth_dropins.webutil.util.json_dumps' imported but unused
tests/test_web.py:17:1: F401 '.testutil' imported but unused
tests/test_web.py:1513:13: E128 continuation line under-indented for visual indent
tests/test_web.py:1514:9: E124 closing bracket does not match visual indentation
tests/testutil.py:106:1: E402 module level import not at top of file
tests/testutil.py:107:1: E402 module level import not at top of file
tests/testutil.py:108:1: E402 module level import not at top of file
tests/testutil.py:109:1: E402 module level import not at top of file
tests/testutil.py:110:1: E402 module level import not at top of file
tests/testutil.py:301:24: E203 whitespace before ':'
tests/testutil.py:301:25: E701 multiple statements on one line (colon)
tests/testutil.py:301:25: E231 missing whitespace after ':'
2023-06-20 11:29:31 -07:00
Ryan Barrett 501406ab7a
bring back the /convert/[src]/[dest]/[url] routes, forgot we'd used them in prod
thanks for reporting @gRegorLove!
2023-06-17 13:48:44 -07:00
Ryan Barrett bbcb8de44a
noop test refactoring: finally unify request context push/pop into testutil 2023-06-15 15:09:03 -07:00
Ryan Barrett c2f5cbd35e
migrate source Protocol in /convert/ endpoints from path to subdomain
for #512, #548
2023-06-14 15:02:59 -07:00
Ryan Barrett cb1f80832f
AP users: add Protocol.ABBREV, Protocol.OTHER_LABELS
for #512
2023-06-11 08:14:17 -07:00
Ryan Barrett 0f19654eb2
/convert/: require user for converting to AS2
since activitypub.postprocess_as2 currently needs g.user.
2023-06-09 09:39:13 -07:00
Ryan Barrett 29f1a1d4ac
AP users: start parameterizing URLs, including UI pages and AP endpoints
user pages now have /[protocol]/ prefix, AP endpoints have /ap/ prefix. for #512
2023-05-30 16:38:58 -07:00
Ryan Barrett 47b04f5574
rename Web.LABEL from 'webmention' to 'web' 2023-05-30 12:15:36 -07:00
Ryan Barrett eaa4e5333a
start to make User subclasses for each protocol
#512
2023-05-26 16:07:36 -07:00
Ryan Barrett a3df7d6d30
make /render redirect to /convert/...
#512
2023-05-24 16:32:00 -07:00
Ryan Barrett 6d976854dc
add /render logic to /convert/... 2023-05-24 16:00:41 -07:00
Ryan Barrett df35ce16cc
AP users: add convert.py and /convert/... endpoint
#512
2023-05-24 15:18:31 -07:00