Support Diaspora URI scheme

Add 'id' and 'target_id' to Diaspora entities. Refactor retrieve content fetcher to use the Diaspora URI scheme based ID.
merge-requests/130/head
Jason Robinson 2017-10-22 14:40:12 +03:00
rodzic e343369f5b
commit a65b040969
12 zmienionych plików z 243 dodań i 73 usunięć

Wyświetl plik

@ -6,27 +6,32 @@
* Added base entity `Share` which maps to a `DiasporaReshare` for the Diaspora protocol. ([related issue](https://github.com/jaywink/federation/issues/94))
The `Share` entity supports all the properties that a Diaspora reshare does. Additionally two other properties are supported: `raw_content` and `entity_type`. The former can be used for a "quoted share" case where the sharer adds their own note to the share. The latter can be used to reference the type of object that was shared, to help the receiver, if it is not sharing a `Post` entity. The value must be a base entity class name.
* Entities have two new properties: `id` and `target_id`.
Diaspora entity ID's are in the form of the [Diaspora URI scheme](https://diaspora.github.io/diaspora_federation/federation/diaspora_scheme.html), where it is possible to construct an ID from the entity. In the future, ActivityPub object ID's will be found in these properties.
* New high level fetcher function `federation.fetchers.retrieve_remote_content`. ([related issue](https://github.com/jaywink/federation/issues/103))
This function takes the following parameters:
* `entity_class` - Base entity class to fetch (for example `Post`).
* `id` - Object ID. Currently since only Diaspora is supported and the ID is expected to be in format `<guid>@<domain.tld>`.
* `id` - Object ID. For Diaspora, the only supported protocol at the moment, this is in the [Diaspora URI](https://diaspora.github.io/diaspora_federation/federation/diaspora_scheme.html) format.
* `sender_key_fetcher` - Optional function that takes a profile `handle` and returns a public key in `str` format. If this is not given, the public key will be fetched from the remote profile over the network.
The given ID will be fetched using the correct entity class specific remote endpoint, validated to be from the correct author against their public key and then an instance of the entity class will be constructed and returned.
The given ID will be fetched from the remote endpoint, validated to be from the correct author against their public key and then an instance of the entity class will be constructed and returned.
* New Diaspora protocol helper `federation.utils.diaspora.retrieve_and_parse_content`. See notes regarding the high level fetcher above.
* New Diaspora protocol helpers in `federation.utils.diaspora`:
* New Diaspora protocol helper `federation.utils.fetch_public_key`. Given a `handle` as a parameter, will fetch the remote profile and return the `public_key` from it.
* `retrieve_and_parse_content`. See notes regarding the high level fetcher above.
* `fetch_public_key`. Given a `handle` as a parameter, will fetch the remote profile and return the `public_key` from it.
* `parse_diaspora_uri`. Parses a Diaspora URI scheme string, returns either `None` if parsing fails or a `tuple` of `handle`, `entity_type` and `guid`.
### Changed
* Refactoring for Diaspora `MagicEnvelope` class.
The class init now also allows passing in parameters to construct and verify MagicEnvelope instances. The order of init parameters has not been changed, but they are now all optional. When creating a class instance, one should always pass in the necessary parameters depnding on whether the class instance will be used for building a payload or verifying an incoming payload. See class docstring for details.
* Diaspora procotol receive flow now uses the `MagicEnvelope` class to verify payloads.
* Diaspora procotol receive flow now uses the `MagicEnvelope` class to verify payloads. No functional changes regarding verification otherwise.
* Diaspora protocol receive flow now fetches the sender public key over the network if a `sender_key_fetcher` function is not passed in. Previously an error would be raised.

Wyświetl plik

@ -109,6 +109,7 @@ Diaspora
.. autofunction:: federation.utils.diaspora.fetch_public_key
.. autofunction:: federation.utils.diaspora.get_fetch_content_endpoint
.. autofunction:: federation.utils.diaspora.get_public_endpoint
.. autofunction:: federation.utils.diaspora.parse_diaspora_uri
.. autofunction:: federation.utils.diaspora.parse_profile_from_hcard
.. autofunction:: federation.utils.diaspora.retrieve_and_parse_content
.. autofunction:: federation.utils.diaspora.retrieve_and_parse_profile

Wyświetl plik

@ -30,6 +30,14 @@ class BaseEntity:
self.__class__.__name__, key
))
@property
def id(self):
"""Global network ID.
Future expansion: Convert later into an attribute which with ActivityPub will have the 'id' directly.
"""
return
def validate(self):
"""Do validation.
@ -120,6 +128,14 @@ class TargetGUIDMixin(BaseEntity):
super().__init__(*args, **kwargs)
self._required += ["target_guid"]
@property
def target_id(self):
"""Global network target ID.
Future expansion: convert to attribute when ActivityPub is supported.
"""
return
def validate_target_guid(self):
if len(self.target_guid) < 16:
raise ValueError("Target GUID must be at least 16 characters")

Wyświetl plik

@ -1,17 +1,58 @@
import importlib
from lxml import etree
from federation.entities.base import (
Comment, Post, Reaction, Relationship, Profile, Retraction, BaseEntity, Follow, Share)
Comment, Post, Reaction, Relationship, Profile, Retraction, BaseEntity, Follow, Share, Image,
)
from federation.entities.diaspora.utils import format_dt, struct_to_xml, get_base_attributes, add_element_to_doc
from federation.exceptions import SignatureVerificationError
from federation.protocols.diaspora.signatures import verify_relayable_signature, create_relayable_signature
from federation.utils.diaspora import retrieve_and_parse_profile
CLASS_TO_TAG_MAPPING = {
Comment: "comment",
Follow: "contact",
Image: "photo",
Post: "status_message",
Profile: "profile",
Reaction: "like",
Relationship: "request",
Retraction: "retraction",
Share: "reshare",
}
class DiasporaEntityMixin(BaseEntity):
# Normally outbound document is generated from entity. Store one here if at some point we already have a doc
outbound_doc = None
@property
def id(self):
"""Diaspora URI scheme format ID.
Only available for entities that own a handle and a guid.
"""
try:
# noinspection PyUnresolvedReferences
return "diaspora://%s/%s/%s" % (self.handle, self._tag_name, self.guid)
except AttributeError:
return None
# noinspection PyUnresolvedReferences
@property
def target_id(self):
"""Diaspora URI scheme format target ID.
Only available for entities that own a target_handle, target_guid and entity_type.
"""
try:
cls_module = importlib.import_module("federation.entities.base")
cls = getattr(cls_module, self.entity_type)
return "diaspora://%s/%s/%s" % (self.target_handle, CLASS_TO_TAG_MAPPING[cls], self.target_guid)
except (AttributeError, ImportError, KeyError):
return None
def to_xml(self):
"""Override in subclasses."""
raise NotImplementedError
@ -66,8 +107,10 @@ class DiasporaRelayableMixin(DiasporaEntityMixin):
class DiasporaComment(DiasporaRelayableMixin, Comment):
"""Diaspora comment."""
_tag_name = "comment"
def to_xml(self):
element = etree.Element("comment")
element = etree.Element(self._tag_name)
struct_to_xml(element, [
{"guid": self.guid},
{"parent_guid": self.target_guid},
@ -82,9 +125,11 @@ class DiasporaComment(DiasporaRelayableMixin, Comment):
class DiasporaPost(DiasporaEntityMixin, Post):
"""Diaspora post, ie status message."""
_tag_name = "status_message"
def to_xml(self):
"""Convert to XML message."""
element = etree.Element("status_message")
element = etree.Element(self._tag_name)
struct_to_xml(element, [
{"raw_message": self.raw_content},
{"guid": self.guid},
@ -98,11 +143,12 @@ class DiasporaPost(DiasporaEntityMixin, Post):
class DiasporaLike(DiasporaRelayableMixin, Reaction):
"""Diaspora like."""
_tag_name = "like"
reaction = "like"
def to_xml(self):
"""Convert to XML message."""
element = etree.Element("like")
element = etree.Element(self._tag_name)
struct_to_xml(element, [
{"target_type": "Post"},
{"guid": self.guid},
@ -117,11 +163,12 @@ class DiasporaLike(DiasporaRelayableMixin, Reaction):
class DiasporaRequest(DiasporaEntityMixin, Relationship):
"""Diaspora legacy request."""
_tag_name = "request"
relationship = "sharing"
def to_xml(self):
"""Convert to XML message."""
element = etree.Element("request")
element = etree.Element(self._tag_name)
struct_to_xml(element, [
{"sender_handle": self.handle},
{"recipient_handle": self.target_handle},
@ -134,10 +181,11 @@ class DiasporaContact(DiasporaEntityMixin, Follow):
Note we don't implement 'sharing' at the moment so just send it as the same as 'following'.
"""
_tag_name = "contact"
def to_xml(self):
"""Convert to XML message."""
element = etree.Element("contact")
element = etree.Element(self._tag_name)
struct_to_xml(element, [
{"author": self.handle},
{"recipient": self.target_handle},
@ -149,10 +197,11 @@ class DiasporaContact(DiasporaEntityMixin, Follow):
class DiasporaProfile(DiasporaEntityMixin, Profile):
"""Diaspora profile."""
_tag_name = "profile"
def to_xml(self):
"""Convert to XML message."""
element = etree.Element("profile")
element = etree.Element(self._tag_name)
struct_to_xml(element, [
{"diaspora_handle": self.handle},
{"first_name": self.name},
@ -181,6 +230,7 @@ class DiasporaProfile(DiasporaEntityMixin, Profile):
class DiasporaRetraction(DiasporaEntityMixin, Retraction):
"""Diaspora Retraction."""
_tag_name = "retraction"
mapped = {
"Like": "Reaction",
"Photo": "Image",
@ -189,7 +239,7 @@ class DiasporaRetraction(DiasporaEntityMixin, Retraction):
def to_xml(self):
"""Convert to XML message."""
element = etree.Element("retraction")
element = etree.Element(self._tag_name)
struct_to_xml(element, [
{"author": self.handle},
{"target_guid": self.target_guid},
@ -216,8 +266,10 @@ class DiasporaRetraction(DiasporaEntityMixin, Retraction):
class DiasporaReshare(DiasporaEntityMixin, Share):
"""Diaspora Reshare."""
_tag_name = "reshare"
def to_xml(self):
element = etree.Element("reshare")
element = etree.Element(self._tag_name)
struct_to_xml(element, [
{"author": self.handle},
{"guid": self.guid},

Wyświetl plik

@ -3,27 +3,16 @@ from datetime import datetime
from lxml import etree
from federation.entities.base import Image, Relationship, Post, Reaction, Comment, Profile, Retraction, Follow, Share
from federation.entities.base import Comment, Follow, Image, Post, Profile, Reaction, Relationship, Retraction, Share
from federation.entities.diaspora.entities import (
DiasporaPost, DiasporaComment, DiasporaLike, DiasporaRequest, DiasporaProfile, DiasporaRetraction,
DiasporaRelayableMixin, DiasporaContact, DiasporaReshare)
DiasporaComment, DiasporaContact, DiasporaLike, DiasporaPost,
DiasporaProfile, DiasporaRelayableMixin, DiasporaRequest, DiasporaReshare, DiasporaRetraction,
)
from federation.protocols.diaspora.signatures import get_element_child_info
from federation.utils.diaspora import retrieve_and_parse_profile
logger = logging.getLogger("federation")
BASE_MAPPINGS = {
Comment: "comment",
Follow: "contact",
Image: "photo",
Post: "status_message",
Profile: "profile",
Reaction: "like",
Relationship: "request",
Retraction: "retraction",
Share: "reshare",
}
MAPPINGS = {
"status_message": DiasporaPost,
"photo": Image,

Wyświetl plik

@ -1,20 +1,19 @@
import importlib
def retrieve_remote_content(entity_class, id, sender_key_fetcher=None):
def retrieve_remote_content(id, sender_key_fetcher=None):
"""Retrieve remote content and return an Entity object.
Currently, due to no other protocols supported, always use the Diaspora protocol.
:param entity_class: Federation entity class (from ``federation.entity.base``).
:param id: ID of the remote entity, in format``guid@domain.tld``.
:param id: ID of the remote entity.
:param sender_key_fetcher: Function to use to fetch sender public key. If not given, network will be used
to fetch the profile and the key. Function must take handle as only parameter and return a public key.
:returns: Entity class instance or ``None``
"""
protocol_name = "diaspora"
utils = importlib.import_module("federation.utils.%s" % protocol_name)
return utils.retrieve_and_parse_content(entity_class, id, sender_key_fetcher=sender_key_fetcher)
return utils.retrieve_and_parse_content(id, sender_key_fetcher=sender_key_fetcher)
def retrieve_remote_profile(handle):

Wyświetl plik

@ -2,9 +2,9 @@ from unittest.mock import Mock
import pytest
from federation.entities.diaspora.entities import DiasporaPost
# noinspection PyUnresolvedReferences
from federation.tests.fixtures.diaspora import *
from federation.tests.fixtures.keys import get_dummy_private_key
from federation.tests.fixtures.payloads import DIASPORA_PUBLIC_PAYLOAD
@pytest.fixture(autouse=True)
@ -23,16 +23,6 @@ def disable_network_calls(monkeypatch):
monkeypatch.setattr("requests.get", Mock(return_value=MockResponse))
@pytest.fixture
def diaspora_public_payload():
return DIASPORA_PUBLIC_PAYLOAD
@pytest.fixture
def diasporapost():
return DiasporaPost()
@pytest.fixture
def private_key():
return get_dummy_private_key()

Wyświetl plik

@ -110,6 +110,42 @@ class TestEntitiesConvertToXML:
assert etree.tostring(result).decode("utf-8") == converted
class TestEntityAttributes:
def test_comment_ids(self, diasporacomment):
assert diasporacomment.id == "diaspora://handle/comment/guid"
assert not diasporacomment.target_id
def test_contact_ids(self, diasporacontact):
assert not diasporacontact.id
assert not diasporacontact.target_id
def test_like_ids(self, diasporalike):
assert diasporalike.id == "diaspora://handle/like/guid"
assert not diasporalike.target_id
def test_post_ids(self, diasporapost):
assert diasporapost.id == "diaspora://handle/status_message/guid"
assert not diasporapost.target_id
def test_profile_ids(self, diasporaprofile):
assert diasporaprofile.id == "diaspora://bob@example.com/profile/"
assert not diasporaprofile.target_id
def test_request_ids(self, diasporarequest):
assert not diasporarequest.id
assert not diasporarequest.target_id
def test_reshare_ids(self, diasporareshare):
assert diasporareshare.id == "diaspora://%s/reshare/%s" % (diasporareshare.handle, diasporareshare.guid)
assert diasporareshare.target_id == "diaspora://%s/status_message/%s" % (
diasporareshare.target_handle, diasporareshare.target_guid
)
def test_retraction_ids(self, diasporaretraction):
assert not diasporaretraction.id
assert not diasporaretraction.target_id
class TestDiasporaProfileFillExtraAttributes:
def test_raises_if_no_handle(self):
attrs = {"foo": "bar"}

Wyświetl plik

@ -0,0 +1,68 @@
import pytest
from federation.entities.diaspora.entities import (
DiasporaPost, DiasporaComment, DiasporaLike, DiasporaRequest, DiasporaProfile, DiasporaRetraction,
DiasporaContact, DiasporaReshare,
)
from federation.tests.factories.entities import ShareFactory
from federation.tests.fixtures.payloads import DIASPORA_PUBLIC_PAYLOAD
__all__ = ("diasporacomment", "diasporacontact", "diasporalike", "diasporapost", "diasporaprofile",
"diasporareshare", "diasporarequest", "diasporaretraction", "diaspora_public_payload")
@pytest.fixture
def diaspora_public_payload():
return DIASPORA_PUBLIC_PAYLOAD
@pytest.fixture
def diasporacomment():
return DiasporaComment(
raw_content="raw_content", guid="guid", target_guid="target_guid", handle="handle",
signature="signature"
)
@pytest.fixture
def diasporacontact():
return DiasporaContact(handle="alice@example.com", target_handle="bob@example.org", following=True)
@pytest.fixture
def diasporalike():
return DiasporaLike(guid="guid", target_guid="target_guid", handle="handle", signature="signature")
@pytest.fixture
def diasporapost():
return DiasporaPost(
raw_content="raw_content", guid="guid", handle="handle", public=True,
provider_display_name="Socialhome"
)
@pytest.fixture
def diasporaprofile():
return DiasporaProfile(
handle="bob@example.com", raw_content="foobar", name="Bob Bobertson", public=True,
tag_list=["socialfederation", "federation"], image_urls={
"large": "urllarge", "medium": "urlmedium", "small": "urlsmall"
}
)
@pytest.fixture
def diasporareshare():
base_entity = ShareFactory()
return DiasporaReshare.from_base(base_entity)
@pytest.fixture
def diasporarequest():
return DiasporaRequest(handle="bob@example.com", target_handle="alice@example.com", relationship="following")
@pytest.fixture
def diasporaretraction():
return DiasporaRetraction(handle="bob@example.com", target_guid="x" * 16, entity_type="Post")

Wyświetl plik

@ -1,6 +1,5 @@
from unittest.mock import patch, Mock
from federation.entities.base import Post
from federation.fetchers import retrieve_remote_profile, retrieve_remote_content
@ -9,9 +8,9 @@ class TestRetrieveRemoteContent:
def test_calls_diaspora_retrieve_and_parse_content(self, mock_import):
mock_retrieve = Mock()
mock_import.return_value = mock_retrieve
retrieve_remote_content(Post, "1234@example.com", sender_key_fetcher=sum)
retrieve_remote_content("diaspora://user@example.com/status_message/1234", sender_key_fetcher=sum)
mock_retrieve.retrieve_and_parse_content.assert_called_once_with(
Post, "1234@example.com", sender_key_fetcher=sum,
"diaspora://user@example.com/status_message/1234", sender_key_fetcher=sum,
)

Wyświetl plik

@ -5,13 +5,14 @@ from urllib.parse import quote
import pytest
from lxml import html
from federation.entities.base import Profile, Post
from federation.entities.base import Profile
from federation.hostmeta.generators import DiasporaWebFinger, DiasporaHostMeta, generate_hcard
from federation.tests.fixtures.payloads import DIASPORA_PUBLIC_PAYLOAD
from federation.utils.diaspora import (
retrieve_diaspora_hcard, retrieve_diaspora_webfinger, retrieve_diaspora_host_meta, _get_element_text_or_none,
_get_element_attr_or_none, parse_profile_from_hcard, retrieve_and_parse_profile, retrieve_and_parse_content,
get_fetch_content_endpoint, fetch_public_key)
get_fetch_content_endpoint, fetch_public_key, parse_diaspora_uri,
)
@patch("federation.utils.diaspora.retrieve_and_parse_profile", autospec=True)
@ -27,6 +28,13 @@ def test_get_fetch_content_endpoint():
"https://example.com/fetch/status_message/1234"
def test_parse_diaspora_uri():
assert parse_diaspora_uri("diaspora://user@example.com/spam/eggs") == ("user@example.com", "spam", "eggs")
assert parse_diaspora_uri("diaspora://user@example.com/spam/eggs@spam") == ("user@example.com", "spam", "eggs@spam")
assert not parse_diaspora_uri("https://user@example.com/spam/eggs")
assert not parse_diaspora_uri("spam and eggs")
class TestRetrieveDiasporaHCard:
@patch("federation.utils.diaspora.retrieve_diaspora_webfinger", return_value=None)
def test_retrieve_webfinger_is_called(self, mock_retrieve):
@ -120,23 +128,23 @@ class TestRetrieveAndParseContent:
@patch("federation.utils.diaspora.fetch_document", return_value=(None, 404, None))
@patch("federation.utils.diaspora.get_fetch_content_endpoint", return_value="https://example.com/fetch/spam/eggs")
def test_calls_fetch_document(self, mock_get, mock_fetch):
retrieve_and_parse_content(Post, "1234@example.com")
retrieve_and_parse_content("diaspora://user@example.com/spam/eggs")
mock_fetch.assert_called_once_with("https://example.com/fetch/spam/eggs")
@patch("federation.utils.diaspora.fetch_document", return_value=(None, 404, None))
@patch("federation.utils.diaspora.get_fetch_content_endpoint")
def test_calls_get_fetch_content_endpoint(self, mock_get, mock_fetch):
retrieve_and_parse_content(Post, "1234@example.com")
mock_get.assert_called_once_with("example.com", "status_message", "1234")
retrieve_and_parse_content("diaspora://user@example.com/spam/eggs")
mock_get.assert_called_once_with("example.com", "spam", "eggs")
mock_get.reset_mock()
retrieve_and_parse_content(Post, "fooobar@1234@example.com")
mock_get.assert_called_once_with("example.com", "status_message", "fooobar@1234")
retrieve_and_parse_content("diaspora://user@example.com/spam/eggs@spam")
mock_get.assert_called_once_with("example.com", "spam", "eggs@spam")
@patch("federation.utils.diaspora.fetch_document", return_value=(DIASPORA_PUBLIC_PAYLOAD, 200, None))
@patch("federation.utils.diaspora.get_fetch_content_endpoint", return_value="https://example.com/fetch/spam/eggs")
@patch("federation.utils.diaspora.handle_receive", return_value=("sender", "protocol", ["entity"]))
def test_calls_handle_receive(self, mock_handle, mock_get, mock_fetch):
entity = retrieve_and_parse_content(Post, "1234@example.com", sender_key_fetcher=sum)
entity = retrieve_and_parse_content("diaspora://user@example.com/spam/eggs", sender_key_fetcher=sum)
mock_handle.assert_called_once_with(DIASPORA_PUBLIC_PAYLOAD, sender_key_fetcher=sum)
assert entity == "entity"
@ -144,16 +152,12 @@ class TestRetrieveAndParseContent:
@patch("federation.utils.diaspora.get_fetch_content_endpoint", return_value="https://example.com/fetch/spam/eggs")
def test_raises_on_fetch_error(self, mock_get, mock_fetch):
with pytest.raises(Exception):
retrieve_and_parse_content(Post, "1234@example.com")
def test_raises_on_unknown_entity(self):
with pytest.raises(ValueError):
retrieve_and_parse_content(dict, "1234@example.com")
retrieve_and_parse_content("diaspora://user@example.com/spam/eggs")
@patch("federation.utils.diaspora.fetch_document", return_value=(None, 404, None))
@patch("federation.utils.diaspora.get_fetch_content_endpoint", return_value="https://example.com/fetch/spam/eggs")
def test_returns_on_404(self, mock_get, mock_fetch):
result = retrieve_and_parse_content(Post, "1234@example.com")
result = retrieve_and_parse_content("diaspora://user@example.com/spam/eggs")
assert not result

Wyświetl plik

@ -108,6 +108,22 @@ def _get_element_attr_or_none(document, selector, attribute):
return None
def parse_diaspora_uri(uri):
"""Parse Diaspora URI scheme string.
See: https://diaspora.github.io/diaspora_federation/federation/diaspora_scheme.html
:return: tuple of (handle, entity_type, guid) or ``None``.
"""
if not uri.startswith("diaspora://"):
return
try:
handle, entity_type, guid = uri.replace("diaspora://", "").rsplit("/", maxsplit=2)
except ValueError:
return
return handle, entity_type, guid
def parse_profile_from_hcard(hcard, handle):
"""
Parse all the fields we can from a hCard document to get a Profile.
@ -132,23 +148,18 @@ def parse_profile_from_hcard(hcard, handle):
return profile
def retrieve_and_parse_content(entity_class, id, sender_key_fetcher=None):
def retrieve_and_parse_content(id, sender_key_fetcher=None):
"""Retrieve remote content and return an Entity class instance.
This is basically the inverse of receiving an entity. Instead, we fetch it, then call 'handle_receive'.
:param entity_class: Federation entity class (from ``federation.entity.base``).
:param id: GUID and domain of the remote entity, in format``guid@domain.tld``.
:param id: Diaspora URI scheme format ID.
:param sender_key_fetcher: Function to use to fetch sender public key. If not given, network will be used
to fetch the profile and the key. Function must take handle as only parameter and return a public key.
:returns: Entity object instance or ``None``
:raises: ``ValueError`` if ``entity_class`` is not valid.
"""
from federation.entities.diaspora.mappers import BASE_MAPPINGS
entity_type = BASE_MAPPINGS.get(entity_class)
if not entity_type:
raise ValueError("Unknown entity_class %s" % entity_class)
guid, domain = id.rsplit("@", 1)
handle, entity_type, guid = parse_diaspora_uri(id)
_username, domain = handle.split("@")
url = get_fetch_content_endpoint(domain, entity_type, guid)
document, status_code, error = fetch_document(url)
if status_code == 200: