mirror
/
wagtail
kopia lustrzana https://github.com/wagtail/wagtail
1
0
Forkuj 0
Kod Zgłoszenia Packages Projekty Wydania Wiki Aktywność

Add custom oEmbed finders for Facebook and Instagram (#6550) 

Move code from https://www.codista.com/en/blog/wagtail-instagram-new-oembed-api/ into core

Add custom oembed finder for Facebook

Apply suggestions from @originell's code review

Co-authored-by: Luis Nell <luis.nell@codista.com>

FIXUP More places to change the exception name (and a linter fix)

Extend Instagram/Facebook tests to check HTTP request

Add documentation for omitscript parameter
pull/6596/head
Cynthia Kiser 2020-11-12 18:29:09 -08:00 zatwierdzone przez Matt Westcott
rodzic 28146e0015
commit 2fdb2e8aef
8 zmienionych plików z 423 dodań i 35 usunięć
Pokaż statystyki Ściągnij plik aktualizacji Ściągnij plik porównania Expand all files Collapse all files

2
CHANGELOG.txt
Wyświetl plik

@ -4,6 +4,8 @@ Changelog
2.11.2 (xx.xx.xxxx) - IN DEVELOPMENT
~~~~~~~~~~~~~~~~~~~
* Add custom finder to support Instagram oEmbed API (Luis Nell)
* Add custom finder to support Facebook oEmbed API (Cynthia Kiser)
* Fix: Improve performance of permission check on translations for edit page (Karl Hobley)
* Fix: Gracefully handle missing Locale records on `Locale.get_active` and `.localized` (Matt Westcott)
* Fix: Handle `get_supported_language_variant` returning a language variant not in `LANGUAGES` (Matt Westcott)

1
CONTRIBUTORS.rst
Wyświetl plik

@ -484,6 +484,7 @@ Contributors
* Noah H
* David Bramwell
* Naglis Jonaitis
* Luis Nell
Translators
===========

41
docs/advanced_topics/embeds.rst
Wyświetl plik

@ -191,6 +191,47 @@ For example, this is how you can instruct Youtube to return videos in HTTPS
Wagtail will not try to run any other finder, even if the chosen one didn't
return an embed.
.. _facebook_and_instagram_embeds:
Facebook and Instagram
----------------------
As of October 2020, Facebook deprecated their public oEmbed APIs. If you would
like to embed Facebook or Instagram posts in your site, you will need to
use the new authenticated APIs. This requires you to set up a Facebook
Developer Account and create a Facebook App that includes the oEmbed Product.
Instructions for creating the neccessary app are in the requirements sections of the
`Facebook <https://developers.facebook.com/docs/plugins/oembed>`_
and `Instagram <https://developers.facebook.com/docs/instagram/oembed>`_ documentation.
Once you have your app access tokens (App ID and App Secret), add the Facebook and/or
Instagram finders to your ``WAGTAILEMBEDS_FINDERS`` setting and configure them with
the App ID and App Secret from your app:
.. code-block:: python
WAGTAILEMBEDS_FINDERS = [
{
'class': 'wagtail.embeds.finders.facebook',
'app_id': 'YOUR FACEBOOK APP_ID HERE',
'app_secret': 'YOUR FACEBOOK APP_SECRET HERE',
},
{
'class': 'wagtail.embeds.finders.instagram',
'app_id': 'YOUR INSTAGRAM APP_ID HERE',
'app_secret': 'YOUR INSTAGRAM APP_SECRET HERE',
}
]
By default, Facebook and Instagram embeds include some JavaScript that is necessary to
fully render the embed. In certain cases, this might not be something you want - for
example, if you have multiple Facebook embeds, this would result in multiple script tags.
By passing ``'omitscript': True`` in the configuration, you can indicate that these script
tags should be omitted from the embed HTML. Note that you will then have to take care of
loading this script yourself.
.. _Embedly:
Embed.ly

9
docs/releases/2.11.2.rst
Wyświetl plik

@ -10,6 +10,15 @@ Wagtail 2.11.2 release notes - IN DEVELOPMENT
What's new
==========
Facebook and Instagram embed finders
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Two new embed finders have been added for Facebook and Instagram, to replace the previous configuration
using Facebook's public oEmbed endpoint which was retired in October 2020. These require a Facebook
developer API key - for details of configuring this, see :ref:`facebook_and_instagram_embeds`.
This feature was developed by Luis Nell and Cynthia Kiser.
Bug fixes
~~~~~~~~~

116
wagtail/embeds/finders/facebook.py 100644
Wyświetl plik

@ -0,0 +1,116 @@
import json
import re
from urllib import request as urllib_request
from urllib.error import HTTPError, URLError
from urllib.parse import urlencode
from urllib.request import Request
from wagtail.embeds.exceptions import EmbedException, EmbedNotFoundException
from .base import EmbedFinder
class AccessDeniedFacebookOEmbedException(EmbedException):
pass
class FacebookOEmbedFinder(EmbedFinder):
'''
An embed finder that supports the authenticated Facebook oEmbed Endpoint.
https://developers.facebook.com/docs/plugins/oembed
'''
facebook_video = {
"endpoint": "https://graph.facebook.com/v9.0/oembed_video",
"urls": [
r'^https://(?:www\.)?facebook\.com/.+?/videos/.+$',
r'^https://(?:www\.)?facebook\.com/video\.php\?(?:v|id)=.+$',
r'^https://fb.watch/.+$',
],
}
facebook_post = {
"endpoint": "https://graph.facebook.com/v9.0/oembed_post",
"urls": [
r'^https://(?:www\.)?facebook\.com/.+?/(?:posts|activity)/.+$',
r'^https://(?:www\.)?facebook\.com/photo\.php\?fbid=.+$',
r'^https://(?:www\.)?facebook\.com/(?:photos|questions)/.+$',
r'^https://(?:www\.)?facebook\.com/permalink\.php\?story_fbid=.+$',
r'^https://(?:www\.)?facebook\.com/media/set/?\?set=.+$',
r'^https://(?:www\.)?facebook\.com/notes/.+?/.+?/.+$',
# At the moment, not documented on https://developers.facebook.com/docs/plugins/oembed-endpoints
# Works for posts with a single photo
r'^https://(?:www\.)?facebook\.com/.+?/photos/.+$',
],
}
def __init__(self, omitscript=False, app_id=None, app_secret=None):
# {settings.facebook_APP_ID}|{settings.facebook_APP_SECRET}
self.app_id = app_id
self.app_secret = app_secret
self.omitscript = omitscript
self._endpoints = {}
for provider in [self.facebook_video, self.facebook_post]:
patterns = []
endpoint = provider['endpoint'].replace('{format}', 'json')
for url in provider['urls']:
patterns.append(re.compile(url))
self._endpoints[endpoint] = patterns
def _get_endpoint(self, url):
for endpoint, patterns in self._endpoints.items():
for pattern in patterns:
if re.match(pattern, url):
return endpoint
def accept(self, url):
return self._get_endpoint(url) is not None
def find_embed(self, url, max_width=None):
# Find provider
endpoint = self._get_endpoint(url)
if endpoint is None:
raise EmbedNotFoundException
params = {'url': url, 'format': 'json'}
if max_width:
params['maxwidth'] = max_width
if self.omitscript:
params['omitscript'] = 'true'
# Configure request
request = Request(endpoint + '?' + urlencode(params))
request.add_header('Authorization', f'Bearer {self.app_id}|{self.app_secret}')
# Perform request
try:
r = urllib_request.urlopen(request)
except (HTTPError, URLError) as e:
if isinstance(e, HTTPError) and e.code == 404:
raise EmbedNotFoundException
elif isinstance(e, HTTPError) and e.code in [400, 401, 403]:
raise AccessDeniedFacebookOEmbedException
else:
raise EmbedNotFoundException
oembed = json.loads(r.read().decode('utf-8'))
# Return embed as a dict
return {
'title': oembed['title'] if 'title' in oembed else '',
'author_name': oembed['author_name'] if 'author_name' in oembed else '',
'provider_name': oembed['provider_name'] if 'provider_name' in oembed else 'Facebook',
'type': oembed['type'],
'thumbnail_url': oembed.get('thumbnail_url'),
'width': oembed.get('width'),
'height': oembed.get('height'),
'html': oembed.get('html'),
}
embed_finder_class = FacebookOEmbedFinder

82
wagtail/embeds/finders/instagram.py 100644
Wyświetl plik

@ -0,0 +1,82 @@
import json
import re
from urllib import request as urllib_request
from urllib.error import HTTPError, URLError
from urllib.parse import urlencode
from urllib.request import Request
from wagtail.embeds.exceptions import EmbedException, EmbedNotFoundException
from .base import EmbedFinder
class AccessDeniedInstagramOEmbedException(EmbedException):
pass
class InstagramOEmbedFinder(EmbedFinder):
'''
An embed finder that supports the authenticated Instagram oEmbed Endpoint.
https://developers.facebook.com/docs/instagram/oembed
'''
INSTAGRAM_ENDPOINT = 'https://graph.facebook.com/v9.0/instagram_oembed'
INSTAGRAM_URL_PATTERNS = [r'^https?://(?:www\.)?instagram\.com/p/.+$',
r'^https?://(?:www\.)?instagram\.com/tv/.+$',
]
def __init__(self, omitscript=False, app_id=None, app_secret=None):
# {settings.INSTAGRAM_APP_ID}|{settings.INSTAGRAM_APP_SECRET}
self.app_id = app_id
self.app_secret = app_secret
self.omitscript = omitscript
def accept(self, url):
for pattern in self.INSTAGRAM_URL_PATTERNS:
if re.match(pattern, url):
return True
return False
def find_embed(self, url, max_width=None):
params = {'url': url, 'format': 'json'}
if max_width:
params['maxwidth'] = max_width
if self.omitscript:
params['omitscript'] = 'true'
# Configure request
request = Request(self.INSTAGRAM_ENDPOINT + '?' + urlencode(params))
request.add_header('Authorization', f'Bearer {self.app_id}|{self.app_secret}')
# Perform request
try:
r = urllib_request.urlopen(request)
except (HTTPError, URLError) as e:
if isinstance(e, HTTPError) and e.code == 404:
raise EmbedNotFoundException
elif isinstance(e, HTTPError) and e.code in [400, 401, 403]:
raise AccessDeniedInstagramOEmbedException
else:
raise EmbedNotFoundException
oembed = json.loads(r.read().decode('utf-8'))
# Convert photos into HTML
if oembed['type'] == 'photo':
html = '<img src="%s" alt="">' % (oembed["url"],)
else:
html = oembed.get('html')
# Return embed as a dict
return {
'title': oembed['title'] if 'title' in oembed else '',
'author_name': oembed['author_name'] if 'author_name' in oembed else '',
'provider_name': oembed['provider_name'] if 'provider_name' in oembed else 'Instagram',
'type': oembed['type'],
'thumbnail_url': oembed.get('thumbnail_url'),
'width': oembed.get('width'),
'height': oembed.get('height'),
'html': html,
}
embed_finder_class = InstagramOEmbedFinder

35
wagtail/embeds/oembed_providers.py
Wyświetl plik

@ -149,38 +149,6 @@ photobucket = {
],
}
instagram = {
"endpoint": "https://api.instagram.com/oembed",
"urls": [
r'^https?://instagr\.am/p/.+$',
r'^https?://(?:www\.)?instagram\.com/p/.+$',
],
}
facebook_video = {
"endpoint": "https://www.facebook.com/plugins/video/oembed.{format}",
"urls": [
r'^https://(?:www\.)?facebook\.com/.+?/videos/.+$',
r'^https://(?:www\.)?facebook\.com/video\.php\?(?:v|id)=.+$',
],
}
facebook_post = {
"endpoint": "https://www.facebook.com/plugins/post/oembed.{format}",
"urls": [
r'^https://(?:www\.)?facebook\.com/.+?/(?:posts|activity)/.+$',
r'^https://(?:www\.)?facebook\.com/photo\.php\?fbid=.+$',
r'^https://(?:www\.)?facebook\.com/(?:photos|questions)/.+$',
r'^https://(?:www\.)?facebook\.com/permalink\.php\?story_fbid=.+$',
r'^https://(?:www\.)?facebook\.com/media/set/?\?set=.+$',
r'^https://(?:www\.)?facebook\.com/notes/.+?/.+?/.+$',
# At the moment, not documented on https://developers.facebook.com/docs/plugins/oembed-endpoints
# Works for posts with a single photo
r'^https://(?:www\.)?facebook\.com/.+?/photos/.+$',
],
}
slideshare = {
"endpoint": "https://www.slideshare.net/api/oembed/2",
"urls": [
@ -662,8 +630,7 @@ reddit = {
all_providers = [
speakerdeck, app_net, youtube, deviantart, blip_tv, dailymotion, flikr,
hulu, nfb, qik, revision3, scribd, viddler, vimeo, dotsub, yfrog,
clickthrough, kinomap, photobucket, instagram, facebook_video,
facebook_post, slideshare,
clickthrough, kinomap, photobucket, slideshare,
major_league_gaming, opera, skitch, twitter, soundcloud, collegehumor,
polleverywhere, ifixit, smugmug, github_gist, animoto, rdio, five_min,
five_hundred_px, dipdive, yandex, mixcloud, kickstarter, coub, screenr,

172
wagtail/embeds/tests/test_embeds.py
Wyświetl plik

@ -3,7 +3,7 @@ import unittest
import urllib.request
from unittest.mock import patch
from urllib.error import URLError
from urllib.error import HTTPError, URLError
from django import template
from django.core.exceptions import ValidationError
@ -18,6 +18,10 @@ from wagtail.embeds.exceptions import EmbedNotFoundException, EmbedUnsupportedPr
from wagtail.embeds.finders import get_finders
from wagtail.embeds.finders.embedly import AccessDeniedEmbedlyException, EmbedlyException
from wagtail.embeds.finders.embedly import EmbedlyFinder as EmbedlyFinder
from wagtail.embeds.finders.facebook import AccessDeniedFacebookOEmbedException
from wagtail.embeds.finders.facebook import FacebookOEmbedFinder as FacebookOEmbedFinder
from wagtail.embeds.finders.instagram import AccessDeniedInstagramOEmbedException
from wagtail.embeds.finders.instagram import InstagramOEmbedFinder as InstagramOEmbedFinder
from wagtail.embeds.finders.oembed import OEmbedFinder as OEmbedFinder
from wagtail.embeds.models import Embed
from wagtail.embeds.templatetags.wagtailembeds_tags import embed_tag
@ -77,6 +81,40 @@ class TestGetFinders(TestCase):
self.assertIsInstance(finders[0], OEmbedFinder)
self.assertEqual(finders[0].options, {'foo': 'bar'})
@override_settings(WAGTAILEMBEDS_FINDERS=[
{
'class': 'wagtail.embeds.finders.instagram',
'app_id': '1234567890',
'app_secret': 'abcdefghijklmnop',
},
])
def test_find_instagram_oembed_with_options(self):
finders = get_finders()
self.assertEqual(len(finders), 1)
self.assertIsInstance(finders[0], InstagramOEmbedFinder)
self.assertEqual(finders[0].app_id, '1234567890')
self.assertEqual(finders[0].app_secret, 'abcdefghijklmnop')
# omitscript defaults to False
self.assertEqual(finders[0].omitscript, False)
@override_settings(WAGTAILEMBEDS_FINDERS=[
{
'class': 'wagtail.embeds.finders.facebook',
'app_id': '1234567890',
'app_secret': 'abcdefghijklmnop',
},
])
def test_find_facebook_oembed_with_options(self):
finders = get_finders()
self.assertEqual(len(finders), 1)
self.assertIsInstance(finders[0], FacebookOEmbedFinder)
self.assertEqual(finders[0].app_id, '1234567890')
self.assertEqual(finders[0].app_secret, 'abcdefghijklmnop')
# omitscript defaults to False
self.assertEqual(finders[0].omitscript, False)
class TestEmbeds(TestCase):
def setUp(self):
@ -387,6 +425,138 @@ class TestOembed(TestCase):
self.assertEqual(request.get_full_url().split('?')[0], "https://www.vimeo.com/api/oembed.json")
class TestInstagramOEmbed(TestCase):
def setUp(self):
class DummyResponse:
def read(self):
return b"""{
"type": "something",
"url": "http://www.example.com",
"title": "test_title",
"author_name": "test_author",
"provider_name": "Instagram",
"thumbnail_url": "test_thumbail_url",
"width": "test_width",
"height": "test_height",
"html": "<blockquote class=\\\"instagram-media\\\">Content</blockquote>"
}"""
self.dummy_response = DummyResponse()
def test_instagram_oembed_only_accepts_new_url_patterns(self):
finder = InstagramOEmbedFinder()
self.assertTrue(finder.accept("https://www.instagram.com/p/CHeRxmnDSYe/?utm_source=ig_embed"))
self.assertFalse(finder.accept("https://instagr.am/p/CHeRxmnDSYe/?utm_source=ig_embed"))
@patch('urllib.request.urlopen')
def test_instagram_oembed_return_values(self, urlopen):
urlopen.return_value = self.dummy_response
result = InstagramOEmbedFinder(app_id='123', app_secret='abc').find_embed("https://instagr.am/p/CHeRxmnDSYe/")
self.assertEqual(result, {
'type': 'something',
'title': 'test_title',
'author_name': 'test_author',
'provider_name': 'Instagram',
'thumbnail_url': 'test_thumbail_url',
'width': 'test_width',
'height': 'test_height',
'html': '<blockquote class="instagram-media">Content</blockquote>'
})
# check that a request was made with the expected URL / authentication
request = urlopen.call_args[0][0]
# check that a request was made with the expected URL / authentication
request = urlopen.call_args[0][0]
self.assertEqual(
request.get_full_url(),
"https://graph.facebook.com/v9.0/instagram_oembed?url=https%3A%2F%2Finstagr.am%2Fp%2FCHeRxmnDSYe%2F&format=json"
)
self.assertEqual(request.get_header('Authorization'), "Bearer 123|abc")
def test_instagram_request_denied_401(self):
err = HTTPError("https://instagr.am/p/CHeRxmnDSYe/", code=401, msg='invalid credentials', hdrs={}, fp=None)
config = {'side_effect': err}
with patch.object(urllib.request, 'urlopen', **config):
self.assertRaises(AccessDeniedInstagramOEmbedException, InstagramOEmbedFinder().find_embed,
"https://instagr.am/p/CHeRxmnDSYe/")
def test_instagram_request_not_found(self):
err = HTTPError("https://instagr.am/p/badrequest/", code=404, msg='Not Found', hdrs={}, fp=None)
config = {'side_effect': err}
with patch.object(urllib.request, 'urlopen', **config):
self.assertRaises(EmbedNotFoundException, InstagramOEmbedFinder().find_embed,
"https://instagr.am/p/CHeRxmnDSYe/")
def test_instagram_failed_request(self):
config = {'side_effect': URLError(reason="Testing error handling")}
with patch.object(urllib.request, 'urlopen', **config):
self.assertRaises(EmbedNotFoundException, InstagramOEmbedFinder().find_embed,
"https://instagr.am/p/CHeRxmnDSYe/")
class TestFacebookOEmbed(TestCase):
def setUp(self):
class DummyResponse:
def read(self):
return b"""{
"type": "something",
"url": "http://www.example.com",
"title": "test_title",
"author_name": "test_author",
"provider_name": "Facebook",
"thumbnail_url": "test_thumbail_url",
"width": "test_width",
"height": "test_height",
"html": "<blockquote class=\\\"facebook-media\\\">Content</blockquote>"
}"""
self.dummy_response = DummyResponse()
def test_facebook_oembed_accepts_various_url_patterns(self):
finder = FacebookOEmbedFinder()
self.assertTrue(finder.accept("https://www.facebook.com/testuser/posts/10157389310497085"))
self.assertTrue(finder.accept("https://fb.watch/ABC123eew/"))
@patch('urllib.request.urlopen')
def test_facebook_oembed_return_values(self, urlopen):
urlopen.return_value = self.dummy_response
result = FacebookOEmbedFinder(app_id='123', app_secret='abc').find_embed("https://fb.watch/ABC123eew/")
self.assertEqual(result, {
'type': 'something',
'title': 'test_title',
'author_name': 'test_author',
'provider_name': 'Facebook',
'thumbnail_url': 'test_thumbail_url',
'width': 'test_width',
'height': 'test_height',
'html': '<blockquote class="facebook-media">Content</blockquote>'
})
# check that a request was made with the expected URL / authentication
request = urlopen.call_args[0][0]
self.assertEqual(
request.get_full_url(),
"https://graph.facebook.com/v9.0/oembed_video?url=https%3A%2F%2Ffb.watch%2FABC123eew%2F&format=json"
)
self.assertEqual(request.get_header('Authorization'), "Bearer 123|abc")
def test_facebook_request_denied_401(self):
err = HTTPError("https://fb.watch/ABC123eew/", code=401, msg='invalid credentials', hdrs={}, fp=None)
config = {'side_effect': err}
with patch.object(urllib.request, 'urlopen', **config):
self.assertRaises(AccessDeniedFacebookOEmbedException, FacebookOEmbedFinder().find_embed,
"https://fb.watch/ABC123eew/")
def test_facebook_request_not_found(self):
err = HTTPError("https://fb.watch/ABC123eew/", code=404, msg='Not Found', hdrs={}, fp=None)
config = {'side_effect': err}
with patch.object(urllib.request, 'urlopen', **config):
self.assertRaises(EmbedNotFoundException, FacebookOEmbedFinder().find_embed,
"https://fb.watch/ABC123eew/")
def test_facebook_failed_request(self):
config = {'side_effect': URLError(reason="Testing error handling")}
with patch.object(urllib.request, 'urlopen', **config):
self.assertRaises(EmbedNotFoundException, FacebookOEmbedFinder().find_embed,
"https://fb.watch/ABC123eew/")
class TestEmbedTag(TestCase):
@patch('wagtail.embeds.embeds.get_embed')
def test_direct_call(self, get_embed):