Remove duplicates before applying unique constraint on embed hash

CHANGELOG.txt

@@ -17,6 +17,7 @@ Changelog
  * Fix: Ensure aliases are published when the source page is published (Karl Hobley)
  * Fix: Make page privacy rules apply to aliases (Karl Hobley)
  * Fix: Prevent error when saving embeds that do not include a thumbnail URL (Cynthia Kiser)
+ * Fix: Ensure that duplicate embed records are deleted when upgrading (Matt Westcott)
  * Fix: Prevent failure when running `manage.py dumpdata` with no arguments (Matt Westcott)
 
 

docs/releases/2.12.1.rst

@@ -16,4 +16,5 @@ Bug fixes
  * Ensure aliases are published when the source page is published (Karl Hobley)
  * Make page privacy rules apply to aliases (Karl Hobley)
  * Prevent error when saving embeds that do not include a thumbnail URL (Cynthia Kiser)
+ * Ensure that duplicate embed records are deleted when upgrading (Matt Westcott)
  * Prevent failure when running ``manage.py dumpdata`` with no arguments (Matt Westcott)

wagtail/embeds/migrations/0007_populate_hash.py

@@ -1,4 +1,5 @@
 from django.db import migrations
+from django.db.models import Count, Min
 
 from wagtail.embeds.embeds import get_embed_hash
 
@@ -28,6 +29,14 @@ def migrate_forwards(apps, schema_editor):
     if batch:
         Embed.objects.bulk_update(batch, ["hash"])
 
+    # delete duplicates
+    duplicates = Embed.objects.values('hash').annotate(
+        hash_count=Count('id'), min_id=Min('id')
+    ).filter(hash_count__gt=1)
+    for dup in duplicates:
+        # for each duplicated hash, delete all except the one with the lowest id
+        Embed.objects.filter(hash=dup['hash']).exclude(id=dup['min_id']).delete()
+
 
 class Migration(migrations.Migration):