ipydrawio/notebooks/Notebook to Diagram.ipynb

622 wiersze
19 KiB
Plaintext

{
"cells": [
{
"cell_type": "markdown",
"id": "0f682c4f-2262-44d1-a17c-56427fd9e204",
"metadata": {},
"source": [
"# Notebook to Diagram\n",
"\n",
"While the drag-and-drop behavior of MIME outputs works well, it is untenable at scale, and doesn't handle some features very well."
]
},
{
"cell_type": "markdown",
"id": "18249538-b224-4d78-8f46-4f2841cb21f6",
"metadata": {},
"source": [
"> # A Diagram Notebook\n",
"\n",
"> There _might_ be a compelling reason to store a drawio diagram in a notebook. This has one."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "46854767-3eef-4b17-9cf2-df639ca47ae1",
"metadata": {},
"outputs": [],
"source": [
"import pandas\n",
"\n",
"%config InlineBackend.figure_formats = ['svg']"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "279402d4-649c-4e18-9834-1d8b829fee84",
"metadata": {},
"outputs": [],
"source": [
"pandas.util.testing.makeDataFrame().plot();"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1025d74e-d652-4e82-899f-c9923cc72688",
"metadata": {},
"outputs": [],
"source": [
"import copy\n",
"import difflib\n",
"import html\n",
"import os\n",
"from pathlib import Path\n",
"from uuid import uuid4\n",
"\n",
"import black\n",
"import bleach\n",
"import IPython"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "16e420c3-0402-4c19-9bb7-dd9922cbcd16",
"metadata": {},
"outputs": [],
"source": [
"import lxml.etree as ET"
]
},
{
"cell_type": "markdown",
"id": "abf613e2-b543-4882-b8b1-f4be51b9c8a2",
"metadata": {},
"source": [
"## What XML Library?\n",
"\n",
"`xml.etree` is part of the standard library, but `lxml.etree` is faster at scale. Let's prefer the latter, and offers some nice API improvements."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "34298b4b-f24d-4b14-908c-4c0482fd4ec5",
"metadata": {},
"outputs": [],
"source": [
"import nbformat\n",
"import requests"
]
},
{
"cell_type": "markdown",
"id": "c4f881b0-333a-40b9-8ca3-970263cbf629",
"metadata": {},
"source": [
"# A Notebook\n",
"\n",
"Using this notebook is as good as any."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "921cb55f-7e53-40a0-a7d3-b710b15abb52",
"metadata": {},
"outputs": [],
"source": [
"from ipydrawio_widgets import Diagram\n",
"from lxml.builder import E\n",
"from nbconvert.exporters.html import TemplateExporter\n",
"from nbconvert.filters import markdown2html_mistune\n",
"from pygments import highlight\n",
"from pygments.formatters import HtmlFormatter\n",
"from pygments.lexers import get_lexer_by_name\n",
"\n",
"this_notebook = nbformat.reads(\n",
" Path(os.environ.get(\"NOTEBOOK\", \"Notebook to Diagram.ipynb\")).read_text(\n",
" encoding=\"utf-8\",\n",
" ),\n",
" as_version=4,\n",
")\n",
"this_notebook.cells[0]"
]
},
{
"cell_type": "markdown",
"id": "ef847468-7ab7-4c76-80b3-ed6bccf96b34",
"metadata": {},
"source": [
"## Exporter\n",
"\n",
"We want to tie into the `nbconvert` pipeline pretty directly. We'll do it in-line for now."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "81f64be0-5e61-481c-9950-ec488762a2d9",
"metadata": {},
"outputs": [],
"source": [
"from ipydrawio.constants import A_SHORT_DRAWIO\n",
"\n",
"A_SHORT_DRAWIO"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d1b716a4-3cdd-4a12-be25-ae52abbca012",
"metadata": {},
"outputs": [],
"source": [
"def a_style(**kwargs):\n",
" return {\"style\": \"\".join([f\"{k}={v};\" for k, v in kwargs.items()])}"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1de60028-0755-4b95-a0e9-5a0a065a464a",
"metadata": {},
"outputs": [],
"source": [
"wikimedia = \"https://tools-static.wmflabs.org/fontcdn/css?family=Architects+Daughter\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2b7fbf0c-45c9-45d2-8c83-3955d955578c",
"metadata": {},
"outputs": [],
"source": [
"def a_note_style(**kwargs):\n",
" _defaults_ = kwargs.pop(\n",
" \"_defaults_\",\n",
" {\n",
" \"shape\": \"note\",\n",
" \"backgroundOutline\": 1,\n",
" \"darkOpacity\": 0.05,\n",
" \"fillColor\": \"#FFF9B2\",\n",
" \"strokeColor\": \"none\",\n",
" \"fillStyle\": \"solid\",\n",
" \"direction\": \"west\",\n",
" \"gradientDirection\": \"north\",\n",
" \"gradientColor\": \"#FFF2A1\",\n",
" \"sketch\": 1,\n",
" \"shadow\": 1,\n",
" \"size\": 20,\n",
" \"fontSize\": 24,\n",
" \"jiggle\": 2,\n",
" \"pointerEvents\": 1,\n",
" \"hachureGap\": 4,\n",
" \"whiteSpace\": \"wrap\",\n",
" \"fontFamily\": \"Architects Daughter\",\n",
" \"fontSource\": \"https%3A%2F%2Ftools-static.wmflabs.org%2Ffontcdn%2Fcss%3Ffamily%3DArchitects%2BDaughter\",\n",
" },\n",
" )\n",
" style = dict(**_defaults_)\n",
" style.update(**kwargs)\n",
" return a_style(**style)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9776b2d7-b1b6-4796-97a7-8bd1ec5ed00a",
"metadata": {},
"outputs": [],
"source": [
"def a_note_geometry(**kwargs):\n",
" _defaults_ = kwargs.get(\n",
" \"_defaults_\",\n",
" dict(x=\"0\", y=\"0\", width=\"500\", height=\"300\", **{\"as\": \"geometry\"}),\n",
" )\n",
" mx_kwargs = dict(_defaults_)\n",
" mx_kwargs.pop(\"defaults\", None)\n",
" mx_kwargs.update(**kwargs)\n",
" return E.mxGeometry(**mx_kwargs)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "686748a1-b11d-466f-b576-2b91cdc90a0e",
"metadata": {},
"outputs": [],
"source": [
"def a_note(value=None, geometry=None, vertex=\"1\", parent=\"1\", **kwargs):\n",
" value = value or kwargs.pop(\"value\", None)\n",
" _defaults_ = kwargs.pop(\n",
" \"_defaults_\",\n",
" {\n",
" \"id\": f\"{uuid4()}\",\n",
" \"value\": value,\n",
" },\n",
" )\n",
" mx_kwargs = dict(_defaults_)\n",
" mx_kwargs.update(**kwargs)\n",
" style = mx_kwargs.pop(\"style\", {})\n",
" if isinstance(style, dict):\n",
" mx_kwargs.update(a_note_style(**style))\n",
" mx_kwargs.update(\n",
" vertex=kwargs.get(\"vertex\", \"1\"),\n",
" parent=kwargs.get(\"parent\", \"1\"),\n",
" )\n",
" geometry = a_note_geometry(**(geometry or {}))\n",
" return E.mxCell(geometry, **mx_kwargs)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "65c28612-2cec-4200-a125-f5333658718f",
"metadata": {},
"outputs": [],
"source": [
"A_CARD = (\n",
" \"\"\"\n",
"<mxCell id=\"64\" value=\"CARD A\" style=\"shape=note;backgroundOutline=1;darkOpacity=0.05;fillColor=#FFF9B2;strokeColor=none;fillStyle=solid;direction=west;gradientDirection=north;gradientColor=#FFF2A1;sketch=1;shadow=1;size=20;fontSize=24;jiggle=2;pointerEvents=1;hachureGap=4;whiteSpace=wrap;fontFamily=Architects Daughter;fontSource=https%3A%2F%2Ftools-static.wmflabs.org%2Ffontcdn%2Fcss%3Ffamily%3DArchitects%2BDaughter;\" vertex=\"1\" parent=\"1\">\n",
" <mxGeometry x=\"0\" y=\"0\" width=\"500\" height=\"300\" as=\"geometry\"/>\n",
"</mxCell>\"\"\"\n",
").strip()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7ed371b0-5c47-4178-8438-cafde963629c",
"metadata": {},
"outputs": [],
"source": [
"def test_a_note():\n",
" note = a_note(value=\"CARD A\", id=\"64\")\n",
" xml = ET.tostring(note, encoding=str, pretty_print=True)\n",
" lines = [xml.splitlines(), A_CARD.splitlines()]\n",
" diff = \"\\n\".join([*difflib.unified_diff(*lines)])\n",
" if diff:\n",
" htmldiff = difflib.HtmlDiff()\n",
" IPython.display.display(IPython.display.HTML(htmldiff.make_file(*lines)))\n",
" assert not diff, diff"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dc81e433-40bb-419d-adac-92772c5cdbb4",
"metadata": {},
"outputs": [],
"source": [
"test_a_note()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a910f0d5-119f-4c0d-8a93-2954706b266e",
"metadata": {},
"outputs": [],
"source": [
"ET.tostring(a_note(\"A Note\"))"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "94cdc65f-a1db-42b1-8005-c4e402a4c0d2",
"metadata": {},
"outputs": [],
"source": [
"class MXTools:\n",
" page_height = 850\n",
" page_width = 1100\n",
"\n",
" def empty_mx(self):\n",
" return E.mxfile(\n",
" E.diagram(\n",
" E.mxGraphModel(\n",
" E.root(\n",
" E.mxCell(id=\"0\"),\n",
" E.mxCell(id=\"1\", parent=\"0\"),\n",
" ),\n",
" dx=\"1450\",\n",
" dy=\"467\",\n",
" grid=\"1\",\n",
" gridSize=\"10\",\n",
" guides=\"1\",\n",
" tooltips=\"1\",\n",
" connect=\"1\",\n",
" arrows=\"1\",\n",
" fold=\"1\",\n",
" page=\"1\",\n",
" pageScale=\"1\",\n",
" pageWidth=f\"{self.page_height}\",\n",
" pageHeight=f\"{self.page_width}\",\n",
" math=\"0\",\n",
" shadow=\"0\",\n",
" ),\n",
" id=\"x\",\n",
" name=\"Page-1\",\n",
" ),\n",
" version=\"14.6.11\",\n",
" )\n",
"\n",
" def style(self, **kwargs):\n",
" return {\"style\": \";\".join([f\"{k}={v}\" for k, v in kwargs.items()]) + \";\"}\n",
"\n",
" @property\n",
" def text_style(self):\n",
" return self.style(\n",
" rounded=0,\n",
" verticalAlign=\"top\",\n",
" autosize=1,\n",
" align=\"left\",\n",
" whiteSpace=\"wrap\",\n",
" fillColor=\"black\",\n",
" strokeColor=\"none\",\n",
" html=\"1\",\n",
" )\n",
"\n",
" def svg_style(self, svg):\n",
" return self.style(\n",
" shape=\"image\",\n",
" verticalLabelPosition=\"bottom\",\n",
" labelBackgroundColor=\"#ffffff\",\n",
" verticalAlign=\"top\",\n",
" aspect=\"fixed\",\n",
" imageAspect=0,\n",
" image=f\"data:image/svg+xml,{svg}\",\n",
" )"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "39d3f631-5553-4309-bbc3-f669f1d09e73",
"metadata": {},
"outputs": [],
"source": [
"DEFAULT_FORMATTER = HtmlFormatter(\n",
" noclasses=True,\n",
" style=\"monokai\",\n",
" wrapcode=True,\n",
" cssstyles=\"background: rgba(0,0,0,0.9); padding: 0.1em 0.5em; border-radius: 0.5em\",\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6f060d3f-91c1-42e8-92f7-a53dde45bb32",
"metadata": {},
"outputs": [],
"source": [
"class NoteCellExporter(TemplateExporter, MXTools):\n",
" export_from_notebook = \"Diagram\"\n",
" formatter = DEFAULT_FORMATTER\n",
" lexer = get_lexer_by_name(\"ipython\")\n",
"\n",
" def _file_extension_default(self):\n",
" \"\"\"\n",
" The new file extension is ``.test_ext``\n",
" \"\"\"\n",
" return \".dio\"\n",
"\n",
" def from_notebook_node(self, nb, resources=None, **kw):\n",
" nb_copy = copy.deepcopy(nb)\n",
" resources = self._init_resources(resources)\n",
"\n",
" if \"language\" in nb[\"metadata\"]:\n",
" resources[\"language\"] = nb[\"metadata\"][\"language\"].lower()\n",
"\n",
" # Preprocess\n",
" nb_copy, resources = self._preprocess(nb_copy, resources)\n",
"\n",
" tree = self.empty_mx()\n",
" tree.xpath(\"//root\")\n",
" prev = tree.xpath(\"//mxCell\")[-1]\n",
" prev_id = 1\n",
" col = 0\n",
" row = 0\n",
" w = 500\n",
" h = 300\n",
" p = 25\n",
"\n",
" def card_geo():\n",
" return dict(\n",
" x=f\"{col * (w + p)}\",\n",
" y=f\"{row * (h + p)}\",\n",
" width=f\"{w}\",\n",
" height=f\"{h}\",\n",
" **{\"as\": \"geometry\"},\n",
" )\n",
"\n",
" for cell in nb_copy[\"cells\"]:\n",
" if cell[\"cell_type\"] == \"markdown\":\n",
" row += 1\n",
" col = 0\n",
" mxc = a_note(\n",
" value=self.markdown_escaped(cell),\n",
" id=f\"{prev_id + 1}\",\n",
" style={\n",
" \"html\": \"1\",\n",
" \"fontFamily\": \"Architects Daughter\",\n",
" \"fontSource\": \"https://tools-static.wmflabs.org/fontcdn/css?family=Architects+Daughter\",\n",
" },\n",
" geometry={\"y\": f\"{row * (h + p)}\", \"x\": f\"{col * (w + p)}\"},\n",
" )\n",
" prev.addnext(mxc)\n",
" prev = mxc\n",
" prev_id += 1\n",
" col += 1\n",
" elif cell[\"cell_type\"] == \"code\":\n",
" mxc = E.mxCell(\n",
" a_note_geometry(**card_geo()),\n",
" id=f\"{prev_id + 1}\",\n",
" parent=\"1\",\n",
" vertex=\"1\",\n",
" value=self.source_escaped(cell),\n",
" **self.text_style,\n",
" )\n",
" prev.addnext(mxc)\n",
" prev = mxc\n",
" prev_id += 1\n",
" col += 1\n",
" for output in cell.outputs:\n",
" if \"data\" not in output:\n",
" continue\n",
" svg = output.data.get(\"image/svg+xml\")\n",
" if svg:\n",
" mxc = E.mxCell(\n",
" a_note_geometry(**card_geo()),\n",
" id=f\"{prev_id + 1}\",\n",
" parent=\"1\",\n",
" vertex=\"1\",\n",
" value=\"\",\n",
" **self.svg_style(self.svg_escaped(svg)),\n",
" )\n",
" prev.addnext(mxc)\n",
" prev = mxc\n",
" prev_id += 1\n",
" col += 1\n",
" return (\n",
" ET.tostring(tree, encoding=str, pretty_print=True).replace(\"&amp;\", \"&\"),\n",
" resources,\n",
" )\n",
"\n",
" def source_escaped(self, cell):\n",
" try:\n",
" source = black.format_str(cell.source, mode=black.FileMode(line_length=60))\n",
" except:\n",
" source = cell.source\n",
" return html.escape(highlight(source, self.lexer, self.formatter))\n",
"\n",
" allowed_tags = {\n",
" *bleach.sanitizer.ALLOWED_TAGS,\n",
" *{f\"h{i}\" for i in range(7)},\n",
" \"div\",\n",
" \"pre\",\n",
" \"span\",\n",
" \"blockquote\",\n",
" \"strong\",\n",
" } ^ {\"a\"}\n",
"\n",
" def markdown_escaped(self, cell):\n",
" return html.escape(\n",
" bleach.clean(\n",
" markdown2html_mistune(cell.source),\n",
" tags=self.allowed_tags,\n",
" strip=True,\n",
" ),\n",
" )\n",
"\n",
" def svg_escaped(self, some_data):\n",
" return requests.utils.quote(some_data, safe=\"!*()\")\n",
"\n",
" def svg_size(self, some_data):\n",
" width = 100\n",
" height = 100\n",
" et = ET.fromstring(some_data.encode(\"utf-8\"))\n",
" width = et.xpath(\"/@width\")\n",
" height = et.xpath(\"/@height\")\n",
" return f\"\"\"width=\"{width}\" height=\"{height}\" \"\"\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e83d569e-9e6e-40b8-a652-dcbc1a75a049",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"export_diagram = NoteCellExporter()\n",
"(body, resources) = export_diagram.from_notebook_node(this_notebook)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3ef7fb6c-477d-4b44-a8ff-78d3cd681044",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"diagram = Diagram(layout={\"height\": \"400px\", \"width\": \"100%\"})\n",
"new_params = dict(**diagram.url_params)\n",
"new_params.update(ui=\"sketch\", format=\"0\")\n",
"diagram.url_params = new_params\n",
"diagram"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e0da8b9d-7248-48dd-88ec-012694718e2c",
"metadata": {},
"outputs": [],
"source": [
"diagram.source.value = body"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "110841eb-5ebe-47f4-8f70-0c15af693e59",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"print(ET.tostring(ET.fromstring(diagram.source.value), encoding=str, pretty_print=True))"
]
},
{
"cell_type": "markdown",
"id": "b654746b-256b-4761-886d-6a6abecb5a8a",
"metadata": {},
"source": [
"## Packaging\n",
"\n",
"Not appearing here, but our `setup.cfg` should be upgraded to something like:\n",
"\n",
"\n",
"```ini\n",
"[options.entry_points]\n",
"nbconvert.exporters =\n",
" ipydrawio = ipydrawio.exporters:DiagramExporter\n",
"```"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.2"
}
},
"nbformat": 4,
"nbformat_minor": 5
}