ipydrawio/docs/tutorials/working-with-custom-libraries/index.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e9738510-4036-46e9-b728-4590dbd2c738",
   "metadata": {},
   "source": [
    "# Working with Custom Shape Libraries\n",
    "\n",
    "When working with the `ipydrawio` TypeScript API, custom shape libraries can be added by providing a fully-resolved, absolute URL to an `.xml` file in `clibs`. \n",
    "\n",
    "From the [Widget API](../../Diagram%20Widget.ipynb), this is somewhat more complicated, but might be worth it for certain custom cases such as [issue #80](https://github.com/deathbeds/ipydrawio/issues/80), where a fully-kernel-driven solution is desirable, despite the _gotchas_ (see below)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ff64bdc4-2dd5-4738-affc-36dab7c82584",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import base64\n",
    "import json\n",
    "import urllib.parse\n",
    "import zlib\n",
    "\n",
    "import ipydrawio"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f54b3ce-a88a-4a89-bebb-3074ad6dfee4",
   "metadata": {},
   "source": [
    "## A Library\n",
    "A library is, at its core, a list of shape descriptions. The best way to learn more about these is investigating the [drawio documentation](https://www.diagrams.net/blog/custom-libraries), which covers building them interactively. When done, you'd end up with some data like this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8c582ec1-23ca-4e76-8733-cf8d8d7d8b46",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "library = [\n",
    "    {\n",
    "        \"w\": 80,\n",
    "        \"h\": 80,\n",
    "        \"aspect\": \"fixed\",\n",
    "        \"title\": \"Source\",\n",
    "        \"xml\": '<mxGraphModel><root><mxCell id=\"0\"/><mxCell id=\"1\" parent=\"0\"/><object label=\"\" type=\"Source\" interArrivalTime=\"\" id=\"2\"><mxCell style=\"rhombus;whiteSpace=wrap;html=1;\" vertex=\"1\" parent=\"1\"><mxGeometry width=\"80\" height=\"80\" as=\"geometry\"/></mxCell></object></root></mxGraphModel>',\n",
    "    },\n",
    "    {\n",
    "        \"w\": 80,\n",
    "        \"h\": 80,\n",
    "        \"aspect\": \"fixed\",\n",
    "        \"title\": \"Queue\",\n",
    "        \"xml\": '<mxGraphModel><root><mxCell id=\"0\"/><mxCell id=\"1\" parent=\"0\"/><object label=\"\" type=\"Queue\" capacity=\"\" id=\"2\"><mxCell style=\"ellipse;whiteSpace=wrap;html=1;\" vertex=\"1\" parent=\"1\"><mxGeometry width=\"80\" height=\"80\" as=\"geometry\"/></mxCell></object></root></mxGraphModel>',\n",
    "    },\n",
    "    {\n",
    "        \"w\": 80,\n",
    "        \"h\": 80,\n",
    "        \"aspect\": \"fixed\",\n",
    "        \"title\": \"Machine\",\n",
    "        \"xml\": '<mxGraphModel><root><mxCell id=\"0\"/><mxCell id=\"1\" parent=\"0\"/><object label=\"\" type=\"Machine\" processingTime=\"\" id=\"2\"><mxCell style=\"whiteSpace=wrap;html=1;\" vertex=\"1\" parent=\"1\"><mxGeometry width=\"80\" height=\"80\" as=\"geometry\"/></mxCell></object></root></mxGraphModel>',\n",
    "    },\n",
    "    {\n",
    "        \"w\": 80,\n",
    "        \"h\": 80,\n",
    "        \"aspect\": \"fixed\",\n",
    "        \"title\": \"Exit\",\n",
    "        \"xml\": '<mxGraphModel><root><mxCell id=\"0\"/><mxCell id=\"1\" parent=\"0\"/><object label=\"\" type=\"Exit\" id=\"2\"><mxCell style=\"rhombus;whiteSpace=wrap;html=1;\" vertex=\"1\" parent=\"1\"><mxGeometry width=\"80\" height=\"80\" as=\"geometry\"/></mxCell></object></root></mxGraphModel>',\n",
    "    },\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "708fb970-bade-40a2-a495-b861ebadfbd3",
   "metadata": {},
   "source": [
    "## JSON Encoding"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c5665cf-9a04-4f28-b27d-7bd9cf880b2a",
   "metadata": {},
   "source": [
    "Each element in the list has an `xml` attribute which is, as it suggests, XML, and must be carefully escaped. Because this will go through a URL parser, an XML parser, a JSON parser, and then another XML parser, the recommended approach is to use drawio's semi-convoluted base64/zlib technique."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3e9a519c-a868-4d71-885e-fcafe41915fa",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "zlib_opts = {\"wbits\": -15}\n",
    "\n",
    "\n",
    "def inflate(deflated):\n",
    "    infl = zlib.decompressobj(**zlib_opts)\n",
    "    return urllib.parse.unquote(\n",
    "        infl.decompress(base64.b64decode(deflated)) + infl.flush(),\n",
    "    )\n",
    "\n",
    "\n",
    "def deflate(inflated):\n",
    "    defl = zlib.compressobj(**zlib_opts)\n",
    "    return base64.b64encode(\n",
    "        defl.compress(urllib.parse.quote(inflated).encode(\"utf-8\")) + defl.flush(),\n",
    "    ).decode(\"utf-8\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a33277f-abd0-4b97-a803-ad2ed82a6222",
   "metadata": {},
   "source": [
    "Again, due to the number of parsers involved, it's best to avoid extra spaces in the data URI."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "63445943-ecea-441e-87f5-7d2f361b7c4d",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "library_json = json.dumps(\n",
    "    [dict(shape.items(), xml=deflate(shape[\"xml\"])) for shape in library],\n",
    "    separators=(\",\", \":\"),\n",
    ")\n",
    "library_json"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad6be53d-0d24-4e5e-99b4-33c9aaf531de",
   "metadata": {},
   "source": [
    "## XML Encoding\n",
    "\n",
    "The JSON is wrapped inside an XML document, with a top-level tag of `mxlibrary`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "82d193a9-2366-4df4-9263-84b65fa96c68",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "library_xml = f\"\"\"<mxlibrary>{library_json}</mxlibrary><!-- /my library -->\"\"\"\n",
    "library_xml"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76502de9-f2cb-4485-a438-acb6a06a8ca5",
   "metadata": {},
   "source": [
    "This, in turn, must be transformed into a [Data URI](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URIs). The whole thing can't be `base64` encoded, because drawio expects a semicolon-separated list of ids. \n",
    "\n",
    "> **GOTCHA**: Use of data URIs relies on a **NASTY PATCH** applied when packaging `@deathbeds/jupyterlab-drawio-webpack`: by default, the upstream would rewrite this into a proxied request, which `ipydrawio` don't support. Usually, the name of the library will be derived from the filename, which is usually the last path component after the `/` ... in this case, the _whole document_ is the path, so we make do with some hacks."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bcd588f0-100e-4dd9-96a7-ea0b16bc6a0f",
   "metadata": {},
   "source": [
    "## URL Encoding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3f062362-8691-42d9-b9d7-f4a29cf6c405",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "library_data_uri = f\"data:application/xml,{library_xml}\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca0d0acc-7186-4f81-83cf-fd93da83f59c",
   "metadata": {},
   "source": [
    "## URL Params\n",
    "Finally, the most reliable means of communicating with drawio is via its [URL parameters](https://www.diagrams.net/doc/faq/supported-url-parameters), exposed on the widget as `url_params`\n",
    "\n",
    "> **GOTCHA** `url_params` should be set before the widget is displayed to avoid extra dialogs. \n",
    "\n",
    "The `clibs` parameters accepts a list of \"library keys,\" each with different formats. We are interest in `U` (for `URL`) library.\n",
    "\n",
    "> Some others that might be worth exploring some time include `L` for `Local`, which works with an `IndexedDB` instance... but is not guaranteed to be configured by the time a document loads.\n",
    "\n",
    "Note, we also override the `stealth` default... `stealth` isn't _strictly_ going to worsen the privacy posture, as all of the other providers are still disabled."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5a89ad37-caf4-4174-90f0-ba51f7c98c3b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "url_params = dict(ipydrawio.Diagram._default_url_params(None))\n",
    "url_params.update(\n",
    "    clibs=f\"U{library_data_uri}\",\n",
    "    stealth=\"0\",\n",
    ")\n",
    "url_params"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40668d7d-0d2d-4727-add8-ef8d494f0b23",
   "metadata": {},
   "source": [
    "### More URL Params\n",
    "\n",
    "A number of other parameters can be useful for custom embedding purposes, such as using a `min`imal `ui`, hiding the default `libs`, disabling additional `p`lugins."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "938af1fe-a2b5-42a0-beee-c93d910b7dfd",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "url_params.update(ui=\"min\", libs=\"0\", p=\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b7169ce-668f-426c-8146-1c27aa4ef66d",
   "metadata": {},
   "source": [
    "## The Widget"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "91835331-2f68-48af-9ca1-c2512f25ded3",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "d = ipydrawio.Diagram(url_params=url_params, layout={\"height\": \"800px\"})\n",
    "d"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbb0e474-649d-4e33-96c3-5f4f3bad56bd",
   "metadata": {},
   "source": [
    "## Use The Source\n",
    "\n",
    "We should now be able to use the desired shapes in the diagram. Unlike `url_params`, the `value` of the diagram's `source` can be updated immediately."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5daa299b-08d8-4ad2-b35e-d599157612e4",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "d.source.value = \"\"\"<mxfile version=\"15.8.7\" type=\"embed\">\n",
    "    <diagram id=\"x\" name=\"My Diagram\">\n",
    "        <mxGraphModel dx=\"1687\" dy=\"681\" grid=\"1\" gridSize=\"10\" guides=\"1\" tooltips=\"1\" connect=\"1\" arrows=\"1\" fold=\"1\" page=\"1\" pageScale=\"1\" pageWidth=\"850\" pageHeight=\"1100\" math=\"0\" shadow=\"0\"><root><mxCell id=\"0\"/><mxCell id=\"1\" parent=\"0\"/><object label=\"apple\" type=\"Source\" interArrivalTime=\"\" id=\"2\"><mxCell style=\"rhombus;whiteSpace=wrap;html=1;\" vertex=\"1\" parent=\"1\"><mxGeometry x=\"100\" y=\"90\" width=\"80\" height=\"80\" as=\"geometry\"/></mxCell></object><object label=\"banana\" type=\"Queue\" capacity=\"\" id=\"3\"><mxCell style=\"ellipse;whiteSpace=wrap;html=1;\" vertex=\"1\" parent=\"1\"><mxGeometry x=\"250\" y=\"90\" width=\"80\" height=\"80\" as=\"geometry\"/></mxCell></object><object label=\"cherry\" type=\"Machine\" processingTime=\"\" id=\"4\"><mxCell style=\"whiteSpace=wrap;html=1;\" vertex=\"1\" parent=\"1\"><mxGeometry x=\"385\" y=\"90\" width=\"80\" height=\"80\" as=\"geometry\"/></mxCell></object><object label=\"date\" type=\"Exit\" id=\"5\"><mxCell style=\"rhombus;whiteSpace=wrap;html=1;\" vertex=\"1\" parent=\"1\"><mxGeometry x=\"510\" y=\"90\" width=\"80\" height=\"80\" as=\"geometry\"/></mxCell></object></root>\n",
    "        </mxGraphModel>\n",
    "    </diagram>\n",
    "</mxfile>\"\"\""
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}