From fa4de7551cbaf5e08f022d106605252d2a4332ec Mon Sep 17 00:00:00 2001
From: Simon Willison <swillison@gmail.com>
Date: Sat, 31 Oct 2020 14:37:58 -0700
Subject: [PATCH] Binary data documentation, closes #1047

---
 docs/binary_data.png | Bin 0 -> 5572 bytes
 docs/binary_data.rst |  68 +++++++++++++++++++++++++++++++++++++++++++
 docs/changelog.rst   |   2 +-
 docs/csv_export.rst  |   2 +-
 docs/index.rst       |   1 +
 5 files changed, 71 insertions(+), 2 deletions(-)
 create mode 100644 docs/binary_data.png
 create mode 100644 docs/binary_data.rst

diff --git a/docs/binary_data.png b/docs/binary_data.png
new file mode 100644
index 0000000000000000000000000000000000000000..2c5d0bdda89fc03e1632c9843ac9a11321e367bb
GIT binary patch
literal 5572
zcmd5=S5OlSmj&q^r9~p3f^-lR3B8DP5>QY<IuQhcP)s0Hr3gqznwS965~WKqh2A^T
zg^)<^O+cC;{NK#~whueIJF_$Y%RP6_+&OdRyquZ)&J3c@#>~%5MMcGCXz<_>71br!
zKYKSL%|FwnH2d@)FfcQ*&^`Why1nyjW@T%bywEl{QQ9_~Qq@&lT<lYqkNQ*`kY5*@
zgnnF_<d9tE7Z7e7R-o>Wg1UIfd!<XdCf_%-lT$Lg!hes3hDL2&QGtqz2W0p_+X6}2
z%uWdZsKohZ--{0z-a_2Dz`QsIc+*a82O>-w>#XJ947xNz3@XLk75EeM3Wi<(d#HG8
zVI*s9Mnl>)Nt-~+3_p8)q<gwgxY(F2n(*HChkiznF8Mz5>s#83rgrU0z1p=aDmG7Q
zhjIElO8t5jS>4o@P~bd6>+~7;o|v^O>rNP3*`M>)DDm2eY6F-p6adzv3Ss?8F?Em_
z=#259kA}RT%4@mtivgqy2)_S$*N639(ia1ek6P%N4yCj<SQn?i#KwWyyQXA!;k@~0
zpF1~Js1#<2Ymm1v8e`k#fVVkNsG3Uf2P^yf{T2OgJDT@NxfRn30Li5km{H%(T4(y)
z30?eIzU#9-9!kHS4^4TN%h{w*+i6q+O&$A7_2~q~bv@9RgbV}RwX)&ZorJtc$*{fe
zxvv71<U@$G8Zfx(8I!8e6@|;PaYIFci(m;zqRcm=Ct_ZOePRzNoPvBHlcc>~)6++6
zmsRE0FyzgKMW?8nn;*P=^;y9<TFYCvGxz7-E6c!7@!!xLvXn?uggz^@<?ZHVeljC?
zbdDawOb=nySCjQ}a#q26C`V~~a$LJ9SOp7=6xA37(;iU~gri=rFqaUWhZ94PW_Kxx
zEsBj^@3VYGk-aH#rXhpqkEO%6veY)7JlY`txf0#g!DhZ5qalbWz87>GrS_m+{6$29
ze3mm}zuNZDk@dii=(*u{#mBM%)|?1+UGZRm7LKA3H?Hp11{*vk{dfz#u6-EDC>^=D
z5bMth732k#v#;gbbQ&L|nxrr3c<FIeY<E~Jd6aj@I`2r^)Gb}Fd&Kp(93W}CWBWr!
zEu&>0?JqJN_$}_MfoP`$lUm<GxQ$^+{XG-G-ZanD1;ZB(X9f3g=2o3eFm&td(W*$Y
zIXPI-sr$BOJHv4*HL-(!2yV(+n~1e03VbS3i_AOd_S>NU;rQ~2B3w|VAZ9SSY&Gu#
z9A#g@CPmC*=qTfeW0$m5?>RP)p4mXEXf{5|{AzQ{;tOo>%h)^&v4OrT?WtWkEpMS>
zJvkz?DCe~-pgvY_OC;<?&wRJ8d!Cudv|%&FKU*GtJDrx9zSGN|04}lavNWIkqIRK_
zJ4M54bNjv=lN%X7#KuXAayaZX6SgLP1yb$^og(fQ@#6_1inUY6<{zuZ!(cszpB3C8
z-0ij$_SY#d#UCNFo&qv?x`<a*r<&sP)(!<I=QbCqb%;?PXg9GXHLZA6kjqC(m4<jV
zW6l5<K7-B>Y`fHGK$At%Or8t$!1l0G5$D1}WTF&5gj#`kLUWpsxDc&z8AFi=lb4_2
zrL)?IJ&!5<RrtB_F)62jtifUDKilie`7RT#KL_9uTJ>8sZyZ2QuI6vEiC+}%{XKpU
zJ0{7fPG(gK_-)8~P@4QkEkR~{vA`&yM0>l)<{hi>6t^4!Eh632m_h~qU7k#2uq|k^
z#Zie;V$FxUo}#kSJ5{yr7#&QTlhvx&DIEt~j>)u~!-Xi_ee20m%}~SnK5>Y1I0x`@
zsgEcRfE=xh&UdhLi`emp^s^-a1tCUO8`q~7mPt*vIQo#oOB!?dAj&`yuF@PZe@FsH
z?9njucA4#CrJKru8~*HAvr!MiQ%U+?;vR|Yg;warKA)H5k%3ZP+#~(t34#4Cf1cS!
z{ijAB_8je%T_3XOsvY~VrVn!E!&nA65D|0mCRi}560mx1&R6Y4Pn1lpl3|x{_5o0c
zrOVyP635ZW{#QPI6goLcxJ2!*Evy){qoP{Ki}g(vQZnw!h+wD|(5WrC{yGoHimgK!
z9~rSkyh#E_(ZpJV#I7n)`#JRo`t<QF{XOX&XS*zDDirkeIv&Vk<>o2}wJJ%Bv8tS4
z;1Oy!+mP%|57h;wEQJr?X&@?_qh-k(X*(Ho&m}_-4@I*247zY|=+7!dyK*uwp_O&l
zSc)^-^qaAze;-@_60tN|?E?y4`s1rhT^T^qj_$;MjwF6~t8h46-QkpRDmRFyd|<RI
z^!bcXN9N6pxRqW2VXx)6Rj8`tSR0`DZwd3`PvyExoV>qx9+|QxU}@Y>-GxR<#G*Pc
zB33JEo$rkvv=9B)R`{9JHadHz#W&J&q52WVLx1soc5|jhVy5DTfEp$oBAhKh(gKxS
zRT#M{{2xP{y-LZ93vT(hInB)T{pt$>WFcHr28uS}6{0N`OIxgd(vic$4xQ41fKE=2
zDzMI*@cg|49*#g5dO;>wvxh5<pLa4KdG(?j0>Rmvj)5Dnu!Yr}&#0?>toCDIo1_3+
z%I}?(HBnQqqG`_E9vo)ceiak?H}oTH4;d&$@lYO#S1uSKZ8N2;zWW7PwuRr-uFd_y
z#ylOgPH%7b8y@wmd?$ap0yCWn*Rf}hDSnSXA<@Z9T_U^|WwC#ggjg_&YaseGFosJN
z@LY%y*Iuko<d$F5N5Dq1KBI7P@lkpRPcWagVu&s~Ye=t5<6|OAD1UBasNulk@GjQA
zA%@EmCu^_XX?sqTmm=zro)+K!{xE$Ak@#|G7bzw5+wk@Zu_veLu>ZG~9Z20qH*B3=
zQ2c`={6|PfxwkjT9Dw{&RZt%Hev$7|Hqy0mLs_sYAe2t7xAhnQST*z$vO@HPyd~0J
z_=@6E+YlToIRp*~wX?GZt+U(^UEcUc?6FDG!gV`D%l{D00+OYx9bX(?=``xn!X@N!
z!{d;k=Q9pArYK)K%5#;8^pi3J?V?z*SLBGPu`P2R%GgTZh8~f)TbPo1Lh@X_`B3t<
z7oxf-7*1IJoc|_)jrzrNHT^bAVzITFnIollO;J^$J(@hv_X8#?)YtZvKGrvQ4aUD4
zf)h>nQ=gv?3plMs2-{E|+-Z*u>Z{MG22!q>+?u0qv>>-4vU52+J?NC!H!v=v3%@0+
z&_iHiScRFmK#ZlGb*mR)+j>aGv*Fm5u4)`ilpg{oEU^9211V1}wMM+UHT(%h1J)Yg
z(Go61)lKER?z+c7Y5qbMg>u^F+;D&Q^0?S*<<}0+ILt|17o^^P#Rb5bGtFoN8bF}H
z>@P<6LE$i8<+7!(ZahilWQVf`8ML4Y10{b@h3i4wT$Ezy5i?>)s*z``Z|3eTD~iZ^
z$cL=^+P^*lmFQ8eN}BXqxD#$Fc3W*{iOe_o4upQFi{fQCFcUr(ABlS?@O#DB;{-#J
z`GZ}>>{~d1K_8e)H(a`CSLtDE3hQjMT$gy2Dq(SRy!6i5apYo{AK^~xRkOp-34bt*
z!8aqHBh1anl`!v^siA?QTMPvO9&b|E%hx){VkQKGHsZSg1kF}!ui%JhCD)*I$+kDP
zz9-@2{wdW67u>oI9)SSi<7zS1By~ntD$1v_2wqm{Ob`3IllgRZ&Ba2XWrD=$L(Uk~
zMjp_(aozW!h{ldy6*2Ff+ZR}O)IAETg=`$$_<lF_mBcW2^-`1$om@RW$Eo@5sKVon
zU53UrDWd$Z*(hEnit7_l&1MvQzXgk+Xz|M5q`YVRs!(>9JTu_f9S1=f3i9Q=fh|$d
z1-=U)7!zAst;GNZ6~OUz-Zh69tJTh0j;)frqg9uzWt-JUWY9Q+hS$Ms^u&bX?nQ0L
z=P0MD{!F>Z+;i^Lj{F?oQRg<*RXrS0*Q9^p)1&juUu+bM-wHpsdppBnsf-05BqD1h
z73U9YE?L5sA%|MK3HnH$R^u)`ES%lF(pKf<Pq27uK4>@rjzDSXk2zB4dnT9rYdFEH
zyu=pI+h(hCCe(=nv$a-^#Q|m;tmm|hYNj0n%kvF-7w=Z>pMmDbWv<JN45-pPDU_Z^
z;GBx;uqbgAM~XwG^)MKIQp;0QcY>5AFJ=NVy4L55)8#FtCh2qik$FvM<K>++4c7D1
zCf=u=B90bF+UC(Fu?9t|RH}G6z1=Bu=lCb9eABiRJMjgt;_2!O_SU0^M;Y-m<)e1%
z0}^x<4y9+Zm90_8YDiESNQ|p~*yT)H{W+Fm8#GfqYPUVmLu)S@rTM`7sJy{z-0k+!
z^7HU$p6cAuff=UrGL@~RpJ<a`oyt;ENjJ)>{g*;YAVCCBg$mMyE?Z9pS{Tt~G2~Ww
zSMUFc=<kD*=G?qPKe4RMUhsf%k`Ya9C94yi1xGA`=GxelLBQIqJP(|IEk1?)KZQ6a
zboc2tI{e#QuJOgm)P*H>_k~M5gk9)2I>+q&_h%RGJ2q+8IvmAgb8U??p$y+tE-tXD
zp-UgdAxF|?5v}Bp>xPWfx+*C$+P#63PsidNZ#=;>@s04C1}4RX*pxTHPXLyy!0!WP
zqS3Wcep-^!i_e66_T<ps!cucy0Qmp{ukBc~mSzjvp6a~I%G=f?pSC?%bUXO_L#{KN
zV|#n;<>(s+4EVQ&^A%0_zp}7AX&kJwvfbtKX}lYlJtEd{`;}5-W7ojA`j*YjVl>dd
zvYqS<b@)jO;&N+qT1M%a=#h**E_}tTefMdbpBc)LqG@D;vHLmRI8V6g5-N`|>S%^4
z{1qTuoac{}ibU)6S{6jT9ou<arNLIj4a!S6;{{n3k+39wnR8m0pkR%hk7s+cW$Nm0
zZzMt;TMx#AVV57G#G1?xbAV(8eXCN*oTZl<*BDfniW(3!Nrtbn+ai!h<jy-mY0htD
zW<)5-_9atKMM^kdn3pn(ZtaYLGzg!0L5Ryz-Ymzl#4?}Lc&G#~;|nvCpW2K3%w$S)
zm|dyvj-+izkxQdg%;v0*NpczuwV1p=x?(k1ruVW|TH7!F;a9KHZT$oV<2X?>r$o9#
zM4j6##tT=$gTLy|)6Mcv3?ac|9D5uK1QdApz8?+4=+Jxyx6KD*jYrNA`p?bnq-?&G
z{5<s8R}jyajS^z^0{qz58FvAtA05;-rcZrF?^=9Soy^2j;2q$ZC$oC7^f;Y%GlgsR
zW5^Wwz2bgK>xVSeh4sszr=wk#zKP|G_?jEN*6BZ?av6s{cb5R%AajPoHzr|BO3Sdu
zO(~nmMiFQ3cZl}fPIiTNl_&^4y-UhRXPNJGR<uG518YZSG5Nf&3HTc78iCT6cz0p+
ztOP$rU0voS+CP~zk3sDz3F(f5;{D0ZQdf!?Vh$#mbF02vx3krhbquufpcXI{CL|a~
zr<A7tT#%+_x7sX**;YAm@RZ+mb~%h`Xp<3enS9o^;OgQ@Vwt7K^eWy{f;YxL6p8i^
zI0c>4c$RplQrCDUppKvGOt&IagYQgH$9}iHvJ94ifqf>m-2bwY$Er$sCdUk6Vf<Fw
z_M(GKg*3RxY6+uHS?Hd4q>2t3rp-ECaALtFZs)KWY+rDG)snGOYCgmQW5CGeO#?-W
zmzhSpdAl1^ZIV(j$_`=T;!mCXr5$Nu`;VZ(`V)(9Ag6!dS0G0>{=f`8?L2_;g^Gge
z$qWl%3A#Xh9{1|P^0M?S{WlZAgG<Pp_ah?azjbp1(!hqkwq%=&ocKSnq0Abt)^DxL
z(gtKlg?!#s$`ejq9qaXc3-GlBY!d3b41@av(m#FsD)5A;R+6d^i}I)jR>ozkhg*C5
z2h{Zw=L`eQE7nKH@n2ZgYYqf%@F`cOaNfqiR_j_`K{;?|zR8nR<Vw$WUdbBcKd(SG
z0VY5$86J=s_;KGuDiIHn%9*`!d;7ow)%*RB>Mc`^YDIu4i}^>2mW~i8u74>5p9f)O
zL|z2?;qK^Ol4bz;16`dz8at<-55+b5q6^TZlf)nvl48<_MLl3V))N#v%^k45KySHd
z;@x+>aBS>pb6?>OhEpRB7&y;0`^0~!+4$R<>_SxMw7CCS7{i~KAl=Q>uJn@39H4x=
zv^SSa>~vka+g9@g)A0lExE<@r&~;OaIIOx{4*Z5|%Iv2Es`#oe@ij|-{uit|hAn6z
zV-TJ>t>BcoVe4_*i7f)`!^-b0C}KB@;li%@J{9G~-%uT_(vER>?7C@lSLTPh=k$PY
zAHK7du!;I`r<UHg0JSH9(0W*-3JZkQ9Q=hf0ib?IO1sy#h5035+H7roy)}|4a<blS
z&bP0Rm2r#1sJjG<x0N{vgCnY1ztqG)(Tfzq_k`8GCZB56dC>~a{knj3VLbR}c<iPj
z$-l%RaF*V1_JMbs!w<K}_@zlU@y3MQ-~NryKcD6=>p99{zPQ=r#8ifoURhn-v5Wz0
zW(*x1%6c*>xxY3~O=8Pq{Pcd}<*?fL1V30~GO2nx?B+QJzx5Xwc`)KyujRE;qmzwJ
z-Q|ZaJ<Ds;`|Sczo#>eYcqL?m+I12-HH(hQ(0=&TYWYUqC}7WFX!`jRh${J?i=@OU
zKRnp!dY}|mV0+Oc9`{rAo`>jCeGv1Gweps?m}c#+g_t-0M$A{moON0VR6u^B(>%M`
ztyH*^ge>M#N<g$*0!<N@uwN4a?h9WavkDWJp`L2vOz2J(r_uzhO4z8Se*FJT1b_|M
z6uqHw99z5l+UT{9YSFGKoSS@}sK4{#|0ejqaAm5rrJ3<A{K-R&8+ZSC%TgKYLLQXg
HcL@I%(U9HO

literal 0
HcmV?d00001

diff --git a/docs/binary_data.rst b/docs/binary_data.rst
new file mode 100644
index 00000000..593cf78d
--- /dev/null
+++ b/docs/binary_data.rst
@@ -0,0 +1,68 @@
+.. _binary:
+
+=============
+ Binary data
+=============
+
+SQLite tables can contain binary data in ``BLOB`` columns.
+
+Datasette includes special handling for these binary values. The Datasette interface detects binary values and provides a link to download their content, for example on https://latest.datasette.io/fixtures/binary_data
+
+.. image:: binary_data.png
+   :width: 311px
+   :alt: Screenshot showing download links next to binary data in the table view
+
+Binary data is represented in ``.json`` exports using Base64 encoding.
+
+https://latest.datasette.io/fixtures/binary_data.json?_shape=array
+
+.. code-block:: json
+
+    [
+        {
+            "rowid": 1,
+            "data": {
+                "$base64": true,
+                "encoded": "FRwCx60F/g=="
+            }
+        },
+        {
+            "rowid": 2,
+            "data": {
+                "$base64": true,
+                "encoded": "FRwDx60F/g=="
+            }
+        },
+        {
+            "rowid": 3,
+            "data": null
+        }
+    ]
+
+.. _binary_linking:
+
+Linking to binary downloads
+---------------------------
+
+The ``.blob`` output format is used to return binary data. It requires a ``_blob_column=`` querystring argument specifying which BLOB column should be downloaded, for example:
+
+https://latest.datasette.io/fixtures/binary_data/1.blob?_blob_column=data
+
+This output format can also be used to return binary data from an arbitrary SQL query. Since such queries do not specify an exact row, an additional ``?_blob_hash=`` parameter can be used to specify the SHA-256 hash of the value that is being linked to.
+
+Consider the query ``select data from binary_data`` - `demonstrated here <https://latest.datasette.io/fixtures?sql=select+data+from+binary_data>`__.
+
+That page links to the binary value downloads. Those links look like this:
+
+https://latest.datasette.io/fixtures.blob?sql=select+data+from+binary_data&_blob_column=data&_blob_hash=f3088978da8f9aea479ffc7f631370b968d2e855eeb172bea7f6c7a04262bb6d
+
+These ``.blob`` links are also returned in the ``.csv`` exports Datasette provides for binary tables and queries, since the CSV format does not have a mechanism for representing binary data.
+
+Binary plugins
+--------------
+
+Several Datasette plugins are available that change the way Datasette treats binary data.
+
+- `datasette-render-binary <https://github.com/simonw/datasette-render-binary>`__ modifies
+- https://github.com/simonw/datasette-render-images
+- https://github.com/simonw/datasette-media
\ No newline at end of file
diff --git a/docs/changelog.rst b/docs/changelog.rst
index 262400c8..fc566a37 100644
--- a/docs/changelog.rst
+++ b/docs/changelog.rst
@@ -9,7 +9,7 @@ Changelog
 0.51a2 (2020-10-30)
 -------------------
 
-- New :ref:`plugin_hook_load_template` plugin hook. (`#1042 <https://github.com/simonw/datasette/issues/1042>`__)
+- New ``load_template`` plugin hook. (`#1042 <https://github.com/simonw/datasette/issues/1042>`__)
 - New :ref:`permissions_debug_menu` permission. (`#1068 <https://github.com/simonw/datasette/issues/1068>`__)
 
 .. _v0_51_a1:
diff --git a/docs/csv_export.rst b/docs/csv_export.rst
index 9b7f8188..b5cc599a 100644
--- a/docs/csv_export.rst
+++ b/docs/csv_export.rst
@@ -1,6 +1,6 @@
 .. _csv_export:
 
-CSV Export
+CSV export
 ==========
 
 Any Datasette table, view or custom SQL query can be exported as CSV.
diff --git a/docs/index.rst b/docs/index.rst
index 9096efd9..6b55da8c 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -46,6 +46,7 @@ Contents
    authentication
    performance
    csv_export
+   binary_data
    facets
    full_text_search
    spatialite