mirror
/
datasette
kopia lustrzana https://github.com/simonw/datasette
1
0
Forkuj 0
Kod Zgłoszenia Packages Projekty Wydania Wiki Aktywność
2 404 Commity
134 Gałęzie
158 Tagi
6,5 GiB
Wykres commitów

899 Commity (json_renderer_refactor)

Autor SHA1 Wiadomość Data
Simon Willison eae103a82b Write errors to stderr, closes #1131 2020-12-04 21:21:11 -08:00
Simon Willison 42efb799ea Fixed invalid test for generated columns, refs #1119 2020-12-04 21:20:12 -08:00
Simon Willison 13c960c03b Test is no longer order dependent, closes #1123 2020-12-02 16:49:55 -08:00
Simon Willison a45a3dff3e Fix for OPTIONS request against /db, closes #1100 2020-12-02 16:49:55 -08:00
Simon Willison 88ac538b41 transfer-encoding: chunked for DB downloads, refs #749 
This should get >32MB downloads working on Cloud Run.
 2020-12-02 15:47:37 -08:00
Simon Willison 17cbbb1f7f
generated_columns table in fixtures.py, closes #1119 2020-11-30 16:28:02 -08:00
Simon Willison 461670a0b8
Support for generated columns 
* Support for generated columns, closes #1116
* Show SQLite version in pytest report header
* Use table_info() if SQLite < 3.26.0
* Cache sqlite_version() rather than re-calculate every time
* Adjust test_database_page for SQLite 3.26.0 or higher
 2020-11-30 13:29:57 -08:00
Simon Willison dea3c508b3 Revert "Support for generated columns, closes #1116" - it failed CI 
This reverts commit 37f87b5e52.
 2020-11-30 12:09:32 -08:00
Simon Willison 37f87b5e52 Support for generated columns, closes #1116 2020-11-30 12:01:15 -08:00
Simon Willison 4777362bf2 Work around CI bug with ensure_eventloop, refs #1115 2020-11-29 12:19:24 -08:00
Simon Willison 09033c08be Suggest --load-extension=spatialite, closes #1115 2020-11-29 12:13:16 -08:00
Simon Willison deb0be4ae5 Fix bug where compound foreign keys produced broken links, closes #1098 2020-11-29 11:30:17 -08:00
Simon Willison 37d18a5bce datasette publish cloudrun --apt-get-install, closes #1110 2020-11-24 19:05:35 -08:00
Simon Willison f2e2bfcdd9 Renamed datasette.config() to .setting(), closes #1107 2020-11-24 14:06:32 -08:00
Simon Willison 5a77f7a649 Updated docs renaming config to settings 
- config.html is now settings.html
- ConfigOption in app.py is now Setting
- updated documentation unit tests

Refs #1106
 2020-11-24 13:22:33 -08:00
Simon Willison 33eadb8782 config.json is now settings.json, closes #1104 2020-11-24 12:37:29 -08:00
Simon Willison 2a3d5b720b Redirect /-/config to /-/settings, closes #1103 2020-11-24 12:19:14 -08:00
Simon Willison 3159263f05 New --setting to replace --config, closes #992 2020-11-24 12:01:47 -08:00
Simon Willison 30e64c8d3b
Use f-strings in place of .format() 
Code transformed like so:

    pip install flynt
    flynt .
    black .
 2020-11-15 15:24:22 -08:00
Simon Willison e8e0a6f284
Use FTS4 in fixtures 
Closes #1081
 2020-11-11 16:02:58 -08:00
Simon Willison 2a981e2ac1 Blank foreign key labels now show as hyphens, closes #1086 2020-11-11 15:44:04 -08:00
Simon Willison 7b19492070 database_actions() plugin hook, closes #1077 2020-11-02 10:27:25 -08:00
Simon Willison 6bb41c4b33 Fix for test_paginate_using_link_header 2020-10-31 13:48:39 -07:00
Simon Willison a4ca26a265 Address PrefixedUrlString bug in #1075 2020-10-31 13:35:47 -07:00
Simon Willison bf18b9ba17 Stop using plugin-example.com, closes #1074 2020-10-31 12:47:42 -07:00
Simon Willison 84bc7244c1 datasette.client now applies base_url, closes #1026 2020-10-31 12:29:42 -07:00
Simon Willison 7a67bc7a56 datasette.urls methods will not apply base_url prefix twice, refs #1026 2020-10-31 12:11:40 -07:00
Simon Willison 11eb1e026f datasette.urls.table(..., format="json"), closes #1035 
Also improved tests for datasette.urls and added format= to some other methods
 2020-10-31 11:16:28 -07:00
Simon Willison b84cfe1b08 Confirm table actions work on views, closes #1067 2020-10-31 10:40:09 -07:00
Simon Willison d6db47f5c1 Deploy demo plugins to latest.datasette.io, refs #1074 2020-10-31 10:36:46 -07:00
Simon Willison f0a740ac21 Remove load_plugin hook - closes #1073 
Refs #1042

This reverts commit 81dea4b07a.
 2020-10-31 09:21:22 -07:00
Simon Willison 81dea4b07a
load_template() plugin hook 
Closes #1042
 2020-10-30 10:47:18 -07:00
Simon Willison 222f79bb4c debug-menu permission, closes #1068 
Also added tests for navigation menu logic.
 2020-10-30 08:41:57 -07:00
Simon Willison 2f7731e9e5 table_actions() plugin hook plus menu, closes #1066 
Refs #690
 2020-10-29 22:16:41 -07:00
Simon Willison 18a64fbb29
Navigation menu plus menu_links() hook 
Closes #1064, refs #690.
 2020-10-29 20:45:15 -07:00
Simon Willison 1a861be19e Fixed test_max_csv_mb test that I just broke, refs #1063 2020-10-29 15:58:40 -07:00
Simon Willison 178b7e8749 .csv now links to .blob downloads 
Closes #1063, closes #1034
 2020-10-29 15:47:32 -07:00
Simon Willison 78b3eeaad9
.blob output renderer 
* _blob_hash= checking plus refactored to use new BadRequest class, refs #1050
* Replace BlobView with new .blob renderer, closes #1050
* .blob downloads on arbitrary queries, closes #1051
 2020-10-29 15:01:38 -07:00
Simon Willison 89519f9a37 Fixed bug with download of BLOB null, refs #1050 2020-10-28 21:05:40 -07:00
Simon Willison cefd058c1c
New explicit versioning mechanism 
Closes #1054
 2020-10-28 20:38:15 -07:00
Simon Willison 7d9fedc176 Cascading permissions for .db download, closes #1058 2020-10-27 20:15:41 -07:00
Simon Willison c3aba4aa98 --cors for /name.db downloads, refs #1057 2020-10-27 13:39:57 -07:00
Simon Willison e5f5034bcd Fixed broken footer test 2020-10-27 12:39:55 -07:00
Simon Willison f5dbe61a45 -o now opens to most relevant page, closes #976 2020-10-25 22:06:20 -07:00
Simon Willison 5db7ae3ce1 Link to BLOB downloads, closes #1046 2020-10-24 17:13:14 -07:00
Simon Willison 5a15197960
/db/table/-/blob/pk/column.blob download URL, refs #1036 2020-10-24 16:09:18 -07:00
Simon Willison 10c35bd371 urls.static_plugins() method, closes #1033 
Also documented how to package static assets and templates in plugins, closes #575
 2020-10-24 13:03:40 -07:00
Simon Willison 7f728d4a37 Extra tests for datasette.urls, refs #1025 2020-10-24 12:21:23 -07:00
Simon Willison 29a977a74e New app_client_base_url_prefix fixture 2020-10-24 12:03:24 -07:00
Simon Willison 091441a444 Fixed remaining places that needed datasette.urls, closes #1025 2020-10-19 22:21:19 -07:00
Simon Willison 310c3a3e05 New datasette.urls URL builders, refs #904 2020-10-19 17:33:59 -07:00
Simon Willison 6aa5886379 --load-extension=spatialite shortcut, closes #1028 2020-10-19 15:37:43 -07:00
Simon Willison f3a087a578 Edit SQL button on canned queries, closes #1019 2020-10-13 20:44:18 -07:00
Simon Willison e34e84901d Link: HTTP header pagination, closes #1014 2020-10-10 17:18:45 -07:00
Simon Willison 7e70643852 Removed --debug option, which didn't do anything - closes #814 2020-10-10 16:39:38 -07:00
Simon Willison 7239175f63 Fixed broken column header links, closes #1011 2020-10-09 20:51:56 -07:00
Simon Willison 9f6dd985bc Fix broken CSV/JSON export on query page, refs #1010 2020-10-09 17:39:45 -07:00
Simon Willison c13d184704 Emergency fix for broken links in 0.50, closes #1010 2020-10-09 17:33:13 -07:00
Simon Willison 896cc2c6ac Replace MockRequest with Request.fake() 
Close #1004
 2020-10-09 09:26:17 -07:00
Simon Willison 8f97b9b58e
datasette.client internal requests mechanism 
Closes #943

* Datasette now requires httpx>=0.15
* Support OPTIONS without 500, closes #1001
* Added internals tests for datasette.client methods
* Datasette's own test mechanism now uses httpx to simulate requests
* Tests simulate HTTP 1.1 now
* Added base_url in a bunch more places
* Mark some tests as xfail - will remove that when new httpx release ships: #1005
 2020-10-09 09:11:24 -07:00
Simon Willison 7249ac5ca0 Support OPTIONS without 500, closes #1001 2020-10-08 18:43:53 -07:00
Simon Willison e4554c37b7 datasette publish heroku --tar option, closes #969 2020-10-08 16:30:46 -07:00
Simon Willison 86823ae6f7 Default to Uvicorn workers=1, refs #999 2020-10-08 16:16:55 -07:00
Simon Willison b47ac37114 Applied Black 2020-10-07 15:51:25 -07:00
Simon Willison 5070425817 Fix handling of nested custom page wildcard paths, closes #996 2020-10-07 15:51:11 -07:00
Simon Willison b37431976c custom pages tests templates now in repo 2020-10-07 15:16:41 -07:00
Simon Willison e02f6c1300 Tests for db.table_columns() and db.table_column_details() 2020-10-06 14:02:30 -07:00
Simon Willison 5a184a5d21 Display column type in column action menu, closes #993 
Also added new documented db.table_column_details() introspection method.
 2020-10-05 17:32:10 -07:00
Simon Willison 0f2626868b Much improved column menu display logic, refs #981 
* Menu links now take into account existing querystring
* No longer shows facet option for primary key columns
* Conditionally displays sort/sort-desc if already sorted
* Does not show facet option if already faceted by this
 2020-09-30 16:01:37 -07:00
Simon Willison 97c71c3a3b Fixed test for column sorting, refs #981 2020-09-30 14:51:10 -07:00
Simon Willison 5b8b8ae597 Handle \r\n correctly in CSS escapes, refs #980 2020-09-29 12:16:30 -07:00
Simon Willison c11383e628 Fix rendering glitch with columns on mobile, closes #978 2020-09-28 15:42:50 -07:00
Simon Willison cac051bb8a Fix for 'open' bug, closes #973 2020-09-22 08:39:48 -07:00
Simon Willison a980199e61 New -o option for opening Datasette in your browser, closes #970 2020-09-22 07:26:47 -07:00
Simon Willison 448d13ea6b Fix for MagicParameters error with no POST body, closes #967 2020-09-15 13:12:57 -07:00
Simon Willison 72ac2fd32c JSON API for writable canned queries, closes #880 2020-09-14 14:23:18 -07:00
Simon Willison 894999a14e Improved test for JSON POST, refs #880 2020-09-14 13:25:09 -07:00
Simon Willison 896fce228f Canned query writes support JSON POST body, refs #880 2020-09-14 13:18:15 -07:00
Simon Willison 699be7dea9 raise_404() function for use in custom templates, closes #964 2020-09-14 10:39:25 -07:00
Simon Willison cc77fcd133 Optional path parameters for custom pages, closes #944 2020-09-13 19:34:43 -07:00
Simon Willison ea340cf320 Correctly persist selected facets in hidden fields 
Closes #963
 2020-09-12 14:54:01 -07:00
Simon Willison 20b1de86a1 Fix for test I broke in #947 2020-09-11 15:04:23 -07:00
Simon Willison d02f6151da datasette --get status code for error pages, closes #947 2020-09-11 14:32:54 -07:00
Simon Willison ca5c405d0f New 'datasette --pdb' option, closes #962 2020-09-11 11:37:55 -07:00
Simon Willison a648bb82ba Upgrade to Black 20.8b1, closes #958 2020-09-02 15:24:55 -07:00
Simon Willison 799ecae948 register_output_renderer can now return Response, closes #953 2020-08-27 21:02:50 -07:00
Simon Willison 69033c6ec4 datasette install --upgrade option, closes #945 2020-08-19 10:20:41 -07:00
Simon Willison 8e7e6458a6 Fix bug with ?_nl=on and binary data, closes #914 2020-08-16 11:26:49 -07:00
Simon Willison 3a4c8ed36a Added columns argument to various extra_ plugin hooks, closes #938 2020-08-16 11:09:53 -07:00
Simon Willison 94ae840fe3 Plugin tests now start with test_hook_ 2020-08-16 10:49:33 -07:00
Simon Willison 2da4144c57 Applied Black 2020-08-16 10:35:14 -07:00
Simon Willison ac69d151c3 Test that plugin hooks are documented with correct arguments 2020-08-16 10:33:44 -07:00
Simon Willison e3639247cd Standard arguments for extra_ plugin hooks, closes #939 2020-08-16 09:50:23 -07:00
Simon Willison 41ddc19756
Docs now live at docs.datasette.io (#937) 2020-08-15 16:57:05 -07:00
Simon Willison af12f45c2b Documentation and tests for db.is_mutable 2020-08-15 16:27:32 -07:00
Simon Willison b86f94883b
Don't hang in db.execute_write_fn() if connection fails 
Closes #935

Refs https://github.com/simonw/latest-datasette-with-all-plugins/issues/3
 2020-08-15 15:35:31 -07:00
Simon Willison 45414f8412 --get now calls startup() plugin hooks, closes #934 2020-08-15 13:52:41 -07:00
Simon Willison e139a7619f
'datasette --get' option, closes #926 
Also made a start on the datasette.utils.testing module, refs #898
 2020-08-11 17:24:40 -07:00
Simon Willison afdeda8216 Use runpy in install/uninstall, refs #928 2020-08-11 16:54:52 -07:00
Simon Willison 01fe5b7401 datasette install / datasette uninstall commands, closes #925 2020-08-11 15:32:06 -07:00
Simon Willison 7f10f0f766 Fix for security issue #918 2020-08-09 09:03:35 -07:00
Simon Willison 0748a65a22 Fixed content-disposition header on DB download, closes #909 2020-07-29 14:34:22 -07:00
Simon Willison 092874202c Improvements to allow block logic and debug tool 
true and false allow block values are now supported, closes #906

Added a bunch of demo links to the documentation, refs #908
 2020-07-24 17:04:06 -07:00
Simon Willison 12c0bc09cc /-/allow-debug tool, closes #908 2020-07-24 15:55:10 -07:00
Simon Willison 213e6a8926 content-length for DB downloads, closes #905 2020-07-21 21:52:35 -07:00
Simon Willison 1f6a134369 await request.post_body() method, closes #897 2020-07-17 13:12:35 -07:00
Amjith Ramanujam ea99a4431c
Only load Python files from plugins-dir 
Pull request #890. Thanks, @amjith!

* Load only python files from plugins-dir
* Add a test to verify non-python files are not loaded as plugins
 2020-07-02 20:08:32 -07:00
Simon Willison 57879dc8b3 Better titles for canned query pages, closes #887 2020-07-01 17:23:37 -07:00
Simon Willison 1bae24691f Only show 'log out' if ds_cookie present, closes #884 2020-07-01 14:25:59 -07:00
Simon Willison 549b1c2063 New forbidden() plugin hook, closes #812 2020-06-30 21:17:38 -07:00
Simon Willison 3ec5b1abf6 CSRF tests for canned query POST, closes #835 2020-06-30 20:08:00 -07:00
Simon Willison d6e03b0430 Cascading view permissions, closes #832 
- If you have table permission but not database permission you can now view the table page
- New BaseView.check_permissions() method
 2020-06-30 16:40:50 -07:00
Simon Willison 9ac6292614 _header_x now defaults to empty string 
Prior to this a request to e.g. https://latest.datasette.io/fixtures/magic_parameters
which did not include a User-Agent header would trigger a 500 error.
 2020-06-30 15:00:17 -07:00
Simon Willison 2115d7e345 Logout link in nav, refs #875 2020-06-29 11:40:40 -07:00
Simon Willison 51427323e6 Add message when user logs out, refs #840 2020-06-29 11:31:35 -07:00
Simon Willison 35aee82c60 Fixed 500 error with /favicon.ico, closes #874 2020-06-28 21:27:11 -07:00
Simon Willison 22d932fafc /-/logout page for logging out of ds_actor cookie 
Refs #840
 2020-06-28 21:17:58 -07:00
Simon Willison a8a5f81372 Made show_messages available to plugins, closes #864 2020-06-28 17:50:47 -07:00
Simon Willison 7ac4936cec .add_message() now works inside plugins, closes #864 
Refs #870
 2020-06-28 17:25:35 -07:00
Simon Willison 0991ea75cc Renamed _timestamp to _now, refs #842, closes #871 2020-06-28 12:47:28 -07:00
Simon Willison 1f55a4a2b6 Release notes for 0.45a3 2020-06-27 20:22:49 -07:00
Simon Willison 335f26a0f7 /fixtures/magic_parameters demo, refs #842 2020-06-27 20:11:01 -07:00
Simon Willison 563f5a2d3a
Magic parameters for canned queries 
Closes #842

Includes a new plugin hook, register_magic_parameters()
 2020-06-27 19:58:16 -07:00
Simon Willison 4b142862f2 Support non-async view functions, closes #867 2020-06-27 11:30:34 -07:00
Simon Willison 1a5b7d318f Fixed test I broke in #863 2020-06-23 21:17:30 -07:00
Simon Willison 28bb1c5189 csrftoken() now works with .render_template(), closes #863 2020-06-23 20:23:50 -07:00
Simon Willison 751e7b4af7 Update tests for new plugin_hooks.rst, refs #687 2020-06-21 19:41:07 -07:00
Simon Willison b59b92b1b0 Fix for tests - order was inconsistent, refs #852 2020-06-18 16:52:16 -07:00
Simon Willison 6c26345836 New plugin hook: canned_queries(), refs #852 2020-06-18 16:35:15 -07:00
Simon Willison 6151c25a5a Respect existing scope["actor"] if set, closes #854 2020-06-18 11:37:28 -07:00
Simon Willison d2aef9f7ef Test illustrating POST against register_routes(), closes #853 2020-06-18 09:21:15 -07:00
Simon Willison a4ad5a504c Workaround for 'Too many open files' in test runs, refs #846 2020-06-13 17:26:18 -07:00
Simon Willison 0e49842e22 datasette/actor_auth_cookie.py coverae to 100%, refs #841 2020-06-13 11:29:14 -07:00
Simon Willison d60bd6ad13 Update plugin tests, refs #834 2020-06-13 11:15:33 -07:00
Simon Willison 72ae975156 Added test for async startup hook, refs #834 2020-06-13 10:58:32 -07:00
Simon Willison 09a3479a54 New "startup" plugin hook, closes #834 2020-06-13 10:55:41 -07:00
Simon Willison 308bcc8805 Fixed test_permissions_debug 2020-06-11 17:25:12 -07:00
Simon Willison fba8ff6e76 "$env": "X" mechanism now works with nested lists, closes #837 2020-06-11 17:21:48 -07:00
Simon Willison f39f111331 Fixed actor_matches_allow bug, closes #836 2020-06-11 15:47:19 -07:00
Simon Willison 29c5ff493a view-instance permission for debug URLs, closes #833 2020-06-11 15:14:51 -07:00
Simon Willison 98632f0a87
--secret command for datasette publish 
Closes #787
 2020-06-11 09:02:03 -07:00
Simon Willison 198545733b Document that "allow": {} denies all 
https://github.com/simonw/datasette/issues/831#issuecomment-642324847
 2020-06-10 16:56:53 -07:00
Simon Willison 57e812d5de ds_author cookie can now expire, closes #829 
Refs https://github.com/simonw/datasette-auth-github/issues/62#issuecomment-642152076
 2020-06-10 12:39:54 -07:00
Simon Willison 008e2f63c2 response.set_cookie(), closes #795 2020-06-09 15:19:37 -07:00
Simon Willison f240970b83 Fixed tests/fixtures.py, closes #804 2020-06-09 12:58:12 -07:00
Simon Willison 5ef3b7b0c9 Applied Black 
Refs #825
 2020-06-09 12:25:48 -07:00
Simon Willison 7633b9ab24 unauthenticated: true method plus allow block docs, closes #825 2020-06-09 10:01:03 -07:00
Simon Willison eb3ec279be
Test for anonymous: true, refs #825 2020-06-08 23:33:06 -07:00
Simon Willison fac8e93815 request.url_vars property, closes #822 2020-06-08 20:40:00 -07:00
Simon Willison db660db463 Docs + unit tests for Response, closes #821 2020-06-08 20:32:10 -07:00
Simon Willison f5e79adf26
register_routes() plugin hook (#819) 
Fixes #215
 2020-06-08 20:12:06 -07:00
Simon Willison 647c5ff0f3 Fixed broken CSS on 404 page, closes #777 2020-06-08 17:35:23 -07:00
Simon Willison 49d6d2f7b0 allow_sql block to control execute-sql upermission in metadata.json, closes #813 
Also removed the --config allow_sql:0 mechanism in favour of the new allow_sql block.
 2020-06-08 17:05:44 -07:00
Simon Willison 040fc0546f Updated tests, refs #817 2020-06-08 12:02:56 -07:00
Simon Willison 799c5d5357 Renamed resource_identifier to resource, refs #817 2020-06-08 11:59:53 -07:00
Simon Willison c9f1ec616e Removed resource_type from permissions system, closes #817 
Refs #811, #699
 2020-06-08 11:51:03 -07:00
Simon Willison 5598c5de01 Database list on index page respects table/view permissions, refs #811 2020-06-08 11:34:14 -07:00
Simon Willison dcec89270a View list respects view-table permission, refs #811 
Also makes a small change to the /fixtures.json JSON:

    "views": ["view_name"]

Is now:

    "views": [{"name": "view_name", "private": true}]
 2020-06-08 11:20:59 -07:00
Simon Willison 9ac27f67fe Show padlock on private query page, refs #811 2020-06-08 11:13:32 -07:00
Simon Willison aa420009c0 Show padlock on private table page, refs #811 2020-06-08 11:07:11 -07:00
Simon Willison 2a8b39800f Updated tests, refs #811 2020-06-08 07:50:06 -07:00
Simon Willison 3ce7f2e7da Show padlock on private database page, refs #811 2020-06-08 07:23:10 -07:00
Simon Willison 1cf86e5ecc Show padlock on private index page, refs #811 2020-06-08 07:18:47 -07:00
Simon Willison cc218fa9be Move assert_permissions_checked() calls from test_html.py to test_permissions.py, refs #811 2020-06-08 07:02:31 -07:00
Simon Willison 9397d71834 Implemented view-table, refs #811 2020-06-07 21:47:22 -07:00
Simon Willison b26292a458 Test that view-query is respected by query list, refs #811 2020-06-07 20:56:49 -07:00
Simon Willison 9b42e1a4f5 view-database permission 
Also now using 🔒 to indicate private resources - resources that
would not be available to the anonymous user. Refs #811
 2020-06-07 20:50:37 -07:00
Simon Willison 613fa551a1 Removed view-row permission, for the moment - refs #811 
https://github.com/simonw/datasette/issues/811#issuecomment-640338347
 2020-06-07 20:14:27 -07:00
Simon Willison cd92e4fe2a Fixed test name, this executes view-query, not execute-sql - refs #811 2020-06-07 14:33:56 -07:00
Simon Willison 8571ce388a Implemented view-instance permission, refs #811 2020-06-07 14:30:39 -07:00
Simon Willison ece0ba6f4b Test + default impl for view-query permission, refs #811 2020-06-07 14:23:16 -07:00
Simon Willison abc7339124 Nicer pattern for make_app_client() in tests, closes #395 2020-06-07 14:14:10 -07:00
Simon Willison a1e801453a Renamed execute-query permission to execute-sql, refs #811 2020-06-07 13:20:59 -07:00
Simon Willison 4340845754 Nested permission checks for all views, refs #811 2020-06-07 13:03:08 -07:00
Simon Willison 86dec9e8ff Added permission check to every view, closes #808 2020-06-06 22:30:36 -07:00
Simon Willison 3359d54a4e Use cookies when accessing csrftoken_from 2020-06-06 12:33:08 -07:00
Simon Willison 966eec7f75 Check permissions on canned query page, refs #800 2020-06-06 12:27:00 -07:00
Simon Willison 070838bfa1 Better test for Vary header 2020-06-06 12:26:19 -07:00
Simon Willison 3f83d4632a Respect query permissions on database page, refs #800 2020-06-06 12:05:22 -07:00
Simon Willison 14f6b4d200 actor_matches_allow utility function, refs #800 2020-06-06 11:39:11 -07:00
Simon Willison 75c143a84c Fixed /-/plugins?all=1, refs #802 2020-06-05 16:55:08 -07:00
Simon Willison f786033a5f Fixed 'datasette plugins' command, with tests - closes #802 2020-06-05 16:46:37 -07:00
Simon Willison 033a1bb22c Removed rogue print() from test 2020-06-05 12:06:43 -07:00
Simon Willison 84a9c4ff75
CSRF protection (#798) 
Closes #793.

* Rename RequestParameters to MultiParams, refs #799
* Allow tuples as well as lists in MultiParams, refs #799
* Use csrftokens when running tests, refs #799
* Use new csrftoken() function, refs https://github.com/simonw/asgi-csrf/issues/7
* Check for Vary: Cookie hedaer, refs https://github.com/simonw/asgi-csrf/issues/8
 2020-06-05 12:05:57 -07:00
Simon Willison d96ac1d52c Allow tuples as well as lists in MultiParams, refs #799 2020-06-05 11:01:06 -07:00
Simon Willison 0da7f49b24 Rename RequestParameters to MultiParams, refs #799 2020-06-05 10:52:50 -07:00
Simon Willison 9cb44be42f Docs and tests for "params", closes #797 2020-06-03 14:04:40 -07:00
Simon Willison aa82d03704
Basic writable canned queries 
Refs #698. First working version of this feature.

* request.post_vars() no longer discards empty values
 2020-06-03 08:16:50 -07:00
Simon Willison 0934844c0b request.post_vars() no longer discards empty values 2020-06-03 06:48:39 -07:00
Simon Willison a7137dfe06 /-/plugins now shows details of hooks, closes #794 
Also added /-/plugins?all=1 parameter to see default plugins.
 2020-06-02 14:49:28 -07:00
Simon Willison 5278c04682 More consistent use of response.text/response.json in tests, closes #792 2020-06-02 14:29:12 -07:00
Simon Willison 4fa7cf6853 Flash messages mechanism, closes #790 2020-06-02 14:12:18 -07:00
Simon Willison dfdbdf378a Added /-/permissions debug tool, closes #788 
Also started the authentication.rst docs page, refs #786.

Part of authentication work, refs #699.
 2020-05-31 22:00:36 -07:00
Simon Willison 9f3d4aba31 --root option and /-/auth-token view, refs #784 2020-05-31 18:16:42 -07:00
Simon Willison fa27e44fe0 datasette.sign() and datasette.unsign() methods, refs #785 2020-05-31 18:16:42 -07:00
Simon Willison 1fc6ceefb9 Added /-/actor.json - refs #699 
Also added JSON highlighting to introspection documentation.
 2020-05-31 18:16:42 -07:00
Simon Willison 9315bacf6f Implemented datasette.permission_allowed(), refs #699 2020-05-31 18:16:42 -07:00
Simon Willison 461c82838d Implemented actor_from_request with tests, refs #699 
Also added datasette argument to permission_allowed hook
 2020-05-31 18:16:42 -07:00
Simon Willison 4d798ca0e3 Added test for db.mtime_ns 2020-05-30 11:17:20 -07:00
Simon Willison de1cde65a6 Moved request tests to test_internals_request.py 2020-05-30 10:45:11 -07:00
Simon Willison ca56c226a9 Renamed test_database.py to test_internals_database.py 
Also added a db fixture to remove some boilerplate.
 2020-05-30 07:33:02 -07:00
Simon Willison 31fb006a9b Added datasette.get_database() method 
Refs #576
 2020-05-30 07:29:59 -07:00
Simon Willison 81be31322a New implementation for RequestParams 
- no longer subclasses dict
- request.args[key] now returns first item, not all items
- removed request.raw_args entirely

Closes #774
 2020-05-29 16:22:22 -07:00
Simon Willison 84616a2364 request.args.getlist() returns [] if missing, refs #774 
Also added some unit tests for request.args
 2020-05-29 15:51:30 -07:00
Simon Willison 7bb30c1f11 request.url now respects force_https_urls, closes #781 2020-05-28 10:10:06 -07:00
Simon Willison 5ab411c733 can_render mechanism for register_output_renderer, closes #770 2020-05-27 22:57:05 -07:00
Simon Willison 75cd432e5a Ability to set custom table/view page size in metadata, closes #751 2020-05-27 22:00:04 -07:00
Simon Willison 510c1989d4 Removed xfail, refs #773 2020-05-27 21:11:53 -07:00
Simon Willison 6d95cb4f91 Unit test for register_facet_classes plugin, closes #773 
I was a bit lazy with this one. I didn't hook up a test for the facet_results mechanism.
The custom facet hook isn't a great design so I will probably rethink it at some point
in the future anyway.
 2020-05-27 21:09:16 -07:00
Simon Willison defead17a4 Test for publish_subcommand hook, refs #773 2020-05-27 20:30:32 -07:00
Simon Willison cbeea23d00 Test for prepare_jinja2_environment, refs #773 2020-05-27 20:13:32 -07:00
Simon Willison 57f48b8416 Made register_output_renderer callback optionally awaitable, closes #776 2020-05-27 19:43:30 -07:00
Simon Willison 52c4387c7d Redesigned register_output_renderer plugin hook, closes #581 2020-05-27 19:21:41 -07:00
Simon Willison 446e5de65d Refactored test plugins into tests/plugins, closes #775 2020-05-27 17:57:25 -07:00
Simon Willison ad88c9b3f3 Mechanism for adding a default URL fragment to a canned query 
Closes #767
 2020-05-27 14:52:03 -07:00
Simon Willison da87e963bf
Test that plugin hooks are unit tested (xfail) 
This currently fails using xfail. Closes 771.
 2020-05-27 13:16:02 -07:00
Simon Willison 41a0cd7b6a call_with_supported_arguments() util, refs #581 2020-05-27 12:25:52 -07:00
Simon Willison 5ea8c6d1cd type-pk instead of type-link CSS class, closes #729 2020-05-14 22:55:20 -07:00
Simon Willison 504196341c Visually distinguish float/int columns, closes #729 2020-05-14 22:51:39 -07:00
Simon Willison 2694ddcf14 Test for .execute_fn(), refs #685 2020-05-08 10:29:17 -07:00
Simon Willison 4433306c18
Improvements + docs for db.execute() and Results class 
* Including new results.first() and results.single_value() methods. Closes #685
 2020-05-08 09:05:46 -07:00
Simon Willison 0784f2ef9d Allow specific pragma functions, closes #761 2020-05-06 10:18:31 -07:00
Simon Willison cc872b1f50 Fixed rogue output in tests, closes #755 2020-05-04 11:42:01 -07:00
Simon Willison 9424687e9e Consistently return charset utf-8, closes #752 2020-05-04 10:42:10 -07:00
Simon Willison 450d2e2896 Fixed pytest warning about TestClient class 2020-05-04 10:42:10 -07:00
Simon Willison cef23e8861 Started pattern portfolio at /-/patterns, refs #151 2020-05-02 20:05:25 -07:00
Simon Willison b3aa5f4313 Added 'not like' table filter, refs #750 2020-05-02 12:04:54 -07:00
Simon Willison 1d91ab71d4 Directory configuration mode supports metadata.yaml, closes #747 2020-04-30 11:47:41 -07:00
Simon Willison 89c4ddd482 403 for static directory listing, closes #740 2020-04-27 11:29:04 -07:00
Simon Willison 25014ca25e
Configuration directory mode, closes #731 2020-04-27 09:30:24 -07:00
Simon Willison 1b7b66c465 Make request available when rendering custom pages, closes #738 2020-04-26 12:01:46 -07:00
Simon Willison 304e7b1d9f
Mechanism for creating custom pages using templates 
Closes #648
 2020-04-26 11:46:43 -07:00
Simon Willison 15e2321804 Extra body CSS class for canned queries, closes #727 2020-04-15 14:07:28 -07:00
Simon Willison d349d57cdf Smarter merging of metadata and extra_metadata, closes #724 2020-04-10 11:34:09 -07:00
Simon Willison d55fe8cdfc Fixed bug with Templates considered comment, closes #689 2020-04-05 12:38:33 -07:00
Simon Willison e89b0ef2f9 Expose extra_template_vars in _contex=1, refs #693 2020-04-05 11:49:15 -07:00
Simon Willison 09253817de Fix for missing view_name bug, closes #716 2020-04-05 11:28:20 -07:00
Simon Willison e0e7a0facf Removed Zeit Now v1 support, closes #710 2020-04-04 16:04:33 -07:00
Simon Willison b07312c2b3 dedent SQL for neighborhood_search fixture 
Makes this page a little prettier:
https://latest.datasette.io/fixtures/neighborhood_search
 2020-04-02 17:54:27 -07:00
Simon Willison 6717c719dd
--metadata accepts YAML as well as JSON - closes #713 2020-04-02 12:30:53 -07:00
Simon Willison 2aaad72789 Refactor template setup into Datasette constructor 
Closes #707
 2020-03-26 18:12:43 -07:00
Simon Willison 6aa516d82d Run base_url tests against /fixtures/facetable too, refs #712 2020-03-25 19:31:22 -07:00
Simon Willison 7656fd64d8
base_url configuration setting, closes #394 
* base_url configuration setting
* base_url works for static assets as well
 2020-03-24 17:18:43 -07:00
Simon Willison 5f4aeb1f19 Removed documentation for Zeit Now v1, refs #710 2020-03-24 15:45:24 -07:00
Simon Willison a498d0fe65 Fix bug with over-riding default sort, closes #702 2020-03-21 19:40:29 -07:00
Simon Willison 236aa065b2 "sort" and "sort_desc" metadata properties, closes #702 2020-03-21 19:28:35 -07:00
Simon Willison e1a817411a Bump to click 7.1.1 to fix flaky tests 2020-03-21 18:47:51 -07:00
Simon Willison 2c0e1e09bc Show sort arrow on primary key by default 
Closes #677. Refs #702.
 2020-03-21 16:57:37 -07:00
Simon Willison a000c80d50 await Request(scope, receive).post_vars() method, closes #700 
Needed for #698
 2020-03-16 19:47:37 -07:00
Simon Willison e1b5339fdf Do not look for templates_path in default plugins 
Closes #697
 2020-03-08 16:11:18 -07:00
Simon Willison ddd11b3ddd --memory option for publish cloudrun, refs #694 2020-03-05 17:34:36 -06:00
Simon Willison 7f5a330377
Don't count rows on homepage for DBs > 100MB (#688) 
Closes #649.
 2020-02-28 17:08:29 -08:00
Kevin Keogh 3041c6b641
Use inspect-file, if possible, for total row count (#666) 
For large tables, counting the number of rows in the table can take a
significant amount of time. Instead, where an inspect-file is provided
for an immutable database, look up the row-count for a plain count(*).

Thanks, @kevindkeogh
 2020-02-25 12:19:29 -08:00
Simon Willison 6cb65555f4
?_searchmode=raw option (#686) 2020-02-24 21:56:03 -08:00
Simon Willison a093c5f79f
.execute_write() and .execute_write_fn() methods on Database (#683) 
Closes #682.
 2020-02-24 20:45:07 -08:00
Simon Willison 7c6a9c3529 Better tests for prepare_connection() plugin hook, refs #678 2020-02-21 18:27:07 -08:00
Simon Willison d3f2fade88 Refactored run_sanity_checks to check_connection(conn), refs #674 2020-02-15 09:56:48 -08:00
Simon Willison cf5f4386ef Run black against everything, not just tests and datasette dirs 2020-02-13 15:02:10 -08:00
Jay Graves 33a12c8ae5
Allow leading comments in SQL input field (#653) 
Thanks, @jaywgraves!
 2020-02-04 18:13:24 -08:00
Simon Willison 67fc9c5720
--port argument for datasette package, plus tests - closes #661 
From pull request #663
 2020-01-29 14:46:43 -08:00
Katie McLaughlin 34d77d780f gcloud run is now GA, s/beta// (#660) 
Thanks, @glasnt
 2020-01-21 15:28:11 -08:00
Simon Willison 3c861f363d _search= queries now correctly escaped, fixes #651 
Queries with reserved words or characters according to the SQLite
FTS5 query language could cause errors.

Queries are now escaped like so:

    dog cat => "dog" "cat"
 2019-12-29 18:48:30 +00:00
Simon Willison d54318fc7f Added template_debug setting, closes #654 2019-12-22 16:04:45 +00:00
Simon Willison d6b6c9171f Include asyncio task information in /-/threads debug page 2019-12-04 22:47:17 -08:00
Simon Willison d3e1c3017e Display 0 results, closes #637 2019-11-22 22:07:01 -08:00
Simon Willison fd137da7f8 Suggest column facet only if at least one count > 1 
Fixes #638
 2019-11-21 16:56:55 -08:00
Simon Willison 440a70428c Include rowid in filter select, closes #636 2019-11-19 15:01:10 -08:00
Simon Willison 8c642f04e0
Render templates using Jinja async mode 
Closes #628
 2019-11-14 15:14:22 -08:00
Simon Willison f524510230 Fix "publish heroku" + upgrade to use Python 3.8.0 
Closes #633. Closes #632.
 2019-11-13 08:42:47 -08:00
Simon Willison d977fbadf7
datasette publish uses python:3.8 base Docker image, closes #629 2019-11-11 22:03:09 -08:00
Simon Willison cf7776d36f
Support Python 3.8, stop supporting Python 3.5 (#627) 
* Upgrade to uvicorn 0.10.4
* Drop support for Python 3.5
* Bump all dependencies to latest releases
* Update docs to reflect we no longer support 3.5
* Removed code that skipped black unit test on 3.5

Closes #622
 2019-11-11 21:09:11 -08:00
Simon Willison 28c4a6db5b CREATE INDEX statements on table page, closes #618 2019-11-09 17:29:36 -08:00
Simon Willison 83fc5165ac Improved UI for publish cloudrun, closes #608 2019-11-07 18:48:39 -08:00
Tobias Kunze 931bfc6661 Handle spaces in DB names (#590) 
Closes #503 - thanks, @rixx
 2019-11-04 15:16:30 -08:00
Simon Willison 52fa79c607 Use select colnames, not select * for table view - refs #615 2019-11-04 15:03:48 -08:00
Simon Willison 9db22cdf18 pk__notin= filter, closes #614 2019-11-03 20:11:55 -08:00
Simon Willison 2bf7ce5f51 Fix CSV export for nullable foreign keys, closes #612 2019-11-02 16:12:46 -07:00
Simon Willison 14da70525b Don't show 'None' as label for nullable foreign key, closes #406 2019-11-02 15:29:40 -07:00
Simon Willison 7152e76eda Don't suggest array facet if column is only [], closes #610 2019-11-01 14:45:59 -07:00
Simon Willison 50287e7c6b Only suggest array facet for arrays of strings - closes #562 2019-11-01 12:37:46 -07:00
Simon Willison e2c390500e Persist _where= in hidden fields, closes #604 2019-10-30 11:49:26 -07:00
Simon Willison 5dd4d2b2d3
Update to latest black (#609) 2019-10-30 11:49:01 -07:00
chris48s f4c0830529 Always pop as_format off args dict (#603) 
Closes #563. Thanks, @chris48s
 2019-10-20 19:03:08 -07:00
Simon Willison b647b5efc2
Fix for /foo v.s. /foo-bar issue, closes #597 
Pull request #599
 2019-10-18 15:51:07 -07:00
Simon Willison b6ad1fdc70 Fixed bug returning non-ascii characters in CSV, closes #584 2019-10-17 22:23:01 -07:00
Simon Willison 3e864b1625 Use --platform=managed for publish cloudrun, closes #587 2019-10-17 14:51:45 -07:00
Tobias Kunze 12cec411ca Display metadata footer on custom SQL queries (#589) 
Closes #408 - thanks, @rixx!
 2019-10-13 20:53:21 -07:00
Tobias Kunze 908fc3999e Sort databases on homepage by argument order - #591 
Closes #585 - thanks, @rixx!
 2019-10-13 20:52:33 -07:00
Simon Willison fffd69ec03 Allow EXPLAIN WITH... - closes #583 2019-10-06 10:23:58 -07:00
Simon Willison 2dc5c8dc25
detect_fts now works with alternative table escaping (#571) 
Fixes #570. See also https://github.com/simonw/sqlite-utils/pull/57
 2019-09-02 17:32:27 -07:00
Simon Willison 90d4f497f9 Fix plus test for unicode characters in custom query name, closes #558 2019-07-13 19:49:24 -07:00
Abdus 74ecf8a7cc Fix static mounts using relative paths and prevent traversal exploits (#554) 
Thanks, @abdusco! Closes #555
 2019-07-11 09:13:19 -07:00
Simon Willison 973f8f139d
--plugin-secret option for datasette publish 
Closes #543

Also added new --show-files option to publish now and publish cloudrun - handy for debugging.
 2019-07-07 19:06:31 -07:00
Simon Willison c5542abba5 Removed ManyToManyFacet for the moment, closes #550 2019-07-07 16:21:11 -07:00
Simon Willison fcfcae21e6
extra_template_vars plugin hook (#542) 
* extra_template_vars plugin hook

Closes #541

* Workaround for cwd bug

Based on https://github.com/pytest-dev/pytest/issues/1235#issuecomment-175295691
 2019-07-05 17:05:56 -07:00
Simon Willison 25ff0a8ba6 Fix for accidentally leaking secrets in /-/metadata, closes #538 2019-07-03 22:47:45 -07:00
Simon Willison a2d4593193
Secret plugin configuration options (#539) 
Closes #538
 2019-07-03 22:36:44 -07:00
Simon Willison 4d2fdafe39 Added asgi_wrapper plugin hook, closes #520 2019-07-02 21:06:25 -07:00
Simon Willison 13e4f9a676 Unit test for _table custom template, refs #521 2019-07-02 20:23:05 -07:00
Simon Willison ba8db9679f
Port Datasette from Sanic to ASGI + Uvicorn (#518) 
Datasette now uses ASGI internally, and no longer depends on Sanic.

It now uses Uvicorn as the underlying HTTP server.

This was thirteen months in the making... for full details see the issue:

https://github.com/simonw/datasette/issues/272

And for a full sequence of commits plus commentary, see the pull request:

https://github.com/simonw/datasette/pull/518
 2019-06-23 20:13:09 -07:00
Simon Willison 35429f9089 Revert "New encode/decode_path_component functions" 
Refs #272

This reverts commit 9fdb47ca95.

Now that ASGI supports raw_path we don't need our own encoding scheme!
 2019-06-18 17:23:27 -07:00
Simon Willison 2a4b892d6c Tidy up with Black 2019-05-27 19:28:47 -07:00
Simon Willison 9c58d048ec Fix pagination when sorted by expanded foreign key 
Closes #489
 2019-05-27 19:23:18 -07:00
Simon Willison de92c05134 Fixed duplicate function name 2019-05-27 18:49:57 -07:00
Simon Willison 839e55bf0a Start of unit tests for Database class, refs #485 2019-05-27 18:42:30 -07:00
Simon Willison 182a3017c2 Sort keys to past tests in Python 3.5 2019-05-25 09:22:02 -07:00
Simon Willison d923d84754 Facet by many-to-many, closes #365 2019-05-25 09:06:32 -07:00
Simon Willison 172da009d8 Added ?_through= table argument, closes #355 
Also added much more interesting many-to-many fixtures - roadside attractions!
 2019-05-22 22:45:12 -07:00
Simon Willison 967230c90e Facet by date, closes #481 2019-05-20 23:09:22 -07:00
Simon Willison 2d14f21f85 Updated tests for date(...) lookup 2019-05-20 23:06:19 -07:00
Simon Willison f4eefdf193 Do not allow downloads of mutable databases - closes #474 2019-05-19 13:41:09 -07:00
Simon Willison 7d8573d672 Rename "datasette publish now" to "datasette publish nowv1" 
Also added an alias so "datasette publish now" continues to work.

Closes #472
 2019-05-19 11:06:51 -07:00
Simon Willison db9dbfb816 ?_hash=1 no longer respected for mutable databases 
Closes #471, refs #419
 2019-05-16 08:10:25 -07:00
Simon Willison 0dad111d24 New introspection endpoint: /-/databases - closes #470 
Refs #419 and #465
 2019-05-16 07:49:34 -07:00
Simon Willison 21b57cdf9a serve --inspect-file=X now populates cached table counts 
Closes #462
 2019-05-16 07:30:32 -07:00
Simon Willison 909e66dfdb Removed .inspect() and /-/inspect.json 
Refs #462

/-/inspect.json may return in some shape in #465
 2019-05-16 07:15:24 -07:00
Simon Willison faf33515b2 Sometimes sort tables by number of relationships, closes #460 2019-05-15 20:23:33 -07:00
Simon Willison 689cf9c139 Index page only shows row counts for smaller databases 
The index page now only shows row counts for immutable databases OR for
databases with less than 30 tables provided it could get a count for
each of those tables in less than 10ms.

Closes #467, Refs #460
 2019-05-15 20:02:33 -07:00
Simon Willison 79303163b2 Row count fix + sort index page databases alphabetically 
Sorting alphabetically should fix a test failure in Python 3.5

Refs #460
 2019-05-15 17:40:10 -07:00
Simon Willison 5d6b2c30f1 Include views on homepage, fix table counts 
If we have less than 5 tables we now also show one or more views in the
summary on the homepage.

Also corrected the logic for the row counts - we now count hidden and
visible tables separately.

Closes #373, Refs #460
 2019-05-15 17:28:07 -07:00
Simon Willison e04511410f Don't show hidden tables on index page, closes #455 
Refs #460. Also bulked out HTML index page unit tests.
 2019-05-14 08:46:57 -07:00
Simon Willison 24bf79d2f0 Run sanity checks, not .inspect(), on startup 
Also fixes tests that did NOT like a call to run_until_complete in the Datasette() constructor.
 2019-05-11 16:22:55 -07:00
Simon Willison da0b3ce2b7 New run_sanity_checks mechanism, for SpatiLite 
Moved VirtualSpatialIndex check into a new mechanism that should allow
us to add further sanity checks in the future.

To test this I've had to commit a binary sample SpatiaLite database to
the repository. I included a build script for creating that database.

Closes #466
 2019-05-11 15:55:30 -07:00
Simon Willison c692cd2911 Fix test ordering 2019-05-11 15:03:52 -07:00
Simon Willison a187c81e0c Black + fix broken test 2019-05-11 14:45:59 -07:00
Simon Willison c0d1b4c322 "datasette inspect foo.db" now just calculates table counts 
Refs #462

* inspect command now just outputs table counts
* test_inspect.py is now only tests for that CLI command
* Updated some relevant documentation
* Removed docs for /-/inspect since that is about to change
 2019-05-11 14:36:57 -07:00
Simon Willison ce09e5d2d3 Fixed tests relating to #459 2019-05-11 13:58:22 -07:00
Simon Willison 05cabaebd9 Pass --token to now alias, refs #459 2019-05-11 13:35:34 -07:00
Simon Willison 09ef305c68 Fixed "datasette publish now ... --alias=x" 
The --alias argument can now be passed more than once.

Also updated our Travis configuration to use this.

Fixes #459
 2019-05-11 13:20:36 -07:00
Simon Willison a0d444837f Finished implementation of ?_trace=1 debug tool 
I redesigned the JSON output and added a handy "traceback" key showing
three relevant lines of the current traceback for each logged query.

Closes #435
 2019-05-11 12:06:22 -07:00
Simon Willison 2db977d8f3 tests/fixtures.py can now write out plugins too 
This command:

    python tests/fixtures.py \
        fixtures.db \
        metadata.json \
        fixtures-plugins/

Will now create the fixtures.db and metadata.json files, AND create
a folder called fixtures-plugins/ containing two test plugins.

You can then run it like this:

    datasette fixtures.db \
        -m metadata.json --plugins-dir=fixtures-plugins/
 2019-05-09 09:44:21 -07:00
Simon Willison 9fdb47ca95 New encode/decode_path_component functions 
ASGI cannot differentiate between / and %2F in a URL, so we need an
alternative scheme for encoding the names of tables that contain special
characters such as /

For background, see
    https://github.com/django/asgiref/issues/51#issuecomment-450603464

Some examples:

    "table/and/slashes" => "tableU+002FandU+002Fslashes"
    "~table" => "U+007Etable"
    "+bobcats!" => "U+002Bbobcats!"
    "U+007Etable" => "UU+002B007Etable"
 2019-05-09 09:44:21 -07:00
Simon Willison ec2db345e9 Fixed crash on /:memory: page 2019-05-05 14:01:14 -04:00
Simon Willison 831515b834 Respect --cors for error pages, closes #453 2019-05-05 08:12:03 -04:00
Simon Willison 35d6ee2790
Apply black to everything, enforce via unit tests (#449) 
I've run the black code formatting tool against everything:

    black tests datasette setup.py

I also added a new unit test, in tests/test_black.py, which will fail if the code does not
conform to black's exacting standards.

This unit test only runs on Python 3.6 or higher, because black itself doesn't run on 3.5.
 2019-05-03 22:15:14 -04:00
Simon Willison 93c65d29ba Fixed 500 error on /-/metadata page 2019-05-03 16:42:59 -04:00
Simon Willison 01b3de5b66 Unit test for binary data display, refs #442 2019-05-03 12:45:06 -04:00
Simon Willison bade9daeb2 Docs for 'datasette publish cloudrun', refs #434 2019-05-03 12:16:58 -04:00
Romain Primet 75a21fc2a1 datasette publish cloudrun (#434) - thanks, @rprimet 
New publish subcommand that publishes using the
new Google Cloud Run platform.

    datasette publish cloudrun database.db
 2019-05-03 09:59:01 -04:00
Simon Willison 53d2f00b73 Implemented ArrayFacet, closes #359 2019-05-02 20:21:27 -04:00
Simon Willison ea66c45df9
Extract facet code out into a new plugin hook, closes #427 (#445) 
Datasette previously only supported one type of faceting: exact column value counting.

With this change, faceting logic is extracted out into one or more separate classes which can implement other patterns of faceting - this is discussed in #427, but potential upcoming facet types include facet-by-date, facet-by-JSON-array, facet-by-many-2-many and more.

A new plugin hook, register_facet_classes, can be used by plugins to add in additional facet classes.

Each class must implement two methods: suggest(), which scans columns in the table to decide if they might be worth suggesting for faceting, and facet_results(), which executes the facet operation and returns results ready to be displayed in the UI.
 2019-05-02 17:11:26 -07:00
Simon Willison efc93b8ab5 Entirely removed table_rows_count table property 
We were not displaying this anywhere, and it is now expensive to calculate.

Refs #419, #420
 2019-05-01 22:29:47 -07:00
Simon Willison ad9de774f7 Added missing file, refs #438 2019-05-01 22:10:23 -07:00
Simon Willison f553a67021 Don't load setuptools plugins during test runs 
Uses pattern from https://docs.pytest.org/en/latest/example/simple.html#detect-if-running-from-within-a-pytest-run

Closes #438
 2019-05-01 22:09:03 -07:00
Simon Willison 50d2d1aac9 Fixed bug where metadata.json hidden tables were ignored 2019-05-01 17:54:48 -07:00
Simon Willison e7151ccccf Index page no longer uses inspect data - refs #420 
Also introduced a mechanism whereby table counts are calculated against a time limit
but immutable databases have their table counts calculated on server startup.
 2019-05-01 17:39:39 -07:00
Simon Willison 669fa21a71 Include request duration in traces 2019-05-01 16:27:14 -07:00
Russ Garrett cf406c0754 New plugin hook: register_output_renderer hook (#441) 
Thanks @russss!

* Add register_output_renderer hook

This changeset refactors out the JSON renderer and then adds a hook and
dispatcher system to allow custom output renderers to be registered.

The CSV output renderer is untouched because supporting streaming
renderers through this system would be significantly more complex, and
probably not worthwhile.

We can't simply allow hooks to be called at request time because we need
a list of supported file extensions when the request is being routed in
order to resolve ambiguous database/table names. So, renderers need to
be registered at startup.

I've tried to make this API independent of Sanic's request/response
objects so that this can remain stable during the switch to ASGI. I'm
using dictionaries to keep it simple and to make adding additional
options in the future easy.

Fixes #440
 2019-05-01 16:01:56 -07:00
Simon Willison e4e73a537a Test for ?_trace=1, refs #435 2019-04-21 10:46:56 -07:00
Simon Willison 9c77e6e355 Support multiple filters of the same type 
Closes #288
 2019-04-15 16:44:17 -07:00
Simon Willison 583b22aa28 New ?column__date=yyyy-mm-dd filter 2019-04-15 15:54:54 -07:00
Simon Willison 1c6649b19b New colname__in=x,y,z filter, closes #433 2019-04-15 15:43:22 -07:00
Simon Willison 2c19a27d15 Documentation for filters, plus new documentation unit test 
https://simonwillison.net/2018/Jul/28/documentation-unit-tests/
 2019-04-15 15:41:11 -07:00
Simon Willison 6da567dda9 Extract and refactor filters into filters.py 
This will help in implementing __in as a filter, refs #433
 2019-04-15 14:51:20 -07:00
Simon Willison bc6a9b4564
?_where= parameter on table views, closes #429 
From pull request #430
 2019-04-12 18:37:22 -07:00
Simon Willison e11cb4c664 Persist show/hide state better, closes #425 2019-04-11 22:00:47 -07:00
Simon Willison db74cf0144 ?_fts_table= and ?_fts_pk= arguments, closes #428 2019-04-11 21:21:17 -07:00
Simon Willison 78e45ead4d New ?tags__arraycontains=tag lookup against JSON fields 
Part one of supporting facet-by-JSON-array, refs #359
 2019-04-10 08:27:52 -07:00
Simon Willison 53bf875483 expand_foreign_keys() no longer uses inspect, refs #420 2019-04-06 19:56:07 -07:00
Simon Willison 97331f3435 sortable_columns_for_table() no longer uses inspect() 
Refs #420
 2019-04-06 18:58:51 -07:00
Simon Willison 468c6fd953 DatabaseDownload no longer uses .inspect(), refs #420 2019-03-31 19:04:17 -07:00
Simon Willison 7d0f668556 .resolve_db_name() and .execute() work without inspect 
Refs #420
 2019-03-31 16:51:52 -07:00
Simon Willison 0209a0a344 table_exists() now uses async SQL, refs #420 2019-03-31 11:02:22 -07:00
Simon Willison 6f6d0ff2b4
URL hashing is now off by default - closes #418 
Prior to this commit Datasette would calculate the content hash of every
database and redirect to a URL containing that hash, like so:

    https://v0-27.datasette.io/fixtures => https://v0-27.datasette.io/fixtures-dd88475

This assumed that all databases were opened in immutable mode and were not
expected to change.

This will be changing as a result of #419 - so this commit takes the first step
in implementing that change by changing this default behaviour. Datasette will
now only redirect hash-free URLs under two circumstances:

* The new `hash_urls` config option is set to true (it defaults to false).
* The user passes `?_hash=1` in the URL
 2019-03-17 15:55:04 -07:00
Simon Willison afe9aa3ae0 show/hide link for SQL on custom query page 
Closes #415
 2019-03-14 22:22:35 -07:00
Simon Willison 2855667908 Fix for test failure with Click 7.0 2019-03-14 22:00:13 -07:00
Simon Willison 9743e1d91b Support for :memory: databases 
If you start Datasette with no files, it will connect to :memory: instead.

When starting it with files you can add --memory to also get a :memory: database.
 2019-03-14 20:54:42 -07:00
Simon Willison bf6b0f918d about and about_url metadata options 2019-03-14 20:54:42 -07:00
Simon Willison 4462a5ab28 Show size of database file next to download link, closes #172 2019-02-05 20:58:29 -08:00
Simon Willison 195a5b3634
Heroku --include-vcs-ignore (#407) 
Means `datasette publish heroku` can work under Travis, unlike this failure:

https://travis-ci.org/simonw/fivethirtyeight-datasette/builds/488047550

```
2.25s$ datasette publish heroku fivethirtyeight.db -m metadata.json -n fivethirtyeight-datasette
tar: unrecognized option '--exclude-vcs-ignores'
Try 'tar --help' or 'tar --usage' for more information.
 ▸    Command failed: tar cz -C /tmp/tmpuaxm7i8f --exclude-vcs-ignores --exclude
 ▸    .git --exclude .gitmodules . >
 ▸    /tmp/f49440e0-1bf3-4d3f-9eb0-fbc2967d1fd4.tar.gz
 ▸    tar: unrecognized option '--exclude-vcs-ignores'
 ▸    Try 'tar --help' or 'tar --usage' for more information.
 ▸    
The command "datasette publish heroku fivethirtyeight.db -m metadata.json -n fivethirtyeight-datasette" exited with 0.
```

The fix for that issue is to call the heroku command like this:

    heroku builds:create -a app_name --include-vcs-ignore
 2019-02-05 20:15:46 -08:00
Simon Willison b5dd83981a Export option: _shape=array&_nl=on for newline-delimited JSON 2019-01-27 17:40:23 -08:00
Simon Willison c3a78eb05c app_client() fixture doesn't need to take **kwargs 2019-01-13 14:23:44 -08:00
Simon Willison 8ce7866312 compile_options output in /-/versions, closes #396 2019-01-10 16:44:50 -08:00
Simon Willison 996e8822d2 Fix CSV export hidden form fields, closes #393 2019-01-02 18:43:56 -08:00
Simon Willison a2bfcfc1b1
Fix some regex DeprecationWarnings (#392) 2018-12-28 18:22:27 -08:00
Simon Willison 3de8fac1d3
New make_app_client() pattern 
Because next version of pytest discourages calling fixture factories as functions
 2018-11-20 09:46:47 -08:00
Simon Willison b7257a21bf
Fix json.loads in Python 3.5 
3.5 requires a str, not a bytes

https://travis-ci.org/simonw/datasette/jobs/421660555
 2018-08-28 17:55:30 +01:00
Simon Willison b7c6a9f9bd
extra_css_urls(template, database, table, datasette) 
The extra_css_urls and extra_js_urls hooks now take additional optional
parameters.

Also refactored them out of the Datasette class and into RenderMixin.

Plus improved plugin documentation to explicitly list parameters.
 2018-08-28 11:56:57 +01:00
Simon Willison 2e836f72d9
render_cell(value, column, table, database, datasette) 
The render_cell plugin hook previously was only passed value.

It is now passed (value, column, table, database, datasette).
 2018-08-28 03:03:01 -07:00
Simon Willison 5cf0c6c91c
New plugin hook: extra_body_script 2018-08-28 02:02:49 -07:00
Simon Willison 0a14a4846b
Added plugin_config() method 2018-08-28 01:35:21 -07:00
Simon Willison aae49fef3b
Import pysqlite3 if available, closes #360 (#361) 2018-08-15 17:58:56 -07:00
Simon Willison fe5b6ea95a
Hide 'view and edit SQL' if config.allow_sql turned off 2018-08-05 20:17:17 -07:00
Simon Willison 5629aaca67
sortable_columns also now works with views 2018-08-05 17:29:23 -07:00
Simon Willison 4ac9132240
render_cell(value) plugin hook, closes #352 
New plugin hook for customizing the way cells values are rendered in HTML.

The first full example of this hook in use is https://github.com/simonw/datasette-json-html
 2018-08-04 17:14:56 -07:00
Simon Willison 295d005ca4
Renamed variable, since docs are not written in Markdown 2018-07-27 21:56:51 -07:00
Simon Willison 8128497b79
Only run documented_views() fixture once per session 
Speeds up tests, because previously it ran once per view class.
 2018-07-27 21:49:20 -07:00
Simon Willison 788a542d3c
Docs for IndexView, TableView, RowView, closes #299 
Also removed xfail from test_view_classes_are_documented, so any future *View
classes that are added without documentation will cause the tests to fail.
 2018-07-27 21:21:42 -07:00
Simon Willison e8625695a3
xfail documentation unit tests for view classes, refs #299 
More documentation unit tests. These ones check that every single **View class
imported into the datasette/app.py module are covered by our documentation.

Just one problem: they aren't documented yet. So I'm using the xfail pytest
decorator to mark these tests as allowed-to-fail. When you run the test suite
you now get a report of how many views still need to be documented, but it
doesn't fail the tests.

The output looks something like this:

    $ pytest tests/test_docs.py
    collected 31 items

    tests/test_docs.py ..........................XXXxx.                [100%]

    ============ 26 passed, 2 xfailed, 3 xpassed in 1.06 seconds ============

Once I have documented all the views I will remove the xfail so any future
views that are added without documentation will cause a test failure.

We can detect that a view is documented by looking for ReST label in the docs,
for example:

    .. _IndexView:

Some view classes can be used to power multiple URLs - the JsonDataView class
for example is used to power /-/metadata and /-/config and /-/plugins

In this case, the second part of the label can indicate the variety of page, e.g:

    .. _JsonDataView_metadata:

The test will pass as long as there is at least one label that starts with
_JsonDataView.
 2018-07-27 20:13:26 -07:00
Simon Willison 3b68314d8f
Unit tests for publish now/heroku - closes #348 2018-07-25 22:45:47 -07:00
Simon Willison dbbe707841
publish_subcommand hook + default plugins mechanism, used for publish heroku/now (#349) 
This change introduces a new plugin hook, publish_subcommand, which can be
used to implement new subcommands for the "datasette publish" command family.

I've used this new hook to refactor out the "publish now" and "publish heroku"
implementations into separate modules. I've also added unit tests for these
two publishers, mocking the subprocess.call and subprocess.check_output
functions.

As part of this, I introduced a mechanism for loading default plugins. These
are defined in the new "default_plugins" list inside datasette/app.py

Closes #217 (Plugin support for datasette publish)
Closes #348 (Unit tests for "datasette publish")
Refs #14, #59, #102, #103, #146, #236, #347
 2018-07-25 22:15:59 -07:00
Simon Willison 3ac21c7498
Unit test confirming all plugin hooks are documented 2018-07-25 21:09:59 -07:00
Simon Willison ba64cfb4bc
Fix for Python 3.5 
https://stackoverflow.com/a/42694113/6083
 2018-07-25 10:20:56 -07:00
Simon Willison 9b85c59907
Removed unnecessary print statements from tests 2018-07-24 09:04:06 -07:00
Simon Willison 74ad3ff4af
Ensure --help examples in docs are always up to date, closes #336 
Unit tests now check that docs/*.txt help examples are all up-to-date.

I ran into a problem here in that the terminal_width needed to be more
accurately defined - so I replaced update-docs-help.sh with update-docs-
help.py which hard-codes the terminal width.
 2018-07-24 09:01:23 -07:00
Simon Willison 581b4c97ee
URLify URLs in custom SQL queries, closes #298 2018-07-23 20:56:32 -07:00
Simon Willison 3955771fea
Unit tests for advanced export box HTML, closes #320 2018-07-23 20:38:31 -07:00
Simon Willison 700d83d8ad
?_json_infinity=1 for handling Infinity/-Infinity - fixes #332 2018-07-23 20:07:57 -07:00
Simon Willison f24b49a1a8
New force_https_urls option, refs #333 2018-07-23 08:58:29 -07:00
Simon Willison 6e37f091ed
Support title/description for canned queries, closes #342 
Demo here: https://latest.datasette.io/fixtures/neighborhood_search
 2018-07-15 19:33:30 -07:00