mirror
/
datasette
kopia lustrzana https://github.com/simonw/datasette
1
0
Forkuj 0
Kod Zgłoszenia Packages Projekty Wydania Wiki Aktywność
2 404 Commity
134 Gałęzie
158 Tagi
6,5 GiB
Wykres commitów

899 Commity (json_renderer_refactor)

Autor SHA1 Wiadomość Data
Simon Willison 310c3a3e05 New datasette.urls URL builders, refs #904 2020-10-19 17:33:59 -07:00
Simon Willison 6aa5886379 --load-extension=spatialite shortcut, closes #1028 2020-10-19 15:37:43 -07:00
Simon Willison f3a087a578 Edit SQL button on canned queries, closes #1019 2020-10-13 20:44:18 -07:00
Simon Willison e34e84901d Link: HTTP header pagination, closes #1014 2020-10-10 17:18:45 -07:00
Simon Willison 7e70643852 Removed --debug option, which didn't do anything - closes #814 2020-10-10 16:39:38 -07:00
Simon Willison 7239175f63 Fixed broken column header links, closes #1011 2020-10-09 20:51:56 -07:00
Simon Willison 9f6dd985bc Fix broken CSV/JSON export on query page, refs #1010 2020-10-09 17:39:45 -07:00
Simon Willison c13d184704 Emergency fix for broken links in 0.50, closes #1010 2020-10-09 17:33:13 -07:00
Simon Willison 896cc2c6ac Replace MockRequest with Request.fake() 
Close #1004
 2020-10-09 09:26:17 -07:00
Simon Willison 8f97b9b58e
datasette.client internal requests mechanism 
Closes #943

* Datasette now requires httpx>=0.15
* Support OPTIONS without 500, closes #1001
* Added internals tests for datasette.client methods
* Datasette's own test mechanism now uses httpx to simulate requests
* Tests simulate HTTP 1.1 now
* Added base_url in a bunch more places
* Mark some tests as xfail - will remove that when new httpx release ships: #1005
 2020-10-09 09:11:24 -07:00
Simon Willison 7249ac5ca0 Support OPTIONS without 500, closes #1001 2020-10-08 18:43:53 -07:00
Simon Willison e4554c37b7 datasette publish heroku --tar option, closes #969 2020-10-08 16:30:46 -07:00
Simon Willison 86823ae6f7 Default to Uvicorn workers=1, refs #999 2020-10-08 16:16:55 -07:00
Simon Willison b47ac37114 Applied Black 2020-10-07 15:51:25 -07:00
Simon Willison 5070425817 Fix handling of nested custom page wildcard paths, closes #996 2020-10-07 15:51:11 -07:00
Simon Willison b37431976c custom pages tests templates now in repo 2020-10-07 15:16:41 -07:00
Simon Willison e02f6c1300 Tests for db.table_columns() and db.table_column_details() 2020-10-06 14:02:30 -07:00
Simon Willison 5a184a5d21 Display column type in column action menu, closes #993 
Also added new documented db.table_column_details() introspection method.
 2020-10-05 17:32:10 -07:00
Simon Willison 0f2626868b Much improved column menu display logic, refs #981 
* Menu links now take into account existing querystring
* No longer shows facet option for primary key columns
* Conditionally displays sort/sort-desc if already sorted
* Does not show facet option if already faceted by this
 2020-09-30 16:01:37 -07:00
Simon Willison 97c71c3a3b Fixed test for column sorting, refs #981 2020-09-30 14:51:10 -07:00
Simon Willison 5b8b8ae597 Handle \r\n correctly in CSS escapes, refs #980 2020-09-29 12:16:30 -07:00
Simon Willison c11383e628 Fix rendering glitch with columns on mobile, closes #978 2020-09-28 15:42:50 -07:00
Simon Willison cac051bb8a Fix for 'open' bug, closes #973 2020-09-22 08:39:48 -07:00
Simon Willison a980199e61 New -o option for opening Datasette in your browser, closes #970 2020-09-22 07:26:47 -07:00
Simon Willison 448d13ea6b Fix for MagicParameters error with no POST body, closes #967 2020-09-15 13:12:57 -07:00
Simon Willison 72ac2fd32c JSON API for writable canned queries, closes #880 2020-09-14 14:23:18 -07:00
Simon Willison 894999a14e Improved test for JSON POST, refs #880 2020-09-14 13:25:09 -07:00
Simon Willison 896fce228f Canned query writes support JSON POST body, refs #880 2020-09-14 13:18:15 -07:00
Simon Willison 699be7dea9 raise_404() function for use in custom templates, closes #964 2020-09-14 10:39:25 -07:00
Simon Willison cc77fcd133 Optional path parameters for custom pages, closes #944 2020-09-13 19:34:43 -07:00
Simon Willison ea340cf320 Correctly persist selected facets in hidden fields 
Closes #963
 2020-09-12 14:54:01 -07:00
Simon Willison 20b1de86a1 Fix for test I broke in #947 2020-09-11 15:04:23 -07:00
Simon Willison d02f6151da datasette --get status code for error pages, closes #947 2020-09-11 14:32:54 -07:00
Simon Willison ca5c405d0f New 'datasette --pdb' option, closes #962 2020-09-11 11:37:55 -07:00
Simon Willison a648bb82ba Upgrade to Black 20.8b1, closes #958 2020-09-02 15:24:55 -07:00
Simon Willison 799ecae948 register_output_renderer can now return Response, closes #953 2020-08-27 21:02:50 -07:00
Simon Willison 69033c6ec4 datasette install --upgrade option, closes #945 2020-08-19 10:20:41 -07:00
Simon Willison 8e7e6458a6 Fix bug with ?_nl=on and binary data, closes #914 2020-08-16 11:26:49 -07:00
Simon Willison 3a4c8ed36a Added columns argument to various extra_ plugin hooks, closes #938 2020-08-16 11:09:53 -07:00
Simon Willison 94ae840fe3 Plugin tests now start with test_hook_ 2020-08-16 10:49:33 -07:00
Simon Willison 2da4144c57 Applied Black 2020-08-16 10:35:14 -07:00
Simon Willison ac69d151c3 Test that plugin hooks are documented with correct arguments 2020-08-16 10:33:44 -07:00
Simon Willison e3639247cd Standard arguments for extra_ plugin hooks, closes #939 2020-08-16 09:50:23 -07:00
Simon Willison 41ddc19756
Docs now live at docs.datasette.io (#937) 2020-08-15 16:57:05 -07:00
Simon Willison af12f45c2b Documentation and tests for db.is_mutable 2020-08-15 16:27:32 -07:00
Simon Willison b86f94883b
Don't hang in db.execute_write_fn() if connection fails 
Closes #935

Refs https://github.com/simonw/latest-datasette-with-all-plugins/issues/3
 2020-08-15 15:35:31 -07:00
Simon Willison 45414f8412 --get now calls startup() plugin hooks, closes #934 2020-08-15 13:52:41 -07:00
Simon Willison e139a7619f
'datasette --get' option, closes #926 
Also made a start on the datasette.utils.testing module, refs #898
 2020-08-11 17:24:40 -07:00
Simon Willison afdeda8216 Use runpy in install/uninstall, refs #928 2020-08-11 16:54:52 -07:00
Simon Willison 01fe5b7401 datasette install / datasette uninstall commands, closes #925 2020-08-11 15:32:06 -07:00
Simon Willison 7f10f0f766 Fix for security issue #918 2020-08-09 09:03:35 -07:00
Simon Willison 0748a65a22 Fixed content-disposition header on DB download, closes #909 2020-07-29 14:34:22 -07:00
Simon Willison 092874202c Improvements to allow block logic and debug tool 
true and false allow block values are now supported, closes #906

Added a bunch of demo links to the documentation, refs #908
 2020-07-24 17:04:06 -07:00
Simon Willison 12c0bc09cc /-/allow-debug tool, closes #908 2020-07-24 15:55:10 -07:00
Simon Willison 213e6a8926 content-length for DB downloads, closes #905 2020-07-21 21:52:35 -07:00
Simon Willison 1f6a134369 await request.post_body() method, closes #897 2020-07-17 13:12:35 -07:00
Amjith Ramanujam ea99a4431c
Only load Python files from plugins-dir 
Pull request #890. Thanks, @amjith!

* Load only python files from plugins-dir
* Add a test to verify non-python files are not loaded as plugins
 2020-07-02 20:08:32 -07:00
Simon Willison 57879dc8b3 Better titles for canned query pages, closes #887 2020-07-01 17:23:37 -07:00
Simon Willison 1bae24691f Only show 'log out' if ds_cookie present, closes #884 2020-07-01 14:25:59 -07:00
Simon Willison 549b1c2063 New forbidden() plugin hook, closes #812 2020-06-30 21:17:38 -07:00
Simon Willison 3ec5b1abf6 CSRF tests for canned query POST, closes #835 2020-06-30 20:08:00 -07:00
Simon Willison d6e03b0430 Cascading view permissions, closes #832 
- If you have table permission but not database permission you can now view the table page
- New BaseView.check_permissions() method
 2020-06-30 16:40:50 -07:00
Simon Willison 9ac6292614 _header_x now defaults to empty string 
Prior to this a request to e.g. https://latest.datasette.io/fixtures/magic_parameters
which did not include a User-Agent header would trigger a 500 error.
 2020-06-30 15:00:17 -07:00
Simon Willison 2115d7e345 Logout link in nav, refs #875 2020-06-29 11:40:40 -07:00
Simon Willison 51427323e6 Add message when user logs out, refs #840 2020-06-29 11:31:35 -07:00
Simon Willison 35aee82c60 Fixed 500 error with /favicon.ico, closes #874 2020-06-28 21:27:11 -07:00
Simon Willison 22d932fafc /-/logout page for logging out of ds_actor cookie 
Refs #840
 2020-06-28 21:17:58 -07:00
Simon Willison a8a5f81372 Made show_messages available to plugins, closes #864 2020-06-28 17:50:47 -07:00
Simon Willison 7ac4936cec .add_message() now works inside plugins, closes #864 
Refs #870
 2020-06-28 17:25:35 -07:00
Simon Willison 0991ea75cc Renamed _timestamp to _now, refs #842, closes #871 2020-06-28 12:47:28 -07:00
Simon Willison 1f55a4a2b6 Release notes for 0.45a3 2020-06-27 20:22:49 -07:00
Simon Willison 335f26a0f7 /fixtures/magic_parameters demo, refs #842 2020-06-27 20:11:01 -07:00
Simon Willison 563f5a2d3a
Magic parameters for canned queries 
Closes #842

Includes a new plugin hook, register_magic_parameters()
 2020-06-27 19:58:16 -07:00
Simon Willison 4b142862f2 Support non-async view functions, closes #867 2020-06-27 11:30:34 -07:00
Simon Willison 1a5b7d318f Fixed test I broke in #863 2020-06-23 21:17:30 -07:00
Simon Willison 28bb1c5189 csrftoken() now works with .render_template(), closes #863 2020-06-23 20:23:50 -07:00
Simon Willison 751e7b4af7 Update tests for new plugin_hooks.rst, refs #687 2020-06-21 19:41:07 -07:00
Simon Willison b59b92b1b0 Fix for tests - order was inconsistent, refs #852 2020-06-18 16:52:16 -07:00
Simon Willison 6c26345836 New plugin hook: canned_queries(), refs #852 2020-06-18 16:35:15 -07:00
Simon Willison 6151c25a5a Respect existing scope["actor"] if set, closes #854 2020-06-18 11:37:28 -07:00
Simon Willison d2aef9f7ef Test illustrating POST against register_routes(), closes #853 2020-06-18 09:21:15 -07:00
Simon Willison a4ad5a504c Workaround for 'Too many open files' in test runs, refs #846 2020-06-13 17:26:18 -07:00
Simon Willison 0e49842e22 datasette/actor_auth_cookie.py coverae to 100%, refs #841 2020-06-13 11:29:14 -07:00
Simon Willison d60bd6ad13 Update plugin tests, refs #834 2020-06-13 11:15:33 -07:00
Simon Willison 72ae975156 Added test for async startup hook, refs #834 2020-06-13 10:58:32 -07:00
Simon Willison 09a3479a54 New "startup" plugin hook, closes #834 2020-06-13 10:55:41 -07:00
Simon Willison 308bcc8805 Fixed test_permissions_debug 2020-06-11 17:25:12 -07:00
Simon Willison fba8ff6e76 "$env": "X" mechanism now works with nested lists, closes #837 2020-06-11 17:21:48 -07:00
Simon Willison f39f111331 Fixed actor_matches_allow bug, closes #836 2020-06-11 15:47:19 -07:00
Simon Willison 29c5ff493a view-instance permission for debug URLs, closes #833 2020-06-11 15:14:51 -07:00
Simon Willison 98632f0a87
--secret command for datasette publish 
Closes #787
 2020-06-11 09:02:03 -07:00
Simon Willison 198545733b Document that "allow": {} denies all 
https://github.com/simonw/datasette/issues/831#issuecomment-642324847
 2020-06-10 16:56:53 -07:00
Simon Willison 57e812d5de ds_author cookie can now expire, closes #829 
Refs https://github.com/simonw/datasette-auth-github/issues/62#issuecomment-642152076
 2020-06-10 12:39:54 -07:00
Simon Willison 008e2f63c2 response.set_cookie(), closes #795 2020-06-09 15:19:37 -07:00
Simon Willison f240970b83 Fixed tests/fixtures.py, closes #804 2020-06-09 12:58:12 -07:00
Simon Willison 5ef3b7b0c9 Applied Black 
Refs #825
 2020-06-09 12:25:48 -07:00
Simon Willison 7633b9ab24 unauthenticated: true method plus allow block docs, closes #825 2020-06-09 10:01:03 -07:00
Simon Willison eb3ec279be
Test for anonymous: true, refs #825 2020-06-08 23:33:06 -07:00
Simon Willison fac8e93815 request.url_vars property, closes #822 2020-06-08 20:40:00 -07:00
Simon Willison db660db463 Docs + unit tests for Response, closes #821 2020-06-08 20:32:10 -07:00
Simon Willison f5e79adf26
register_routes() plugin hook (#819) 
Fixes #215
 2020-06-08 20:12:06 -07:00
Simon Willison 647c5ff0f3 Fixed broken CSS on 404 page, closes #777 2020-06-08 17:35:23 -07:00
Simon Willison 49d6d2f7b0 allow_sql block to control execute-sql upermission in metadata.json, closes #813 
Also removed the --config allow_sql:0 mechanism in favour of the new allow_sql block.
 2020-06-08 17:05:44 -07:00
Simon Willison 040fc0546f Updated tests, refs #817 2020-06-08 12:02:56 -07:00
Simon Willison 799c5d5357 Renamed resource_identifier to resource, refs #817 2020-06-08 11:59:53 -07:00
Simon Willison c9f1ec616e Removed resource_type from permissions system, closes #817 
Refs #811, #699
 2020-06-08 11:51:03 -07:00
Simon Willison 5598c5de01 Database list on index page respects table/view permissions, refs #811 2020-06-08 11:34:14 -07:00
Simon Willison dcec89270a View list respects view-table permission, refs #811 
Also makes a small change to the /fixtures.json JSON:

    "views": ["view_name"]

Is now:

    "views": [{"name": "view_name", "private": true}]
 2020-06-08 11:20:59 -07:00
Simon Willison 9ac27f67fe Show padlock on private query page, refs #811 2020-06-08 11:13:32 -07:00
Simon Willison aa420009c0 Show padlock on private table page, refs #811 2020-06-08 11:07:11 -07:00
Simon Willison 2a8b39800f Updated tests, refs #811 2020-06-08 07:50:06 -07:00
Simon Willison 3ce7f2e7da Show padlock on private database page, refs #811 2020-06-08 07:23:10 -07:00
Simon Willison 1cf86e5ecc Show padlock on private index page, refs #811 2020-06-08 07:18:47 -07:00
Simon Willison cc218fa9be Move assert_permissions_checked() calls from test_html.py to test_permissions.py, refs #811 2020-06-08 07:02:31 -07:00
Simon Willison 9397d71834 Implemented view-table, refs #811 2020-06-07 21:47:22 -07:00
Simon Willison b26292a458 Test that view-query is respected by query list, refs #811 2020-06-07 20:56:49 -07:00
Simon Willison 9b42e1a4f5 view-database permission 
Also now using 🔒 to indicate private resources - resources that
would not be available to the anonymous user. Refs #811
 2020-06-07 20:50:37 -07:00
Simon Willison 613fa551a1 Removed view-row permission, for the moment - refs #811 
https://github.com/simonw/datasette/issues/811#issuecomment-640338347
 2020-06-07 20:14:27 -07:00
Simon Willison cd92e4fe2a Fixed test name, this executes view-query, not execute-sql - refs #811 2020-06-07 14:33:56 -07:00
Simon Willison 8571ce388a Implemented view-instance permission, refs #811 2020-06-07 14:30:39 -07:00
Simon Willison ece0ba6f4b Test + default impl for view-query permission, refs #811 2020-06-07 14:23:16 -07:00
Simon Willison abc7339124 Nicer pattern for make_app_client() in tests, closes #395 2020-06-07 14:14:10 -07:00
Simon Willison a1e801453a Renamed execute-query permission to execute-sql, refs #811 2020-06-07 13:20:59 -07:00
Simon Willison 4340845754 Nested permission checks for all views, refs #811 2020-06-07 13:03:08 -07:00
Simon Willison 86dec9e8ff Added permission check to every view, closes #808 2020-06-06 22:30:36 -07:00
Simon Willison 3359d54a4e Use cookies when accessing csrftoken_from 2020-06-06 12:33:08 -07:00
Simon Willison 966eec7f75 Check permissions on canned query page, refs #800 2020-06-06 12:27:00 -07:00
Simon Willison 070838bfa1 Better test for Vary header 2020-06-06 12:26:19 -07:00
Simon Willison 3f83d4632a Respect query permissions on database page, refs #800 2020-06-06 12:05:22 -07:00
Simon Willison 14f6b4d200 actor_matches_allow utility function, refs #800 2020-06-06 11:39:11 -07:00
Simon Willison 75c143a84c Fixed /-/plugins?all=1, refs #802 2020-06-05 16:55:08 -07:00
Simon Willison f786033a5f Fixed 'datasette plugins' command, with tests - closes #802 2020-06-05 16:46:37 -07:00
Simon Willison 033a1bb22c Removed rogue print() from test 2020-06-05 12:06:43 -07:00
Simon Willison 84a9c4ff75
CSRF protection (#798) 
Closes #793.

* Rename RequestParameters to MultiParams, refs #799
* Allow tuples as well as lists in MultiParams, refs #799
* Use csrftokens when running tests, refs #799
* Use new csrftoken() function, refs https://github.com/simonw/asgi-csrf/issues/7
* Check for Vary: Cookie hedaer, refs https://github.com/simonw/asgi-csrf/issues/8
 2020-06-05 12:05:57 -07:00
Simon Willison d96ac1d52c Allow tuples as well as lists in MultiParams, refs #799 2020-06-05 11:01:06 -07:00
Simon Willison 0da7f49b24 Rename RequestParameters to MultiParams, refs #799 2020-06-05 10:52:50 -07:00
Simon Willison 9cb44be42f Docs and tests for "params", closes #797 2020-06-03 14:04:40 -07:00
Simon Willison aa82d03704
Basic writable canned queries 
Refs #698. First working version of this feature.

* request.post_vars() no longer discards empty values
 2020-06-03 08:16:50 -07:00
Simon Willison 0934844c0b request.post_vars() no longer discards empty values 2020-06-03 06:48:39 -07:00
Simon Willison a7137dfe06 /-/plugins now shows details of hooks, closes #794 
Also added /-/plugins?all=1 parameter to see default plugins.
 2020-06-02 14:49:28 -07:00
Simon Willison 5278c04682 More consistent use of response.text/response.json in tests, closes #792 2020-06-02 14:29:12 -07:00
Simon Willison 4fa7cf6853 Flash messages mechanism, closes #790 2020-06-02 14:12:18 -07:00
Simon Willison dfdbdf378a Added /-/permissions debug tool, closes #788 
Also started the authentication.rst docs page, refs #786.

Part of authentication work, refs #699.
 2020-05-31 22:00:36 -07:00
Simon Willison 9f3d4aba31 --root option and /-/auth-token view, refs #784 2020-05-31 18:16:42 -07:00
Simon Willison fa27e44fe0 datasette.sign() and datasette.unsign() methods, refs #785 2020-05-31 18:16:42 -07:00
Simon Willison 1fc6ceefb9 Added /-/actor.json - refs #699 
Also added JSON highlighting to introspection documentation.
 2020-05-31 18:16:42 -07:00
Simon Willison 9315bacf6f Implemented datasette.permission_allowed(), refs #699 2020-05-31 18:16:42 -07:00
Simon Willison 461c82838d Implemented actor_from_request with tests, refs #699 
Also added datasette argument to permission_allowed hook
 2020-05-31 18:16:42 -07:00
Simon Willison 4d798ca0e3 Added test for db.mtime_ns 2020-05-30 11:17:20 -07:00
Simon Willison de1cde65a6 Moved request tests to test_internals_request.py 2020-05-30 10:45:11 -07:00
Simon Willison ca56c226a9 Renamed test_database.py to test_internals_database.py 
Also added a db fixture to remove some boilerplate.
 2020-05-30 07:33:02 -07:00
Simon Willison 31fb006a9b Added datasette.get_database() method 
Refs #576
 2020-05-30 07:29:59 -07:00
Simon Willison 81be31322a New implementation for RequestParams 
- no longer subclasses dict
- request.args[key] now returns first item, not all items
- removed request.raw_args entirely

Closes #774
 2020-05-29 16:22:22 -07:00
Simon Willison 84616a2364 request.args.getlist() returns [] if missing, refs #774 
Also added some unit tests for request.args
 2020-05-29 15:51:30 -07:00
Simon Willison 7bb30c1f11 request.url now respects force_https_urls, closes #781 2020-05-28 10:10:06 -07:00
Simon Willison 5ab411c733 can_render mechanism for register_output_renderer, closes #770 2020-05-27 22:57:05 -07:00
Simon Willison 75cd432e5a Ability to set custom table/view page size in metadata, closes #751 2020-05-27 22:00:04 -07:00
Simon Willison 510c1989d4 Removed xfail, refs #773 2020-05-27 21:11:53 -07:00
Simon Willison 6d95cb4f91 Unit test for register_facet_classes plugin, closes #773 
I was a bit lazy with this one. I didn't hook up a test for the facet_results mechanism.
The custom facet hook isn't a great design so I will probably rethink it at some point
in the future anyway.
 2020-05-27 21:09:16 -07:00
Simon Willison defead17a4 Test for publish_subcommand hook, refs #773 2020-05-27 20:30:32 -07:00
Simon Willison cbeea23d00 Test for prepare_jinja2_environment, refs #773 2020-05-27 20:13:32 -07:00
Simon Willison 57f48b8416 Made register_output_renderer callback optionally awaitable, closes #776 2020-05-27 19:43:30 -07:00
Simon Willison 52c4387c7d Redesigned register_output_renderer plugin hook, closes #581 2020-05-27 19:21:41 -07:00
Simon Willison 446e5de65d Refactored test plugins into tests/plugins, closes #775 2020-05-27 17:57:25 -07:00
Simon Willison ad88c9b3f3 Mechanism for adding a default URL fragment to a canned query 
Closes #767
 2020-05-27 14:52:03 -07:00
Simon Willison da87e963bf
Test that plugin hooks are unit tested (xfail) 
This currently fails using xfail. Closes 771.
 2020-05-27 13:16:02 -07:00
Simon Willison 41a0cd7b6a call_with_supported_arguments() util, refs #581 2020-05-27 12:25:52 -07:00
Simon Willison 5ea8c6d1cd type-pk instead of type-link CSS class, closes #729 2020-05-14 22:55:20 -07:00
Simon Willison 504196341c Visually distinguish float/int columns, closes #729 2020-05-14 22:51:39 -07:00
Simon Willison 2694ddcf14 Test for .execute_fn(), refs #685 2020-05-08 10:29:17 -07:00
Simon Willison 4433306c18
Improvements + docs for db.execute() and Results class 
* Including new results.first() and results.single_value() methods. Closes #685
 2020-05-08 09:05:46 -07:00
Simon Willison 0784f2ef9d Allow specific pragma functions, closes #761 2020-05-06 10:18:31 -07:00
Simon Willison cc872b1f50 Fixed rogue output in tests, closes #755 2020-05-04 11:42:01 -07:00
Simon Willison 9424687e9e Consistently return charset utf-8, closes #752 2020-05-04 10:42:10 -07:00
Simon Willison 450d2e2896 Fixed pytest warning about TestClient class 2020-05-04 10:42:10 -07:00
Simon Willison cef23e8861 Started pattern portfolio at /-/patterns, refs #151 2020-05-02 20:05:25 -07:00
Simon Willison b3aa5f4313 Added 'not like' table filter, refs #750 2020-05-02 12:04:54 -07:00
Simon Willison 1d91ab71d4 Directory configuration mode supports metadata.yaml, closes #747 2020-04-30 11:47:41 -07:00
Simon Willison 89c4ddd482 403 for static directory listing, closes #740 2020-04-27 11:29:04 -07:00
Simon Willison 25014ca25e
Configuration directory mode, closes #731 2020-04-27 09:30:24 -07:00
Simon Willison 1b7b66c465 Make request available when rendering custom pages, closes #738 2020-04-26 12:01:46 -07:00
Simon Willison 304e7b1d9f
Mechanism for creating custom pages using templates 
Closes #648
 2020-04-26 11:46:43 -07:00
Simon Willison 15e2321804 Extra body CSS class for canned queries, closes #727 2020-04-15 14:07:28 -07:00
Simon Willison d349d57cdf Smarter merging of metadata and extra_metadata, closes #724 2020-04-10 11:34:09 -07:00
Simon Willison d55fe8cdfc Fixed bug with Templates considered comment, closes #689 2020-04-05 12:38:33 -07:00
Simon Willison e89b0ef2f9 Expose extra_template_vars in _contex=1, refs #693 2020-04-05 11:49:15 -07:00
Simon Willison 09253817de Fix for missing view_name bug, closes #716 2020-04-05 11:28:20 -07:00
Simon Willison e0e7a0facf Removed Zeit Now v1 support, closes #710 2020-04-04 16:04:33 -07:00
Simon Willison b07312c2b3 dedent SQL for neighborhood_search fixture 
Makes this page a little prettier:
https://latest.datasette.io/fixtures/neighborhood_search
 2020-04-02 17:54:27 -07:00
Simon Willison 6717c719dd
--metadata accepts YAML as well as JSON - closes #713 2020-04-02 12:30:53 -07:00
Simon Willison 2aaad72789 Refactor template setup into Datasette constructor 
Closes #707
 2020-03-26 18:12:43 -07:00
Simon Willison 6aa516d82d Run base_url tests against /fixtures/facetable too, refs #712 2020-03-25 19:31:22 -07:00
Simon Willison 7656fd64d8
base_url configuration setting, closes #394 
* base_url configuration setting
* base_url works for static assets as well
 2020-03-24 17:18:43 -07:00
Simon Willison 5f4aeb1f19 Removed documentation for Zeit Now v1, refs #710 2020-03-24 15:45:24 -07:00
Simon Willison a498d0fe65 Fix bug with over-riding default sort, closes #702 2020-03-21 19:40:29 -07:00
Simon Willison 236aa065b2 "sort" and "sort_desc" metadata properties, closes #702 2020-03-21 19:28:35 -07:00
Simon Willison e1a817411a Bump to click 7.1.1 to fix flaky tests 2020-03-21 18:47:51 -07:00
Simon Willison 2c0e1e09bc Show sort arrow on primary key by default 
Closes #677. Refs #702.
 2020-03-21 16:57:37 -07:00
Simon Willison a000c80d50 await Request(scope, receive).post_vars() method, closes #700 
Needed for #698
 2020-03-16 19:47:37 -07:00
Simon Willison e1b5339fdf Do not look for templates_path in default plugins 
Closes #697
 2020-03-08 16:11:18 -07:00
Simon Willison ddd11b3ddd --memory option for publish cloudrun, refs #694 2020-03-05 17:34:36 -06:00
Simon Willison 7f5a330377
Don't count rows on homepage for DBs > 100MB (#688) 
Closes #649.
 2020-02-28 17:08:29 -08:00
Kevin Keogh 3041c6b641
Use inspect-file, if possible, for total row count (#666) 
For large tables, counting the number of rows in the table can take a
significant amount of time. Instead, where an inspect-file is provided
for an immutable database, look up the row-count for a plain count(*).

Thanks, @kevindkeogh
 2020-02-25 12:19:29 -08:00
Simon Willison 6cb65555f4
?_searchmode=raw option (#686) 2020-02-24 21:56:03 -08:00
Simon Willison a093c5f79f
.execute_write() and .execute_write_fn() methods on Database (#683) 
Closes #682.
 2020-02-24 20:45:07 -08:00
Simon Willison 7c6a9c3529 Better tests for prepare_connection() plugin hook, refs #678 2020-02-21 18:27:07 -08:00
Simon Willison d3f2fade88 Refactored run_sanity_checks to check_connection(conn), refs #674 2020-02-15 09:56:48 -08:00
Simon Willison cf5f4386ef Run black against everything, not just tests and datasette dirs 2020-02-13 15:02:10 -08:00
Jay Graves 33a12c8ae5
Allow leading comments in SQL input field (#653) 
Thanks, @jaywgraves!
 2020-02-04 18:13:24 -08:00
Simon Willison 67fc9c5720
--port argument for datasette package, plus tests - closes #661 
From pull request #663
 2020-01-29 14:46:43 -08:00
Katie McLaughlin 34d77d780f gcloud run is now GA, s/beta// (#660) 
Thanks, @glasnt
 2020-01-21 15:28:11 -08:00
Simon Willison 3c861f363d _search= queries now correctly escaped, fixes #651 
Queries with reserved words or characters according to the SQLite
FTS5 query language could cause errors.

Queries are now escaped like so:

    dog cat => "dog" "cat"
 2019-12-29 18:48:30 +00:00
Simon Willison d54318fc7f Added template_debug setting, closes #654 2019-12-22 16:04:45 +00:00
Simon Willison d6b6c9171f Include asyncio task information in /-/threads debug page 2019-12-04 22:47:17 -08:00
Simon Willison d3e1c3017e Display 0 results, closes #637 2019-11-22 22:07:01 -08:00
Simon Willison fd137da7f8 Suggest column facet only if at least one count > 1 
Fixes #638
 2019-11-21 16:56:55 -08:00
Simon Willison 440a70428c Include rowid in filter select, closes #636 2019-11-19 15:01:10 -08:00
Simon Willison 8c642f04e0
Render templates using Jinja async mode 
Closes #628
 2019-11-14 15:14:22 -08:00
Simon Willison f524510230 Fix "publish heroku" + upgrade to use Python 3.8.0 
Closes #633. Closes #632.
 2019-11-13 08:42:47 -08:00
Simon Willison d977fbadf7
datasette publish uses python:3.8 base Docker image, closes #629 2019-11-11 22:03:09 -08:00
Simon Willison cf7776d36f
Support Python 3.8, stop supporting Python 3.5 (#627) 
* Upgrade to uvicorn 0.10.4
* Drop support for Python 3.5
* Bump all dependencies to latest releases
* Update docs to reflect we no longer support 3.5
* Removed code that skipped black unit test on 3.5

Closes #622
 2019-11-11 21:09:11 -08:00
Simon Willison 28c4a6db5b CREATE INDEX statements on table page, closes #618 2019-11-09 17:29:36 -08:00
Simon Willison 83fc5165ac Improved UI for publish cloudrun, closes #608 2019-11-07 18:48:39 -08:00
Tobias Kunze 931bfc6661 Handle spaces in DB names (#590) 
Closes #503 - thanks, @rixx
 2019-11-04 15:16:30 -08:00
Simon Willison 52fa79c607 Use select colnames, not select * for table view - refs #615 2019-11-04 15:03:48 -08:00
Simon Willison 9db22cdf18 pk__notin= filter, closes #614 2019-11-03 20:11:55 -08:00
Simon Willison 2bf7ce5f51 Fix CSV export for nullable foreign keys, closes #612 2019-11-02 16:12:46 -07:00
Simon Willison 14da70525b Don't show 'None' as label for nullable foreign key, closes #406 2019-11-02 15:29:40 -07:00
Simon Willison 7152e76eda Don't suggest array facet if column is only [], closes #610 2019-11-01 14:45:59 -07:00
Simon Willison 50287e7c6b Only suggest array facet for arrays of strings - closes #562 2019-11-01 12:37:46 -07:00
Simon Willison e2c390500e Persist _where= in hidden fields, closes #604 2019-10-30 11:49:26 -07:00
Simon Willison 5dd4d2b2d3
Update to latest black (#609) 2019-10-30 11:49:01 -07:00
chris48s f4c0830529 Always pop as_format off args dict (#603) 
Closes #563. Thanks, @chris48s
 2019-10-20 19:03:08 -07:00
Simon Willison b647b5efc2
Fix for /foo v.s. /foo-bar issue, closes #597 
Pull request #599
 2019-10-18 15:51:07 -07:00
Simon Willison b6ad1fdc70 Fixed bug returning non-ascii characters in CSV, closes #584 2019-10-17 22:23:01 -07:00
Simon Willison 3e864b1625 Use --platform=managed for publish cloudrun, closes #587 2019-10-17 14:51:45 -07:00
Tobias Kunze 12cec411ca Display metadata footer on custom SQL queries (#589) 
Closes #408 - thanks, @rixx!
 2019-10-13 20:53:21 -07:00
Tobias Kunze 908fc3999e Sort databases on homepage by argument order - #591 
Closes #585 - thanks, @rixx!
 2019-10-13 20:52:33 -07:00
Simon Willison fffd69ec03 Allow EXPLAIN WITH... - closes #583 2019-10-06 10:23:58 -07:00
Simon Willison 2dc5c8dc25
detect_fts now works with alternative table escaping (#571) 
Fixes #570. See also https://github.com/simonw/sqlite-utils/pull/57
 2019-09-02 17:32:27 -07:00
Simon Willison 90d4f497f9 Fix plus test for unicode characters in custom query name, closes #558 2019-07-13 19:49:24 -07:00
Abdus 74ecf8a7cc Fix static mounts using relative paths and prevent traversal exploits (#554) 
Thanks, @abdusco! Closes #555
 2019-07-11 09:13:19 -07:00
Simon Willison 973f8f139d
--plugin-secret option for datasette publish 
Closes #543

Also added new --show-files option to publish now and publish cloudrun - handy for debugging.
 2019-07-07 19:06:31 -07:00
Simon Willison c5542abba5 Removed ManyToManyFacet for the moment, closes #550 2019-07-07 16:21:11 -07:00
Simon Willison fcfcae21e6
extra_template_vars plugin hook (#542) 
* extra_template_vars plugin hook

Closes #541

* Workaround for cwd bug

Based on https://github.com/pytest-dev/pytest/issues/1235#issuecomment-175295691
 2019-07-05 17:05:56 -07:00
Simon Willison 25ff0a8ba6 Fix for accidentally leaking secrets in /-/metadata, closes #538 2019-07-03 22:47:45 -07:00
Simon Willison a2d4593193
Secret plugin configuration options (#539) 
Closes #538
 2019-07-03 22:36:44 -07:00
Simon Willison 4d2fdafe39 Added asgi_wrapper plugin hook, closes #520 2019-07-02 21:06:25 -07:00
Simon Willison 13e4f9a676 Unit test for _table custom template, refs #521 2019-07-02 20:23:05 -07:00
Simon Willison ba8db9679f
Port Datasette from Sanic to ASGI + Uvicorn (#518) 
Datasette now uses ASGI internally, and no longer depends on Sanic.

It now uses Uvicorn as the underlying HTTP server.

This was thirteen months in the making... for full details see the issue:

https://github.com/simonw/datasette/issues/272

And for a full sequence of commits plus commentary, see the pull request:

https://github.com/simonw/datasette/pull/518
 2019-06-23 20:13:09 -07:00
Simon Willison 35429f9089 Revert "New encode/decode_path_component functions" 
Refs #272

This reverts commit 9fdb47ca95.

Now that ASGI supports raw_path we don't need our own encoding scheme!
 2019-06-18 17:23:27 -07:00
Simon Willison 2a4b892d6c Tidy up with Black 2019-05-27 19:28:47 -07:00
Simon Willison 9c58d048ec Fix pagination when sorted by expanded foreign key 
Closes #489
 2019-05-27 19:23:18 -07:00
Simon Willison de92c05134 Fixed duplicate function name 2019-05-27 18:49:57 -07:00
Simon Willison 839e55bf0a Start of unit tests for Database class, refs #485 2019-05-27 18:42:30 -07:00
Simon Willison 182a3017c2 Sort keys to past tests in Python 3.5 2019-05-25 09:22:02 -07:00
Simon Willison d923d84754 Facet by many-to-many, closes #365 2019-05-25 09:06:32 -07:00
Simon Willison 172da009d8 Added ?_through= table argument, closes #355 
Also added much more interesting many-to-many fixtures - roadside attractions!
 2019-05-22 22:45:12 -07:00
Simon Willison 967230c90e Facet by date, closes #481 2019-05-20 23:09:22 -07:00
Simon Willison 2d14f21f85 Updated tests for date(...) lookup 2019-05-20 23:06:19 -07:00
Simon Willison f4eefdf193 Do not allow downloads of mutable databases - closes #474 2019-05-19 13:41:09 -07:00
Simon Willison 7d8573d672 Rename "datasette publish now" to "datasette publish nowv1" 
Also added an alias so "datasette publish now" continues to work.

Closes #472
 2019-05-19 11:06:51 -07:00
Simon Willison db9dbfb816 ?_hash=1 no longer respected for mutable databases 
Closes #471, refs #419
 2019-05-16 08:10:25 -07:00
Simon Willison 0dad111d24 New introspection endpoint: /-/databases - closes #470 
Refs #419 and #465
 2019-05-16 07:49:34 -07:00
Simon Willison 21b57cdf9a serve --inspect-file=X now populates cached table counts 
Closes #462
 2019-05-16 07:30:32 -07:00
Simon Willison 909e66dfdb Removed .inspect() and /-/inspect.json 
Refs #462

/-/inspect.json may return in some shape in #465
 2019-05-16 07:15:24 -07:00
Simon Willison faf33515b2 Sometimes sort tables by number of relationships, closes #460 2019-05-15 20:23:33 -07:00
Simon Willison 689cf9c139 Index page only shows row counts for smaller databases 
The index page now only shows row counts for immutable databases OR for
databases with less than 30 tables provided it could get a count for
each of those tables in less than 10ms.

Closes #467, Refs #460
 2019-05-15 20:02:33 -07:00
Simon Willison 79303163b2 Row count fix + sort index page databases alphabetically 
Sorting alphabetically should fix a test failure in Python 3.5

Refs #460
 2019-05-15 17:40:10 -07:00
Simon Willison 5d6b2c30f1 Include views on homepage, fix table counts 
If we have less than 5 tables we now also show one or more views in the
summary on the homepage.

Also corrected the logic for the row counts - we now count hidden and
visible tables separately.

Closes #373, Refs #460
 2019-05-15 17:28:07 -07:00
Simon Willison e04511410f Don't show hidden tables on index page, closes #455 
Refs #460. Also bulked out HTML index page unit tests.
 2019-05-14 08:46:57 -07:00
Simon Willison 24bf79d2f0 Run sanity checks, not .inspect(), on startup 
Also fixes tests that did NOT like a call to run_until_complete in the Datasette() constructor.
 2019-05-11 16:22:55 -07:00
Simon Willison da0b3ce2b7 New run_sanity_checks mechanism, for SpatiLite 
Moved VirtualSpatialIndex check into a new mechanism that should allow
us to add further sanity checks in the future.

To test this I've had to commit a binary sample SpatiaLite database to
the repository. I included a build script for creating that database.

Closes #466
 2019-05-11 15:55:30 -07:00
Simon Willison c692cd2911 Fix test ordering 2019-05-11 15:03:52 -07:00
Simon Willison a187c81e0c Black + fix broken test 2019-05-11 14:45:59 -07:00
Simon Willison c0d1b4c322 "datasette inspect foo.db" now just calculates table counts 
Refs #462

* inspect command now just outputs table counts
* test_inspect.py is now only tests for that CLI command
* Updated some relevant documentation
* Removed docs for /-/inspect since that is about to change
 2019-05-11 14:36:57 -07:00
Simon Willison ce09e5d2d3 Fixed tests relating to #459 2019-05-11 13:58:22 -07:00
Simon Willison 05cabaebd9 Pass --token to now alias, refs #459 2019-05-11 13:35:34 -07:00
Simon Willison 09ef305c68 Fixed "datasette publish now ... --alias=x" 
The --alias argument can now be passed more than once.

Also updated our Travis configuration to use this.

Fixes #459
 2019-05-11 13:20:36 -07:00
Simon Willison a0d444837f Finished implementation of ?_trace=1 debug tool 
I redesigned the JSON output and added a handy "traceback" key showing
three relevant lines of the current traceback for each logged query.

Closes #435
 2019-05-11 12:06:22 -07:00
Simon Willison 2db977d8f3 tests/fixtures.py can now write out plugins too 
This command:

    python tests/fixtures.py \
        fixtures.db \
        metadata.json \
        fixtures-plugins/

Will now create the fixtures.db and metadata.json files, AND create
a folder called fixtures-plugins/ containing two test plugins.

You can then run it like this:

    datasette fixtures.db \
        -m metadata.json --plugins-dir=fixtures-plugins/
 2019-05-09 09:44:21 -07:00
Simon Willison 9fdb47ca95 New encode/decode_path_component functions 
ASGI cannot differentiate between / and %2F in a URL, so we need an
alternative scheme for encoding the names of tables that contain special
characters such as /

For background, see
    https://github.com/django/asgiref/issues/51#issuecomment-450603464

Some examples:

    "table/and/slashes" => "tableU+002FandU+002Fslashes"
    "~table" => "U+007Etable"
    "+bobcats!" => "U+002Bbobcats!"
    "U+007Etable" => "UU+002B007Etable"
 2019-05-09 09:44:21 -07:00
Simon Willison ec2db345e9 Fixed crash on /:memory: page 2019-05-05 14:01:14 -04:00
Simon Willison 831515b834 Respect --cors for error pages, closes #453 2019-05-05 08:12:03 -04:00
Simon Willison 35d6ee2790
Apply black to everything, enforce via unit tests (#449) 
I've run the black code formatting tool against everything:

    black tests datasette setup.py

I also added a new unit test, in tests/test_black.py, which will fail if the code does not
conform to black's exacting standards.

This unit test only runs on Python 3.6 or higher, because black itself doesn't run on 3.5.
 2019-05-03 22:15:14 -04:00
Simon Willison 93c65d29ba Fixed 500 error on /-/metadata page 2019-05-03 16:42:59 -04:00
Simon Willison 01b3de5b66 Unit test for binary data display, refs #442 2019-05-03 12:45:06 -04:00
Simon Willison bade9daeb2 Docs for 'datasette publish cloudrun', refs #434 2019-05-03 12:16:58 -04:00
Romain Primet 75a21fc2a1 datasette publish cloudrun (#434) - thanks, @rprimet 
New publish subcommand that publishes using the
new Google Cloud Run platform.

    datasette publish cloudrun database.db
 2019-05-03 09:59:01 -04:00
Simon Willison 53d2f00b73 Implemented ArrayFacet, closes #359 2019-05-02 20:21:27 -04:00
Simon Willison ea66c45df9
Extract facet code out into a new plugin hook, closes #427 (#445) 
Datasette previously only supported one type of faceting: exact column value counting.

With this change, faceting logic is extracted out into one or more separate classes which can implement other patterns of faceting - this is discussed in #427, but potential upcoming facet types include facet-by-date, facet-by-JSON-array, facet-by-many-2-many and more.

A new plugin hook, register_facet_classes, can be used by plugins to add in additional facet classes.

Each class must implement two methods: suggest(), which scans columns in the table to decide if they might be worth suggesting for faceting, and facet_results(), which executes the facet operation and returns results ready to be displayed in the UI.
 2019-05-02 17:11:26 -07:00
Simon Willison efc93b8ab5 Entirely removed table_rows_count table property 
We were not displaying this anywhere, and it is now expensive to calculate.

Refs #419, #420
 2019-05-01 22:29:47 -07:00
Simon Willison ad9de774f7 Added missing file, refs #438 2019-05-01 22:10:23 -07:00
Simon Willison f553a67021 Don't load setuptools plugins during test runs 
Uses pattern from https://docs.pytest.org/en/latest/example/simple.html#detect-if-running-from-within-a-pytest-run

Closes #438
 2019-05-01 22:09:03 -07:00
Simon Willison 50d2d1aac9 Fixed bug where metadata.json hidden tables were ignored 2019-05-01 17:54:48 -07:00
Simon Willison e7151ccccf Index page no longer uses inspect data - refs #420 
Also introduced a mechanism whereby table counts are calculated against a time limit
but immutable databases have their table counts calculated on server startup.
 2019-05-01 17:39:39 -07:00
Simon Willison 669fa21a71 Include request duration in traces 2019-05-01 16:27:14 -07:00
Russ Garrett cf406c0754 New plugin hook: register_output_renderer hook (#441) 
Thanks @russss!

* Add register_output_renderer hook

This changeset refactors out the JSON renderer and then adds a hook and
dispatcher system to allow custom output renderers to be registered.

The CSV output renderer is untouched because supporting streaming
renderers through this system would be significantly more complex, and
probably not worthwhile.

We can't simply allow hooks to be called at request time because we need
a list of supported file extensions when the request is being routed in
order to resolve ambiguous database/table names. So, renderers need to
be registered at startup.

I've tried to make this API independent of Sanic's request/response
objects so that this can remain stable during the switch to ASGI. I'm
using dictionaries to keep it simple and to make adding additional
options in the future easy.

Fixes #440
 2019-05-01 16:01:56 -07:00
Simon Willison e4e73a537a Test for ?_trace=1, refs #435 2019-04-21 10:46:56 -07:00
Simon Willison 9c77e6e355 Support multiple filters of the same type 
Closes #288
 2019-04-15 16:44:17 -07:00
Simon Willison 583b22aa28 New ?column__date=yyyy-mm-dd filter 2019-04-15 15:54:54 -07:00
Simon Willison 1c6649b19b New colname__in=x,y,z filter, closes #433 2019-04-15 15:43:22 -07:00
Simon Willison 2c19a27d15 Documentation for filters, plus new documentation unit test 
https://simonwillison.net/2018/Jul/28/documentation-unit-tests/
 2019-04-15 15:41:11 -07:00
Simon Willison 6da567dda9 Extract and refactor filters into filters.py 
This will help in implementing __in as a filter, refs #433
 2019-04-15 14:51:20 -07:00
Simon Willison bc6a9b4564
?_where= parameter on table views, closes #429 
From pull request #430
 2019-04-12 18:37:22 -07:00
Simon Willison e11cb4c664 Persist show/hide state better, closes #425 2019-04-11 22:00:47 -07:00
Simon Willison db74cf0144 ?_fts_table= and ?_fts_pk= arguments, closes #428 2019-04-11 21:21:17 -07:00
Simon Willison 78e45ead4d New ?tags__arraycontains=tag lookup against JSON fields 
Part one of supporting facet-by-JSON-array, refs #359
 2019-04-10 08:27:52 -07:00
Simon Willison 53bf875483 expand_foreign_keys() no longer uses inspect, refs #420 2019-04-06 19:56:07 -07:00
Simon Willison 97331f3435 sortable_columns_for_table() no longer uses inspect() 
Refs #420
 2019-04-06 18:58:51 -07:00
Simon Willison 468c6fd953 DatabaseDownload no longer uses .inspect(), refs #420 2019-03-31 19:04:17 -07:00
Simon Willison 7d0f668556 .resolve_db_name() and .execute() work without inspect 
Refs #420
 2019-03-31 16:51:52 -07:00
Simon Willison 0209a0a344 table_exists() now uses async SQL, refs #420 2019-03-31 11:02:22 -07:00
Simon Willison 6f6d0ff2b4
URL hashing is now off by default - closes #418 
Prior to this commit Datasette would calculate the content hash of every
database and redirect to a URL containing that hash, like so:

    https://v0-27.datasette.io/fixtures => https://v0-27.datasette.io/fixtures-dd88475

This assumed that all databases were opened in immutable mode and were not
expected to change.

This will be changing as a result of #419 - so this commit takes the first step
in implementing that change by changing this default behaviour. Datasette will
now only redirect hash-free URLs under two circumstances:

* The new `hash_urls` config option is set to true (it defaults to false).
* The user passes `?_hash=1` in the URL
 2019-03-17 15:55:04 -07:00
Simon Willison afe9aa3ae0 show/hide link for SQL on custom query page 
Closes #415
 2019-03-14 22:22:35 -07:00
Simon Willison 2855667908 Fix for test failure with Click 7.0 2019-03-14 22:00:13 -07:00
Simon Willison 9743e1d91b Support for :memory: databases 
If you start Datasette with no files, it will connect to :memory: instead.

When starting it with files you can add --memory to also get a :memory: database.
 2019-03-14 20:54:42 -07:00
Simon Willison bf6b0f918d about and about_url metadata options 2019-03-14 20:54:42 -07:00
Simon Willison 4462a5ab28 Show size of database file next to download link, closes #172 2019-02-05 20:58:29 -08:00
Simon Willison 195a5b3634
Heroku --include-vcs-ignore (#407) 
Means `datasette publish heroku` can work under Travis, unlike this failure:

https://travis-ci.org/simonw/fivethirtyeight-datasette/builds/488047550

```
2.25s$ datasette publish heroku fivethirtyeight.db -m metadata.json -n fivethirtyeight-datasette
tar: unrecognized option '--exclude-vcs-ignores'
Try 'tar --help' or 'tar --usage' for more information.
 ▸    Command failed: tar cz -C /tmp/tmpuaxm7i8f --exclude-vcs-ignores --exclude
 ▸    .git --exclude .gitmodules . >
 ▸    /tmp/f49440e0-1bf3-4d3f-9eb0-fbc2967d1fd4.tar.gz
 ▸    tar: unrecognized option '--exclude-vcs-ignores'
 ▸    Try 'tar --help' or 'tar --usage' for more information.
 ▸    
The command "datasette publish heroku fivethirtyeight.db -m metadata.json -n fivethirtyeight-datasette" exited with 0.
```

The fix for that issue is to call the heroku command like this:

    heroku builds:create -a app_name --include-vcs-ignore
 2019-02-05 20:15:46 -08:00
Simon Willison b5dd83981a Export option: _shape=array&_nl=on for newline-delimited JSON 2019-01-27 17:40:23 -08:00
Simon Willison c3a78eb05c app_client() fixture doesn't need to take **kwargs 2019-01-13 14:23:44 -08:00
Simon Willison 8ce7866312 compile_options output in /-/versions, closes #396 2019-01-10 16:44:50 -08:00
Simon Willison 996e8822d2 Fix CSV export hidden form fields, closes #393 2019-01-02 18:43:56 -08:00
Simon Willison a2bfcfc1b1
Fix some regex DeprecationWarnings (#392) 2018-12-28 18:22:27 -08:00
Simon Willison 3de8fac1d3
New make_app_client() pattern 
Because next version of pytest discourages calling fixture factories as functions
 2018-11-20 09:46:47 -08:00
Simon Willison b7257a21bf
Fix json.loads in Python 3.5 
3.5 requires a str, not a bytes

https://travis-ci.org/simonw/datasette/jobs/421660555
 2018-08-28 17:55:30 +01:00
Simon Willison b7c6a9f9bd
extra_css_urls(template, database, table, datasette) 
The extra_css_urls and extra_js_urls hooks now take additional optional
parameters.

Also refactored them out of the Datasette class and into RenderMixin.

Plus improved plugin documentation to explicitly list parameters.
 2018-08-28 11:56:57 +01:00
Simon Willison 2e836f72d9
render_cell(value, column, table, database, datasette) 
The render_cell plugin hook previously was only passed value.

It is now passed (value, column, table, database, datasette).
 2018-08-28 03:03:01 -07:00
Simon Willison 5cf0c6c91c
New plugin hook: extra_body_script 2018-08-28 02:02:49 -07:00
Simon Willison 0a14a4846b
Added plugin_config() method 2018-08-28 01:35:21 -07:00
Simon Willison aae49fef3b
Import pysqlite3 if available, closes #360 (#361) 2018-08-15 17:58:56 -07:00
Simon Willison fe5b6ea95a
Hide 'view and edit SQL' if config.allow_sql turned off 2018-08-05 20:17:17 -07:00
Simon Willison 5629aaca67
sortable_columns also now works with views 2018-08-05 17:29:23 -07:00
Simon Willison 4ac9132240
render_cell(value) plugin hook, closes #352 
New plugin hook for customizing the way cells values are rendered in HTML.

The first full example of this hook in use is https://github.com/simonw/datasette-json-html
 2018-08-04 17:14:56 -07:00
Simon Willison 295d005ca4
Renamed variable, since docs are not written in Markdown 2018-07-27 21:56:51 -07:00
Simon Willison 8128497b79
Only run documented_views() fixture once per session 
Speeds up tests, because previously it ran once per view class.
 2018-07-27 21:49:20 -07:00
Simon Willison 788a542d3c
Docs for IndexView, TableView, RowView, closes #299 
Also removed xfail from test_view_classes_are_documented, so any future *View
classes that are added without documentation will cause the tests to fail.
 2018-07-27 21:21:42 -07:00
Simon Willison e8625695a3
xfail documentation unit tests for view classes, refs #299 
More documentation unit tests. These ones check that every single **View class
imported into the datasette/app.py module are covered by our documentation.

Just one problem: they aren't documented yet. So I'm using the xfail pytest
decorator to mark these tests as allowed-to-fail. When you run the test suite
you now get a report of how many views still need to be documented, but it
doesn't fail the tests.

The output looks something like this:

    $ pytest tests/test_docs.py
    collected 31 items

    tests/test_docs.py ..........................XXXxx.                [100%]

    ============ 26 passed, 2 xfailed, 3 xpassed in 1.06 seconds ============

Once I have documented all the views I will remove the xfail so any future
views that are added without documentation will cause a test failure.

We can detect that a view is documented by looking for ReST label in the docs,
for example:

    .. _IndexView:

Some view classes can be used to power multiple URLs - the JsonDataView class
for example is used to power /-/metadata and /-/config and /-/plugins

In this case, the second part of the label can indicate the variety of page, e.g:

    .. _JsonDataView_metadata:

The test will pass as long as there is at least one label that starts with
_JsonDataView.
 2018-07-27 20:13:26 -07:00
Simon Willison 3b68314d8f
Unit tests for publish now/heroku - closes #348 2018-07-25 22:45:47 -07:00
Simon Willison dbbe707841
publish_subcommand hook + default plugins mechanism, used for publish heroku/now (#349) 
This change introduces a new plugin hook, publish_subcommand, which can be
used to implement new subcommands for the "datasette publish" command family.

I've used this new hook to refactor out the "publish now" and "publish heroku"
implementations into separate modules. I've also added unit tests for these
two publishers, mocking the subprocess.call and subprocess.check_output
functions.

As part of this, I introduced a mechanism for loading default plugins. These
are defined in the new "default_plugins" list inside datasette/app.py

Closes #217 (Plugin support for datasette publish)
Closes #348 (Unit tests for "datasette publish")
Refs #14, #59, #102, #103, #146, #236, #347
 2018-07-25 22:15:59 -07:00
Simon Willison 3ac21c7498
Unit test confirming all plugin hooks are documented 2018-07-25 21:09:59 -07:00
Simon Willison ba64cfb4bc
Fix for Python 3.5 
https://stackoverflow.com/a/42694113/6083
 2018-07-25 10:20:56 -07:00
Simon Willison 9b85c59907
Removed unnecessary print statements from tests 2018-07-24 09:04:06 -07:00
Simon Willison 74ad3ff4af
Ensure --help examples in docs are always up to date, closes #336 
Unit tests now check that docs/*.txt help examples are all up-to-date.

I ran into a problem here in that the terminal_width needed to be more
accurately defined - so I replaced update-docs-help.sh with update-docs-
help.py which hard-codes the terminal width.
 2018-07-24 09:01:23 -07:00
Simon Willison 581b4c97ee
URLify URLs in custom SQL queries, closes #298 2018-07-23 20:56:32 -07:00
Simon Willison 3955771fea
Unit tests for advanced export box HTML, closes #320 2018-07-23 20:38:31 -07:00
Simon Willison 700d83d8ad
?_json_infinity=1 for handling Infinity/-Infinity - fixes #332 2018-07-23 20:07:57 -07:00
Simon Willison f24b49a1a8
New force_https_urls option, refs #333 2018-07-23 08:58:29 -07:00
Simon Willison 6e37f091ed
Support title/description for canned queries, closes #342 
Demo here: https://latest.datasette.io/fixtures/neighborhood_search
 2018-07-15 19:33:30 -07:00
Simon Willison fc9b1cc136
New truncate_cells_html config for truncating cells, closes #330 2018-07-10 09:20:41 -07:00
Simon Willison 6541ce633e
Fix for row pages for tables with / in, closes #325 2018-07-07 22:21:51 -07:00
Simon Willison 64c2fea8df
CSV export now respects --cors, fixes #326 2018-06-23 17:59:37 -07:00
Simon Willison e843ce26e5
Correctly display empty strings in HTML table, closes #314 2018-06-21 08:26:18 -07:00
Simon Willison 120f20cd97
Allow "." in database filenames, closes #302 2018-06-21 08:21:09 -07:00
Simon Willison 97ae66ccab
404s ending in slash redirect to remove that slash, closes #309 2018-06-21 08:13:07 -07:00
Simon Willison 3b53eea382
Fixed incorrect display of compound primary keys with foreign key references 
Closes #319
 2018-06-21 07:56:28 -07:00
Simon Willison 3683a6b626
Docs + example of canned SQL query using || concatenation 
Closes #321
 2018-06-20 21:54:36 -07:00
Simon Willison aeeb50f61b
Correctly display facets with value of 0 - fixes #318 
Also added comprehensive unit test for facet display HTML.
 2018-06-20 21:30:13 -07:00
Simon Willison 83f4ef7ec7
Improved UI for CSV/JSON export, closes #266 2018-06-17 23:05:18 -07:00
Simon Willison fc3660cfad
Streaming mode for downloading all rows as a CSV (#315) 
* table.csv?_stream=1 to download all rows - refs #266

This option causes Datasette to serve ALL rows in the table, by internally
following the _next= pagination links and serving everything out as a stream.

Also added new config option, allow_csv_stream, which can be used to disable
this feature.

* New config option max_csv_mb limiting size of CSV export
 2018-06-17 20:21:02 -07:00
Simon Willison 0d7ba1ba67
Default to _labels=on on JSON/CSV links with foreign keys, refs #266 2018-06-17 15:56:55 -07:00
Simon Willison db1e6bc182
--version-note for datasette, datasette publish and datasette package 
This is a relatively obscure new command-line argument that helps solve the
problem of showing accurate version information in deployed instances of
Datasette even if they were deployed directly from source code.

You can pass --version-note to datasette publish and package and it will then
in turn be passed to datasette when it starts:

    datasette --version-note=hello fixtures.db

Now if you visit /-/versions.json you will see this:

    {
        "datasette": {
            "note": "hello",
            "version": "0+unknown"
        },
        "python": {
            "full": "3.6.5 (default, Jun  6 2018, 19:19:24) \n[GCC 6.3.0 20170516]",
            "version": "3.6.5"
        },
        ...
    }

I plan to use this in some Travis CI configuration, refs #313
 2018-06-17 14:19:39 -07:00
Simon Willison 6a32df334d
Improved fixtures to support publication 
The fixtures database created by our unit tests makes for a good "live" demo
of Datasette in action.

I've improved the metadata it ships with to better support this use-case.

I've also improved the mechanism for writing out fixtures: you can do this:

	python tests/fixtures.py fixtures.db

To get just the fixtures database written out... or you can do this:

	python tests/fixtures.py fixtures.db fixtures.json

To get metadata which you can then serve like so:

	datasette fixtures.db -m fixtures.json

Refs #313
 2018-06-17 12:16:04 -07:00
Simon Willison 0357774c94
Renamed test_tables.db to fixtures.db in unit tests 2018-06-17 11:34:16 -07:00
Simon Willison ed631e690b
?_labels= and ?_label=COL to expand foreign keys in JSON/CSV 
These new querystring arguments can be used to request expanded foreign keys
in both JSON and CSV formats.

?_labels=on turns on expansions for ALL foreign key columns

?_label=COLUMN1&_label=COLUMN2 can be used to pick specific columns to expand

e.g. `Street_Tree_List.json?_label=qSpecies&_label=qLegalStatus`

    {
        "rowid": 233,
        "TreeID": 121240,
        "qLegalStatus": {
            "value" 2,
            "label": "Private"
        }
        "qSpecies": {
            "value": 16,
            "label": "Sycamore"
        }
        "qAddress": "91 Commonwealth Ave",
        ...
    }

The labels option also works for the HTML and CSV views.

HTML defaults to `?_labels=on`, so if you pass `?_labels=off` you can disable
foreign key expansion entirely - or you can use `?_label=COLUMN` to request
just specific columns.

If you expand labels on CSV you get additional columns in the output:

`/Street_Tree_List.csv?_label=qLegalStatus`

    rowid,TreeID,qLegalStatus,qLegalStatus_label...
    1,141565,1,Permitted Site...
    2,232565,2,Undocumented...

I also refactored the existing foreign key expansion code.

Closes #233. Refs #266.
 2018-06-16 15:18:57 -07:00
Simon Willison 5bda4a477c
Fixed CSV tests - Python 3.6.5 and 3.6.3 apparently differ 
The test used to expect CSV to come back like this:

	hello
	world
	""

With the final blank value encoded in quotes.

Judging by Travis failures, this behaviour changed between Python 3.6.3 and
3.6.5:

	https://travis-ci.org/simonw/datasette/jobs/392586661
 2018-06-15 00:01:48 -07:00
Simon Willison 3a79ad98ea
Basic CSV export, refs #266 
Tables and custom SQL query results can now be exported as CSV.

The easiest way to do this is to use the .csv extension, e.g.

	/test_tables/facet_cities.csv

By default this is served as Content-Type: text/plain so you can see it in
your browser. If you want to download the file (using text/csv and with an
appropriate Content-Disposition: attachment header) you can do so like this:

	/test_tables/facet_cities.csv?_dl=1

We link to the CSV and downloadable CSV URLs from the table and query pages.

The links use ?_size=max and so by default will return 1,000 rows.

Also fixes #303 - table names ending in .json or .csv are now detected and
URLs are generated that look like this instead:

	/test_tables/table%2Fwith%2Fslashes.csv?_format=csv

The ?_format= option is available for everything else too, but we link to the
.csv / .json versions in most cases because they are aesthetically pleasing.
 2018-06-14 23:51:23 -07:00
Simon Willison 864328e074
Test client wrapper removing need for gather_request - refs #272 
As part of decoupling from Sanic, this will make it easier to run tests
against ASGI instead.
 2018-06-04 20:53:12 -07:00
Simon Willison 29edbe90ff
New cache_size_kb config for SQLite, closes #304 2018-06-04 09:02:07 -07:00
Simon Willison 9277c6236a
Fixed tests I broke in b18e45158 2018-05-31 07:47:22 -07:00
Simon Willison 969771770f
Upgrade pytest to 3.6.0 
https://github.com/pytest-dev/pytest/issues/1875 made it impossible to declare
a function as a fixture multiple times, which we were doing across different
modules. The fix was to move our @pytest.fixture calls into decorators in the
tests/fixtures.py module.
 2018-05-31 06:42:37 -07:00
Simon Willison 1a8b0213e0
Fixed broken test introduced in b0a95da96 2018-05-29 19:38:21 -07:00
Simon Willison b0a95da963
Show more useful error message for SQL interrupted, closes #142 2018-05-28 14:24:19 -07:00
Simon Willison 98c8f0e728
Moved plugin HTML tests from test_html to test_plugins 2018-05-28 14:23:48 -07:00
Simon Willison 27c10f6482
Missing half of last commit fefb0db8ae 2018-05-28 13:42:48 -07:00
Simon Willison fefb0db8ae
Unit test for 02870e57, closes #291 2018-05-28 13:41:53 -07:00
Simon Willison a4d6acc239
Use scope='session' for all fixtures 
This means they will only be executed once which makes sense
since the database they create is immutable.
 2018-05-28 13:41:06 -07:00
Simon Willison 76d11eb768
New ?_json=colname argument for returning unescaped JSON 
Also extracted docs for special JSON arguments into a new section.

Closes #31
 2018-05-28 11:08:39 -07:00
Simon Willison 7944a8b0de
Added num_sql_threads config option, closes #285 2018-05-26 17:43:22 -07:00
Simon Willison 276913b748
?_shape=arrayfirst, closes #287 2018-05-26 17:32:15 -07:00
Simon Willison b463f60158
?_ttl= parameter and default_cache_ttl config 
Refs #285, Closes #289
 2018-05-26 15:17:33 -07:00
Simon Willison a6afc21aba
Test that ensures all config options are documented 2018-05-26 15:11:16 -07:00
Simon Willison f722b0a730
allow_sql config option to disable custom SQL, closes #284 2018-05-24 22:50:50 -07:00
Simon Willison 50920cfe3d
allow_facet, allow_download, suggest_facets boolean --config 
Refs #284
 2018-05-24 18:12:27 -07:00
Simon Willison 2bfd111d65
Faceting no longer breaks pagination, fixes #282 2018-05-23 06:41:14 -07:00
Simon Willison f6183ff5fa
Renamed --limit to --config, added --help-config, closes #274 
Removed the --page_size= argument to datasette serve in favour of:

    datasette serve --config default_page_size:50 mydb.db

Added new help section:

    $ datasette --help-config
    Config options:
      default_page_size            Default page size for the table view
                                   (default=100)
      max_returned_rows            Maximum rows that can be returned from a table
                                   or custom query (default=1000)
      sql_time_limit_ms            Time limit for a SQL query in milliseconds
                                   (default=1000)
      default_facet_size           Number of values to return for requested facets
                                   (default=30)
      facet_time_limit_ms          Time limit for calculating a requested facet
                                   (default=200)
      facet_suggest_time_limit_ms  Time limit for calculating a suggested facet
                                   (default=50)
 2018-05-20 10:01:49 -07:00
Simon Willison 39426ff0e4
Added /-/limits and /-/limits.json, closes #270 2018-05-17 23:16:28 -07:00
Simon Willison cef9a9a870
--limit= mechanism plus new limits for facets 
Replaced the --max_returned_rows and --sql_time_limit_ms options to
"datasette serve" with a new --limit option, which supports a larger
list of limits.

Example usage:

	datasette serve --limit max_returned_rows:1000 \
		--limit sql_time_limit_ms:2500 \
		--limit default_facet_size:50 \
		--limit facet_time_limit_ms:1000 \
		--limit facet_suggest_time_limit_ms:500

New docs: https://datasette.readthedocs.io/en/latest/limits.html

Closes #270
Closes #264
 2018-05-17 22:08:26 -07:00
Simon Willison b263da78e3 Reliable sort order for facets in Python 3.5, fixing test - refs #255 2018-05-16 08:27:24 -07:00
Simon Willison 6d12580ed7 Foreign key facets are now expanded to labels, refs #255 2018-05-16 08:27:24 -07:00
Simon Willison f4943ca89b _facet selections persist through table form, refs #255 2018-05-16 08:27:24 -07:00
Simon Willison 1dc94f6eaa Facets can now be toggled off again, refs #255 2018-05-16 08:27:24 -07:00
Simon Willison 142a550a99 Facet results now have "truncated" field 
To indicate if there was more than 20 distinct values. Refs #255
 2018-05-16 08:27:24 -07:00
Simon Willison a82175276c _sort/_next links now use new path_with_replaced_args method 2018-05-16 08:27:24 -07:00
Simon Willison 8a4ed052a5 path_with_added_args now works with multiple existing args 2018-05-16 08:27:24 -07:00
Simon Willison f36c9d4b4c Facet toggling now works for integer columns, refs #255 2018-05-16 08:27:24 -07:00
Simon Willison de05cf21aa Facet "selected" key and toggle_url now toggles, refs #255 2018-05-16 08:27:24 -07:00
Simon Willison 2b79f2bdeb path_with_added_args now preserves order in Python 3.5 2018-05-14 00:04:23 -03:00
Simon Willison 4301a8f3ac
Case insensitive querystring comparison, fix Python 3.5 2018-05-12 19:49:37 -03:00
Simon Willison eaaa3ea149
Initial implementation of facets, plus tests and docs 
Refs #255
 2018-05-12 19:29:41 -03:00
Simon Willison 70ff615f1b
utils.path_with_added_args() improvements 
* Now covered by unit tests
* Preserves original order
* Can handle multiple args of the same name, e.g. ?bar=1&bar=2
 2018-05-12 18:44:09 -03:00
Simon Willison 04d21ccd08
/-/versions now includes SQLite fts_versions, closes #252 2018-05-11 10:19:25 -03:00
Simon Willison 801381b765
Default tests to using a longer timelimit 
Every now and then a test will fail in Travis CI on Python 3.5 because it hit
the default 20ms SQL time limit.

Test fixtures now default to a 200ms time limit, and we only use the 20ms time
limit for the specific test that tests query interruption. This should make
our tests on Python 3.5 in Travis much more stable.
 2018-05-05 19:44:32 -03:00
Simon Willison 1259b8ac0b
Support _search_COLUMN=text searches, closes #237 2018-05-05 19:33:08 -03:00
Simon Willison 4d6a568d6c
Unit tests for _search= feature, refs #237 2018-05-05 19:01:14 -03:00
Simon Willison ca290719ed
Show version on /-/plugins page, closes #248 2018-05-04 15:04:33 -03:00
Simon Willison d4da4c92c8
?_size=max option, closes #249 2018-05-04 15:03:40 -03:00
Simon Willison bb87cf8730
Added /-/versions and /-/versions.json, closes #244 
Sample output:

    {
      "python": {
        "version": "3.6.3",
        "full": "3.6.3 (default, Oct  4 2017, 06:09:38) \n[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.37)]"
      },
      "datasette": {
        "version": "0.20"
      },
      "sqlite": {
        "version": "3.23.1",
        "extensions": {
          "json1": null,
          "spatialite": "4.3.0a"
        }
      }
    }
 2018-05-03 11:09:27 -03:00
Simon Willison 690736436b Bump up time limit for test_paginate_tables_and_views 
It was intermittently failing in Travis CI on Python 3.5:

https://travis-ci.org/simonw/datasette/jobs/373713476
 2018-05-01 17:20:39 -07:00
Simon Willison 349e262bb1 Renamed ?_sql_time_limit_ms= to ?_timelimit, closes #242 2018-05-01 17:20:39 -07:00
Simon Willison a4cc5dc813 New ?_shape=array option + tweaks to _shape, closes #245 
* Default is now ?_shape=arrays (renamed from lists)
* New ?_shape=array returns an array of objects as the root object
* Changed ?_shape=object to return the object as the root
* Updated docs
 2018-05-01 17:20:39 -07:00
Simon Willison aa954382c3
FTS tables now detected by inspect(), closes #240 2018-04-28 17:04:32 -07:00
Simon Willison f188ceaa2a
New ?_size=XXX querystring parameter for table view, closes #229 
Also added documentation for all of the _special arguments.

Plus deleted some duplicate logic implementing _group_count.
 2018-04-25 21:47:12 -07:00
Simon Willison 4504d5160b
If max_returned_rows==page_size, increment max_returned_rows 
Fixes #230, where if the two were equal pagination didn't work correctly.
 2018-04-25 21:04:12 -07:00
Simon Willison 02ee31c8b4
New hidden: True option for table metadat, closes #239 2018-04-25 20:42:57 -07:00
Simon Willison 00ab954300
Added CSS class rows-and-columns to main table 2018-04-23 18:31:32 -07:00
Simon Willison f3f4295712
label_column now defined on the table-being-linked-to, fixes #234 2018-04-22 13:46:18 -07:00
Simon Willison f27cabbaf3
label_column option in metadata.json - closes #234 2018-04-22 10:53:16 -07:00
Simon Willison b52171db1e
Plugins can now bundle custom templates, closes #224 
Refs #14
 2018-04-18 22:50:27 -07:00
Simon Willison b55809a1e2
Added /-/metadata /-/plugins /-/inspect, closes #225 2018-04-18 22:25:22 -07:00
Simon Willison 404fa2252b
datasette publish/package --install option, closes #223 
Allows you to specify one or more additional packages to be installed,
useful for deploying plugins.
 2018-04-18 07:55:36 -07:00
Simon Willison 1c36d07dd4
New plugin hooks: extra_css_urls() and extra_js_urls() 
Closes #214
 2018-04-17 20:12:21 -07:00
Simon Willison a5792a8c61
<th> now gets class="col-X" - plus added col-X documentation 
Refs #209
 2018-04-17 19:11:54 -07:00
Simon Willison dfb87d012c
Use to_css_class for table cell column classes 
This ensures that columns with spaces in the name will still
generate usable CSS class names. Refs #209
 2018-04-17 18:19:21 -07:00
Russ Garrett 136a70d887 Add column name classes to <td>s, make PK bold 2018-04-17 18:13:02 -07:00
Russ Garrett 0f782dd8df Additional test asserts 2018-04-17 18:13:02 -07:00
Simon Willison aaf59db570
Longer time limit for test_paginate_compound_keys 
It was failing intermittently in Travis - see #209
 2018-04-17 18:09:48 -07:00
Simon Willison e7c769ef30
Working implementation of #216 which passes the tests 
Reverted commit 5364fa7f33 (where I removed the
code that didn't work).

Added primary keys to order-by clause for sorting to get tests to pass
 2018-04-16 18:41:55 -07:00
Simon Willison 5364fa7f33
Revert #216 until I can get tests to pass in Travis 
Revert "Fix for _sort_desc=sortable_with_nulls test, refs #216"

This reverts commit 07fc2d113e.

Revert "Fixed #216 - paginate correctly when sorting by nullable column"

This reverts commit 2abe539a0f.
 2018-04-16 17:53:37 -07:00
Simon Willison 07fc2d113e
Fix for _sort_desc=sortable_with_nulls test, refs #216 2018-04-16 17:05:03 -07:00
Simon Willison 2abe539a0f
Fixed #216 - paginate correctly when sorting by nullable column 2018-04-16 16:52:50 -07:00
Simon Willison b2955d9065
New --plugins-dir=plugins/ option (#212) 
* New --plugins-dir=plugins/ option

New option causing Datasette to load and evaluate all of the Python files in
the specified directory and register any plugins that are defined in those
files.

This new option is available for the following commands:

    datasette serve mydb.db --plugins-dir=plugins/
    datasette publish now/heroku mydb.db --plugins-dir=plugins/
    datasette package mydb.db --plugins-dir=plugins/

* Unit tests for --plugins-dir=plugins/

Closes #211
 2018-04-15 22:22:01 -07:00
Simon Willison dd4491dd81
Update number of expected tables 2018-04-14 08:16:54 -07:00
Simon Willison 6b15a53cd3
Unit test for unlabelled foreign keys from #207 2018-04-14 08:00:54 -07:00
Simon Willison d72201e883
Added unit test for foreign key links in HTML 
Needed to add a further unit test for #207
 2018-04-14 07:55:27 -07:00
Russ Garrett ed974417ad
Tests for unit filtering 2018-04-14 15:06:52 +01:00
Simon Willison 9f28bbe43d
Better mechanism for handling errors; 404s for missing table/database 
New error mechanism closes #193

404s for missing tables/databesse closes #184

Makes pull request #202 unnecessary.
 2018-04-13 11:17:22 -07:00
Simon Willison 46b237c29a
datasette inspect now finds primary_keys 
Closes #195
 2018-04-09 17:54:12 -07:00
Simon Willison 57b19f09d1
Ability to sort using form fields (for mobile portrait mode) 
We now display sort options as a select box plus a descending checkbox, which
means you can apply sort orders even in portrait mode on a mobile phone where
the column headers are hidden.

Closes #199
 2018-04-09 17:34:32 -07:00
Simon Willison a290f28caa
table_rows => table_rows_count, filtered_table_rows => filtered_table_rows_count 
Renamed properties. Closes #194
 2018-04-08 22:24:24 -07:00
Simon Willison c1d37fdf2b Fixed bug with human filter description, refs #189 
We were showing this:

    201 rows where sorted by sortable_with_nulls

We now show this:

    201 rows sorted by sortable_with_nulls
 2018-04-08 22:10:22 -07:00
Simon Willison b13f0986f2 New sortable_columns option in metadata.json to control sort options 
You can now explicitly set which columns in a table can be used for sorting
using the _sort and _sort_desc arguments using metadata.json:

    {
        "databases": {
            "database1": {
                "tables": {
                    "example_table": {
                        "sortable_columns": [
                            "height",
                            "weight"
                        ]
                    }
                }
            }
        }
    }

Refs #189
 2018-04-08 22:10:22 -07:00
Simon Willison a87df963a0 Error handling for ?_sort and ?_sort_desc 
Verifies that they match an existing column, and only one or the other option
is provided - refs #189

Eses a new DatasetteError exception that closes #193
 2018-04-08 22:10:22 -07:00
Simon Willison 747a801b50 Column headers now link to sort/desc sort - refs #189 2018-04-08 22:10:22 -07:00
Simon Willison 9f2ec39fbc Current sort order now reflected in human filter description 
Plus renamed human_description to human_description_en

Refs #189
 2018-04-08 22:10:22 -07:00
Simon Willison f3a3820ff5 _sort and _sort_desc parameters for table views 
Allows for paginated sorted results based on a specified column.

Refs #189
 2018-04-08 22:10:22 -07:00
Simon Willison 0abd3abacb
New ?_shape=objects/object/lists param for JSON API (#192) 
New _shape= parameter replacing old .jsono extension

Now instead of this:

	/database/table.jsono

We use the _shape parameter like this:

	/database/table.json?_shape=objects

Also introduced a new _shape called 'object' which looks like this:

	/database/table.json?_shape=object

Returning an object for the rows key:

	...
	"rows": {
		"pk1": {
			...
		},
		"pk2": {
			...
		}
	}

Refs #122
 2018-04-03 07:52:54 -07:00
Simon Willison dd0566ff8e
Utility for writing test database fixtures to a .db file 
python tests/fixtures.py /tmp/hello.db

This is useful for making a SQLite database of the test fixtures for
interactive exploration.
 2018-04-03 06:46:11 -07:00
Simon Willison 8f0d44d646
escape_sqlite_table_name => escape_sqlite, handles reserved words 
It can be used for column names as well as table names.

Reserved word list from https://www.sqlite.org/lang_keywords.html
 2018-04-03 06:40:49 -07:00