mirror
/
datasette
kopia lustrzana https://github.com/simonw/datasette
1
0
Forkuj 0
Kod Zgłoszenia Packages Projekty Wydania Wiki Aktywność
1 457 Commity
130 Gałęzie
153 Tagi
6,5 GiB
Wykres commitów

527 Commity (b61f6cceb5682f9154ba72259c0c9c7503a605bf)

Autor SHA1 Wiadomość Data
Simon Willison a1e801453a Renamed execute-query permission to execute-sql, refs #811 2020-06-07 13:20:59 -07:00
Simon Willison 4340845754 Nested permission checks for all views, refs #811 2020-06-07 13:03:08 -07:00
Simon Willison 86dec9e8ff Added permission check to every view, closes #808 2020-06-06 22:30:36 -07:00
Simon Willison 3359d54a4e Use cookies when accessing csrftoken_from 2020-06-06 12:33:08 -07:00
Simon Willison 966eec7f75 Check permissions on canned query page, refs #800 2020-06-06 12:27:00 -07:00
Simon Willison 070838bfa1 Better test for Vary header 2020-06-06 12:26:19 -07:00
Simon Willison 3f83d4632a Respect query permissions on database page, refs #800 2020-06-06 12:05:22 -07:00
Simon Willison 14f6b4d200 actor_matches_allow utility function, refs #800 2020-06-06 11:39:11 -07:00
Simon Willison 75c143a84c Fixed /-/plugins?all=1, refs #802 2020-06-05 16:55:08 -07:00
Simon Willison f786033a5f Fixed 'datasette plugins' command, with tests - closes #802 2020-06-05 16:46:37 -07:00
Simon Willison 033a1bb22c Removed rogue print() from test 2020-06-05 12:06:43 -07:00
Simon Willison 84a9c4ff75
CSRF protection (#798) 
Closes #793.

* Rename RequestParameters to MultiParams, refs #799
* Allow tuples as well as lists in MultiParams, refs #799
* Use csrftokens when running tests, refs #799
* Use new csrftoken() function, refs https://github.com/simonw/asgi-csrf/issues/7
* Check for Vary: Cookie hedaer, refs https://github.com/simonw/asgi-csrf/issues/8
 2020-06-05 12:05:57 -07:00
Simon Willison d96ac1d52c Allow tuples as well as lists in MultiParams, refs #799 2020-06-05 11:01:06 -07:00
Simon Willison 0da7f49b24 Rename RequestParameters to MultiParams, refs #799 2020-06-05 10:52:50 -07:00
Simon Willison 9cb44be42f Docs and tests for "params", closes #797 2020-06-03 14:04:40 -07:00
Simon Willison aa82d03704
Basic writable canned queries 
Refs #698. First working version of this feature.

* request.post_vars() no longer discards empty values
 2020-06-03 08:16:50 -07:00
Simon Willison 0934844c0b request.post_vars() no longer discards empty values 2020-06-03 06:48:39 -07:00
Simon Willison a7137dfe06 /-/plugins now shows details of hooks, closes #794 
Also added /-/plugins?all=1 parameter to see default plugins.
 2020-06-02 14:49:28 -07:00
Simon Willison 5278c04682 More consistent use of response.text/response.json in tests, closes #792 2020-06-02 14:29:12 -07:00
Simon Willison 4fa7cf6853 Flash messages mechanism, closes #790 2020-06-02 14:12:18 -07:00
Simon Willison dfdbdf378a Added /-/permissions debug tool, closes #788 
Also started the authentication.rst docs page, refs #786.

Part of authentication work, refs #699.
 2020-05-31 22:00:36 -07:00
Simon Willison 9f3d4aba31 --root option and /-/auth-token view, refs #784 2020-05-31 18:16:42 -07:00
Simon Willison fa27e44fe0 datasette.sign() and datasette.unsign() methods, refs #785 2020-05-31 18:16:42 -07:00
Simon Willison 1fc6ceefb9 Added /-/actor.json - refs #699 
Also added JSON highlighting to introspection documentation.
 2020-05-31 18:16:42 -07:00
Simon Willison 9315bacf6f Implemented datasette.permission_allowed(), refs #699 2020-05-31 18:16:42 -07:00
Simon Willison 461c82838d Implemented actor_from_request with tests, refs #699 
Also added datasette argument to permission_allowed hook
 2020-05-31 18:16:42 -07:00
Simon Willison 4d798ca0e3 Added test for db.mtime_ns 2020-05-30 11:17:20 -07:00
Simon Willison de1cde65a6 Moved request tests to test_internals_request.py 2020-05-30 10:45:11 -07:00
Simon Willison ca56c226a9 Renamed test_database.py to test_internals_database.py 
Also added a db fixture to remove some boilerplate.
 2020-05-30 07:33:02 -07:00
Simon Willison 31fb006a9b Added datasette.get_database() method 
Refs #576
 2020-05-30 07:29:59 -07:00
Simon Willison 81be31322a New implementation for RequestParams 
- no longer subclasses dict
- request.args[key] now returns first item, not all items
- removed request.raw_args entirely

Closes #774
 2020-05-29 16:22:22 -07:00
Simon Willison 84616a2364 request.args.getlist() returns [] if missing, refs #774 
Also added some unit tests for request.args
 2020-05-29 15:51:30 -07:00
Simon Willison 7bb30c1f11 request.url now respects force_https_urls, closes #781 2020-05-28 10:10:06 -07:00
Simon Willison 5ab411c733 can_render mechanism for register_output_renderer, closes #770 2020-05-27 22:57:05 -07:00
Simon Willison 75cd432e5a Ability to set custom table/view page size in metadata, closes #751 2020-05-27 22:00:04 -07:00
Simon Willison 510c1989d4 Removed xfail, refs #773 2020-05-27 21:11:53 -07:00
Simon Willison 6d95cb4f91 Unit test for register_facet_classes plugin, closes #773 
I was a bit lazy with this one. I didn't hook up a test for the facet_results mechanism.
The custom facet hook isn't a great design so I will probably rethink it at some point
in the future anyway.
 2020-05-27 21:09:16 -07:00
Simon Willison defead17a4 Test for publish_subcommand hook, refs #773 2020-05-27 20:30:32 -07:00
Simon Willison cbeea23d00 Test for prepare_jinja2_environment, refs #773 2020-05-27 20:13:32 -07:00
Simon Willison 57f48b8416 Made register_output_renderer callback optionally awaitable, closes #776 2020-05-27 19:43:30 -07:00
Simon Willison 52c4387c7d Redesigned register_output_renderer plugin hook, closes #581 2020-05-27 19:21:41 -07:00
Simon Willison 446e5de65d Refactored test plugins into tests/plugins, closes #775 2020-05-27 17:57:25 -07:00
Simon Willison ad88c9b3f3 Mechanism for adding a default URL fragment to a canned query 
Closes #767
 2020-05-27 14:52:03 -07:00
Simon Willison da87e963bf
Test that plugin hooks are unit tested (xfail) 
This currently fails using xfail. Closes 771.
 2020-05-27 13:16:02 -07:00
Simon Willison 41a0cd7b6a call_with_supported_arguments() util, refs #581 2020-05-27 12:25:52 -07:00
Simon Willison 5ea8c6d1cd type-pk instead of type-link CSS class, closes #729 2020-05-14 22:55:20 -07:00
Simon Willison 504196341c Visually distinguish float/int columns, closes #729 2020-05-14 22:51:39 -07:00
Simon Willison 2694ddcf14 Test for .execute_fn(), refs #685 2020-05-08 10:29:17 -07:00
Simon Willison 4433306c18
Improvements + docs for db.execute() and Results class 
* Including new results.first() and results.single_value() methods. Closes #685
 2020-05-08 09:05:46 -07:00
Simon Willison 0784f2ef9d Allow specific pragma functions, closes #761 2020-05-06 10:18:31 -07:00
Simon Willison cc872b1f50 Fixed rogue output in tests, closes #755 2020-05-04 11:42:01 -07:00
Simon Willison 9424687e9e Consistently return charset utf-8, closes #752 2020-05-04 10:42:10 -07:00
Simon Willison 450d2e2896 Fixed pytest warning about TestClient class 2020-05-04 10:42:10 -07:00
Simon Willison cef23e8861 Started pattern portfolio at /-/patterns, refs #151 2020-05-02 20:05:25 -07:00
Simon Willison b3aa5f4313 Added 'not like' table filter, refs #750 2020-05-02 12:04:54 -07:00
Simon Willison 1d91ab71d4 Directory configuration mode supports metadata.yaml, closes #747 2020-04-30 11:47:41 -07:00
Simon Willison 89c4ddd482 403 for static directory listing, closes #740 2020-04-27 11:29:04 -07:00
Simon Willison 25014ca25e
Configuration directory mode, closes #731 2020-04-27 09:30:24 -07:00
Simon Willison 1b7b66c465 Make request available when rendering custom pages, closes #738 2020-04-26 12:01:46 -07:00
Simon Willison 304e7b1d9f
Mechanism for creating custom pages using templates 
Closes #648
 2020-04-26 11:46:43 -07:00
Simon Willison 15e2321804 Extra body CSS class for canned queries, closes #727 2020-04-15 14:07:28 -07:00
Simon Willison d349d57cdf Smarter merging of metadata and extra_metadata, closes #724 2020-04-10 11:34:09 -07:00
Simon Willison d55fe8cdfc Fixed bug with Templates considered comment, closes #689 2020-04-05 12:38:33 -07:00
Simon Willison e89b0ef2f9 Expose extra_template_vars in _contex=1, refs #693 2020-04-05 11:49:15 -07:00
Simon Willison 09253817de Fix for missing view_name bug, closes #716 2020-04-05 11:28:20 -07:00
Simon Willison e0e7a0facf Removed Zeit Now v1 support, closes #710 2020-04-04 16:04:33 -07:00
Simon Willison b07312c2b3 dedent SQL for neighborhood_search fixture 
Makes this page a little prettier:
https://latest.datasette.io/fixtures/neighborhood_search
 2020-04-02 17:54:27 -07:00
Simon Willison 6717c719dd
--metadata accepts YAML as well as JSON - closes #713 2020-04-02 12:30:53 -07:00
Simon Willison 2aaad72789 Refactor template setup into Datasette constructor 
Closes #707
 2020-03-26 18:12:43 -07:00
Simon Willison 6aa516d82d Run base_url tests against /fixtures/facetable too, refs #712 2020-03-25 19:31:22 -07:00
Simon Willison 7656fd64d8
base_url configuration setting, closes #394 
* base_url configuration setting
* base_url works for static assets as well
 2020-03-24 17:18:43 -07:00
Simon Willison 5f4aeb1f19 Removed documentation for Zeit Now v1, refs #710 2020-03-24 15:45:24 -07:00
Simon Willison a498d0fe65 Fix bug with over-riding default sort, closes #702 2020-03-21 19:40:29 -07:00
Simon Willison 236aa065b2 "sort" and "sort_desc" metadata properties, closes #702 2020-03-21 19:28:35 -07:00
Simon Willison e1a817411a Bump to click 7.1.1 to fix flaky tests 2020-03-21 18:47:51 -07:00
Simon Willison 2c0e1e09bc Show sort arrow on primary key by default 
Closes #677. Refs #702.
 2020-03-21 16:57:37 -07:00
Simon Willison a000c80d50 await Request(scope, receive).post_vars() method, closes #700 
Needed for #698
 2020-03-16 19:47:37 -07:00
Simon Willison e1b5339fdf Do not look for templates_path in default plugins 
Closes #697
 2020-03-08 16:11:18 -07:00
Simon Willison ddd11b3ddd --memory option for publish cloudrun, refs #694 2020-03-05 17:34:36 -06:00
Simon Willison 7f5a330377
Don't count rows on homepage for DBs > 100MB (#688) 
Closes #649.
 2020-02-28 17:08:29 -08:00
Kevin Keogh 3041c6b641
Use inspect-file, if possible, for total row count (#666) 
For large tables, counting the number of rows in the table can take a
significant amount of time. Instead, where an inspect-file is provided
for an immutable database, look up the row-count for a plain count(*).

Thanks, @kevindkeogh
 2020-02-25 12:19:29 -08:00
Simon Willison 6cb65555f4
?_searchmode=raw option (#686) 2020-02-24 21:56:03 -08:00
Simon Willison a093c5f79f
.execute_write() and .execute_write_fn() methods on Database (#683) 
Closes #682.
 2020-02-24 20:45:07 -08:00
Simon Willison 7c6a9c3529 Better tests for prepare_connection() plugin hook, refs #678 2020-02-21 18:27:07 -08:00
Simon Willison d3f2fade88 Refactored run_sanity_checks to check_connection(conn), refs #674 2020-02-15 09:56:48 -08:00
Simon Willison cf5f4386ef Run black against everything, not just tests and datasette dirs 2020-02-13 15:02:10 -08:00
Jay Graves 33a12c8ae5
Allow leading comments in SQL input field (#653) 
Thanks, @jaywgraves!
 2020-02-04 18:13:24 -08:00
Simon Willison 67fc9c5720
--port argument for datasette package, plus tests - closes #661 
From pull request #663
 2020-01-29 14:46:43 -08:00
Katie McLaughlin 34d77d780f gcloud run is now GA, s/beta// (#660) 
Thanks, @glasnt
 2020-01-21 15:28:11 -08:00
Simon Willison 3c861f363d _search= queries now correctly escaped, fixes #651 
Queries with reserved words or characters according to the SQLite
FTS5 query language could cause errors.

Queries are now escaped like so:

    dog cat => "dog" "cat"
 2019-12-29 18:48:30 +00:00
Simon Willison d54318fc7f Added template_debug setting, closes #654 2019-12-22 16:04:45 +00:00
Simon Willison d6b6c9171f Include asyncio task information in /-/threads debug page 2019-12-04 22:47:17 -08:00
Simon Willison d3e1c3017e Display 0 results, closes #637 2019-11-22 22:07:01 -08:00
Simon Willison fd137da7f8 Suggest column facet only if at least one count > 1 
Fixes #638
 2019-11-21 16:56:55 -08:00
Simon Willison 440a70428c Include rowid in filter select, closes #636 2019-11-19 15:01:10 -08:00
Simon Willison 8c642f04e0
Render templates using Jinja async mode 
Closes #628
 2019-11-14 15:14:22 -08:00
Simon Willison f524510230 Fix "publish heroku" + upgrade to use Python 3.8.0 
Closes #633. Closes #632.
 2019-11-13 08:42:47 -08:00
Simon Willison d977fbadf7
datasette publish uses python:3.8 base Docker image, closes #629 2019-11-11 22:03:09 -08:00
Simon Willison cf7776d36f
Support Python 3.8, stop supporting Python 3.5 (#627) 
* Upgrade to uvicorn 0.10.4
* Drop support for Python 3.5
* Bump all dependencies to latest releases
* Update docs to reflect we no longer support 3.5
* Removed code that skipped black unit test on 3.5

Closes #622
 2019-11-11 21:09:11 -08:00
Simon Willison 28c4a6db5b CREATE INDEX statements on table page, closes #618 2019-11-09 17:29:36 -08:00
Simon Willison 83fc5165ac Improved UI for publish cloudrun, closes #608 2019-11-07 18:48:39 -08:00
Tobias Kunze 931bfc6661 Handle spaces in DB names (#590) 
Closes #503 - thanks, @rixx
 2019-11-04 15:16:30 -08:00
Simon Willison 52fa79c607 Use select colnames, not select * for table view - refs #615 2019-11-04 15:03:48 -08:00
Simon Willison 9db22cdf18 pk__notin= filter, closes #614 2019-11-03 20:11:55 -08:00
Simon Willison 2bf7ce5f51 Fix CSV export for nullable foreign keys, closes #612 2019-11-02 16:12:46 -07:00
Simon Willison 14da70525b Don't show 'None' as label for nullable foreign key, closes #406 2019-11-02 15:29:40 -07:00
Simon Willison 7152e76eda Don't suggest array facet if column is only [], closes #610 2019-11-01 14:45:59 -07:00
Simon Willison 50287e7c6b Only suggest array facet for arrays of strings - closes #562 2019-11-01 12:37:46 -07:00
Simon Willison e2c390500e Persist _where= in hidden fields, closes #604 2019-10-30 11:49:26 -07:00
Simon Willison 5dd4d2b2d3
Update to latest black (#609) 2019-10-30 11:49:01 -07:00
chris48s f4c0830529 Always pop as_format off args dict (#603) 
Closes #563. Thanks, @chris48s
 2019-10-20 19:03:08 -07:00
Simon Willison b647b5efc2
Fix for /foo v.s. /foo-bar issue, closes #597 
Pull request #599
 2019-10-18 15:51:07 -07:00
Simon Willison b6ad1fdc70 Fixed bug returning non-ascii characters in CSV, closes #584 2019-10-17 22:23:01 -07:00
Simon Willison 3e864b1625 Use --platform=managed for publish cloudrun, closes #587 2019-10-17 14:51:45 -07:00
Tobias Kunze 12cec411ca Display metadata footer on custom SQL queries (#589) 
Closes #408 - thanks, @rixx!
 2019-10-13 20:53:21 -07:00
Tobias Kunze 908fc3999e Sort databases on homepage by argument order - #591 
Closes #585 - thanks, @rixx!
 2019-10-13 20:52:33 -07:00
Simon Willison fffd69ec03 Allow EXPLAIN WITH... - closes #583 2019-10-06 10:23:58 -07:00
Simon Willison 2dc5c8dc25
detect_fts now works with alternative table escaping (#571) 
Fixes #570. See also https://github.com/simonw/sqlite-utils/pull/57
 2019-09-02 17:32:27 -07:00
Simon Willison 90d4f497f9 Fix plus test for unicode characters in custom query name, closes #558 2019-07-13 19:49:24 -07:00
Abdus 74ecf8a7cc Fix static mounts using relative paths and prevent traversal exploits (#554) 
Thanks, @abdusco! Closes #555
 2019-07-11 09:13:19 -07:00
Simon Willison 973f8f139d
--plugin-secret option for datasette publish 
Closes #543

Also added new --show-files option to publish now and publish cloudrun - handy for debugging.
 2019-07-07 19:06:31 -07:00
Simon Willison c5542abba5 Removed ManyToManyFacet for the moment, closes #550 2019-07-07 16:21:11 -07:00
Simon Willison fcfcae21e6
extra_template_vars plugin hook (#542) 
* extra_template_vars plugin hook

Closes #541

* Workaround for cwd bug

Based on https://github.com/pytest-dev/pytest/issues/1235#issuecomment-175295691
 2019-07-05 17:05:56 -07:00
Simon Willison 25ff0a8ba6 Fix for accidentally leaking secrets in /-/metadata, closes #538 2019-07-03 22:47:45 -07:00
Simon Willison a2d4593193
Secret plugin configuration options (#539) 
Closes #538
 2019-07-03 22:36:44 -07:00
Simon Willison 4d2fdafe39 Added asgi_wrapper plugin hook, closes #520 2019-07-02 21:06:25 -07:00
Simon Willison 13e4f9a676 Unit test for _table custom template, refs #521 2019-07-02 20:23:05 -07:00
Simon Willison ba8db9679f
Port Datasette from Sanic to ASGI + Uvicorn (#518) 
Datasette now uses ASGI internally, and no longer depends on Sanic.

It now uses Uvicorn as the underlying HTTP server.

This was thirteen months in the making... for full details see the issue:

https://github.com/simonw/datasette/issues/272

And for a full sequence of commits plus commentary, see the pull request:

https://github.com/simonw/datasette/pull/518
 2019-06-23 20:13:09 -07:00
Simon Willison 35429f9089 Revert "New encode/decode_path_component functions" 
Refs #272

This reverts commit 9fdb47ca95.

Now that ASGI supports raw_path we don't need our own encoding scheme!
 2019-06-18 17:23:27 -07:00
Simon Willison 2a4b892d6c Tidy up with Black 2019-05-27 19:28:47 -07:00
Simon Willison 9c58d048ec Fix pagination when sorted by expanded foreign key 
Closes #489
 2019-05-27 19:23:18 -07:00
Simon Willison de92c05134 Fixed duplicate function name 2019-05-27 18:49:57 -07:00
Simon Willison 839e55bf0a Start of unit tests for Database class, refs #485 2019-05-27 18:42:30 -07:00
Simon Willison 182a3017c2 Sort keys to past tests in Python 3.5 2019-05-25 09:22:02 -07:00
Simon Willison d923d84754 Facet by many-to-many, closes #365 2019-05-25 09:06:32 -07:00
Simon Willison 172da009d8 Added ?_through= table argument, closes #355 
Also added much more interesting many-to-many fixtures - roadside attractions!
 2019-05-22 22:45:12 -07:00
Simon Willison 967230c90e Facet by date, closes #481 2019-05-20 23:09:22 -07:00
Simon Willison 2d14f21f85 Updated tests for date(...) lookup 2019-05-20 23:06:19 -07:00
Simon Willison f4eefdf193 Do not allow downloads of mutable databases - closes #474 2019-05-19 13:41:09 -07:00
Simon Willison 7d8573d672 Rename "datasette publish now" to "datasette publish nowv1" 
Also added an alias so "datasette publish now" continues to work.

Closes #472
 2019-05-19 11:06:51 -07:00
Simon Willison db9dbfb816 ?_hash=1 no longer respected for mutable databases 
Closes #471, refs #419
 2019-05-16 08:10:25 -07:00
Simon Willison 0dad111d24 New introspection endpoint: /-/databases - closes #470 
Refs #419 and #465
 2019-05-16 07:49:34 -07:00
Simon Willison 21b57cdf9a serve --inspect-file=X now populates cached table counts 
Closes #462
 2019-05-16 07:30:32 -07:00
Simon Willison 909e66dfdb Removed .inspect() and /-/inspect.json 
Refs #462

/-/inspect.json may return in some shape in #465
 2019-05-16 07:15:24 -07:00
Simon Willison faf33515b2 Sometimes sort tables by number of relationships, closes #460 2019-05-15 20:23:33 -07:00
Simon Willison 689cf9c139 Index page only shows row counts for smaller databases 
The index page now only shows row counts for immutable databases OR for
databases with less than 30 tables provided it could get a count for
each of those tables in less than 10ms.

Closes #467, Refs #460
 2019-05-15 20:02:33 -07:00
Simon Willison 79303163b2 Row count fix + sort index page databases alphabetically 
Sorting alphabetically should fix a test failure in Python 3.5

Refs #460
 2019-05-15 17:40:10 -07:00
Simon Willison 5d6b2c30f1 Include views on homepage, fix table counts 
If we have less than 5 tables we now also show one or more views in the
summary on the homepage.

Also corrected the logic for the row counts - we now count hidden and
visible tables separately.

Closes #373, Refs #460
 2019-05-15 17:28:07 -07:00
Simon Willison e04511410f Don't show hidden tables on index page, closes #455 
Refs #460. Also bulked out HTML index page unit tests.
 2019-05-14 08:46:57 -07:00
Simon Willison 24bf79d2f0 Run sanity checks, not .inspect(), on startup 
Also fixes tests that did NOT like a call to run_until_complete in the Datasette() constructor.
 2019-05-11 16:22:55 -07:00
Simon Willison da0b3ce2b7 New run_sanity_checks mechanism, for SpatiLite 
Moved VirtualSpatialIndex check into a new mechanism that should allow
us to add further sanity checks in the future.

To test this I've had to commit a binary sample SpatiaLite database to
the repository. I included a build script for creating that database.

Closes #466
 2019-05-11 15:55:30 -07:00
Simon Willison c692cd2911 Fix test ordering 2019-05-11 15:03:52 -07:00
Simon Willison a187c81e0c Black + fix broken test 2019-05-11 14:45:59 -07:00
Simon Willison c0d1b4c322 "datasette inspect foo.db" now just calculates table counts 
Refs #462

* inspect command now just outputs table counts
* test_inspect.py is now only tests for that CLI command
* Updated some relevant documentation
* Removed docs for /-/inspect since that is about to change
 2019-05-11 14:36:57 -07:00
Simon Willison ce09e5d2d3 Fixed tests relating to #459 2019-05-11 13:58:22 -07:00
Simon Willison 05cabaebd9 Pass --token to now alias, refs #459 2019-05-11 13:35:34 -07:00
Simon Willison 09ef305c68 Fixed "datasette publish now ... --alias=x" 
The --alias argument can now be passed more than once.

Also updated our Travis configuration to use this.

Fixes #459
 2019-05-11 13:20:36 -07:00
Simon Willison a0d444837f Finished implementation of ?_trace=1 debug tool 
I redesigned the JSON output and added a handy "traceback" key showing
three relevant lines of the current traceback for each logged query.

Closes #435
 2019-05-11 12:06:22 -07:00
Simon Willison 2db977d8f3 tests/fixtures.py can now write out plugins too 
This command:

    python tests/fixtures.py \
        fixtures.db \
        metadata.json \
        fixtures-plugins/

Will now create the fixtures.db and metadata.json files, AND create
a folder called fixtures-plugins/ containing two test plugins.

You can then run it like this:

    datasette fixtures.db \
        -m metadata.json --plugins-dir=fixtures-plugins/
 2019-05-09 09:44:21 -07:00
Simon Willison 9fdb47ca95 New encode/decode_path_component functions 
ASGI cannot differentiate between / and %2F in a URL, so we need an
alternative scheme for encoding the names of tables that contain special
characters such as /

For background, see
    https://github.com/django/asgiref/issues/51#issuecomment-450603464

Some examples:

    "table/and/slashes" => "tableU+002FandU+002Fslashes"
    "~table" => "U+007Etable"
    "+bobcats!" => "U+002Bbobcats!"
    "U+007Etable" => "UU+002B007Etable"
 2019-05-09 09:44:21 -07:00
Simon Willison ec2db345e9 Fixed crash on /:memory: page 2019-05-05 14:01:14 -04:00
Simon Willison 831515b834 Respect --cors for error pages, closes #453 2019-05-05 08:12:03 -04:00
Simon Willison 35d6ee2790
Apply black to everything, enforce via unit tests (#449) 
I've run the black code formatting tool against everything:

    black tests datasette setup.py

I also added a new unit test, in tests/test_black.py, which will fail if the code does not
conform to black's exacting standards.

This unit test only runs on Python 3.6 or higher, because black itself doesn't run on 3.5.
 2019-05-03 22:15:14 -04:00
Simon Willison 93c65d29ba Fixed 500 error on /-/metadata page 2019-05-03 16:42:59 -04:00
Simon Willison 01b3de5b66 Unit test for binary data display, refs #442 2019-05-03 12:45:06 -04:00
Simon Willison bade9daeb2 Docs for 'datasette publish cloudrun', refs #434 2019-05-03 12:16:58 -04:00
Romain Primet 75a21fc2a1 datasette publish cloudrun (#434) - thanks, @rprimet 
New publish subcommand that publishes using the
new Google Cloud Run platform.

    datasette publish cloudrun database.db
 2019-05-03 09:59:01 -04:00
Simon Willison 53d2f00b73 Implemented ArrayFacet, closes #359 2019-05-02 20:21:27 -04:00
Simon Willison ea66c45df9
Extract facet code out into a new plugin hook, closes #427 (#445) 
Datasette previously only supported one type of faceting: exact column value counting.

With this change, faceting logic is extracted out into one or more separate classes which can implement other patterns of faceting - this is discussed in #427, but potential upcoming facet types include facet-by-date, facet-by-JSON-array, facet-by-many-2-many and more.

A new plugin hook, register_facet_classes, can be used by plugins to add in additional facet classes.

Each class must implement two methods: suggest(), which scans columns in the table to decide if they might be worth suggesting for faceting, and facet_results(), which executes the facet operation and returns results ready to be displayed in the UI.
 2019-05-02 17:11:26 -07:00
Simon Willison efc93b8ab5 Entirely removed table_rows_count table property 
We were not displaying this anywhere, and it is now expensive to calculate.

Refs #419, #420
 2019-05-01 22:29:47 -07:00
Simon Willison ad9de774f7 Added missing file, refs #438 2019-05-01 22:10:23 -07:00
Simon Willison f553a67021 Don't load setuptools plugins during test runs 
Uses pattern from https://docs.pytest.org/en/latest/example/simple.html#detect-if-running-from-within-a-pytest-run

Closes #438
 2019-05-01 22:09:03 -07:00
Simon Willison 50d2d1aac9 Fixed bug where metadata.json hidden tables were ignored 2019-05-01 17:54:48 -07:00
Simon Willison e7151ccccf Index page no longer uses inspect data - refs #420 
Also introduced a mechanism whereby table counts are calculated against a time limit
but immutable databases have their table counts calculated on server startup.
 2019-05-01 17:39:39 -07:00
Simon Willison 669fa21a71 Include request duration in traces 2019-05-01 16:27:14 -07:00
Russ Garrett cf406c0754 New plugin hook: register_output_renderer hook (#441) 
Thanks @russss!

* Add register_output_renderer hook

This changeset refactors out the JSON renderer and then adds a hook and
dispatcher system to allow custom output renderers to be registered.

The CSV output renderer is untouched because supporting streaming
renderers through this system would be significantly more complex, and
probably not worthwhile.

We can't simply allow hooks to be called at request time because we need
a list of supported file extensions when the request is being routed in
order to resolve ambiguous database/table names. So, renderers need to
be registered at startup.

I've tried to make this API independent of Sanic's request/response
objects so that this can remain stable during the switch to ASGI. I'm
using dictionaries to keep it simple and to make adding additional
options in the future easy.

Fixes #440
 2019-05-01 16:01:56 -07:00
Simon Willison e4e73a537a Test for ?_trace=1, refs #435 2019-04-21 10:46:56 -07:00
Simon Willison 9c77e6e355 Support multiple filters of the same type 
Closes #288
 2019-04-15 16:44:17 -07:00
Simon Willison 583b22aa28 New ?column__date=yyyy-mm-dd filter 2019-04-15 15:54:54 -07:00
Simon Willison 1c6649b19b New colname__in=x,y,z filter, closes #433 2019-04-15 15:43:22 -07:00
Simon Willison 2c19a27d15 Documentation for filters, plus new documentation unit test 
https://simonwillison.net/2018/Jul/28/documentation-unit-tests/
 2019-04-15 15:41:11 -07:00
Simon Willison 6da567dda9 Extract and refactor filters into filters.py 
This will help in implementing __in as a filter, refs #433
 2019-04-15 14:51:20 -07:00
Simon Willison bc6a9b4564
?_where= parameter on table views, closes #429 
From pull request #430
 2019-04-12 18:37:22 -07:00
Simon Willison e11cb4c664 Persist show/hide state better, closes #425 2019-04-11 22:00:47 -07:00
Simon Willison db74cf0144 ?_fts_table= and ?_fts_pk= arguments, closes #428 2019-04-11 21:21:17 -07:00
Simon Willison 78e45ead4d New ?tags__arraycontains=tag lookup against JSON fields 
Part one of supporting facet-by-JSON-array, refs #359
 2019-04-10 08:27:52 -07:00
Simon Willison 53bf875483 expand_foreign_keys() no longer uses inspect, refs #420 2019-04-06 19:56:07 -07:00
Simon Willison 97331f3435 sortable_columns_for_table() no longer uses inspect() 
Refs #420
 2019-04-06 18:58:51 -07:00
Simon Willison 468c6fd953 DatabaseDownload no longer uses .inspect(), refs #420 2019-03-31 19:04:17 -07:00
Simon Willison 7d0f668556 .resolve_db_name() and .execute() work without inspect 
Refs #420
 2019-03-31 16:51:52 -07:00
Simon Willison 0209a0a344 table_exists() now uses async SQL, refs #420 2019-03-31 11:02:22 -07:00
Simon Willison 6f6d0ff2b4
URL hashing is now off by default - closes #418 
Prior to this commit Datasette would calculate the content hash of every
database and redirect to a URL containing that hash, like so:

    https://v0-27.datasette.io/fixtures => https://v0-27.datasette.io/fixtures-dd88475

This assumed that all databases were opened in immutable mode and were not
expected to change.

This will be changing as a result of #419 - so this commit takes the first step
in implementing that change by changing this default behaviour. Datasette will
now only redirect hash-free URLs under two circumstances:

* The new `hash_urls` config option is set to true (it defaults to false).
* The user passes `?_hash=1` in the URL
 2019-03-17 15:55:04 -07:00
Simon Willison afe9aa3ae0 show/hide link for SQL on custom query page 
Closes #415
 2019-03-14 22:22:35 -07:00
Simon Willison 2855667908 Fix for test failure with Click 7.0 2019-03-14 22:00:13 -07:00
Simon Willison 9743e1d91b Support for :memory: databases 
If you start Datasette with no files, it will connect to :memory: instead.

When starting it with files you can add --memory to also get a :memory: database.
 2019-03-14 20:54:42 -07:00
Simon Willison bf6b0f918d about and about_url metadata options 2019-03-14 20:54:42 -07:00
Simon Willison 4462a5ab28 Show size of database file next to download link, closes #172 2019-02-05 20:58:29 -08:00
Simon Willison 195a5b3634
Heroku --include-vcs-ignore (#407) 
Means `datasette publish heroku` can work under Travis, unlike this failure:

https://travis-ci.org/simonw/fivethirtyeight-datasette/builds/488047550

```
2.25s$ datasette publish heroku fivethirtyeight.db -m metadata.json -n fivethirtyeight-datasette
tar: unrecognized option '--exclude-vcs-ignores'
Try 'tar --help' or 'tar --usage' for more information.
 ▸    Command failed: tar cz -C /tmp/tmpuaxm7i8f --exclude-vcs-ignores --exclude
 ▸    .git --exclude .gitmodules . >
 ▸    /tmp/f49440e0-1bf3-4d3f-9eb0-fbc2967d1fd4.tar.gz
 ▸    tar: unrecognized option '--exclude-vcs-ignores'
 ▸    Try 'tar --help' or 'tar --usage' for more information.
 ▸    
The command "datasette publish heroku fivethirtyeight.db -m metadata.json -n fivethirtyeight-datasette" exited with 0.
```

The fix for that issue is to call the heroku command like this:

    heroku builds:create -a app_name --include-vcs-ignore
 2019-02-05 20:15:46 -08:00
Simon Willison b5dd83981a Export option: _shape=array&_nl=on for newline-delimited JSON 2019-01-27 17:40:23 -08:00
Simon Willison c3a78eb05c app_client() fixture doesn't need to take **kwargs 2019-01-13 14:23:44 -08:00