* table.csv?_stream=1 to download all rows - refs #266
This option causes Datasette to serve ALL rows in the table, by internally
following the _next= pagination links and serving everything out as a stream.
Also added new config option, allow_csv_stream, which can be used to disable
this feature.
* New config option max_csv_mb limiting size of CSV export
When a new tagged version is pushed, this should:
* bdist_wheel it and release to https://pypi.python.org/pypi/datasette
* Set an alias of v0-22-1.datasette.io for the deployed demo
If the tests pass in Travis CI, deploy an instance containing Datasette's test
fixtures to https://latest.datasette.io/
Also set up an alias of truncated-commit-hash.datasette.io
This is a relatively obscure new command-line argument that helps solve the
problem of showing accurate version information in deployed instances of
Datasette even if they were deployed directly from source code.
You can pass --version-note to datasette publish and package and it will then
in turn be passed to datasette when it starts:
datasette --version-note=hello fixtures.db
Now if you visit /-/versions.json you will see this:
{
"datasette": {
"note": "hello",
"version": "0+unknown"
},
"python": {
"full": "3.6.5 (default, Jun 6 2018, 19:19:24) \n[GCC 6.3.0 20170516]",
"version": "3.6.5"
},
...
}
I plan to use this in some Travis CI configuration, refs #313
The fixtures database created by our unit tests makes for a good "live" demo
of Datasette in action.
I've improved the metadata it ships with to better support this use-case.
I've also improved the mechanism for writing out fixtures: you can do this:
python tests/fixtures.py fixtures.db
To get just the fixtures database written out... or you can do this:
python tests/fixtures.py fixtures.db fixtures.json
To get metadata which you can then serve like so:
datasette fixtures.db -m fixtures.json
Refs #313
These new querystring arguments can be used to request expanded foreign keys
in both JSON and CSV formats.
?_labels=on turns on expansions for ALL foreign key columns
?_label=COLUMN1&_label=COLUMN2 can be used to pick specific columns to expand
e.g. `Street_Tree_List.json?_label=qSpecies&_label=qLegalStatus`
{
"rowid": 233,
"TreeID": 121240,
"qLegalStatus": {
"value" 2,
"label": "Private"
}
"qSpecies": {
"value": 16,
"label": "Sycamore"
}
"qAddress": "91 Commonwealth Ave",
...
}
The labels option also works for the HTML and CSV views.
HTML defaults to `?_labels=on`, so if you pass `?_labels=off` you can disable
foreign key expansion entirely - or you can use `?_label=COLUMN` to request
just specific columns.
If you expand labels on CSV you get additional columns in the output:
`/Street_Tree_List.csv?_label=qLegalStatus`
rowid,TreeID,qLegalStatus,qLegalStatus_label...
1,141565,1,Permitted Site...
2,232565,2,Undocumented...
I also refactored the existing foreign key expansion code.
Closes#233. Refs #266.
The test used to expect CSV to come back like this:
hello
world
""
With the final blank value encoded in quotes.
Judging by Travis failures, this behaviour changed between Python 3.6.3 and
3.6.5:
https://travis-ci.org/simonw/datasette/jobs/392586661
Tables and custom SQL query results can now be exported as CSV.
The easiest way to do this is to use the .csv extension, e.g.
/test_tables/facet_cities.csv
By default this is served as Content-Type: text/plain so you can see it in
your browser. If you want to download the file (using text/csv and with an
appropriate Content-Disposition: attachment header) you can do so like this:
/test_tables/facet_cities.csv?_dl=1
We link to the CSV and downloadable CSV URLs from the table and query pages.
The links use ?_size=max and so by default will return 1,000 rows.
Also fixes#303 - table names ending in .json or .csv are now detected and
URLs are generated that look like this instead:
/test_tables/table%2Fwith%2Fslashes.csv?_format=csv
The ?_format= option is available for everything else too, but we link to the
.csv / .json versions in most cases because they are aesthetically pleasing.
New command-line argument which causes SpatiaLite to be installed and
configured for the published Datasette.
datasette publish now --spatialite mydb.db
https://github.com/pytest-dev/pytest/issues/1875 made it impossible to declare
a function as a fixture multiple times, which we were doing across different
modules. The fix was to move our @pytest.fixture calls into decorators in the
tests/fixtures.py module.
Simon Willison