David Brochart
908161aecb
Dot gets added later
2019-11-07 13:26:26 +01:00
David Brochart
39eeb82342
Raise ValueError if not able to checkout ref
2019-11-06 09:32:08 +01:00
David Brochart
97c8bfb68f
Clone and check out branch in two steps
2019-11-05 18:53:45 +01:00
David Brochart
56e6c116f3
Add test
2019-11-05 18:01:06 +01:00
David Brochart
e4e9c1fdce
Fix submodule check out
2019-11-05 15:10:19 +01:00
Scott Black
1257a2c910
update downloading message to include full url
2019-10-14 13:56:29 -06:00
Scott Black
16411fc35e
update formatting
2019-10-01 12:59:06 -06:00
Scott Black
d657453b3f
update hydroshare version fetch to assume timestamp is in utc
2019-10-01 11:16:34 -06:00
Scott Black
3a936412af
use datetime for iso parsing
2019-09-30 16:59:01 -06:00
Scott Black
8dd9b5b928
update date parser to parse hydroshare date with timezone
2019-09-30 15:57:27 -06:00
Scott Black
dddc45acfb
reformatting with lint
2019-09-30 13:50:24 -06:00
Scott Black
21e61f3099
add hydroshare resource versioning and raise exception fetch fails
2019-09-25 14:37:19 -06:00
Scott Black
7fcb67f06d
raise exceptions during failure and add modify date version
2019-09-25 14:00:47 -06:00
Scott Black
2508cc2104
fix _urlretrieve class method
2019-09-25 10:58:45 -06:00
Scott Black
d89f3a66aa
update hydroshare content provider to doi and add tests
2019-09-25 09:59:44 -06:00
Scott Black
0549577a51
Merge branch 'master' of git://github.com/jupyter/repo2docker
2019-09-24 13:37:50 -06:00
Daniel Nüst
3bdf7041dc
sort and pretty print dataverse.json
...
so that we can track changes and only update the lines of the files that need updating
2019-09-18 08:54:48 +02:00
Kacper Kowalik (Xarthisius)
4df4fd6ab4
Download individual files instead of zip bundle
2019-09-17 15:44:56 -05:00
Kacper Kowalik (Xarthisius)
a2f8228b15
Match DV hosts based on netloc instead of url
2019-09-16 15:03:12 -05:00
Kacper Kowalik (Xarthisius)
30375d13df
Use the persistent id from the url regardless whether it's a DOI or not
2019-09-16 14:29:47 -05:00
Kacper Kowalik (Xarthisius)
14b758d327
Improve debug logging
2019-09-16 09:17:15 -05:00
Kacper Kowalik (Xarthisius)
331a610324
Add Dataverse content provider
2019-09-13 13:42:41 -05:00
Daniel Nüst
a40e1799bd
fix content_id for Figshare
2019-09-11 13:45:16 +02:00
Daniel Nüst
f1ec646d2e
add support for Figshare versions > 9
2019-09-11 13:18:20 +02:00
Daniel Nüst
698b8a1d7e
try to improve test coverage
2019-09-11 13:17:48 +02:00
Daniel Nüst
b2b75600c0
handle versions in Figshare properly
2019-09-09 08:37:02 +02:00
Daniel Nüst
b2b6030492
initialize hosts as instance variable
2019-09-08 21:39:48 +02:00
Daniel Nüst
e6cd4b19d9
reuse hosts definition in Zenodo tests
2019-09-08 14:51:17 +02:00
Daniel Nüst
0e4029bb8d
add Figshare content provider
...
moving common functions to a DoiProvider
2019-09-08 14:51:17 +02:00
Scott Black
ebbc9e0233
update messaging
2019-08-23 11:16:39 -06:00
Scott Black
b068f8d9c1
extend wait time and cleanup messaging
2019-08-23 11:09:26 -06:00
Scott Black
970c7ece20
actually reconnect
2019-08-23 11:06:02 -06:00
Scott Black
97eee3b8c0
allow for bag creation
2019-08-23 10:56:52 -06:00
Scott Black
8817fec7ac
handle traliing slash in url
2019-08-22 17:24:52 -06:00
Scott Black
9927faaafd
extract from contents directory
2019-08-22 17:10:55 -06:00
Scott Black
fdb92525c6
only use data/contents directory
2019-08-22 17:00:55 -06:00
Scott Black
316b1a0f34
use request.urlretrieve
2019-08-22 16:41:31 -06:00
Scott Black
e04bc18248
remove unused imports
2019-08-22 16:33:25 -06:00
Scott Black
5f574d3a2c
initial checkin of hydroshare content provider
2019-08-22 16:30:48 -06:00
Tim Head
dd8ba8ec18
Remove print statement and unused import
2019-07-03 06:53:59 +02:00
Tom Morrell
93147888c5
Reformatting and better tests
2019-06-20 13:22:17 -07:00
Tom Morrell
87f33e3b6a
Full generalization and support for CaltechDATA
2019-06-18 15:11:06 -07:00
Tom Morrell
d2016bb722
rename
2019-06-13 12:22:31 -07:00
Tom Morrell
f35e2a0382
More general doi to url
2019-06-13 11:27:24 -07:00
Tom Morrell
10fc5b5f36
Formatting
2019-06-13 11:14:16 -07:00
Tom Morrell
2db395ac87
Initial, incomplete generalization
2019-06-13 11:09:34 -07:00
Tim Head
8b004e06dc
Apply auto formatting
...
Apologies to anyone finding this commit via git blame or log
run the autoformatting by
pre-commit run --all-files
2019-06-03 21:37:25 +02:00
Tim Head
363c962efd
Add a urlopen helper to Zenodo content provider
...
Use a helper function to inject a default user-agent header into every
request we make.
2019-05-29 22:27:09 +02:00
Tim Head
e99c80799d
Add comment about supporting Zenodo instances not at zenodo.org
2019-05-29 22:27:09 +02:00
Tim Head
9bda1152fb
Update docstrings in Zenodo provider
2019-05-29 22:27:09 +02:00
Tim Head
ec8659e62f
Add tests for Zenodo content provider
2019-05-29 22:27:09 +02:00
Tim Head
2ed3818f99
Better handling of software archives
...
Unpack a single ZIP file, then move it to the root of the directory
2019-05-29 22:27:09 +02:00
Tim Head
dce6c1e8d7
Add basic Zenodo content provider
2019-05-29 22:27:09 +02:00
Alistair Miles
62005f6f15
ensure git submodules are updated and initialised
2019-04-08 08:09:19 -05:00
Gladys Nalvarte
7f169b8282
Updated log info for local content provider
2019-01-10 11:48:29 +01:00
Tim Head
e7018d7ca5
Add caching of already built repositories
...
Add tests for image caching
Adjust tests and main app for cached builds
Remove obsolete command-line handling
Remove print statement from test
Fix subdirectory handling
Put back exception instead of sys.exit()
2018-12-21 11:20:51 +01:00
James Bourbeau
3cadb1832d
Another f-string
2018-12-18 09:07:52 -06:00
yuvipanda
8f56061e69
Don't call sys.exit from inside the class
...
Instead, we raise exceptions that can be caught and
handled by calling code instead
2018-12-14 13:15:33 -08:00
Yuvi Panda
aeec658149
Revert "[MRG] Start reusing existing docker images if content hasn't changed"
2018-12-05 11:51:22 -08:00
Tim Head
b9e182791e
Use a subset of the git SHA1 to make image names unique
2018-11-29 18:22:05 +01:00
Tim Head
39c2ae276b
Start reusing existing docker images if content hasn't changed
...
Content providers can specify a "content ID" which is used to identify
versions of the content. The ID is used in the docker image name and if
we find an existing image for a given content ID the build step is
skipped.
2018-11-04 13:23:04 +01:00
Tim Head
c7dc5e1841
Remove explicit passing of logger instance
2018-10-16 08:04:13 +02:00
Tim Head
e5c7fdd341
Use the local content provider
...
This let's us remove some special casing around local directories from
the build process.
2018-10-16 08:04:13 +02:00
Tim Head
866dd4f800
Start using content providers
2018-10-16 08:04:13 +02:00
yuvipanda
18f11b8c1f
[WIP] base contentproviders commit
...
Very much a WIP, but some thougts on how we can support more
than just git.
2018-10-16 08:04:13 +02:00