Wykres commitów

107 Commity (main)

Autor SHA1 Wiadomość Data
Scott Black 3a936412af use datetime for iso parsing 2019-09-30 16:59:01 -06:00
Scott Black 8dd9b5b928 update date parser to parse hydroshare date with timezone 2019-09-30 15:57:27 -06:00
Scott Black dddc45acfb reformatting with lint 2019-09-30 13:50:24 -06:00
Scott Black 21e61f3099 add hydroshare resource versioning and raise exception fetch fails 2019-09-25 14:37:19 -06:00
Scott Black 7fcb67f06d raise exceptions during failure and add modify date version 2019-09-25 14:00:47 -06:00
Scott Black 2508cc2104 fix _urlretrieve class method 2019-09-25 10:58:45 -06:00
Scott Black d89f3a66aa update hydroshare content provider to doi and add tests 2019-09-25 09:59:44 -06:00
Scott Black 0549577a51 Merge branch 'master' of git://github.com/jupyter/repo2docker 2019-09-24 13:37:50 -06:00
Daniel Nüst 3bdf7041dc sort and pretty print dataverse.json
so that we can track changes and only update the lines of the files that need updating
2019-09-18 08:54:48 +02:00
Kacper Kowalik (Xarthisius) 4df4fd6ab4
Download individual files instead of zip bundle 2019-09-17 15:44:56 -05:00
Kacper Kowalik (Xarthisius) a2f8228b15
Match DV hosts based on netloc instead of url 2019-09-16 15:03:12 -05:00
Kacper Kowalik (Xarthisius) 30375d13df
Use the persistent id from the url regardless whether it's a DOI or not 2019-09-16 14:29:47 -05:00
Kacper Kowalik (Xarthisius) 14b758d327
Improve debug logging 2019-09-16 09:17:15 -05:00
Kacper Kowalik (Xarthisius) 331a610324
Add Dataverse content provider 2019-09-13 13:42:41 -05:00
Daniel Nüst a40e1799bd fix content_id for Figshare 2019-09-11 13:45:16 +02:00
Daniel Nüst f1ec646d2e add support for Figshare versions > 9 2019-09-11 13:18:20 +02:00
Daniel Nüst 698b8a1d7e try to improve test coverage 2019-09-11 13:17:48 +02:00
Daniel Nüst b2b75600c0 handle versions in Figshare properly 2019-09-09 08:37:02 +02:00
Daniel Nüst b2b6030492 initialize hosts as instance variable 2019-09-08 21:39:48 +02:00
Daniel Nüst e6cd4b19d9 reuse hosts definition in Zenodo tests 2019-09-08 14:51:17 +02:00
Daniel Nüst 0e4029bb8d add Figshare content provider
moving common functions to a DoiProvider
2019-09-08 14:51:17 +02:00
Scott Black ebbc9e0233 update messaging 2019-08-23 11:16:39 -06:00
Scott Black b068f8d9c1 extend wait time and cleanup messaging 2019-08-23 11:09:26 -06:00
Scott Black 970c7ece20 actually reconnect 2019-08-23 11:06:02 -06:00
Scott Black 97eee3b8c0 allow for bag creation 2019-08-23 10:56:52 -06:00
Scott Black 8817fec7ac handle traliing slash in url 2019-08-22 17:24:52 -06:00
Scott Black 9927faaafd extract from contents directory 2019-08-22 17:10:55 -06:00
Scott Black fdb92525c6 only use data/contents directory 2019-08-22 17:00:55 -06:00
Scott Black 316b1a0f34 use request.urlretrieve 2019-08-22 16:41:31 -06:00
Scott Black e04bc18248 remove unused imports 2019-08-22 16:33:25 -06:00
Scott Black 5f574d3a2c initial checkin of hydroshare content provider 2019-08-22 16:30:48 -06:00
Tim Head dd8ba8ec18 Remove print statement and unused import 2019-07-03 06:53:59 +02:00
Tom Morrell 93147888c5 Reformatting and better tests 2019-06-20 13:22:17 -07:00
Tom Morrell 87f33e3b6a Full generalization and support for CaltechDATA 2019-06-18 15:11:06 -07:00
Tom Morrell d2016bb722 rename 2019-06-13 12:22:31 -07:00
Tom Morrell f35e2a0382 More general doi to url 2019-06-13 11:27:24 -07:00
Tom Morrell 10fc5b5f36 Formatting 2019-06-13 11:14:16 -07:00
Tom Morrell 2db395ac87 Initial, incomplete generalization 2019-06-13 11:09:34 -07:00
Tim Head 8b004e06dc Apply auto formatting
Apologies to anyone finding this commit via git blame or log

run the autoformatting by

    pre-commit run --all-files
2019-06-03 21:37:25 +02:00
Tim Head 363c962efd Add a urlopen helper to Zenodo content provider
Use a helper function to inject a default user-agent header into every
request we make.
2019-05-29 22:27:09 +02:00
Tim Head e99c80799d Add comment about supporting Zenodo instances not at zenodo.org 2019-05-29 22:27:09 +02:00
Tim Head 9bda1152fb Update docstrings in Zenodo provider 2019-05-29 22:27:09 +02:00
Tim Head ec8659e62f Add tests for Zenodo content provider 2019-05-29 22:27:09 +02:00
Tim Head 2ed3818f99 Better handling of software archives
Unpack a single ZIP file, then move it to the root of the directory
2019-05-29 22:27:09 +02:00
Tim Head dce6c1e8d7 Add basic Zenodo content provider 2019-05-29 22:27:09 +02:00
Alistair Miles 62005f6f15
ensure git submodules are updated and initialised 2019-04-08 08:09:19 -05:00
Gladys Nalvarte 7f169b8282 Updated log info for local content provider 2019-01-10 11:48:29 +01:00
Tim Head e7018d7ca5 Add caching of already built repositories
Add tests for image caching

Adjust tests and main app for cached builds

Remove obsolete command-line handling

Remove print statement from test

Fix subdirectory handling

Put back exception instead of sys.exit()
2018-12-21 11:20:51 +01:00
James Bourbeau 3cadb1832d Another f-string 2018-12-18 09:07:52 -06:00
yuvipanda 8f56061e69 Don't call sys.exit from inside the class
Instead, we raise exceptions that can be caught and
handled by calling code instead
2018-12-14 13:15:33 -08:00
Yuvi Panda aeec658149
Revert "[MRG] Start reusing existing docker images if content hasn't changed" 2018-12-05 11:51:22 -08:00
Tim Head b9e182791e Use a subset of the git SHA1 to make image names unique 2018-11-29 18:22:05 +01:00
Tim Head 39c2ae276b Start reusing existing docker images if content hasn't changed
Content providers can specify a "content ID" which is used to identify
versions of the content. The ID is used in the docker image name and if
we find an existing image for a given content ID the build step is
skipped.
2018-11-04 13:23:04 +01:00
Tim Head c7dc5e1841 Remove explicit passing of logger instance 2018-10-16 08:04:13 +02:00
Tim Head e5c7fdd341 Use the local content provider
This let's us remove some special casing around local directories from
the build process.
2018-10-16 08:04:13 +02:00
Tim Head 866dd4f800 Start using content providers 2018-10-16 08:04:13 +02:00
yuvipanda 18f11b8c1f [WIP] base contentproviders commit
Very much a WIP, but some thougts on how we can support more
than just git.
2018-10-16 08:04:13 +02:00