kopia lustrzana https://github.com/jupyterhub/repo2docker
Merge branch 'master' into enab-python2
commit
cb766589e1
|
@ -0,0 +1,10 @@
|
||||||
|
# show coverage in CI status, not as a comment. Never consider it a failure.
|
||||||
|
comment: off
|
||||||
|
coverage:
|
||||||
|
status:
|
||||||
|
project:
|
||||||
|
default:
|
||||||
|
target: 0%
|
||||||
|
patch:
|
||||||
|
default:
|
||||||
|
target: 0%
|
|
@ -1,4 +1,13 @@
|
||||||
# Local Development
|
# Contributing to repo2docker development
|
||||||
|
|
||||||
|
This document covers:
|
||||||
|
|
||||||
|
- Setting up for Local Development
|
||||||
|
- Running Tests
|
||||||
|
- Updating and Freezing BuildPack Dependencies
|
||||||
|
- Creating a Release
|
||||||
|
|
||||||
|
## Setting up for Local Development
|
||||||
|
|
||||||
To develop & test repo2docker locally, you need:
|
To develop & test repo2docker locally, you need:
|
||||||
|
|
||||||
|
@ -9,7 +18,7 @@ To develop & test repo2docker locally, you need:
|
||||||
5. Your favorite text editor
|
5. Your favorite text editor
|
||||||
6. A recent version of [Docker Community Edition](https://www.docker.com/community-edition)
|
6. A recent version of [Docker Community Edition](https://www.docker.com/community-edition)
|
||||||
|
|
||||||
## Clone the repository
|
### Clone the repository
|
||||||
|
|
||||||
First, you need to get a copy of the repo2docker git repository on your local
|
First, you need to get a copy of the repo2docker git repository on your local
|
||||||
disk.
|
disk.
|
||||||
|
@ -21,7 +30,7 @@ git clone https://github.com/jupyter/repo2docker
|
||||||
This will clone repo2docker into a directory called `repo2docker`. You can
|
This will clone repo2docker into a directory called `repo2docker`. You can
|
||||||
make that your current directory with `cd repo2docker`.
|
make that your current directory with `cd repo2docker`.
|
||||||
|
|
||||||
## Set up local virtual environment
|
### Set up a local virtual environment
|
||||||
|
|
||||||
After cloning the repository (or your fork of the repo), you should set up an
|
After cloning the repository (or your fork of the repo), you should set up an
|
||||||
isolated environment to install libraries required for running / developing
|
isolated environment to install libraries required for running / developing
|
||||||
|
@ -37,7 +46,7 @@ pip3 install -r dev-requirements.txt
|
||||||
|
|
||||||
This should install all the libraries required for testing & running repo2docker!
|
This should install all the libraries required for testing & running repo2docker!
|
||||||
|
|
||||||
## Verify that docker is installed and running
|
### Verify that docker is installed and running
|
||||||
|
|
||||||
If you do not already have [Docker](https://www.docker.com/), you should be able
|
If you do not already have [Docker](https://www.docker.com/), you should be able
|
||||||
to download and install it for your operating system using the links from the
|
to download and install it for your operating system using the links from the
|
||||||
|
@ -74,7 +83,9 @@ Then you are good to go!
|
||||||
## Running tests
|
## Running tests
|
||||||
|
|
||||||
We have a lot of tests for various cases supported by repo2docker in the `tests/`
|
We have a lot of tests for various cases supported by repo2docker in the `tests/`
|
||||||
subdirectory. These use [py.test](https://docs.pytest.org/).
|
subdirectory. If you fix a bug or add new functionality consider adding a new
|
||||||
|
test to prevent the bug from coming back. These use
|
||||||
|
[py.test](https://docs.pytest.org/).
|
||||||
|
|
||||||
You can run all the tests with:
|
You can run all the tests with:
|
||||||
|
|
||||||
|
@ -88,10 +99,12 @@ If you want to run a specific test, you can do so with:
|
||||||
py.test -s tests/<path-to-test>
|
py.test -s tests/<path-to-test>
|
||||||
```
|
```
|
||||||
|
|
||||||
# Updating libraries installed for all repos
|
## Update and Freeze BuildPack Dependencies
|
||||||
|
|
||||||
For both the `conda` and `virtualenv` (`pip`) base environments, we install specific
|
### Updating libraries installed for all repos
|
||||||
pinned versions of all dependencies. We explicitly list the dependencies
|
|
||||||
|
For both the `conda` and `virtualenv` (`pip`) base environments in the **Conda BuildPack** and **Python BuildPack**,
|
||||||
|
we install specific pinned versions of all dependencies. We explicitly list the dependencies
|
||||||
we want, then *freeze* them at commit time to explicitly list all the
|
we want, then *freeze* them at commit time to explicitly list all the
|
||||||
transitive dependencies at current versions. This way, we know that
|
transitive dependencies at current versions. This way, we know that
|
||||||
all dependencies will have the exact same version installed at all times.
|
all dependencies will have the exact same version installed at all times.
|
||||||
|
@ -106,54 +119,68 @@ must follow these steps (with more detailed information in the sections below):
|
||||||
|
|
||||||
See the subsections below for more detailed instructions.
|
See the subsections below for more detailed instructions.
|
||||||
|
|
||||||
## Virtualenv dependencies
|
### Virtualenv dependencies
|
||||||
|
|
||||||
There are two files related to virtualenv dependencies:
|
1. There are two files related to virtualenv dependencies. Edit as needed.
|
||||||
|
|
||||||
1. `repo2docker/buildpacks/python/requirements.txt`
|
- `repo2docker/buildpacks/python/requirements.txt`
|
||||||
|
|
||||||
Contains list of packages to install in Python3 virtualenvs,
|
Contains list of packages to install in Python3 virtualenvs,
|
||||||
which are the default. **This where all Notebook versions &
|
which are the default. **This where all Notebook versions &
|
||||||
notebook extensions (such as JupyterLab / nteract) go**.
|
notebook extensions (such as JupyterLab / nteract) go**.
|
||||||
|
|
||||||
2. `repo2docker/buildpacks/python/requirements2.txt`
|
- `repo2docker/buildpacks/python/requirements2.txt`
|
||||||
|
|
||||||
Contains list of packages to install in Python2 virtualenvs, which
|
Contains list of packages to install in Python2 virtualenvs, which
|
||||||
can be specifically requested by users. **This only needs `IPyKernel`
|
can be specifically requested by users. **This only needs `IPyKernel`
|
||||||
and kernel related libraries** - Notebook / Notebook Extension need
|
and kernel related libraries** Notebook / Notebook Extension need
|
||||||
not be installed here.
|
not be installed here.
|
||||||
|
|
||||||
After you edit either of these files to add a new package / bump version on
|
2. After you edit either of these files to add a new package / bump version on
|
||||||
an existing package, run `./repo2docker/buildpacks/python/freeze.bash`.
|
an existing package, run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./repo2docker/buildpacks/python/freeze.bash
|
||||||
|
```
|
||||||
|
|
||||||
This script will resolve dependencies and write them to the respective `.frozen.txt`
|
This script will resolve dependencies and write them to the respective `.frozen.txt`
|
||||||
files. You will need Python3 and Python2 with virtualenv to run the script.
|
files.
|
||||||
|
|
||||||
All the `.txt` files in `repo2docker/buildpacks/python/` should be committed to git.
|
Note: If you do not have Python3 and Python2 with virtualenv, the script
|
||||||
|
will create and build Docker containers to process the frozen files.
|
||||||
|
|
||||||
## Conda dependencies
|
3. All the `.txt` files in `repo2docker/buildpacks/python/` should be committed to git.
|
||||||
|
|
||||||
There are two files related to conda dependencies:
|
4. Make a pull request.
|
||||||
|
|
||||||
1. `repo2docker/buildpacks/conda/environment.yml`
|
### Conda dependencies
|
||||||
|
|
||||||
|
1. There are two files related to conda dependencies. Edit as needed.
|
||||||
|
|
||||||
|
- `repo2docker/buildpacks/conda/environment.yml`
|
||||||
|
|
||||||
Contains list of packages to install in Python3 conda environments,
|
Contains list of packages to install in Python3 conda environments,
|
||||||
which are the default. **This is where all Notebook versions &
|
which are the default. **This is where all Notebook versions &
|
||||||
notebook extensions (such as JupyterLab / nteract) go**.
|
notebook extensions (such as JupyterLab / nteract) go**.
|
||||||
|
|
||||||
2. `repo2docker/buildpacks/conda/environment.py-2.7.yml`
|
- `repo2docker/buildpacks/conda/environment.py-2.7.yml`
|
||||||
|
|
||||||
Contains list of packages to install in Python2 conda environments, which
|
Contains list of packages to install in Python2 conda environments, which
|
||||||
can be specifically requested by users. **This only needs `IPyKernel`
|
can be specifically requested by users. **This only needs `IPyKernel`
|
||||||
and kernel related libraries** - Notebook / Notebook Extension need
|
and kernel related libraries**. Notebook / Notebook Extension need
|
||||||
not be installed here.
|
not be installed here.
|
||||||
|
|
||||||
Once you edit either of these files to add a new package / bump version on
|
2. Once you edit either of these files to add a new package / bump version on
|
||||||
an existing package, you should then run `./repo2docker/buildpacks/conda/freeze.py`.
|
an existing package, you should then run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python ./repo2docker/buildpacks/conda/freeze.py
|
||||||
|
```
|
||||||
|
|
||||||
This script will resolve dependencies and write them to the respective `.frozen.yml`
|
This script will resolve dependencies and write them to the respective `.frozen.yml`
|
||||||
files. You will need `docker` installed to run this script.
|
files. You will need `docker` installed to run this script.
|
||||||
|
|
||||||
After the freeze script finishes, a number of files will have been created.
|
3. After the freeze script finishes, a number of files will have been created.
|
||||||
Commit the following subset of files to git:
|
Commit the following subset of files to git:
|
||||||
|
|
||||||
```
|
```
|
||||||
|
@ -165,18 +192,20 @@ repo2docker/buildpacks/conda/environment.py-3.5.frozen.yml
|
||||||
repo2docker/buildpacks/conda/environment.py-3.6.frozen.yml
|
repo2docker/buildpacks/conda/environment.py-3.6.frozen.yml
|
||||||
```
|
```
|
||||||
|
|
||||||
|
4. Make a pull request.
|
||||||
|
|
||||||
## Make a Pull Request
|
### Make a Pull Request
|
||||||
|
|
||||||
Once you've made the commit, please make a Pull Request to the `jupyter/repo2docker`
|
Once you've made the commit, please make a Pull Request to the `jupyter/repo2docker`
|
||||||
repository, with a description of what versions were bumped / what new packages were
|
repository, with a description of what versions were bumped / what new packages were
|
||||||
added and why.
|
added and why. If you fix a bug or add new functionality consider adding a new
|
||||||
|
test to prevent the bug from coming back/the feature breaking in the future.
|
||||||
|
|
||||||
# Release Process
|
## Creating a Release
|
||||||
|
|
||||||
We try to make a release of repo2docker every few months if possible.
|
We try to make a release of repo2docker every few months if possible.
|
||||||
|
|
||||||
## Access
|
## Obtain access credentials
|
||||||
|
|
||||||
To release repo2docker, you will need proper access credentials prior to beginning the process.
|
To release repo2docker, you will need proper access credentials prior to beginning the process.
|
||||||
|
|
||||||
|
@ -186,7 +215,7 @@ To release repo2docker, you will need proper access credentials prior to beginni
|
||||||
|
|
||||||
If you do not have access to any of these, please contact a current maintainer of the project!
|
If you do not have access to any of these, please contact a current maintainer of the project!
|
||||||
|
|
||||||
## Steps
|
## Release Process Steps
|
||||||
|
|
||||||
1. Make a PR bumping version number of repo2docker in the
|
1. Make a PR bumping version number of repo2docker in the
|
||||||
`setup.py` file (like https://github.com/jupyter/repo2docker/pull/221),
|
`setup.py` file (like https://github.com/jupyter/repo2docker/pull/221),
|
||||||
|
|
33
README.md
33
README.md
|
@ -1,11 +1,10 @@
|
||||||
# jupyter-repo2docker
|
# jupyter-repo2docker
|
||||||
|
|
||||||
|
|
||||||
[](https://travis-ci.org/jupyter/repo2docker)
|
[](https://travis-ci.org/jupyter/repo2docker)
|
||||||
[](http://repo2docker.readthedocs.io/en/latest/?badge=latest)
|
[](http://repo2docker.readthedocs.io/en/latest/?badge=latest)
|
||||||
|
|
||||||
**jupyter-repo2docker** takes as input a repository source, such as a GitHub repo. It then builds, runs, and/or pushes Docker images built from
|
**jupyter-repo2docker** takes as input a repository source, such as a GitHub
|
||||||
that source.
|
repo. It then builds, runs, and/or pushes Docker images built from that source.
|
||||||
|
|
||||||
See the [repo2docker documentation](http://repo2docker.readthedocs.io)
|
See the [repo2docker documentation](http://repo2docker.readthedocs.io)
|
||||||
for more information.
|
for more information.
|
||||||
|
@ -16,15 +15,17 @@ for more information.
|
||||||
is recommended.
|
is recommended.
|
||||||
2. Python 3.4+.
|
2. Python 3.4+.
|
||||||
|
|
||||||
|
Supported on Linux and macOS. [See documentation note about Windows support.](http://repo2docker.readthedocs.io/en/latest/install.html#note-about-windows-support)
|
||||||
|
|
||||||
## Installation
|
## Installation
|
||||||
|
|
||||||
To install from pypi, the python packaging index:
|
To install from PyPI, the python packaging index, using `pip`:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pip install jupyter-repo2docker
|
pip install jupyter-repo2docker
|
||||||
```
|
```
|
||||||
|
|
||||||
To install from source and start contributing:
|
To install from source:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/jupyter/repo2docker.git
|
git clone https://github.com/jupyter/repo2docker.git
|
||||||
|
@ -34,29 +35,29 @@ pip install -e .
|
||||||
|
|
||||||
## Usage
|
## Usage
|
||||||
|
|
||||||
The core feature of repo2docker is to fetch a repo (from github or locally), build a container
|
The core feature of repo2docker is to fetch a repo (from github or locally),
|
||||||
image based on the specifications found in the repo & optionally launch a local Jupyter Notebook
|
build a container image based on the specifications found in the repo &
|
||||||
you can use to explore it.
|
optionally launch a local Jupyter Notebook you can use to explore it.
|
||||||
|
|
||||||
**Note that Docker needs to be running on your machine for this to work.**
|
**Note that Docker needs to be running on your machine for this to work.**
|
||||||
|
|
||||||
Example:
|
Example:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
jupyter-repo2docker https://github.com/jakevdp/PythonDataScienceHandbook
|
jupyter-repo2docker https://github.com/norvig/pytudes
|
||||||
```
|
```
|
||||||
|
|
||||||
After building (it might take a while!), it should output in your terminal something like:
|
After building (it might take a while!), it should output in your terminal
|
||||||
|
something like:
|
||||||
|
|
||||||
|
```bash
|
||||||
```
|
|
||||||
Copy/paste this URL into your browser when you connect for the first time,
|
Copy/paste this URL into your browser when you connect for the first time,
|
||||||
to login with a token:
|
to login with a token:
|
||||||
http://0.0.0.0:36511/?token=f94f8fabb92e22f5bfab116c382b4707fc2cade56ad1ace0
|
http://0.0.0.0:36511/?token=f94f8fabb92e22f5bfab116c382b4707fc2cade56ad1ace0
|
||||||
```
|
```
|
||||||
|
|
||||||
If you copy paste that URL into your browser you will see a Jupyter Notebook with the
|
If you copy paste that URL into your browser you will see a Jupyter Notebook
|
||||||
contents of the repository you had just built!
|
with the contents of the repository you had just built!
|
||||||
|
|
||||||
For more information on how to use ``repo2docker``, see the
|
For more information on how to use ``repo2docker``, see the
|
||||||
[usage guide](http://repo2docker.readthedocs.io/en/latest/usage.html).
|
[usage guide](http://repo2docker.readthedocs.io/en/latest/usage.html).
|
||||||
|
@ -64,8 +65,8 @@ For more information on how to use ``repo2docker``, see the
|
||||||
## Repository specifications
|
## Repository specifications
|
||||||
|
|
||||||
Repo2Docker looks for configuration files in the source repository to
|
Repo2Docker looks for configuration files in the source repository to
|
||||||
determine how the Docker image should be built.
|
determine how the Docker image should be built. It is philosophically similar
|
||||||
It is philosophically similar to [Heroku Build Packs](https://devcenter.heroku.com/articles/buildpacks).
|
to [Heroku Build Packs](https://devcenter.heroku.com/articles/buildpacks).
|
||||||
|
|
||||||
For a list of the configuration files that ``repo2docker`` can use,
|
For a list of the configuration files that ``repo2docker`` can use,
|
||||||
see the [usage guide](http://repo2docker.readthedocs.io/en/latest/usage.html).
|
see the [usage guide](http://repo2docker.readthedocs.io/en/latest/usage.html).
|
||||||
|
|
|
@ -1,6 +1,4 @@
|
||||||
"""
|
"""BuildPack for conda environments"""
|
||||||
Buildpack for conda environments
|
|
||||||
"""
|
|
||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
|
|
||||||
|
@ -15,16 +13,48 @@ HERE = os.path.dirname(os.path.abspath(__file__))
|
||||||
|
|
||||||
|
|
||||||
class CondaBuildPack(BaseImage):
|
class CondaBuildPack(BaseImage):
|
||||||
|
"""A conda BuildPack.
|
||||||
|
|
||||||
|
Uses miniconda since it is more lightweight than Anaconda.
|
||||||
|
|
||||||
|
"""
|
||||||
def get_env(self):
|
def get_env(self):
|
||||||
|
"""Return environment variables to be set.
|
||||||
|
|
||||||
|
We set `CONDA_DIR` to the conda install directory and
|
||||||
|
the `NB_PYTHON_PREFIX` to the location of the jupyter binary.
|
||||||
|
|
||||||
|
"""
|
||||||
return super().get_env() + [
|
return super().get_env() + [
|
||||||
('CONDA_DIR', '${APP_BASE}/conda'),
|
('CONDA_DIR', '${APP_BASE}/conda'),
|
||||||
('NB_PYTHON_PREFIX', '${CONDA_DIR}'),
|
('NB_PYTHON_PREFIX', '${CONDA_DIR}'),
|
||||||
]
|
]
|
||||||
|
|
||||||
def get_path(self):
|
def get_path(self):
|
||||||
|
"""Return paths (including conda environment path) to be added to
|
||||||
|
the PATH environment variable.
|
||||||
|
|
||||||
|
"""
|
||||||
return super().get_path() + ['${CONDA_DIR}/bin']
|
return super().get_path() + ['${CONDA_DIR}/bin']
|
||||||
|
|
||||||
def get_build_scripts(self):
|
def get_build_scripts(self):
|
||||||
|
"""
|
||||||
|
Return series of build-steps common to all Python 3 repositories.
|
||||||
|
|
||||||
|
All scripts here should be independent of contents of the repository.
|
||||||
|
|
||||||
|
This sets up through `install-miniconda.bash` (found in this directory):
|
||||||
|
|
||||||
|
- a directory for the conda environment and its ownership by the
|
||||||
|
notebook user
|
||||||
|
- a Python 3 interpreter for the conda environment
|
||||||
|
- a Python 3 jupyter kernel
|
||||||
|
- a frozen base set of requirements, including:
|
||||||
|
- support for Jupyter widgets
|
||||||
|
- support for JupyterLab
|
||||||
|
- support for nteract
|
||||||
|
|
||||||
|
"""
|
||||||
return super().get_build_scripts() + [
|
return super().get_build_scripts() + [
|
||||||
(
|
(
|
||||||
"root",
|
"root",
|
||||||
|
@ -41,6 +71,20 @@ class CondaBuildPack(BaseImage):
|
||||||
}
|
}
|
||||||
|
|
||||||
def get_build_script_files(self):
|
def get_build_script_files(self):
|
||||||
|
"""
|
||||||
|
Dict of files to be copied to the container image for use in building.
|
||||||
|
|
||||||
|
This is copied before the `build_scripts` & `assemble_scripts` are
|
||||||
|
run, so can be executed from either of them.
|
||||||
|
|
||||||
|
It's a dictionary where the key is the source file path in the host
|
||||||
|
system, and the value is the destination file path inside the
|
||||||
|
container image.
|
||||||
|
|
||||||
|
This currently adds a frozen set of Python requirements to the dict
|
||||||
|
of files.
|
||||||
|
|
||||||
|
"""
|
||||||
files = {
|
files = {
|
||||||
'conda/install-miniconda.bash': '/tmp/install-miniconda.bash',
|
'conda/install-miniconda.bash': '/tmp/install-miniconda.bash',
|
||||||
}
|
}
|
||||||
|
@ -69,10 +113,11 @@ class CondaBuildPack(BaseImage):
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def python_version(self):
|
def python_version(self):
|
||||||
"""
|
"""Detect the Python version for a given `environment.yml`
|
||||||
Detect the Python version for a given environment.yml
|
|
||||||
|
Will return 'x.y' if version is found (e.g '3.6'),
|
||||||
|
or a Falsy empty string '' if not found.
|
||||||
|
|
||||||
Will return 'x.y' if found, or Falsy '' if not.
|
|
||||||
"""
|
"""
|
||||||
environment_yml = self.binder_path('environment.yml')
|
environment_yml = self.binder_path('environment.yml')
|
||||||
if not os.path.exists(environment_yml):
|
if not os.path.exists(environment_yml):
|
||||||
|
@ -109,6 +154,8 @@ class CondaBuildPack(BaseImage):
|
||||||
return self.python_version and self.python_version.split('.')[0] == '2'
|
return self.python_version and self.python_version.split('.')[0] == '2'
|
||||||
|
|
||||||
def get_assemble_scripts(self):
|
def get_assemble_scripts(self):
|
||||||
|
"""Return series of build-steps specific to this source repository.
|
||||||
|
"""
|
||||||
assembly_scripts = []
|
assembly_scripts = []
|
||||||
environment_yml = self.binder_path('environment.yml')
|
environment_yml = self.binder_path('environment.yml')
|
||||||
env_name = 'kernel' if self.py2 else 'root'
|
env_name = 'kernel' if self.py2 else 'root'
|
||||||
|
@ -124,4 +171,6 @@ class CondaBuildPack(BaseImage):
|
||||||
return super().get_assemble_scripts() + assembly_scripts
|
return super().get_assemble_scripts() + assembly_scripts
|
||||||
|
|
||||||
def detect(self):
|
def detect(self):
|
||||||
|
"""Check if current repo should be built with the Conda BuildPack.
|
||||||
|
"""
|
||||||
return os.path.exists(self.binder_path('environment.yml')) and super().detect()
|
return os.path.exists(self.binder_path('environment.yml')) and super().detect()
|
||||||
|
|
|
@ -1,5 +1,5 @@
|
||||||
# AUTO GENERATED FROM environment.py-3.6.yml, DO NOT MANUALLY MODIFY
|
# AUTO GENERATED FROM environment.py-3.6.yml, DO NOT MANUALLY MODIFY
|
||||||
# Frozen on 2018-03-13 17:43:53 UTC
|
# Frozen on 2018-03-20 00:47:27 UTC
|
||||||
name: r2d
|
name: r2d
|
||||||
channels:
|
channels:
|
||||||
- conda-forge
|
- conda-forge
|
||||||
|
@ -30,23 +30,23 @@ dependencies:
|
||||||
- nbconvert=5.3.1=py_1
|
- nbconvert=5.3.1=py_1
|
||||||
- nbformat=4.4.0=py36_0
|
- nbformat=4.4.0=py36_0
|
||||||
- ncurses=5.9=10
|
- ncurses=5.9=10
|
||||||
- notebook=5.4.0=py36_0
|
- notebook=5.4.1=py36_0
|
||||||
- openssl=1.0.2n=0
|
- openssl=1.0.2n=0
|
||||||
- pandoc=2.1.2=0
|
- pandoc=2.1.3=0
|
||||||
- pandocfilters=1.4.1=py36_0
|
- pandocfilters=1.4.1=py36_0
|
||||||
- parso=0.1.1=py_0
|
- parso=0.1.1=py_0
|
||||||
- pexpect=4.4.0=py36_0
|
- pexpect=4.4.0=py36_0
|
||||||
- pickleshare=0.7.4=py36_0
|
- pickleshare=0.7.4=py36_0
|
||||||
- pip=9.0.1=py36_1
|
- pip=9.0.2=py36_0
|
||||||
- prompt_toolkit=1.0.15=py36_0
|
- prompt_toolkit=1.0.15=py36_0
|
||||||
- ptyprocess=0.5.2=py36_0
|
- ptyprocess=0.5.2=py36_0
|
||||||
- pygments=2.2.0=py36_0
|
- pygments=2.2.0=py36_0
|
||||||
- python=3.6.4=0
|
- python=3.6.4=0
|
||||||
- python-dateutil=2.6.1=py36_0
|
- python-dateutil=2.7.0=py_0
|
||||||
- pyzmq=17.0.0=py36_3
|
- pyzmq=17.0.0=py36_3
|
||||||
- readline=7.0=0
|
- readline=7.0=0
|
||||||
- send2trash=1.5.0=py_0
|
- send2trash=1.5.0=py_0
|
||||||
- setuptools=38.5.2=py36_0
|
- setuptools=39.0.1=py36_0
|
||||||
- simplegeneric=0.8.1=py36_0
|
- simplegeneric=0.8.1=py36_0
|
||||||
- six=1.11.0=py36_1
|
- six=1.11.0=py36_1
|
||||||
- sqlite=3.20.1=2
|
- sqlite=3.20.1=2
|
||||||
|
|
|
@ -1,5 +1,5 @@
|
||||||
# AUTO GENERATED FROM environment.py-2.7.yml, DO NOT MANUALLY MODIFY
|
# AUTO GENERATED FROM environment.py-2.7.yml, DO NOT MANUALLY MODIFY
|
||||||
# Frozen on 2018-03-13 17:38:07 UTC
|
# Frozen on 2018-03-20 00:42:45 UTC
|
||||||
name: r2d
|
name: r2d
|
||||||
channels:
|
channels:
|
||||||
- conda-forge
|
- conda-forge
|
||||||
|
@ -24,16 +24,16 @@ dependencies:
|
||||||
- pathlib2=2.3.0=py27_0
|
- pathlib2=2.3.0=py27_0
|
||||||
- pexpect=4.4.0=py27_0
|
- pexpect=4.4.0=py27_0
|
||||||
- pickleshare=0.7.4=py27_0
|
- pickleshare=0.7.4=py27_0
|
||||||
- pip=9.0.1=py27_1
|
- pip=9.0.2=py27_0
|
||||||
- prompt_toolkit=1.0.15=py27_0
|
- prompt_toolkit=1.0.15=py27_0
|
||||||
- ptyprocess=0.5.2=py27_0
|
- ptyprocess=0.5.2=py27_0
|
||||||
- pygments=2.2.0=py27_0
|
- pygments=2.2.0=py27_0
|
||||||
- python=2.7.14=4
|
- python=2.7.14=4
|
||||||
- python-dateutil=2.6.1=py27_0
|
- python-dateutil=2.7.0=py_0
|
||||||
- pyzmq=17.0.0=py27_3
|
- pyzmq=17.0.0=py27_3
|
||||||
- readline=7.0=0
|
- readline=7.0=0
|
||||||
- scandir=1.7=py27_0
|
- scandir=1.7=py27_0
|
||||||
- setuptools=38.5.2=py27_0
|
- setuptools=39.0.1=py27_0
|
||||||
- simplegeneric=0.8.1=py27_0
|
- simplegeneric=0.8.1=py27_0
|
||||||
- singledispatch=3.4.0.3=py27_0
|
- singledispatch=3.4.0.3=py27_0
|
||||||
- six=1.11.0=py27_1
|
- six=1.11.0=py27_1
|
||||||
|
|
|
@ -1,5 +1,5 @@
|
||||||
# AUTO GENERATED FROM environment.py-3.5.yml, DO NOT MANUALLY MODIFY
|
# AUTO GENERATED FROM environment.py-3.5.yml, DO NOT MANUALLY MODIFY
|
||||||
# Frozen on 2018-03-13 17:40:14 UTC
|
# Frozen on 2018-03-20 00:43:53 UTC
|
||||||
name: r2d
|
name: r2d
|
||||||
channels:
|
channels:
|
||||||
- conda-forge
|
- conda-forge
|
||||||
|
@ -30,23 +30,23 @@ dependencies:
|
||||||
- nbconvert=5.3.1=py_1
|
- nbconvert=5.3.1=py_1
|
||||||
- nbformat=4.4.0=py35_0
|
- nbformat=4.4.0=py35_0
|
||||||
- ncurses=5.9=10
|
- ncurses=5.9=10
|
||||||
- notebook=5.4.0=py35_0
|
- notebook=5.4.1=py35_0
|
||||||
- openssl=1.0.2n=0
|
- openssl=1.0.2n=0
|
||||||
- pandoc=2.1.2=0
|
- pandoc=2.1.3=0
|
||||||
- pandocfilters=1.4.1=py35_0
|
- pandocfilters=1.4.1=py35_0
|
||||||
- parso=0.1.1=py_0
|
- parso=0.1.1=py_0
|
||||||
- pexpect=4.4.0=py35_0
|
- pexpect=4.4.0=py35_0
|
||||||
- pickleshare=0.7.4=py35_0
|
- pickleshare=0.7.4=py35_0
|
||||||
- pip=9.0.1=py35_1
|
- pip=9.0.2=py35_0
|
||||||
- prompt_toolkit=1.0.15=py35_0
|
- prompt_toolkit=1.0.15=py35_0
|
||||||
- ptyprocess=0.5.2=py35_0
|
- ptyprocess=0.5.2=py35_0
|
||||||
- pygments=2.2.0=py35_0
|
- pygments=2.2.0=py35_0
|
||||||
- python=3.5.5=0
|
- python=3.5.5=0
|
||||||
- python-dateutil=2.6.1=py35_0
|
- python-dateutil=2.7.0=py_0
|
||||||
- pyzmq=17.0.0=py35_3
|
- pyzmq=17.0.0=py35_3
|
||||||
- readline=7.0=0
|
- readline=7.0=0
|
||||||
- send2trash=1.5.0=py_0
|
- send2trash=1.5.0=py_0
|
||||||
- setuptools=38.5.2=py35_0
|
- setuptools=39.0.1=py35_0
|
||||||
- simplegeneric=0.8.1=py35_0
|
- simplegeneric=0.8.1=py35_0
|
||||||
- six=1.11.0=py35_1
|
- six=1.11.0=py35_1
|
||||||
- sqlite=3.20.1=2
|
- sqlite=3.20.1=2
|
||||||
|
|
|
@ -1,5 +1,5 @@
|
||||||
# AUTO GENERATED FROM environment.py-3.6.yml, DO NOT MANUALLY MODIFY
|
# AUTO GENERATED FROM environment.py-3.6.yml, DO NOT MANUALLY MODIFY
|
||||||
# Frozen on 2018-03-13 17:43:53 UTC
|
# Frozen on 2018-03-20 00:47:27 UTC
|
||||||
name: r2d
|
name: r2d
|
||||||
channels:
|
channels:
|
||||||
- conda-forge
|
- conda-forge
|
||||||
|
@ -30,23 +30,23 @@ dependencies:
|
||||||
- nbconvert=5.3.1=py_1
|
- nbconvert=5.3.1=py_1
|
||||||
- nbformat=4.4.0=py36_0
|
- nbformat=4.4.0=py36_0
|
||||||
- ncurses=5.9=10
|
- ncurses=5.9=10
|
||||||
- notebook=5.4.0=py36_0
|
- notebook=5.4.1=py36_0
|
||||||
- openssl=1.0.2n=0
|
- openssl=1.0.2n=0
|
||||||
- pandoc=2.1.2=0
|
- pandoc=2.1.3=0
|
||||||
- pandocfilters=1.4.1=py36_0
|
- pandocfilters=1.4.1=py36_0
|
||||||
- parso=0.1.1=py_0
|
- parso=0.1.1=py_0
|
||||||
- pexpect=4.4.0=py36_0
|
- pexpect=4.4.0=py36_0
|
||||||
- pickleshare=0.7.4=py36_0
|
- pickleshare=0.7.4=py36_0
|
||||||
- pip=9.0.1=py36_1
|
- pip=9.0.2=py36_0
|
||||||
- prompt_toolkit=1.0.15=py36_0
|
- prompt_toolkit=1.0.15=py36_0
|
||||||
- ptyprocess=0.5.2=py36_0
|
- ptyprocess=0.5.2=py36_0
|
||||||
- pygments=2.2.0=py36_0
|
- pygments=2.2.0=py36_0
|
||||||
- python=3.6.4=0
|
- python=3.6.4=0
|
||||||
- python-dateutil=2.6.1=py36_0
|
- python-dateutil=2.7.0=py_0
|
||||||
- pyzmq=17.0.0=py36_3
|
- pyzmq=17.0.0=py36_3
|
||||||
- readline=7.0=0
|
- readline=7.0=0
|
||||||
- send2trash=1.5.0=py_0
|
- send2trash=1.5.0=py_0
|
||||||
- setuptools=38.5.2=py36_0
|
- setuptools=39.0.1=py36_0
|
||||||
- simplegeneric=0.8.1=py36_0
|
- simplegeneric=0.8.1=py36_0
|
||||||
- six=1.11.0=py36_1
|
- six=1.11.0=py36_1
|
||||||
- sqlite=3.20.1=2
|
- sqlite=3.20.1=2
|
||||||
|
|
|
@ -3,6 +3,6 @@ dependencies:
|
||||||
- ipywidgets==7.1.1
|
- ipywidgets==7.1.1
|
||||||
- jupyterlab==0.31.5
|
- jupyterlab==0.31.5
|
||||||
- tornado==4.5.3
|
- tornado==4.5.3
|
||||||
- notebook==5.4.0
|
- notebook==5.4.1
|
||||||
- pip:
|
- pip:
|
||||||
- nteract_on_jupyter==1.5.0
|
- nteract_on_jupyter==1.5.0
|
||||||
|
|
|
@ -21,7 +21,7 @@ from ruamel.yaml import YAML
|
||||||
|
|
||||||
MINICONDA_VERSION = '4.3.27'
|
MINICONDA_VERSION = '4.3.27'
|
||||||
# need conda ≥ 4.4 to avoid bug adding spurious pip dependencies
|
# need conda ≥ 4.4 to avoid bug adding spurious pip dependencies
|
||||||
CONDA_VERSION = '4.4.8'
|
CONDA_VERSION = '4.4.11'
|
||||||
|
|
||||||
HERE = pathlib.Path(os.path.dirname(os.path.abspath(__file__)))
|
HERE = pathlib.Path(os.path.dirname(os.path.abspath(__file__)))
|
||||||
|
|
||||||
|
|
|
@ -1,5 +1,4 @@
|
||||||
"""
|
"""Generates a variety of Dockerfiles based on an input matrix
|
||||||
Generates a variety of Dockerfiles based on an input matrix
|
|
||||||
"""
|
"""
|
||||||
import os
|
import os
|
||||||
import docker
|
import docker
|
||||||
|
@ -7,17 +6,21 @@ from .base import BuildPack
|
||||||
|
|
||||||
|
|
||||||
class DockerBuildPack(BuildPack):
|
class DockerBuildPack(BuildPack):
|
||||||
|
"""Docker BuildPack"""
|
||||||
dockerfile = "Dockerfile"
|
dockerfile = "Dockerfile"
|
||||||
|
|
||||||
def detect(self):
|
def detect(self):
|
||||||
|
"""Check if current repo should be built with the Docker BuildPack"""
|
||||||
return os.path.exists(self.binder_path('Dockerfile'))
|
return os.path.exists(self.binder_path('Dockerfile'))
|
||||||
|
|
||||||
def render(self):
|
def render(self):
|
||||||
|
"""Render the Dockerfile using by reading it from the source repo"""
|
||||||
Dockerfile = self.binder_path('Dockerfile')
|
Dockerfile = self.binder_path('Dockerfile')
|
||||||
with open(Dockerfile) as f:
|
with open(Dockerfile) as f:
|
||||||
return f.read()
|
return f.read()
|
||||||
|
|
||||||
def build(self, image_spec, memory_limit, build_args):
|
def build(self, image_spec, memory_limit, build_args):
|
||||||
|
"""Build a Docker image based on the Dockerfile in the source repo."""
|
||||||
limits = {
|
limits = {
|
||||||
# Always disable memory swap for building, since mostly
|
# Always disable memory swap for building, since mostly
|
||||||
# nothing good can come of that.
|
# nothing good can come of that.
|
||||||
|
|
|
@ -1,5 +1,10 @@
|
||||||
"""
|
"""Generates Dockerfiles from the legacy Binder Dockerfiles
|
||||||
Generates a variety of Dockerfiles based on an input matrix
|
based on `andrewosh/binder-base`.
|
||||||
|
|
||||||
|
The Dockerfile is amended to add the contents of the repository
|
||||||
|
to the image and install a supported version of the notebook
|
||||||
|
and IPython kernel.
|
||||||
|
|
||||||
"""
|
"""
|
||||||
import os
|
import os
|
||||||
import shutil
|
import shutil
|
||||||
|
@ -7,7 +12,7 @@ from textwrap import dedent
|
||||||
from ..docker import DockerBuildPack
|
from ..docker import DockerBuildPack
|
||||||
|
|
||||||
class LegacyBinderDockerBuildPack(DockerBuildPack):
|
class LegacyBinderDockerBuildPack(DockerBuildPack):
|
||||||
|
"""Legacy build pack for compatibility to first version of Binder."""
|
||||||
dockerfile = '._binder.Dockerfile'
|
dockerfile = '._binder.Dockerfile'
|
||||||
|
|
||||||
legacy_prependix = dedent(r"""
|
legacy_prependix = dedent(r"""
|
||||||
|
@ -39,6 +44,13 @@ class LegacyBinderDockerBuildPack(DockerBuildPack):
|
||||||
""")
|
""")
|
||||||
|
|
||||||
def render(self):
|
def render(self):
|
||||||
|
"""Render buildpack into a Dockerfile.
|
||||||
|
|
||||||
|
Render legacy image source (andrewosh/binder-base at a specific commit)
|
||||||
|
and then prependix. Render appendix (post-build commands) at the end of
|
||||||
|
the Dockerfile.
|
||||||
|
|
||||||
|
"""
|
||||||
segments = [
|
segments = [
|
||||||
'FROM andrewosh/binder-base@sha256:eabde24f4c55174832ed8795faa40cea62fc9e2a4a9f1ee1444f8a2e4f9710ee',
|
'FROM andrewosh/binder-base@sha256:eabde24f4c55174832ed8795faa40cea62fc9e2a4a9f1ee1444f8a2e4f9710ee',
|
||||||
self.legacy_prependix,
|
self.legacy_prependix,
|
||||||
|
@ -52,12 +64,27 @@ class LegacyBinderDockerBuildPack(DockerBuildPack):
|
||||||
return '\n'.join(segments)
|
return '\n'.join(segments)
|
||||||
|
|
||||||
def get_build_script_files(self):
|
def get_build_script_files(self):
|
||||||
|
"""
|
||||||
|
Dict of files to be copied to the container image for use in building.
|
||||||
|
|
||||||
|
This is copied before the `build_scripts` & `assemble_scripts` are
|
||||||
|
run, so can be executed from either of them.
|
||||||
|
|
||||||
|
It's a dictionary where the key is the source file path in the host
|
||||||
|
system, and the value is the destination file path inside the
|
||||||
|
container image.
|
||||||
|
|
||||||
|
This currently adds a frozen set of Python requirements to the dict
|
||||||
|
of files.
|
||||||
|
|
||||||
|
"""
|
||||||
return {
|
return {
|
||||||
'legacy/root.frozen.yml': '/tmp/root.frozen.yml',
|
'legacy/root.frozen.yml': '/tmp/root.frozen.yml',
|
||||||
'legacy/python3.frozen.yml': '/tmp/python3.frozen.yml',
|
'legacy/python3.frozen.yml': '/tmp/python3.frozen.yml',
|
||||||
}
|
}
|
||||||
|
|
||||||
def build(self, image_spec, memory_limit, build_args):
|
def build(self, image_spec, memory_limit, build_args):
|
||||||
|
"""Build a legacy Docker image."""
|
||||||
with open(self.dockerfile, 'w') as f:
|
with open(self.dockerfile, 'w') as f:
|
||||||
f.write(self.render())
|
f.write(self.render())
|
||||||
for env in ('root', 'python3'):
|
for env in ('root', 'python3'):
|
||||||
|
@ -70,6 +97,8 @@ class LegacyBinderDockerBuildPack(DockerBuildPack):
|
||||||
return super().build(image_spec, memory_limit, build_args)
|
return super().build(image_spec, memory_limit, build_args)
|
||||||
|
|
||||||
def detect(self):
|
def detect(self):
|
||||||
|
"""Check if current repo should be built with the Legacy BuildPack.
|
||||||
|
"""
|
||||||
try:
|
try:
|
||||||
with open('Dockerfile', 'r') as f:
|
with open('Dockerfile', 'r') as f:
|
||||||
for line in f:
|
for line in f:
|
||||||
|
|
|
@ -1,5 +1,5 @@
|
||||||
# AUTO GENERATED FROM requirements.txt, DO NOT MANUALLY MODIFY
|
# AUTO GENERATED FROM requirements.txt, DO NOT MANUALLY MODIFY
|
||||||
# Frozen on Thu Mar 15 11:51:16 UTC 2018
|
# Frozen on Tue Mar 20 00:18:43 UTC 2018
|
||||||
bleach==2.1.3
|
bleach==2.1.3
|
||||||
decorator==4.2.1
|
decorator==4.2.1
|
||||||
entrypoints==0.2.3
|
entrypoints==0.2.3
|
||||||
|
@ -19,7 +19,7 @@ MarkupSafe==1.0
|
||||||
mistune==0.8.3
|
mistune==0.8.3
|
||||||
nbconvert==5.3.1
|
nbconvert==5.3.1
|
||||||
nbformat==4.4.0
|
nbformat==4.4.0
|
||||||
notebook==5.4.0
|
notebook==5.4.1
|
||||||
nteract-on-jupyter==1.5.0
|
nteract-on-jupyter==1.5.0
|
||||||
pandocfilters==1.4.2
|
pandocfilters==1.4.2
|
||||||
parso==0.1.1
|
parso==0.1.1
|
||||||
|
|
|
@ -1,4 +1,4 @@
|
||||||
notebook==5.4.0
|
notebook==5.4.1
|
||||||
tornado==4.5.3
|
tornado==4.5.3
|
||||||
ipywidgets==7.1.1
|
ipywidgets==7.1.1
|
||||||
jupyterlab==0.31.5
|
jupyterlab==0.31.5
|
||||||
|
|
|
@ -1,5 +1,5 @@
|
||||||
# AUTO GENERATED FROM requirements2.txt, DO NOT MANUALLY MODIFY
|
# AUTO GENERATED FROM requirements2.txt, DO NOT MANUALLY MODIFY
|
||||||
# Frozen on Thu Mar 15 11:52:00 UTC 2018
|
# Frozen on Tue Mar 20 00:20:49 UTC 2018
|
||||||
backports-abc==0.5
|
backports-abc==0.5
|
||||||
backports.shutil-get-terminal-size==1.0.0
|
backports.shutil-get-terminal-size==1.0.0
|
||||||
certifi==2018.1.18
|
certifi==2018.1.18
|
||||||
|
|
Ładowanie…
Reference in New Issue