6.6 KiB
Local Development
To develop & test repo2docker locally, you need:
- Familiarity with using a command line terminal
- A computer running macOS / Linux
- Some knowledge of git
- At least python 3.4
- Your favorite text editor
- A recent version of Docker Community Edition
Clone the repository
First, you need to get a copy of the repo2docker git repository on your local disk.
git clone https://github.com/jupyter/repo2docker
This will clone repo2docker into a directory called repo2docker
. You can
make that your current directory with cd repo2docker
.
Set up local virtual environment
After cloning the repository (or your fork of the repo), you should set up an
isolated environment to install libraries required for running / developing
repo2docker. There are many ways to do this, and a virtual environment
is
one of them.
python3 -m venv .
source bin/activate
pip3 install -e .
pip3 install -r dev-requirements.txt
This should install all the libraries required for testing & running repo2docker!
Verify that docker is installed and running
If you do not already have Docker, you should be able to download and install it for your operating system using the links from the official website. After you have installed it, you can verify that it is working by running the following commands:
docker version
It should output something like:
Client:
Version: 17.09.0-ce
API version: 1.32
Go version: go1.8.3
Git commit: afdb6d4
Built: Tue Sep 26 22:42:45 2017
OS/Arch: linux/amd64
Server:
Version: 17.09.0-ce
API version: 1.32 (minimum version 1.12)
Go version: go1.8.3
Git commit: afdb6d4
Built: Tue Sep 26 22:41:24 2017
OS/Arch: linux/amd64
Experimental: false
Then you are good to go!
Running tests
We have a lot of tests for various cases supported by repo2docker in the tests/
subdirectory. These use py.test.
You can run all the tests with:
py.test -s tests/*
If you want to run a specific test, you can do so with:
py.test -s tests/<path-to-test>
Updating libraries installed for all repos
For both the conda and virtualenv base environments, we install specific pinned versions of all dependencies. We explicitly list the dependencies we want, then freeze them at commit time to explicitly list all the transitive dependencies at current versions. This way, we know that all dependencies will have the exact same version installed at all times.
If you update a library in the base environment, you need to update it for both the virtualenv and conda environments before submitting a PR.
Virtualenv dependencies
There are two files related to virtualenv dependencies:
-
repo2docker/buildpacks/python/requirements.txt
Contains list of packages to install in Python3 virtualenvs, which are the default. This should be where all Notebook versions & notebook extensions (such as JupyterLab / nteract) go.
-
repo2docker/buildpacks/python/requirements2.txt
Contains list of packages to install in Python2 virtualenvs, which can be specifically requested by users. This only needs
IPyKernel
and kernel related libraries - Notebook / Notebook Extension need not be installed here.
Once you edit either of these files to add a new package / bump version on
an existing package, you should then run ./repo2docker/buildpacks/python/freeze.bash
.
This script will resolve dependencies and write them to the respective .frozen.txt
files. You will need Python3 and Python2 with virtualenv to run the script.
All the .txt files in repo2docker/buildpacks/python/
should be comitted to git.
Conda dependencies
There are two files related to conda dependencies:
-
repo2docker/buildpacks/conda/environment.yml
Contains list of packages to install in Python3 conda environments, which are the default. This should be where all Notebook versions & notebook extensions (such as JupyterLab / nteract) go.
-
repo2docker/buildpacks/conda/environment.py-2.7.yml
Contains list of packages to install in Python2 conda environments, which can be specifically requested by users. This only needs
IPyKernel
and kernel related libraries - Notebook / Notebook Extension need not be installed here.
Once you edit either of these files to add a new package / bump version on
an existing package, you should then run ./repo2docker/buildpacks/conda/freeze.py
.
This script will resolve dependencies and write them to the respective .frozen.yml
files. You will need docker
installed to run this script.
The following files should be comitted to git:
- repo2docker/buildpacks/conda/environment.frozen.yml
- repo2docker/buildpacks/conda/environment.py-2.7.yml
- repo2docker/buildpacks/conda/environment.py-3.6.frozen.yml
- repo2docker/buildpacks/conda/environment.py-2.7.frozen.yml
- repo2docker/buildpacks/conda/environment.py-3.5.frozen.yml
- repo2docker/buildpacks/conda/environment.yml
Make a Pull Request
Once you've made the commit, please make a Pull Request to the jupyter/repo2docker
repository, with a description of what versions were bumped / what new packages were
added and why.
Release Process
We try to make a release of repo2docker every few months if possible.
Access
To release repo2docker, you will need proper access credentials prior to beginning the process.
- Access to the PyPI package for repo2docker
- Access to push tags to the jupyter/repo2docker repository
- Acess to push images to dockerhub on jupyter/repo2docker
If you do not have access to any of these, please contact a current maintainer of the project!
Steps
-
Make a PR bumping version number of repo2docker in the
setup.py
file (like https://github.com/jupyter/repo2docker/pull/221), get it merged, and make sure your local checkout is the same asmaster
on GitHub. -
In your environment, install packages needed to make releases:
pip install wheel twine
-
Clean out the
dist
directory and then build thewheel
andtar.gz
files:rm -f dist/* python setup.py sdist bdist_wheel
-
Once tests pass, time to upload!
twine upload dist/*
This might ask for your PyPI username and password.
-
Make a git tag and push it to GitHub:
git tag -a v<version> git push official --tags
-
Tag and push a docker image:
docker build -t jupyter/repo2docker:v<version> . docker push jupyter/repo2docker:v<version>