kopia lustrzana https://github.com/jupyterhub/repo2docker
251 wiersze
8.5 KiB
Markdown
251 wiersze
8.5 KiB
Markdown
# Contributing to repo2docker development
|
|
|
|
This document covers:
|
|
|
|
- Setting up for Local Development
|
|
- Running Tests
|
|
- Updating and Freezing BuildPack Dependencies
|
|
- Merging a Pull Request
|
|
- Creating a Release
|
|
|
|
## Setting up for Local Development
|
|
|
|
To develop & test repo2docker locally, you need:
|
|
|
|
1. Familiarity with using a command line terminal
|
|
2. A computer running macOS / Linux
|
|
3. Some knowledge of git
|
|
4. At least python 3.4
|
|
5. Your favorite text editor
|
|
6. A recent version of [Docker Community Edition](https://www.docker.com/community-edition)
|
|
|
|
### Clone the repository
|
|
|
|
First, you need to get a copy of the repo2docker git repository on your local
|
|
disk.
|
|
|
|
```bash
|
|
git clone https://github.com/jupyter/repo2docker
|
|
```
|
|
|
|
This will clone repo2docker into a directory called `repo2docker`. You can
|
|
make that your current directory with `cd repo2docker`.
|
|
|
|
### Set up a local virtual environment
|
|
|
|
After cloning the repository (or your fork of the repo), you should set up an
|
|
isolated environment to install libraries required for running / developing
|
|
repo2docker. There are many ways to do this, and a `virtual environment` is
|
|
one of them.
|
|
|
|
```bash
|
|
python3 -m venv .
|
|
source bin/activate
|
|
pip3 install -e .
|
|
pip3 install -r dev-requirements.txt
|
|
```
|
|
|
|
This should install all the libraries required for testing & running repo2docker!
|
|
|
|
### Verify that docker is installed and running
|
|
|
|
If you do not already have [Docker](https://www.docker.com/), you should be able
|
|
to download and install it for your operating system using the links from the
|
|
[official website](https://www.docker.com/community-edition). After you have
|
|
installed it, you can verify that it is working by running the following commands:
|
|
|
|
```bash
|
|
docker version
|
|
```
|
|
|
|
It should output something like:
|
|
|
|
```
|
|
Client:
|
|
Version: 17.09.0-ce
|
|
API version: 1.32
|
|
Go version: go1.8.3
|
|
Git commit: afdb6d4
|
|
Built: Tue Sep 26 22:42:45 2017
|
|
OS/Arch: linux/amd64
|
|
|
|
Server:
|
|
Version: 17.09.0-ce
|
|
API version: 1.32 (minimum version 1.12)
|
|
Go version: go1.8.3
|
|
Git commit: afdb6d4
|
|
Built: Tue Sep 26 22:41:24 2017
|
|
OS/Arch: linux/amd64
|
|
Experimental: false
|
|
```
|
|
|
|
Then you are good to go!
|
|
|
|
## Running tests
|
|
|
|
We have a lot of tests for various cases supported by repo2docker in the `tests/`
|
|
subdirectory. If you fix a bug or add new functionality consider adding a new
|
|
test to prevent the bug from coming back. These use
|
|
[py.test](https://docs.pytest.org/).
|
|
|
|
You can run all the tests with:
|
|
|
|
```bash
|
|
py.test -s tests/*
|
|
```
|
|
|
|
If you want to run a specific test, you can do so with:
|
|
|
|
```bash
|
|
py.test -s tests/<path-to-test>
|
|
```
|
|
|
|
## Update and Freeze BuildPack Dependencies
|
|
|
|
### Updating libraries installed for all repos
|
|
|
|
For both the `conda` and `virtualenv` (`pip`) base environments in the **Conda BuildPack** and **Python BuildPack**,
|
|
we install specific pinned versions of all dependencies. We explicitly list the dependencies
|
|
we want, then *freeze* them at commit time to explicitly list all the
|
|
transitive dependencies at current versions. This way, we know that
|
|
all dependencies will have the exact same version installed at all times.
|
|
|
|
To update one of the dependencies shared across all `repo2docker` builds, you
|
|
must follow these steps (with more detailed information in the sections below):
|
|
|
|
* Make sure you have [Docker](https://www.docker.com/) running on your computer
|
|
* Bump the version number in `virtualenv` ([link](https://github.com/jupyter/repo2docker/blob/master/CONTRIBUTING.md#virtualenv-dependencies))
|
|
* Bump the version number in the `conda` environment ([link](https://github.com/jupyter/repo2docker/blob/master/CONTRIBUTING.md#conda-dependencies))
|
|
* Make a pull request with your changes ([link](https://github.com/jupyter/repo2docker/blob/master/CONTRIBUTING.md#make-a-pull-request))
|
|
|
|
See the subsections below for more detailed instructions.
|
|
|
|
### Virtualenv dependencies
|
|
|
|
1. There are two files related to virtualenv dependencies. Edit as needed.
|
|
|
|
- `repo2docker/buildpacks/python/requirements.txt`
|
|
|
|
Contains list of packages to install in Python3 virtualenvs,
|
|
which are the default. **This where all Notebook versions &
|
|
notebook extensions (such as JupyterLab / nteract) go**.
|
|
|
|
- `repo2docker/buildpacks/python/requirements2.txt`
|
|
|
|
Contains list of packages to install in Python2 virtualenvs, which
|
|
can be specifically requested by users. **This only needs `IPyKernel`
|
|
and kernel related libraries** Notebook / Notebook Extension need
|
|
not be installed here.
|
|
|
|
2. After you edit either of these files to add a new package / bump version on
|
|
an existing package, run:
|
|
|
|
```bash
|
|
./repo2docker/buildpacks/python/freeze.bash
|
|
```
|
|
|
|
This script will resolve dependencies and write them to the respective `.frozen.txt`
|
|
files.
|
|
|
|
Note: If you do not have Python3 and Python2 with virtualenv, the script
|
|
will create and build Docker containers to process the frozen files.
|
|
|
|
3. All the `.txt` files in `repo2docker/buildpacks/python/` should be committed to git.
|
|
|
|
4. Make a pull request.
|
|
|
|
### Conda dependencies
|
|
|
|
1. There are two files related to conda dependencies. Edit as needed.
|
|
|
|
- `repo2docker/buildpacks/conda/environment.yml`
|
|
|
|
Contains list of packages to install in Python3 conda environments,
|
|
which are the default. **This is where all Notebook versions &
|
|
notebook extensions (such as JupyterLab / nteract) go**.
|
|
|
|
- `repo2docker/buildpacks/conda/environment.py-2.7.yml`
|
|
|
|
Contains list of packages to install in Python2 conda environments, which
|
|
can be specifically requested by users. **This only needs `IPyKernel`
|
|
and kernel related libraries**. Notebook / Notebook Extension need
|
|
not be installed here.
|
|
|
|
2. Once you edit either of these files to add a new package / bump version on
|
|
an existing package, you should then run:
|
|
|
|
```bash
|
|
python ./repo2docker/buildpacks/conda/freeze.py
|
|
```
|
|
|
|
This script will resolve dependencies and write them to the respective `.frozen.yml`
|
|
files. You will need `docker` installed to run this script.
|
|
|
|
3. After the freeze script finishes, a number of files will have been created.
|
|
Commit the following subset of files to git:
|
|
|
|
```
|
|
repo2docker/buildpacks/conda/environment.yml
|
|
repo2docker/buildpacks/conda/environment.frozen.yml
|
|
repo2docker/buildpacks/conda/environment.py-2.7.yml
|
|
repo2docker/buildpacks/conda/environment.py-2.7.frozen.yml
|
|
repo2docker/buildpacks/conda/environment.py-3.5.frozen.yml
|
|
repo2docker/buildpacks/conda/environment.py-3.6.frozen.yml
|
|
```
|
|
|
|
4. Make a pull request.
|
|
|
|
### Make a Pull Request
|
|
|
|
Once you've made the commit, please make a Pull Request to the `jupyter/repo2docker`
|
|
repository, with a description of what versions were bumped / what new packages were
|
|
added and why. If you fix a bug or add new functionality consider adding a new
|
|
test to prevent the bug from coming back/the feature breaking in the future.
|
|
|
|
|
|
## Merging a Pull Request
|
|
|
|
There are not a lot of rules around merging a Pull Request (PR), we rely on
|
|
individuals to be responsible and tread softly when doing so. Below a few
|
|
standard procedures that have proven useful over time that we do follow:
|
|
|
|
* do not merge your own PR
|
|
* wait for travis to complete
|
|
* check if test coverage has gone up or down, consider discussing additional
|
|
tests to keep coverage at the same level or even increase it
|
|
* do use merge commits instead of merge-by-squashing/-rebasing. This makes it
|
|
easier to find all changes since the last deployment `git log --merges --pretty=format:"%h %<(10,trunc)%an %<(15)%ar %s" <deployed-revision>..`
|
|
* [when you merge do deploy to mybinder.org](http://mybinder-sre.readthedocs.io/en/latest/deployment/how.html)
|
|
|
|
|
|
|
|
## Creating a Release
|
|
|
|
We try to make a release of repo2docker every few months if possible.
|
|
|
|
## Obtain access credentials
|
|
|
|
To release repo2docker, you will need proper access credentials prior to beginning the process.
|
|
|
|
1. Access to the PyPI package for repo2docker
|
|
2. Access to push tags to the jupyter/repo2docker repository
|
|
3. Acess to push images to dockerhub on jupyter/repo2docker
|
|
|
|
If you do not have access to any of these, please contact a current maintainer of the project!
|
|
|
|
## Release Process Steps
|
|
|
|
1. Make a PR bumping version number of repo2docker in the
|
|
`setup.py` file and `reop2docker/__init__.py` ,
|
|
get it merged, and make sure your local checkout is the
|
|
same as `master` on GitHub.
|
|
|
|
2. Make a new release on GitHub. When the tag is create travis will build
|
|
and deploy that tag as the latest release.
|
|
|
|
3. Tag and push a docker image:
|
|
```bash
|
|
docker build -t jupyter/repo2docker:v<version> .
|
|
docker push jupyter/repo2docker:v<version>
|
|
```
|