Merge remote-tracking branch 'upstream/master'

pull/413/head
Derek Ludwig 2018-10-15 10:21:41 -07:00
commit 2d2738b6db
58 zmienionych plików z 1149 dodań i 377 usunięć

3
.coveragerc 100644
Wyświetl plik

@ -0,0 +1,3 @@
[run]
# this file comes from versioneer and we don't test it
omit = */_version.py

Wyświetl plik

@ -17,18 +17,18 @@ script:
# possible issues with MANIFEST.in
- pushd tests;
if [ ${REPO_TYPE} == "r" ]; then
travis_wait pytest --cov repo2docker -v ${REPO_TYPE};
travis_wait pytest --cov repo2docker -v ${REPO_TYPE} || exit 1;
else
travis_retry pytest --cov repo2docker -v ${REPO_TYPE};
travis_retry pytest --cov repo2docker -v ${REPO_TYPE} || exit 1;
fi;
popd;
- pip install -r docs/doc-requirements.txt
- pushd docs;
make html;
make html || exit 1;
popd;
after_success:
- pip install codecov
- codecov
- pushd tests && codecov && popd
jobs:
include:
- stage: deploy

56
CHANGES.rst 100644
Wyświetl plik

@ -0,0 +1,56 @@
Version x.y.z (unreleased)
==========================
Release date: unknown
New features
------------
- Editable mode: allows editing a local repository from a live container [#421]
- Change log added [#426]
API changes
-----------
Bug fixes
---------
Version 0.6
===========
Released 2018-09-09
Version 0.5
===========
Released 2018-02-07
Version 0.4.1
=============
Released 2018-09-06
Version 0.2
===========
Released 2018-05-25
Version 0.1.1
=============
Released 2017-04-19
Version 0.1
===========
Released 2017-04-14

Wyświetl plik

@ -2,12 +2,44 @@
This document covers:
- Process for making a code contribution
- Setting up for Local Development
- Running Tests
- Updating and Freezing BuildPack Dependencies
- Merging a Pull Request
- Updating the change log
- Creating a Release
## Process for making a code contribution
This outlines the process for getting changes to the code of
repo2docker merged. This serves as information on when a PR is "done".
Contributions should follow these guidelines:
* all changes by pull request (PR);
* please prefix the title of your pull request with `[MRG]` if the contribution
is complete and should be subjected to a detailed review;
* create a PR as early as possible, marking it with `[WIP]` while you work on
it (good to avoid duplicated work, get broad review of functionality or API,
or seek collaborators);
* a PR solves one problem (do not mix problems together in one PR) with the
minimal set of changes;
* describe why you are proposing the changes you are proposing;
* try to not rush changes (the definition of rush depends on how big your
changes are);
* someone else has to merge your PR;
* new code needs to come with a test;
* apply [PEP8](https://www.python.org/dev/peps/pep-0008/) as much
as possible, but not too much;
* no merging if travis is red;
* do use merge commits instead of merge-by-squashing/-rebasing. This makes it
easier to find all changes since the last deployment `git log --merges --pretty=format:"%h %<(10,trunc)%an %<(15)%ar %s" <deployed-revision>..`
* [when you merge do deploy to mybinder.org](http://mybinder-sre.readthedocs.io/en/latest/deployment/how.html)
These are not hard rules to be enforced by :police_car: but instead guidelines.
## Setting up for Local Development
To develop & test repo2docker locally, you need:
@ -43,6 +75,7 @@ python3 -m venv .
source bin/activate
pip3 install -e .
pip3 install -r dev-requirements.txt
pip3 install -r docs/doc-requirements.txt
```
This should install all the libraries required for testing & running repo2docker!
@ -81,6 +114,7 @@ Server:
Then you are good to go!
## Running tests
We have a lot of tests for various cases supported by repo2docker in the `tests/`
@ -159,53 +193,67 @@ See the subsections below for more detailed instructions.
repo2docker/buildpacks/conda/environment.py-3.6.frozen.yml
```
4. Make a pull request.
5. Make a pull request; see details below.
6. Once the pull request is approved (but not yet merged), Update the
change log (details below) and commit the change log, then update
the pull request.
### Change log
To add your change to the change log, find the relevant Feature/Bug
fix/API change section for the next release near the top of the file;
then add one or two sentences as a new bullet point about your
changes. Include the pull request or issue number between square
brackets at the end.
Some details:
- versioning follows the x.y.z, major.minor.bugfix numbering
- bug fixes go into the next bugfix release. If there isn't any, you
can create a new section (see point below). Don't worry if you're
not sure about that, and think it should go into a next major or
minor release: an admin will let you know, or move the change later
to the appropriate section
- API changes should preferably go into the next major release, unless
they are backward compatible (for example, a deprecated function
keyword): then they can go into the next minor release. For release
with major release 0, non-backward compatible breaking changes are
also fine for the next minor release.
- new features should go into the next minor release.
- if there is no section for the appropriate release, you can add one:
follow the versioning scheme, by simply increasing the relevant
number for one of the major /minor/bugfix numbers, appropriate for
your change (see the above bullet points); add the release
section. Then add three subsections: new features, api changes, and
bug fixes. Leave out the sections that are not appropriate for the
newlye added release section.
Release candidate versions in the change log are only temporary, and
should be superseded by either a next release candidate, or the final
release for that version (bugfix version 0).
### Make a Pull Request
Once you've made the commit, please make a Pull Request to the `jupyter/repo2docker`
Once you've made the commit, please make a Pull Request to the `jupyterhub/repo2docker`
repository, with a description of what versions were bumped / what new packages were
added and why. If you fix a bug or add new functionality consider adding a new
test to prevent the bug from coming back/the feature breaking in the future.
## Merging a Pull Request
There are not a lot of rules around merging a Pull Request (PR), we rely on
individuals to be responsible and tread softly when doing so. Below a few
standard procedures that have proven useful over time that we do follow:
* do not merge your own PR
* wait for Travis to complete
* check if test coverage has gone up or down, consider discussing additional
tests to keep coverage at the same level or even increase it
* do use merge commits instead of merge-by-squashing/-rebasing. This makes it
easier to find all changes since the last deployment `git log --merges --pretty=format:"%h %<(10,trunc)%an %<(15)%ar %s" <deployed-revision>..`
* [when you merge do deploy to mybinder.org](http://mybinder-sre.readthedocs.io/en/latest/deployment/how.html)
## Creating a Release
We try to make a release of repo2docker every few months if possible.
## Obtain access credentials
We follow semantic versioning.
To release repo2docker, you will need proper access credentials prior to beginning the process.
Check hat the Change log is ready and then tag a new release on GitHub.
1. Access to the PyPI package for repo2docker
2. Access to push tags to the jupyter/repo2docker repository
3. Access to push images to dockerhub on jupyter/repo2docker
If you do not have access to any of these, please contact a current maintainer of the project!
## Release Process Steps
1. Make a new release on GitHub. When the tag is create travis will build
and deploy that tag as the latest release.
2. Tag and push a docker image:
```bash
docker build -t jupyter/repo2docker:v<version> .
docker push jupyter/repo2docker:v<version>
```
When the travis run completes check that the new release is available on PyPI.

Wyświetl plik

@ -1,15 +1,19 @@
# jupyter-repo2docker
# repo2docker
[![Build Status](https://travis-ci.org/jupyter/repo2docker.svg?branch=master)](https://travis-ci.org/jupyter/repo2docker)
[![Documentation Status](https://readthedocs.org/projects/repo2docker/badge/?version=latest)](http://repo2docker.readthedocs.io/en/latest/?badge=latest)
**jupyter-repo2docker** takes as input a repository source, such as a GitHub
repository. It then builds, runs, and/or pushes Docker images built from that source.
`repo2docker` fetches a git repository and builds a container image based on
the configuration files found in the repository.
See the [repo2docker documentation](http://repo2docker.readthedocs.io)
for more information.
## Prerequisites
See the [contributing guide](CONTRIBUTING.md) for information on contributing to
repo2docker.
## Using repo2docker
### Prerequisites
1. Docker to build & run the repositories. The [community edition](https://store.docker.com/search?type=edition&offering=community)
is recommended.
@ -17,9 +21,11 @@ for more information.
Supported on Linux and macOS. [See documentation note about Windows support.](http://repo2docker.readthedocs.io/en/latest/install.html#note-about-windows-support)
## Installation
### Installation
To install from PyPI, the python packaging index, using `pip`:
This a quick guide to installing `repo2docker`, see our documentation for [a full guide](https://repo2docker.readthedocs.io/en/latest/install.html).
To install from PyPI:
```bash
pip install jupyter-repo2docker
@ -33,11 +39,11 @@ cd repo2docker
pip install -e .
```
## Usage
### Usage
The core feature of repo2docker is to fetch a repository (from GitHub or locally),
The core feature of repo2docker is to fetch a git repository (from GitHub or locally),
build a container image based on the specifications found in the repository &
optionally launch a local Jupyter Notebook you can use to explore it.
optionally launch the container that you can use to explore the repository.
**Note that Docker needs to be running on your machine for this to work.**
@ -62,11 +68,13 @@ with the contents of the repository you had just built!
For more information on how to use ``repo2docker``, see the
[usage guide](http://repo2docker.readthedocs.io/en/latest/usage.html).
## Repository specifications
Repo2Docker looks for configuration files in the source repository to
determine how the Docker image should be built. It is philosophically similar
to [Heroku Build Packs](https://devcenter.heroku.com/articles/buildpacks).
determine how the Docker image should be built. For a list of the configuration
files that ``repo2docker`` can use, see the
[complete list of configuration files](https://repo2docker.readthedocs.io/en/latest/config_files.html).
For a list of the configuration files that ``repo2docker`` can use,
see the [usage guide](http://repo2docker.readthedocs.io/en/latest/usage.html).
The philosophy of repo2docker is inspired by
[Heroku Build Packs](https://devcenter.heroku.com/articles/buildpacks).

Wyświetl plik

@ -1,3 +1,11 @@
img.logo {
width:100%
}
div.body p.caption {
font-size: 1.5em;
}
h2 a.toc-backref {
text-decoration: none;
}

Wyświetl plik

@ -44,7 +44,7 @@ source_parsers = {
}
def setup(app):
app.add_stylesheet('custom.css') # may also be an URL
app.add_stylesheet('custom.css') # may also be a URL
# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
@ -187,39 +187,3 @@ epub_copyright = copyright
# A list of files that should not be packed into the epub file.
epub_exclude_files = ['search.html']
# Build the repo2docker test syntax
import os
s = ''
FILE_ORDER_PREFERENCE = ['System', 'Python', 'Julia', 'Docker', 'Misc']
SKIP_FOLDERS = ['binder-dir']
FILES_DICT = {ii: [] for ii in FILE_ORDER_PREFERENCE}
for root, _, files in os.walk(os.path.join('..', '..', 'tests')):
depth = len(root.split('/')) - 3
if 'README.rst' not in files or depth > 2:
continue
if any(folder in root for folder in SKIP_FOLDERS):
continue
header = files.pop(files.index('README.rst'))
with open(os.path.join(root, header), 'r') as ff:
this_s = ff.read() + '\n\n'
title = this_s.split('\n')[0]
for ifile in files:
filename = os.path.basename(ifile)
if filename == 'verify':
continue
with open(os.path.join(root, ifile), 'r') as ff:
lines = ff.readlines()
lines = [' ' + line for line in lines]
this_s += 'File: ``{}``\n{}\n\n**Contents**::\n\n'.format(
filename, '~' * (len(filename) + 10))
this_s += '\n'.join(lines)
this_s += '\n\n'
usename = [ii for ii in FILE_ORDER_PREFERENCE if ii in title]
usename = 'Misc' if len(usename) == 0 else usename[0]
FILES_DICT[usename].append(this_s)
for key in FILE_ORDER_PREFERENCE:
for istring in FILES_DICT[key]:
s += istring
with open('./test_file_text.txt', 'w') as ff:
ff.write(s)

Wyświetl plik

@ -1,9 +1,8 @@
.. NOTE: the header characters are different in this file because it is 'included' in another site
.. see https://raw.githubusercontent.com/jupyterhub/binder/master/doc/using.rst
.. _config-files:
===================
Configuration Files
-------------------
===================
``repo2docker`` looks for configuration files in the repository being built
to determine how to build it. In general, ``repo2docker`` uses the same
@ -13,16 +12,8 @@ rather than creating new custom configuration files.
A number of ``repo2docker`` configuration files can be combined to compose more
complex setups.
``repo2docker`` will look for configuration files in either:
* A folder named ``binder/`` in the root of the repository.
* The root directory of the repository.
If the folder ``binder/`` is located at the top level of the repository,
**only configuration files in the** ``binder/`` **folder will be considered**.
The `binder examples <https://github.com/binder-examples>`_ organization on
GitHub contains a list of sample repositories for common configurations
GitHub contains a list of sample repositories for common configurations
that ``repo2docker`` can build with various configuration files such as
Python and R installation in a repository.
@ -32,64 +23,71 @@ Below is a list of supported configuration files (roughly in the order of build
:local:
:depth: 1
``Dockerfile``
~~~~~~~~~~~~~~
In the majority of cases, providing your own Dockerfile is not necessary as the base
images provide core functionality, compact image sizes, and efficient builds. We recommend
trying the other configuration files before deciding to use your own Dockerfile.
With Dockerfiles, a regular Docker build will be performed.
**If a Dockerfile is present, all other configuration files will be ignored.**
See the `Binder Documentation <https://mybinder.readthedocs.io/en/latest/dockerfile.html>`_ for
best-practices with Dockerfiles.
.. _environment-yml:
``environment.yml``
~~~~~~~~~~~~~~~~~~~
``environment.yml`` - Install a Python environment
==================================================
``environment.yml`` is the standard configuration file used by Anaconda, conda,
and miniconda that lets you install Python packages.
You can also install files from pip in your ``environment.yml`` as well.
Our example `environment.yml <https://github.com/binder-examples/python-conda_pip/blob/master/environment.yml>`_
shows how one can specify a conda environment for repo2docker.
and miniconda that lets you install packages in the data analytics stack (it
primarily installs Python packages, though can be used to install a range of
non-Python packages as well).
.. note::
You can install files from pip in your ``environment.yml`` as well.
For example, see the `binder-examples environment.yml
<https://github.com/binder-examples/python-conda_pip/blob/master/environment.yml>`_
file.
You can also specify which Python version to install in your built environment
with ``environment.yml``. By default, ``repo2docker`` **installs
Python 3.6** with your ``environment.yml`` unless you include the version of
Python in the file. ``conda`` supports Python versions 3.6, 3.5, 3.4, and 2.7.
``repo2docker`` support is best with Python 3.6, 3.5, and 2.7. If you include
a Python version in a ``runtime.txt`` file in addition to your
``environment.yml``, your ``runtime.txt`` **will be ignored**.
``repo2docker`` support is best with Python 3.6, 3.5, and 2.7.
``requirements.txt``
~~~~~~~~~~~~~~~~~~~~
.. warning::
If you include a Python version in a ``runtime.txt`` file in addition to your
``environment.yml``, your ``runtime.txt`` will be ignored.
``requirements.txt`` - Install a Python environment
===================================================
This specifies a list of Python packages that should be installed in your
environment. Our
`requirements.txt example <https://github.com/binder-examples/requirements/blob/master/requirements.txt>`_
on GitHub shows a typical requirements file.
``REQUIRE``
~~~~~~~~~~~
``setup.py`` - Install Python packages
======================================
To install your repository like a Python package, you may include a
``setup.py`` file. repo2docker installs ``setup.py`` files by running
``pip install -e .``.
``REQUIRE`` - Install a Julia environment
=========================================
This specifies a list of Julia packages. Repositories with a ``REQUIRE`` file
**must also contain an** ``environment.yml`` **file**. To see an example of a
Julia repository with ``REQUIRE`` and ``environment.yml``,
visit `binder-examples/julia-python <https://github.com/binder-examples/julia-python>`_.
``install.R``
~~~~~~~~~~~~~
``install.R`` - Install an R/RStudio environment
================================================
This is used to install R libraries pinned to a specific snapshot on
`MRAN <https://mran.microsoft.com/documents/rro/reproducibility>`_.
To set the date of the snapshot add a runtime.txt_.
For an example ``install.R`` file, visit our `example install.R file <https://github.com/binder-examples/r/blob/master/install.R>`_.
``apt.txt``
~~~~~~~~~~~
``apt.txt`` - Install packages with apt-get
===========================================
A list of Debian packages that should be installed. The base image used is usually the latest released
version of Ubuntu.
@ -98,33 +96,34 @@ We use ``apt.txt``, for example, to install LaTeX in our
`example apt.txt for LaTeX <https://github.com/binder-examples/latex/blob/master/apt.txt>`_.
``setup.py``
~~~~~~~~~~~~
``DESCRIPTION`` - Install an R package
======================================
To install your repository like a Python package, you may include a
``setup.py`` file. repo2docker installs ``setup.py`` files by running
``pip install -e .``.
To install your repository like an R package, you may include a
``DESCRIPTION`` file. repo2docker installs the package and dependencies
from the ``DESCRIPTION`` by running ``devtools:install_git(".")``.
While one can specify dependencies in ``setup.py``,
repo2docker **requires configuration files such as** ``environment.yml`` or
``requirements.txt`` to install dependencies during the build process.
You also need to have a ``runtime.txt`` file that is formatted as
``r-<YYYY>-<MM>-<DD>``, where YYYY-MM-DD is a snapshot of MRAN that will be
used for your R installation.
.. _postBuild:
``postBuild``
~~~~~~~~~~~~~
``postBuild`` - Run code after installing the environment
=========================================================
A script that can contain arbitrary commands to be run after the whole repository has been built. If you
want this to be a shell script, make sure the first line is ```#!/bin/bash``.
want this to be a shell script, make sure the first line is ``#!/bin/bash``.
An example use-case of ``postBuild`` file is JupyterLab's demo on mybinder.org.
It uses a ``postBuild`` file in a folder called ``binder`` to `prepare
their demo for binder <https://github.com/jupyterlab/jupyterlab-demo/blob/master/binder/postBuild>`_.
.. _start:
``start``
^^^^^^^^^
``start`` - Run code before the user sessions starts
====================================================
A script that can contain simple commands to be run at runtime (as an
`ENTRYPOINT <https://docs.docker.com/engine/reference/builder/#entrypoint>`
@ -138,21 +137,22 @@ should at most take a few seconds to run.
If you only need to run things once during the build phase use :ref:`postBuild`.
.. TODO: Discuss runtime limits, best practices, etc.
Also, point to an example.
.. _runtime.txt:
``runtime.txt``
~~~~~~~~~~~~~~~
``runtime.txt`` - Specifying runtimes
=====================================
This allows you to control the runtime of Python or R.
To use python-2.7: add python-2.7 in runtime.txt file.
To use python-2.7: add ``python-2.7`` in runtime.txt file.
The repository will run in a virtualenv with
Python 2 installed. To see a full example repository, visit our
`Python2 example <https://github.com/binder-examples/python2_runtime/blob/master/runtime.txt>`_.
**Python versions in ``runtime.txt`` are ignored when** ``environment.yml`` **is
**Python versions in** ``runtime.txt`` **are ignored when** ``environment.yml`` **is
present in the same folder**.
repo2docker uses R libraries pinned to a specific snapshot on
@ -163,3 +163,19 @@ used for installing libraries.
To see an example R repository, visit our `R
example in binder-examples <https://github.com/binder-examples/r/blob/master/runtime.txt>`_.
``Dockerfile`` - Advanced environments
======================================
In the majority of cases, providing your own Dockerfile is not necessary as the base
images provide core functionality, compact image sizes, and efficient builds. We recommend
trying the other configuration files before deciding to use your own Dockerfile.
With Dockerfiles, a regular Docker build will be performed.
.. note::
If a Dockerfile is present, all other configuration files will be ignored.
See the `Advanced Binder Documentation <https://mybinder.readthedocs.io/en/latest/tutorials/dockerfile.html>`_ for
best-practices with Dockerfiles.

Wyświetl plik

@ -1,37 +0,0 @@
# Frequently Asked Questions (FAQ)
A collection of frequently asked questions with answers. If you have a question
and have found an answer, send a PR to add it here!
## How should I specify another version of Python 3?
One can specify a Python version in the ``environment.yml`` file of a repository.
## Can I add executable files to the user's PATH?
Yes! With a :ref:`postBuild` file, you can place any files that should be called
from the command line in the folder ``~/.local/``. This folder will be
available in a user's PATH, and can be run from the command line (or as
a subsequent build step.)
## How do I set environment variables?
Use the `-e` or `--env` flag for each variable that you want to define.
For example `jupyter-repo2docker -e VAR1=val1 -e VAR2=val2 ...`
## Can I use repo2docker to bootstrap my own Dockerfile?
No, you can't.
If you pass the `--debug` flag to `repo2docker`, it outputs the intermediate
Dockerfile that is used to build the docker image. While it is tempting to copy
this as a base for your own Dockerfile, that is not supported & in most cases
will not work. The `--debug` output is just our intermediate generated
Dockerfile, and is meant to be built in
[a very specific way](https://github.com/jupyter/repo2docker/blob/master/repo2docker/detectors.py#L381).
Hence the output of `--debug` can not be built with a normal `docker build -t .`
or similar traditional docker command.
Check out the [binder-examples](http://github.com/binder-examples/) GitHub
organization for example repositories you can copy & modify for your own use!

115
docs/source/faq.rst 100644
Wyświetl plik

@ -0,0 +1,115 @@
.. _faq:
Frequently Asked Questions (FAQ)
================================
A collection of frequently asked questions with answers. If you have a question
and have found an answer, send a PR to add it here!
How should I specify another version of Python?
-----------------------------------------------
One can specify a Python version in the ``environment.yml`` file of a repository.
What versions of Python (or R or Julia...) are supported?
---------------------------------------------------------
Python
~~~~~~
Repo2docker officially supports the following versions of Python (specified in environment.yml or runtime.txt):
- 3.7 (added in 0.7)
- 3.6 (default)
- 3.5
Additional versions may work, as long as the
`base environment <https://github.com/jupyter/repo2docker/blob/master/repo2docker/buildpacks/conda/environment.yml>`_
can be installed for your version of Python.
The most likely source of incompatibility is if one of the packages
in the base environment is not packaged for your Python,
either because the version of the package is too new and your chosen Python is too old,
or vice versa.
Additionally, if Python 2.7 is specified,
a separate environment for the kernel will be installed with Python 2.
The notebook server will run in the default Python 3.6 environment.
Julia
~~~~~
The following versions of Julia are supported (specified in REQUIRE):
- 1.0 (added in 0.7)
- 0.7 (added in 0.7)
- 0.6 (default)
R
~
Only R 3.4.4 is currently supported, which is installed via ``apt`` from the
`ubuntu bionic repository <https://packages.ubuntu.com/bionic/r-base>`_.
Can I add executable files to the user's PATH?
----------------------------------------------
Yes! With a :ref:`postBuild` file, you can place any files that should be called
from the command line in the folder ``~/.local/``. This folder will be
available in a user's PATH, and can be run from the command line (or as
a subsequent build step.)
How do I set environment variables?
-----------------------------------
Use the ``-e`` or ``--env`` flag for each variable that you want to define.
For example ``jupyter-repo2docker -e VAR1=val1 -e VAR2=val2 ...``
Can I use repo2docker to bootstrap my own Dockerfile?
-----------------------------------------------------
No, you can't.
If you pass the ``--debug`` flag to ``repo2docker``, it outputs the
intermediate Dockerfile that is used to build the docker image. While
it is tempting to copy this as a base for your own Dockerfile, that is
not supported & in most cases will not work. The ``--debug`` output is
just our intermediate generated Dockerfile, and is meant to be built
in a very specific way. Hence the output of ``--debug`` can not be
built with a normal ``docker build -t .`` or similar traditional
docker command.
Check out the `binder-examples <http://github.com/binder-examples/>`_ GitHub
organization for example repositories you can copy & modify for your own use!
Can I use repo2docker to edit a local host repository within a Docker environment?
----------------------------------------------------------------------------------
Yes: use the ``--editable`` or ``-E`` flag (don't confuse this with
the ``-e`` flag for environment variables), and run repo2docker on a
local repository::
repo2docker -E my-repository/.
This builds a Docker container from the files in that repository
(using, for example, a ``requirements.txt`` or ``install.R`` file),
then runs that container, while connecting the working directory
inside the container to the local repository outside the
container. For example, in case there is a notebook file (``.ipynb``),
this will open in a local webbrowser, and one can edit it and save
it. The resulting notebook is updated in both the Docker container and
the local repository. Once the container is exited, the changed file
will still be in the local repository.
This allows for easy testing of the container while debugging some
items, as well as using a fully customizable container to edit
notebooks (among others).
.. note::
Editable mode is a convenience option that will bind the
repository to the container working directory (usually
``$HOME``). If you need to mount to a different location in
the container, use the ``--volumes`` option instead. Similarly,
for a fully customized user Dockerfile, this option is not
guaranteed to work.

Wyświetl plik

@ -0,0 +1,25 @@
.. _jupyterhub_images:
=============================
Build JupyterHub-ready images
=============================
JupyterHub_ allows multiple
users to collaborate on a shared Jupyter server. ``repo2docker`` can build
Docker images that can be shared within a JupyterHub deployment. For example,
`mybinder.org <https://mybinder.org>`_ uses JupyterHub and ``repo2docker``
to allow anyone to build a Docker image of a git repository online and
share an executable version of the repository with a URL to the built image.
To build JupyterHub_-ready Docker images with ``repo2docker``, the
version of your JupterHub deployment must be included in the
``environment.yml`` or ``requirements.txt`` of the git repositories you
build.
If your instance of JupyterHub uses ``DockerSpawner``, you will need to set its
command to run ``jupyterhub-singleuser`` by adding this line in your
configuration file::
c.DockerSpawner.cmd = ['jupyterhub-singleuser']
.. _JupyterHub: https://github.com/jupyterhub/jupyterhub

Wyświetl plik

@ -0,0 +1,84 @@
.. _languages:
=====================================
Choose languages for your environment
=====================================
You can define many different languages in your configuration files. This
page describes how to use some of the more common ones.
Python
======
Your environment will have Python (and specified dependencies) installed when
you use one of the following configuration files:
* ``requirements.txt``
* ``environment.yml``
Note that by default, the environment will have **Python 3** installed.
Specifying a version of Python
------------------------------
To specify a specific version of Python, you have two options:
* Use ``runtime.txt``. Include a line that specifies the Python version in
this file. This line takes the following form::
python=X.X
For example,::
python=2.7
* Use ``environment.yml``. The Anaconda distribution also lets you define
the Python environment within ``environment.yml``. To do so, add ``python=X.X``
to your dependencies section, like so::
name: python 2.7
dependencies:
- python=2.7
- numpy
The R Language
==============
To ensure that R is installed, you must specify a version of R in a ``runtime.txt``
file. This takes the following form::
r-YYYY-MM-DD
The date corresponds to the state of the MRAN repository at this day. Make sure
that you choose a day with the desired version of your packages. For example,
to use the MRAN repository on January 1st, 2018, add this line to ``runtime.txt``::
r-2018-01-01
Note that to install specific packages with the R environment, you should
use the ``install.R`` configuration file.
Julia
=====
To build an environment with Julia, include a configuration file called
``REQUIRE``. Each line of this file should include a package that you wish
to have installed with Julia. For example, the following contents of ``REQURE``
would install the ``PyPlot`` package with your Julia environment.::
PyPlot
Languages not covered here
==========================
If a language is not "officially" supported by a build pack, it can often be
installed with a ``postBuild`` script. This will run arbitrary ``bash`` commands,
and can be used to download / install a language.
Using multiple languages at once
================================
It may also be possible to combine multiple languages in a single environment.
The details on how to accomplish this with all possible combinations are outside
the scope of this guide. However we recommend that you take a look at the
`Multi-Language Demo <https://github.com/binder-examples/multi-language-demo>`_
repository for some inspiration.

Wyświetl plik

@ -0,0 +1,40 @@
.. _user_interface:
============================
Configure the user interface
============================
You can build several user interfaces into the resulting Docker image.
This is controlled with various :ref:`configuration files <config-files>`.
JupyterLab
----------
You do not need any extra configuration in order to allow the use
of the JupyterLab interface. You can launch JupyterLab from within a user
session by opening the Jupyter Notebook and appending ``/lab`` to the end of the URL
like so:
.. code-block:: none
http(s)://<server:port>/lab
To switch back to the classic notebook, add ``/tree`` to the URL like so:
.. code-block:: none
http(s)://<server:port>/tree
To learn more about URLs in JupyterLab and Jupyter Notebook, visit
`starting JupyterLab <http://jupyterlab.readthedocs.io/en/latest/getting_started/starting.html>`_.
RStudio
-------
The RStudio user interface is automatically enabled a configuration file for
R is detected (an R version specified in ``runtime.txt``). If this is detected,
RStudio will be accessible by appending ``/rstudio`` to the URL, like so:
.. code-block:: none
http(s)://<server:port>/rstudio

Wyświetl plik

@ -1,26 +1,45 @@
jupyter-repo2docker
===================
**jupyter-repo2docker** is a tool to build, run, and push Docker
images from source code repositories. repo2docker fetches a repository
``jupyter-repo2docker`` is a tool to **build, run, and push Docker
images from source code repositories** that run via a Jupyter server.
``repo2docker`` fetches a repository
(e.g., from GitHub or other locations) and builds a container image
based on the configuration files found in the repository. It can be
used to explore a repository locally by building and executing the
constructed image of the repository.
constructed image of the repository, or as a means of building images that
are pushed to a Docker registry.
Please report `Bugs <https://github.com/jupyter/repo2docker/issues>`_,
`ask questions <https://gitter.im/jupyterhub/binder>`_ or
`contribute to the project <https://github.com/jupyter/repo2docker/blob/master/CONTRIBUTING.md>`_.
Site Contents
-------------
.. toctree::
:maxdepth: 1
:maxdepth: 2
:caption: Getting started with repo2docker
install
usage
.. toctree::
:maxdepth: 1
:caption: How-to...
howto/user_interface
howto/languages
howto/jupyterhub_images
.. toctree::
:maxdepth: 2
:caption: Complete list of configuration files
config_files
.. toctree::
:maxdepth: 1
:caption: Advanced and developer information
faq
deploy
design

Wyświetl plik

@ -6,7 +6,7 @@ Installing ``repo2docker``
repo2docker requires Python 3.4 and above on Linux and macOS. See
:ref:`below <windows>` for more information about Windows support.
Prerequisite: docker
Prerequisite: Docker
--------------------
Install `Docker <https://www.docker.com>`_ as it is required
@ -56,28 +56,3 @@ Bash on Windows) provides additional information about Windows and docker.
.. _using Windows and the WSL: https://nickjanetakis.com/blog/setting-up-docker-for-windows-and-wsl-to-work-flawlessly
.. _jupyterhub_ready:
JupyterHub-ready images
-----------------------
JupyterHub_ allows multiple
users to collaborate on a shared Jupyter server. ``repo2docker`` can build
Docker images that can be shared within a JupyterHub deployment. For example,
`mybinder.org <https://mybinder.org>`_ uses JupyterHub and ``repo2docker``
to allow anyone to build a Docker image of a git repository online and
share an executable version of the repository with a URL to the built image.
To build JupyterHub_-ready Docker images with ``repo2docker``, the
version of your JupterHub deployment must be included in the
``environment.yml`` or ``requirements.txt`` of the git repositories you
build.
If your instance of JupyterHub uses ``DockerSpawner``, you will need to set its
command to run ``jupyterhub-singleuser`` by adding this line in your
configuration file::
c.DockerSpawner.cmd = ['jupyterhub-singleuser']
.. _JupyterHub: https://github.com/jupyterhub/jupyterhub

Wyświetl plik

@ -1,60 +1,50 @@
.. _usage:
=====================
Using ``repo2docker``
=====================
`Docker <https://docs.docker.com/>`_ **must be running** in
order to run ``repo2docker``. For more information on installing
``repo2docker``, see :ref:`install`.
.. note::
``repo2docker`` performs two steps:
`Docker <https://docs.docker.com/>`_ **must be running** in
order to run ``repo2docker``. For more information on installing
``repo2docker``, see :ref:`install`.
1. builds a Docker image from a git repo
2. runs a Jupyter server within the image to explore the repository
``repo2docker`` is called with a URL/path to a git repository. It then
performs these steps:
.. To ensure you can run the software in your repository, you must
1. Inspects the repository for :ref:`configuration files <config-files>`. These will be used to build
the environment needed to run the repository.
2. Builds a Docker image with an environment specified in these :ref:`configuration files <config-files>`.
3. Runs a Jupyter server within the image that lets you explore the
repository interactively (optional)
4. Pushes the images to a Docker registry so that it may be accessed remotely
(optional)
Calling repo2docker
===================
repo2docker is called with this command::
jupyter-repo2docker <URL-or-path to repository>
where ``<URL-or-path to repository>`` is a URL or path to the source repository.
where ``<URL-or-path to repository>`` is a URL or path to the source repository
for which you'd like to build an image.
For example, use the following to build an image of Peter Norvig's
Pytudes_::
For example, the following command will build an image of Peter Norvig's
Pytudes_ repository::
jupyter-repo2docker https://github.com/norvig/pytudes
To build a particular branch and commit, use the argument ``--ref`` to
specify the ``branch-name`` or ``commit-hash``::
jupyter-repo2docker https://github.com/norvig/pytudes --ref 9ced85dd9a84859d0767369e58f33912a214a3cf
.. tip::
For reproducible research, we recommend specifying a commit-hash to
deterministically build a fixed version of a repository. Not specifying a
commit-hash will result in the latest commit of the repository being built.
Building the image may take a few minutes.
During building, ``repo2docker``
clones the repository to obtain its contents and inspects the repository for
:ref:`configuration files <config-files>`.
By default, ``repo2docker`` will assume you are using
Python 3.6 unless you include the version of Python in your
:ref:`configuration files <config-files>`. ``repo2docker`` support is best with
Python 2.7, 3.5, and 3.6. In the case of this repository, a Python version is not
specified in their configuration files and Python 3.6 is installed.
Pytudes_
uses a `requirements.txt file <https://github.com/norvig/pytudes/blob/master/requirements.txt>`_
to specify its Python environment. ``repo2docker`` uses ``pip`` to install
dependencies listed in the ``requirement.txt`` in the image. To learn more about
to specify its Python environment. Because of this, ``repo2docker`` will use
``pip`` to install dependencies listed in this ``requirement.txt`` file, and
these will be present in the generated Docker image. To learn more about
configuration files in ``repo2docker`` visit :ref:`config-files`.
.. _Pytudes: https://github.com/norvig/pytudes
When the image is built, a message will be output to your terminal::
Copy/paste this URL into your browser when you connect for the first time,
@ -64,25 +54,41 @@ When the image is built, a message will be output to your terminal::
Pasting the URL into your browser will open Jupyter Notebook with the
dependencies and contents of the source repository in the built image.
Because JupyterLab is a server extension of the classic Jupyter Notebook server,
you can launch JupyterLab by opening Jupyter Notebook and visiting the
```/lab`` to the end of the URL:
.. code-block:: none
Building a specific branch / commit / tag
=========================================
http(s)://<server:port>/<lab-location>/lab
To build a particular branch and commit, use the argument ``--ref`` and
specify the ``branch-name`` or ``commit-hash``. For example::
To switch back to the classic notebook, add ``/tree`` to the URL:
jupyter-repo2docker --ref 9ced85dd9a84859d0767369e58f33912a214a3cf https://github.com/norvig/pytudes
.. code-block:: none
.. tip::
For reproducible research, we recommend specifying a commit-hash to
deterministically build a fixed version of a repository. Not specifying a
commit-hash will result in the latest commit of the repository being built.
http(s)://<server:port>/<lab-location>/tree
To learn more about URLs in JupyterLab and Jupyter Notebook, visit
`starting JupyterLab <http://jupyterlab.readthedocs.io/en/latest/getting_started/starting.html>`_.
Where to put configuration files
================================
``--debug`` and ``--no-build``
------------------------------
``repo2docker`` will look for configuration files in either:
* A folder named ``binder/`` in the root of the repository.
* The root directory of the repository.
If the folder ``binder/`` is located at the top level of the repository,
**only configuration files in the** ``binder/`` **folder will be considered**.
.. note::
``repo2docker`` builds an environment with Python 3.6 by default. If you'd
like a different version, you can specify this in your
:ref:`configuration files <config-files>`.
Debugging repo2docker with ``--debug`` and ``--no-build``
=========================================================
To debug the docker image being built, pass the ``--debug`` parameter:
@ -100,3 +106,5 @@ be used by docker directly.
.. code-block:: bash
jupyter-repo2docker --no-build --debug https://github.com/norvig/pytudes
.. _Pytudes: https://github.com/norvig/pytudes

Wyświetl plik

@ -177,8 +177,11 @@ class Repo2Docker(Application):
def fetch(self, url, ref, checkout_path):
"""Check out a repo using url and ref to the checkout_path location"""
try:
for line in execute_cmd(['git', 'clone', '--recursive', url, checkout_path],
capture=self.json_logs):
cmd = ['git', 'clone', '--recursive']
if not ref:
cmd.extend(['--depth', '1'])
cmd.extend([url, checkout_path])
for line in execute_cmd(cmd, capture=self.json_logs):
self.log.info(line, extra=dict(phase='fetching'))
except subprocess.CalledProcessError:
self.log.error('Failed to clone repository!',
@ -349,6 +352,13 @@ class Repo2Docker(Application):
default=[]
)
argparser.add_argument(
'--editable', '-E',
dest='editable',
action='store_true',
help='Use the local repository in edit mode',
)
argparser.add_argument(
'--appendix',
type=str,
@ -405,6 +415,8 @@ class Repo2Docker(Application):
self.repo = args.repo
self.ref = None
self.cleanup_checkout = False
if args.editable:
self.volumes[os.path.abspath(args.repo)] = '.'
else:
self.repo_type = 'remote'
self.repo = args.repo

Wyświetl plik

@ -543,7 +543,7 @@ class BaseImage(BuildPack):
return []
def get_start_script(self):
start = self.binder_path('start')
start = self.binder_path('./start')
if os.path.exists(start):
return start
return None

Wyświetl plik

@ -1,32 +1,21 @@
# AUTO GENERATED FROM environment.py-3.6.yml, DO NOT MANUALLY MODIFY
# Frozen on 2018-09-07 17:44:07 UTC
# Frozen on 2018-09-17 08:36:41 UTC
name: r2d
channels:
- conda-forge
- defaults
- conda-forge/label/broken
dependencies:
- appdirs=1.4.3=py_1
- asn1crypto=0.24.0=py36_3
- attrs=18.2.0=py_0
- automat=0.7.0=py_1
- backcall=0.1.0=py_0
- bleach=2.1.4=py_1
- bzip2=1.0.6=h470a237_2
- ca-certificates=2018.8.24=ha4d7672_0
- certifi=2018.8.24=py36_1
- cffi=1.11.5=py36h5e8e0c9_1
- constantly=15.1.0=py_0
- cryptography=2.3.1=py36hdffb7b8_0
- cryptography-vectors=2.3.1=py36_0
- decorator=4.3.0=py_0
- defusedxml=0.5.0=py36_0
- entrypoints=0.2.3=py36_2
- gmp=6.1.2=hfc679d8_0
- html5lib=1.0.1=py_0
- hyperlink=17.3.1=py_0
- idna=2.7=py36_2
- incremental=17.5.0=py_0
- ipykernel=4.9.0=py36_0
- ipython=6.5.0=py36_0
- ipython_genutils=0.2.0=py_1
@ -36,7 +25,7 @@ dependencies:
- jsonschema=2.6.0=py36_2
- jupyter_client=5.2.3=py_1
- jupyter_core=4.4.0=py_0
- jupyterlab=0.34.0=py36_0
- jupyterlab=0.34.9=py36_0
- jupyterlab_launcher=0.13.1=py_2
- libffi=3.2.1=hfc679d8_5
- libgcc-ng=7.2.0=hdf63c60_3
@ -44,32 +33,26 @@ dependencies:
- libstdcxx-ng=7.2.0=hdf63c60_3
- markupsafe=1.0=py36h470a237_1
- mistune=0.8.3=py36h470a237_2
- nbconvert=5.4.0=0
- nbconvert=5.4.0=1
- nbformat=4.4.0=py_1
- ncurses=6.1=hfc679d8_1
- notebook=5.6.0=py36_1
- openssl=1.0.2p=h470a237_0
- pandoc=2.2.2=1
- pandoc=1.19.2=0
- pandocfilters=1.4.2=py_1
- parso=0.3.1=py_0
- pexpect=4.6.0=py36_0
- pickleshare=0.7.4=py36_0
- pip=18.0=py36_1
- prometheus_client=0.3.0=py_0
- prometheus_client=0.3.1=py_1
- prompt_toolkit=1.0.15=py_1
- ptyprocess=0.6.0=py36_0
- pyasn1=0.4.4=py_0
- pyasn1-modules=0.2.1=py_0
- pycparser=2.18=py_1
- pygments=2.2.0=py_1
- pyhamcrest=1.9.0=py_2
- pyopenssl=18.0.0=py36_0
- python=3.6.6=h5001a0f_0
- python-dateutil=2.7.3=py_0
- pyzmq=17.1.2=py36hae99301_0
- readline=7.0=haf1bffa_1
- send2trash=1.5.0=py_0
- service_identity=17.0.0=py_0
- setuptools=40.2.0=py36_0
- simplegeneric=0.8.1=py_1
- six=1.11.0=py36_1
@ -79,7 +62,6 @@ dependencies:
- tk=8.6.8=0
- tornado=4.5.3=py36_0
- traitlets=4.3.2=py36_0
- twisted=18.7.0=py36h470a237_1
- wcwidth=0.1.7=py_1
- webencodings=0.5.1=py_1
- wheel=0.31.1=py36_1
@ -87,7 +69,6 @@ dependencies:
- xz=5.2.4=h470a237_1
- zeromq=4.2.5=hfc679d8_5
- zlib=1.2.11=h470a237_3
- zope.interface=4.5.0=py36h470a237_1
- pip:
- nteract-on-jupyter==1.9.6
prefix: /opt/conda/envs/r2d

Wyświetl plik

@ -1,5 +1,5 @@
# AUTO GENERATED FROM environment.py-2.7.yml, DO NOT MANUALLY MODIFY
# Frozen on 2018-09-07 17:36:45 UTC
# Frozen on 2018-09-17 08:27:26 UTC
name: r2d
channels:
- conda-forge

Wyświetl plik

@ -1,32 +1,21 @@
# AUTO GENERATED FROM environment.py-3.5.yml, DO NOT MANUALLY MODIFY
# Frozen on 2018-09-07 17:39:09 UTC
# Frozen on 2018-09-17 08:33:33 UTC
name: r2d
channels:
- conda-forge
- defaults
- conda-forge/label/broken
dependencies:
- appdirs=1.4.3=py_1
- asn1crypto=0.24.0=py35_3
- attrs=18.2.0=py_0
- automat=0.7.0=py_1
- backcall=0.1.0=py_0
- bleach=2.1.4=py_1
- bzip2=1.0.6=h470a237_2
- ca-certificates=2018.8.24=ha4d7672_0
- certifi=2018.8.24=py35_1
- cffi=1.11.5=py35h5e8e0c9_1
- constantly=15.1.0=py_0
- cryptography=2.3.1=py35hdffb7b8_0
- cryptography-vectors=2.3.1=py35_0
- decorator=4.3.0=py_0
- defusedxml=0.5.0=py35_0
- entrypoints=0.2.3=py35_2
- gmp=6.1.2=hfc679d8_0
- html5lib=1.0.1=py_0
- hyperlink=17.3.1=py_0
- idna=2.7=py35_2
- incremental=17.5.0=py_0
- ipykernel=4.9.0=py35_0
- ipython=6.5.0=py35_0
- ipython_genutils=0.2.0=py_1
@ -36,7 +25,7 @@ dependencies:
- jsonschema=2.6.0=py35_2
- jupyter_client=5.2.3=py_1
- jupyter_core=4.4.0=py_0
- jupyterlab=0.34.0=py35_0
- jupyterlab=0.34.9=py35_0
- jupyterlab_launcher=0.13.1=py_2
- libffi=3.2.1=hfc679d8_5
- libgcc-ng=7.2.0=hdf63c60_3
@ -44,32 +33,26 @@ dependencies:
- libstdcxx-ng=7.2.0=hdf63c60_3
- markupsafe=1.0=py35h470a237_1
- mistune=0.8.3=py35h470a237_2
- nbconvert=5.4.0=0
- nbconvert=5.4.0=1
- nbformat=4.4.0=py_1
- ncurses=6.1=hfc679d8_1
- notebook=5.6.0=py35_1
- openssl=1.0.2p=h470a237_0
- pandoc=2.2.2=1
- pandoc=1.19.2=0
- pandocfilters=1.4.2=py_1
- parso=0.3.1=py_0
- pexpect=4.6.0=py35_0
- pickleshare=0.7.4=py35_0
- pip=18.0=py35_1
- prometheus_client=0.3.0=py_0
- prometheus_client=0.3.1=py_1
- prompt_toolkit=1.0.15=py_1
- ptyprocess=0.6.0=py35_0
- pyasn1=0.4.4=py_0
- pyasn1-modules=0.2.1=py_0
- pycparser=2.18=py_1
- pygments=2.2.0=py_1
- pyhamcrest=1.9.0=py_2
- pyopenssl=18.0.0=py35_0
- python=3.5.5=h5001a0f_2
- python-dateutil=2.7.3=py_0
- pyzmq=17.1.2=py35hae99301_0
- readline=7.0=haf1bffa_1
- send2trash=1.5.0=py_0
- service_identity=17.0.0=py_0
- setuptools=40.2.0=py35_0
- simplegeneric=0.8.1=py_1
- six=1.11.0=py35_1
@ -79,7 +62,6 @@ dependencies:
- tk=8.6.8=0
- tornado=4.5.3=py35_0
- traitlets=4.3.2=py35_0
- twisted=18.7.0=py35h470a237_1
- wcwidth=0.1.7=py_1
- webencodings=0.5.1=py_1
- wheel=0.31.1=py35_1
@ -87,7 +69,6 @@ dependencies:
- xz=5.2.4=h470a237_1
- zeromq=4.2.5=hfc679d8_5
- zlib=1.2.11=h470a237_3
- zope.interface=4.5.0=py35h470a237_1
- pip:
- nteract-on-jupyter==1.9.6
prefix: /opt/conda/envs/r2d

Wyświetl plik

@ -1,32 +1,21 @@
# AUTO GENERATED FROM environment.py-3.6.yml, DO NOT MANUALLY MODIFY
# Frozen on 2018-09-07 17:44:07 UTC
# Frozen on 2018-09-17 08:36:41 UTC
name: r2d
channels:
- conda-forge
- defaults
- conda-forge/label/broken
dependencies:
- appdirs=1.4.3=py_1
- asn1crypto=0.24.0=py36_3
- attrs=18.2.0=py_0
- automat=0.7.0=py_1
- backcall=0.1.0=py_0
- bleach=2.1.4=py_1
- bzip2=1.0.6=h470a237_2
- ca-certificates=2018.8.24=ha4d7672_0
- certifi=2018.8.24=py36_1
- cffi=1.11.5=py36h5e8e0c9_1
- constantly=15.1.0=py_0
- cryptography=2.3.1=py36hdffb7b8_0
- cryptography-vectors=2.3.1=py36_0
- decorator=4.3.0=py_0
- defusedxml=0.5.0=py36_0
- entrypoints=0.2.3=py36_2
- gmp=6.1.2=hfc679d8_0
- html5lib=1.0.1=py_0
- hyperlink=17.3.1=py_0
- idna=2.7=py36_2
- incremental=17.5.0=py_0
- ipykernel=4.9.0=py36_0
- ipython=6.5.0=py36_0
- ipython_genutils=0.2.0=py_1
@ -36,7 +25,7 @@ dependencies:
- jsonschema=2.6.0=py36_2
- jupyter_client=5.2.3=py_1
- jupyter_core=4.4.0=py_0
- jupyterlab=0.34.0=py36_0
- jupyterlab=0.34.9=py36_0
- jupyterlab_launcher=0.13.1=py_2
- libffi=3.2.1=hfc679d8_5
- libgcc-ng=7.2.0=hdf63c60_3
@ -44,32 +33,26 @@ dependencies:
- libstdcxx-ng=7.2.0=hdf63c60_3
- markupsafe=1.0=py36h470a237_1
- mistune=0.8.3=py36h470a237_2
- nbconvert=5.4.0=0
- nbconvert=5.4.0=1
- nbformat=4.4.0=py_1
- ncurses=6.1=hfc679d8_1
- notebook=5.6.0=py36_1
- openssl=1.0.2p=h470a237_0
- pandoc=2.2.2=1
- pandoc=1.19.2=0
- pandocfilters=1.4.2=py_1
- parso=0.3.1=py_0
- pexpect=4.6.0=py36_0
- pickleshare=0.7.4=py36_0
- pip=18.0=py36_1
- prometheus_client=0.3.0=py_0
- prometheus_client=0.3.1=py_1
- prompt_toolkit=1.0.15=py_1
- ptyprocess=0.6.0=py36_0
- pyasn1=0.4.4=py_0
- pyasn1-modules=0.2.1=py_0
- pycparser=2.18=py_1
- pygments=2.2.0=py_1
- pyhamcrest=1.9.0=py_2
- pyopenssl=18.0.0=py36_0
- python=3.6.6=h5001a0f_0
- python-dateutil=2.7.3=py_0
- pyzmq=17.1.2=py36hae99301_0
- readline=7.0=haf1bffa_1
- send2trash=1.5.0=py_0
- service_identity=17.0.0=py_0
- setuptools=40.2.0=py36_0
- simplegeneric=0.8.1=py_1
- six=1.11.0=py36_1
@ -79,7 +62,6 @@ dependencies:
- tk=8.6.8=0
- tornado=4.5.3=py36_0
- traitlets=4.3.2=py36_0
- twisted=18.7.0=py36h470a237_1
- wcwidth=0.1.7=py_1
- webencodings=0.5.1=py_1
- wheel=0.31.1=py36_1
@ -87,7 +69,6 @@ dependencies:
- xz=5.2.4=h470a237_1
- zeromq=4.2.5=hfc679d8_5
- zlib=1.2.11=h470a237_3
- zope.interface=4.5.0=py36h470a237_1
- pip:
- nteract-on-jupyter==1.9.6
prefix: /opt/conda/envs/r2d

Wyświetl plik

@ -0,0 +1,74 @@
# AUTO GENERATED FROM environment.py-3.7.yml, DO NOT MANUALLY MODIFY
# Frozen on 2018-09-17 08:29:40 UTC
name: r2d
channels:
- conda-forge
- defaults
- conda-forge/label/broken
dependencies:
- backcall=0.1.0=py_0
- bleach=2.1.4=py_1
- bzip2=1.0.6=h470a237_2
- ca-certificates=2018.8.24=ha4d7672_0
- certifi=2018.4.16=py37_0
- decorator=4.3.0=py_0
- entrypoints=0.2.3=py37_2
- gmp=6.1.2=hfc679d8_0
- html5lib=1.0.1=py_0
- ipykernel=4.8.2=py37_0
- ipython=6.5.0=py37_0
- ipython_genutils=0.2.0=py_1
- jedi=0.12.1=py37_0
- jinja2=2.10=py_1
- jsonschema=2.6.0=py37_2
- jupyter_client=5.2.3=py_1
- jupyter_core=4.4.0=py_0
- jupyterlab_launcher=0.13.1=py_2
- libffi=3.2.1=hfc679d8_5
- libgcc-ng=7.2.0=hdf63c60_3
- libsodium=1.0.16=h470a237_1
- libstdcxx-ng=7.2.0=hdf63c60_3
- markupsafe=1.0=py37h470a237_1
- mistune=0.8.3=py_0
- nbconvert=5.4.0=1
- nbformat=4.4.0=py_1
- ncurses=6.1=hfc679d8_1
- notebook=5.6.0=py37_1
- openssl=1.0.2p=h470a237_0
- pandoc=1.19.2=0
- pandocfilters=1.4.2=py_1
- parso=0.3.1=py_0
- pexpect=4.6.0=py37_0
- pickleshare=0.7.4=py37_0
- pip=18.0=py37_1
- prometheus_client=0.3.1=py_1
- prompt_toolkit=1.0.15=py_1
- ptyprocess=0.6.0=py37_0
- pygments=2.2.0=py_1
- python=3.7.0=h5001a0f_1
- python-dateutil=2.7.3=py_0
- pyzmq=17.1.2=py37hae99301_0
- readline=7.0=haf1bffa_1
- send2trash=1.5.0=py_0
- setuptools=40.2.0=py37_0
- simplegeneric=0.8.1=py_1
- six=1.11.0=py37_1
- sqlite=3.24.0=h2f33b56_1
- terminado=0.8.1=py37_1
- testpath=0.3.1=py37_1
- tk=8.6.8=0
- traitlets=4.3.2=py37_0
- wcwidth=0.1.7=py_1
- webencodings=0.5.1=py_1
- wheel=0.31.1=py37_1
- widgetsnbextension=3.4.1=py37_0
- xz=5.2.4=h470a237_1
- zeromq=4.2.5=hfc679d8_5
- zlib=1.2.11=h470a237_3
- defusedxml=0.5.0=py37_1
- ipywidgets=7.2.1=py37_0
- jupyterlab=0.34.9=py37_0
- tornado=4.5.3=py37_0
- pip:
- nteract-on-jupyter==1.9.6
prefix: /opt/conda/envs/r2d

Wyświetl plik

@ -0,0 +1,11 @@
# AUTO GENERATED FROM environment.yml, DO NOT MANUALLY MODIFY
# Generated on 2018-09-17 08:29:40 UTC
dependencies:
- python=3.7.*
- ipywidgets==7.2.1
- jupyterlab==0.34.9
- nbconvert==5.4.0
- tornado==4.5.3
- notebook==5.6.0
- pip:
- nteract_on_jupyter==1.9.6

Wyświetl plik

@ -1,7 +1,8 @@
dependencies:
- python=3.6.*
- ipywidgets==7.2.1
- jupyterlab==0.34.0
- jupyterlab==0.34.9
- nbconvert==5.4.0
- tornado==4.5.3
- notebook==5.6.0
- pip:

Wyświetl plik

@ -100,7 +100,7 @@ def set_python(py_env_file, py):
if __name__ == '__main__':
# allow specifying which Pythons to update on argv
pys = sys.argv[1:] or ('2.7', '3.5', '3.6')
pys = sys.argv[1:] or ('2.7', '3.7', '3.5', '3.6')
for py in pys:
env_file = ENV_FILE_T.format(py=py)
set_python(env_file, py)

Wyświetl plik

@ -1,6 +1,6 @@
"""Generates a Dockerfile based on an input matrix for Julia"""
import os
from .conda import CondaBuildPack
from ..conda import CondaBuildPack
class JuliaBuildPack(CondaBuildPack):
@ -12,6 +12,44 @@ class JuliaBuildPack(CondaBuildPack):
See https://github.com/JuliaPy/PyCall.jl/issues/410
"""
minor_julias = {
'0.6': '0.6.4',
'0.7': '0.7.0',
'1.0': '1.0.0',
}
major_julias = {
'1': '1.0.0',
}
@property
def julia_version(self):
require = self.binder_path('REQUIRE')
try:
with open(require) as f:
julia_version_line = f.readline().strip() # First line is optionally a julia version
except FileNotFoundError:
julia_version_line = ''
if not julia_version_line.startswith('julia '):
# not a Julia version line.
# use the default Julia.
self._julia_version = self.minor_julias['0.6']
return self._julia_version
julia_version_info = julia_version_line.split(' ', 1)[1].split('.')
julia_version = ''
if len(julia_version_info) == 1:
julia_version = self.major_julias[julia_version_info[0]]
elif len(julia_version_info) == 2:
# get major.minor
julia_version = self.minor_julias['.'.join(julia_version_info)]
else:
# use supplied julia version
julia_version = '.'.join(julia_version_info)
self._julia_version = julia_version
return self._julia_version
def get_build_env(self):
"""Get additional environment settings for Julia and Jupyter
@ -33,9 +71,10 @@ class JuliaBuildPack(CondaBuildPack):
"""
return super().get_build_env() + [
('JULIA_PATH', '${APP_BASE}/julia'),
('JULIA_HOME', '${JULIA_PATH}/bin'),
('JULIA_HOME', '${JULIA_PATH}/bin'), # julia <= 0.6
('JULIA_BINDIR', '${JULIA_HOME}'), # julia >= 0.7
('JULIA_PKGDIR', '${JULIA_PATH}/pkg'),
('JULIA_VERSION', '0.6.0'),
('JULIA_VERSION', self.julia_version),
('JUPYTER', '${NB_PYTHON_PREFIX}/bin/jupyter')
]
@ -80,8 +119,8 @@ class JuliaBuildPack(CondaBuildPack):
# HACK: Can't seem to tell IJulia to install in sys-prefix
# FIXME: Find way to get it to install under /srv and not $HOME?
r"""
julia -e 'Pkg.init(); Pkg.add("IJulia"); using IJulia;' && \
mv ${HOME}/.local/share/jupyter/kernels/julia-0.6 ${NB_PYTHON_PREFIX}/share/jupyter/kernels/julia-0.6
julia -e 'if (VERSION > v"0.7-") using Pkg; else Pkg.init(); end; Pkg.add("IJulia"); using IJulia;' && \
mv ${HOME}/.local/share/jupyter/kernels/julia-${JULIA_VERSION%[.-]*} ${NB_PYTHON_PREFIX}/share/jupyter/kernels/julia-${JULIA_VERSION%[.-]*}
"""
)
]
@ -98,19 +137,25 @@ class JuliaBuildPack(CondaBuildPack):
require = self.binder_path('REQUIRE')
return super().get_assemble_scripts() + [(
"${NB_USER}",
# Pre-compile all libraries if they've opted into it.
# `using {libraryname}` does the right thing
# Install and pre-compile all libraries if they've opted into it.
# In v0.6, Pkg.resolve() installs all the packages, but in v0.7+, we
# have to manually Pkg.add() each of them (since the REQUIRES file
# format is deprecated).
# The precompliation is done via `using {libraryname}`.
r"""
cat "%(require)s" >> ${JULIA_PKGDIR}/v0.6/REQUIRE && \
julia -e ' \
Pkg.resolve(); \
for pkg in keys(Pkg.Reqs.parse("%(require)s")) \
pkg != "julia" && eval(:(using $(Symbol(pkg)))) \
end \
'
""" % { "require" : require }
julia /tmp/install-repo-dependencies.jl "%(require)s"
""" % {"require": require}
# TODO: For some reason, `rm`ing the file fails with permission denied.
# && rm /tmp/install-repo-dependencies.jl
)]
def get_build_script_files(self):
files = {
'julia/install-repo-dependencies.jl': '/tmp/install-repo-dependencies.jl',
}
files.update(super().get_build_script_files())
return files
def detect(self):
"""
Check if current repo should be built with the Julia Build pack
@ -122,4 +167,5 @@ class JuliaBuildPack(CondaBuildPack):
Instead we just check if the path to `REQUIRE` exists
"""
# TODO(nhdaly): Add support for Project.toml here as well.
return os.path.exists(self.binder_path('REQUIRE'))

Wyświetl plik

@ -0,0 +1,36 @@
#!/bin/julia
# Path to potential REQUIRE file is passed on the command line.
require_file = ARGS[1]
if VERSION < v"0.7-"
pkg_dir = "$(ENV["JULIA_PKGDIR"])/v$(VERSION.major).$(VERSION.minor)"
open("$pkg_dir/REQUIRE", "a") do io
write(io, read(require_file))
end
Pkg.resolve();
Reqs = Pkg.Reqs
else
using Pkg
Reqs = Pkg.Pkg2.Reqs
emptyversionlower = v"0.0.0-"
for reqline in Reqs.read(require_file)
if reqline isa Reqs.Requirement
pkg = String(reqline.package)
if pkg == "julia" continue end
version = try; reqline.versions.intervals[1].lower; catch; emptyversionlower; end
if version != emptyversionlower
Pkg.add(PackageSpec(name=pkg, version=version))
else
Pkg.add(pkg)
end
end
end
end
# Precompile the packages
for reqline in Reqs.read(require_file)
if reqline isa Reqs.Requirement
pkg = reqline.package
pkg != "julia" && eval(:(using $(Symbol(pkg))))
end
end

Wyświetl plik

@ -69,7 +69,16 @@ class RBuildPack(PythonBuildPack):
"""
# If no date is found, then self.checkpoint_date will be False
# Otherwise, it'll be a date object, which will evaluate to True
return bool(self.checkpoint_date)
if self.checkpoint_date:
return True
description_R = 'DESCRIPTION'
if not os.path.exists('binder') and os.path.exists(description_R):
if not self.checkpoint_date:
# no R snapshot date set through runtime.txt
# set the R runtime to the latest date that is guaranteed to be on MRAN across timezones
self._checkpoint_date = datetime.date.today() - datetime.timedelta(days=2)
self._runtime = "r-{}".format(str(self._checkpoint_date))
return True
def get_path(self):
"""
@ -257,4 +266,13 @@ class RBuildPack(PythonBuildPack):
)
]
description_R = 'DESCRIPTION'
if not os.path.exists('binder') and os.path.exists(description_R):
assemble_scripts += [
(
"${NB_USER}",
'R --quiet -e "devtools::install_local(getwd())"'
)
]
return assemble_scripts

Wyświetl plik

@ -3,7 +3,6 @@ from functools import partial
import re
import shutil
import subprocess
import sys
from traitlets import Integer, TraitError

Wyświetl plik

@ -1 +1 @@
Test that node 10 is installed and runnable.
Test that node 10 and npm 6 are installed and runnable.

Wyświetl plik

@ -2,6 +2,9 @@
set -e
which node
node --version
node --version | grep v10
which npm
npm --version | grep 6.2
npm --version
npm --version | grep 6.4

Wyświetl plik

@ -0,0 +1,2 @@
dependencies:
- python=3.7

Wyświetl plik

@ -0,0 +1,4 @@
#!/usr/bin/env python
import sys
assert sys.version_info[:2] == (3, 7), sys.version

Wyświetl plik

@ -103,11 +103,10 @@ class Repo2DockerTest(pytest.Function):
class LocalRepo(pytest.File):
def collect(self):
yield Repo2DockerTest(
self.fspath.basename, self,
'build', self,
args=[
'--appendix', 'RUN echo "appendix" > /tmp/appendix',
self.fspath.dirname,
'./verify',
],
)
yield Repo2DockerTest(
@ -115,6 +114,7 @@ class LocalRepo(pytest.File):
args=[
'--appendix', 'RUN echo "appendix" > /tmp/appendix',
self.fspath.dirname,
'./verify',
],
)

Wyświetl plik

@ -0,0 +1,13 @@
FROM python:3.5
RUN pip install --no-cache notebook
CMD "/bin/sh"
ADD change.sh /usr/local/bin/change.sh
ARG NB_UID
ENV HOME /tmp
WORKDIR ${HOME}
USER $NB_UID

Wyświetl plik

@ -0,0 +1,19 @@
Docker - Edit mode
------------------
Using the --editable option with a local repository, one can modify a
file or create a new file in the container, and this change is
reflected in the respective host directory. It is essentially a
shortcut for `--mount
type=bind,source=<local-host-repository>,target=.` (where the target
resolves into the container working directory).
This is tested by running the change.sh script inside the container
(using the 'cmd' argument to the Repo2Docker app), which creates a new
file, and then verifying on the host side the new file is created with
the proper contents.
In practice, this can be used to run a notebook from inside a
container (which provides the proper environment), making changes as
necessary, which are then immediately reflected in the host
repository.

Wyświetl plik

@ -0,0 +1,6 @@
#!/bin/bash
cat <<EOF > newfile
new contents
EOF
exit 0

Wyświetl plik

@ -0,0 +1,5 @@
Julia - REQUIRE: test version for julia v1.0
---------------
Starting with julia v0.7 and up, the package manager has changed, so this tests
that the julia version can be specified and packages installed correctly.

Wyświetl plik

@ -0,0 +1,4 @@
julia 1
# Julia packages:
Compat

Wyświetl plik

@ -0,0 +1,17 @@
#!/usr/bin/env julia
# Verify the version:
if VERSION < v"1"
exit(1)
end
try
# Test that the package was installed.
using Compat
# Verify that the environment variables are set correctly for julia 1.0+
@assert "julia" ∈ readdir(Sys.BINDIR)
catch
exit(1)
end
exit(0)

Wyświetl plik

@ -0,0 +1,22 @@
Julia - REQUIRE: packages and julia version number
---------------
To specify dependencies in Julia, include a REQUIRE file.
To specify a version of Julia to be installed, the first line of the file must
follow the format `julia <version-number>`. For example:
```
julia 0.7
```
To add package dependencies, the rest of the file can include lines that list
the names of packages you'd like to be installed. For example:
```
PyPlot
DataFrames
```
Note that `IJulia` is installed by default.
These packages will all be installed, and then precompiled via `using`.

Wyświetl plik

@ -0,0 +1,4 @@
julia 0.6.3
# Julia packages:
Compat

Wyświetl plik

@ -0,0 +1,17 @@
#!/usr/bin/env julia
# Verify the version:
if VERSION != v"0.6.3"
exit(1)
end
try
# Test that the package was installed.
using Compat
# Verify that the environment variables are set correctly for julia <= 0.6
@assert "julia" ∈ readdir(Base.JULIA_HOME)
catch
exit(1)
end
exit(0)

Wyświetl plik

@ -2,7 +2,7 @@
using PyPlot
# Make sure we are re-using the same conda python
if PyCall.libpython != "/srv/conda/lib/libpython3.6m"
if ! startswith(PyCall.libpython, ENV["NB_PYTHON_PREFIX"] * "/lib")
println("Not re-using conda python! Using " * PyCall.libpython * " instead")
exit(1)
end
@ -16,4 +16,3 @@ if isfile("graph.png")
else
exit(1)
end

Wyświetl plik

@ -0,0 +1,7 @@
Package: binderdescription
Version: 0.1
Date: 2018-09-18
Title: Binder R DESCRIPTION support
Description: Test that automatically building R packages works
Author: Bastian Greshake Tzovaras <bgreshake@googlemail.com>
Maintainer: Bastian Greshake Tzovaras <bgreshake@googlemail.com>

Wyświetl plik

@ -0,0 +1,2 @@
# Export all names
exportPattern(".")

Wyświetl plik

@ -0,0 +1,4 @@
print_something <- function()
{
print('blah');
}

Wyświetl plik

@ -0,0 +1,4 @@
# Test that `repo2docker` will build an R package
If a `DESCRIPTION` file is present along with an `R`-requiring `runtime.txt`
, repo2docker should build the R package that is in the repository.

Wyświetl plik

@ -0,0 +1,2 @@
#!/usr/bin/env Rscript
library('binderdescription')

Wyświetl plik

@ -0,0 +1,103 @@
"""
Test that a clone depth of 1 is used and good enough when no refspec is used
Note: the tests don't actually run the container. Building the
container requires a specific repository and commit to be checked out,
and that is the only thing that is tested.
"""
import subprocess
from repo2docker.app import Repo2Docker
URL = "https://github.com/binderhub-ci-repos/repo2docker-ci-clone-depth"
def test_clone_depth():
"""Test a remote repository, without a refspec"""
app = Repo2Docker()
argv = [URL]
app.initialize(argv)
app.debug = True
app.run = False
app.cleanup_checkout = False
app.start() # This just build the image and does not run it.
# Building the image has already put us in the cloned repository directory
cmd = ['git', 'rev-parse', 'HEAD']
p = subprocess.run(cmd, stdout=subprocess.PIPE)
assert p.stdout.strip() == b'703322e9c6635ba1835d3b92eafbabeca0042c3e'
cmd = ['git', 'rev-list', '--count', 'HEAD']
p = subprocess.run(cmd, stdout=subprocess.PIPE)
assert p.stdout.strip() == b'1'
with open('COMMIT') as fp:
assert fp.read() == '100\n'
def test_clone_depth_full():
"""Test a remote repository, with a refspec of 'master'"""
app = Repo2Docker()
argv = ['--ref', 'master', URL]
app.initialize(argv)
app.debug = True
app.run = False
app.cleanup_checkout = False
app.start() # This just build the image and does not run it.
# Building the image has already put us in the cloned repository directory
cmd = ['git', 'rev-parse', 'HEAD']
p = subprocess.run(cmd, stdout=subprocess.PIPE)
assert p.stdout.strip() == b'703322e9c6635ba1835d3b92eafbabeca0042c3e'
cmd = ['git', 'rev-list', '--count', 'HEAD']
p = subprocess.run(cmd, stdout=subprocess.PIPE)
assert p.stdout.strip() == b'100'
with open('COMMIT') as fp:
assert fp.read() == '100\n'
def test_clone_depth_full2():
"""Test a remote repository, with a refspec of the master commit hash"""
app = Repo2Docker()
argv = ['--ref', '703322e', URL]
app.initialize(argv)
app.debug = True
app.run = False
app.cleanup_checkout = False
app.start() # This just build the image and does not run it.
# Building the image has already put us in the cloned repository directory
cmd = ['git', 'rev-parse', 'HEAD']
p = subprocess.run(cmd, stdout=subprocess.PIPE)
assert p.stdout.strip() == b'703322e9c6635ba1835d3b92eafbabeca0042c3e'
cmd = ['git', 'rev-list', '--count', 'HEAD']
p = subprocess.run(cmd, stdout=subprocess.PIPE)
assert p.stdout.strip() == b'100'
with open('COMMIT') as fp:
assert fp.read() == '100\n'
def test_clone_depth_mid():
"""Test a remote repository, with a refspec of a commit hash halfway"""
app = Repo2Docker()
argv = ['--ref', '8bc4f21', URL]
app.initialize(argv)
app.debug = True
app.run = False
app.cleanup_checkout = False
app.start() # This just build the image and does not run it.
# Building the image has already put us in the cloned repository directory
cmd = ['git', 'rev-parse', 'HEAD']
p = subprocess.run(cmd, stdout=subprocess.PIPE)
assert p.stdout.strip() == b'8bc4f216856f86f6fc25a788b744b93b87e9ba48'
cmd = ['git', 'rev-list', '--count', 'HEAD']
p = subprocess.run(cmd, stdout=subprocess.PIPE)
assert p.stdout.strip() == b'50'
with open('COMMIT') as fp:
assert fp.read() == '50\n'

Wyświetl plik

@ -0,0 +1,56 @@
import os
import time
import re
import tempfile
from conftest import make_test_func
from repo2docker.app import Repo2Docker
DIR = os.path.join(os.path.dirname(__file__), 'dockerfile', 'editable')
def test_editable(run_repo2docker):
"""Run a local repository in edit mode. Verify a new file has been
created afterwards"""
newfile = os.path.join(DIR, 'newfile')
try:
# If the file didn't get properly cleaned up last time, we
# need to do that now
os.remove(newfile)
except FileNotFoundError:
pass
argv = ['--editable', DIR, '/usr/local/bin/change.sh']
run_repo2docker(argv)
try:
with open(newfile) as fp:
contents = fp.read()
assert contents == "new contents\n"
finally:
os.remove(newfile)
def test_editable_by_host():
"""Test whether a new file created by the host environment, is
detected in the container"""
app = Repo2Docker()
app.initialize(['--editable', DIR])
app.run = False
app.start() # This just build the image and does not run it.
container = app.start_container()
# give the container a chance to start
time.sleep(1)
try:
with tempfile.NamedTemporaryFile(dir=DIR, prefix='testfile', suffix='.txt'):
status, output = container.exec_run(['sh', '-c', 'ls testfile????????.txt'])
assert status == 0
assert re.match(br'^testfile\w{8}\.txt\n$', output) is not None
# File should be removed in the container as well
status, output = container.exec_run(['sh', '-c', 'ls testfile????????.txt'])
assert status != 1
assert re.match(br'^testfile\w{8}\.txt\n$', output) is None
finally:
# stop the container
container.stop()
app.wait_for_container(container)

Wyświetl plik

@ -1,8 +1,5 @@
System - Post-build scripts
---------------------------
It is possible to run scripts after you've built the environment specified in
your other files. This could be used to, for example, download data or run
some configuration scripts.
In this example, we download and install a Jupyter Notebook extension.
postBuild and start
-------------------
This test checks that we can use a postBuild and start script
at the same time.

Wyświetl plik

@ -1,2 +1,6 @@
#!/bin/bash
# this value should not be visible in `verify`
export TEST_START_VAR="var is set by postBuild"
jupyter nbextension enable --py --sys-prefix ipyleaflet

Wyświetl plik

@ -0,0 +1,4 @@
#!/bin/bash
export TEST_START_VAR="var is set"
exec "$@"

Wyświetl plik

@ -2,7 +2,8 @@
set -euo pipefail
jupyter nbextension list | grep 'jupyter-leaflet' | grep enabled
if [ "$TEST_START_VAR" != "var is set" ]
# set value of TEST_START_VAR to empty string when it is not defined
if [ "${TEST_START_VAR:-}" != "var is set" ]
then
echo "TEST_START_VAR is not set"
exit 1

Wyświetl plik

@ -1,7 +1,8 @@
#!/bin/bash
set -euo pipefail
if [ "$TEST_START_VAR" != "var is set" ]
# set value of TEST_START_VAR to empty string when it is not defined
if [ "${TEST_START_VAR:-}" != "var is set" ]
then
echo "TEST_START_VAR is not set"
exit 1