updating index, usage, and install docs

pull/409/head
Chris Holdgraf 2018-09-17 19:19:50 -04:00
rodzic 46f056a4e2
commit d6eea937ce
7 zmienionych plików z 251 dodań i 180 usunięć

Wyświetl plik

@ -1,3 +1,7 @@
img.logo {
width:100%
}
div.body p.caption {
font-size: 1.5em;
}

Wyświetl plik

@ -44,7 +44,7 @@ source_parsers = {
}
def setup(app):
app.add_stylesheet('custom.css') # may also be an URL
app.add_stylesheet('_static/custom.css') # may also be an URL
# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
@ -188,38 +188,31 @@ epub_copyright = copyright
# A list of files that should not be packed into the epub file.
epub_exclude_files = ['search.html']
# Build the repo2docker test syntax
import os
s = ''
FILE_ORDER_PREFERENCE = ['System', 'Python', 'Julia', 'Docker', 'Misc']
SKIP_FOLDERS = ['binder-dir']
FILES_DICT = {ii: [] for ii in FILE_ORDER_PREFERENCE}
for root, _, files in os.walk(os.path.join('..', '..', 'tests')):
depth = len(root.split('/')) - 3
if 'README.rst' not in files or depth > 2:
continue
if any(folder in root for folder in SKIP_FOLDERS):
continue
header = files.pop(files.index('README.rst'))
with open(os.path.join(root, header), 'r') as ff:
this_s = ff.read() + '\n\n'
title = this_s.split('\n')[0]
for ifile in files:
filename = os.path.basename(ifile)
if filename == 'verify':
continue
with open(os.path.join(root, ifile), 'r') as ff:
lines = ff.readlines()
lines = [' ' + line for line in lines]
this_s += 'File: ``{}``\n{}\n\n**Contents**::\n\n'.format(
filename, '~' * (len(filename) + 10))
this_s += '\n'.join(lines)
this_s += '\n\n'
usename = [ii for ii in FILE_ORDER_PREFERENCE if ii in title]
usename = 'Misc' if len(usename) == 0 else usename[0]
FILES_DICT[usename].append(this_s)
for key in FILE_ORDER_PREFERENCE:
for istring in FILES_DICT[key]:
s += istring
with open('./test_file_text.txt', 'w') as ff:
ff.write(s)
# Update the example_repos text file
# import requests
#
# config_files = ['start', 'postBuild', 'apt.txt', 'runtime.txt',
# 'environment.yml', 'requirements.txt', 'setup.py',
# 'REQUIRE', 'install.R', 'Dockerfile']
# config_dict = {ii: [] for ii in config_files}
# repos = requests.get('https://api.github.com/orgs/jupyterhub/repos').json()
# repo_names = [ii['full_name'].split('/')[-1] for ii in repos]
# repo_descriptions = {name: repo['description'] for repo, name in zip(repos, repo_names)}
#
# files = {}
# for repo_name in repo_names:
# this_files = requests.get('https://api.github.com/repos/binder-examples/{}/contents'.format(repo_name)).json()
# for ifile in this_files:
# if ifile['name'] in config_dict:
# config_dict[ifile['name']].append(repo_name)
#
# s = ''
# for file_name, repos in config_dict.items():
# header_chars = '=' * len(file_name)
# s = '{}\n{}\n\n'.format(file_name, header_chars)
# for repo_name in repos:
# this_url = "https://github.com/binder-examples/{}".format(repo_name)
# this_desc = repo_descriptions['repo_name']
# s += '`{} <{}>`_\n'.format(this_desc, this_url)
# with open('example_repos.txt', 'w') as ff:
# ff.write(s)

Wyświetl plik

@ -1,9 +1,8 @@
.. NOTE: the header characters are different in this file because it is 'included' in another site
.. see https://raw.githubusercontent.com/jupyterhub/binder/master/doc/using.rst
.. _config-files:
===================
Configuration Files
-------------------
===================
``repo2docker`` looks for configuration files in the repository being built
to determine how to build it. In general, ``repo2docker`` uses the same
@ -13,14 +12,6 @@ rather than creating new custom configuration files.
A number of ``repo2docker`` configuration files can be combined to compose more
complex setups.
``repo2docker`` will look for configuration files in either:
* A folder named ``binder/`` in the root of the repository.
* The root directory of the repository.
If the folder ``binder/`` is located at the top level of the repository,
**only configuration files in the** ``binder/`` **folder will be considered**.
The `binder examples <https://github.com/binder-examples>`_ organization on
GitHub contains a list of sample repositories for common configurations
that ``repo2docker`` can build with various configuration files such as
@ -32,74 +23,45 @@ Below is a list of supported configuration files (roughly in the order of build
:local:
:depth: 1
``Dockerfile``
~~~~~~~~~~~~~~
In the majority of cases, providing your own Dockerfile is not necessary as the base
images provide core functionality, compact image sizes, and efficient builds. We recommend
trying the other configuration files before deciding to use your own Dockerfile.
With Dockerfiles, a regular Docker build will be performed.
**If a Dockerfile is present, all other configuration files will be ignored.**
See the `Binder Documentation <https://mybinder.readthedocs.io/en/latest/dockerfile.html>`_ for
best-practices with Dockerfiles.
.. _environment-yml:
``environment.yml``
~~~~~~~~~~~~~~~~~~~
``environment.yml`` - Install a Python environment
==================================================
``environment.yml`` is the standard configuration file used by Anaconda, conda,
and miniconda that lets you install Python packages.
You can also install files from pip in your ``environment.yml`` as well.
Our example `environment.yml <https://github.com/binder-examples/python-conda_pip/blob/master/environment.yml>`_
shows how one can specify a conda environment for repo2docker.
and miniconda that lets you install packages in the data analytics stack (it
primarily installs Python packages, though can be used to install a range of
non-Python packages as well).
.. note::
You can install files from pip in your ``environment.yml`` as well.
For example, see the `binder-examples environment.yml
<https://github.com/binder-examples/python-conda_pip/blob/master/environment.yml>`_
file.
You can also specify which Python version to install in your built environment
with ``environment.yml``. By default, ``repo2docker`` **installs
Python 3.6** with your ``environment.yml`` unless you include the version of
Python in the file. ``conda`` supports Python versions 3.6, 3.5, 3.4, and 2.7.
``repo2docker`` support is best with Python 3.6, 3.5, and 2.7. If you include
a Python version in a ``runtime.txt`` file in addition to your
``environment.yml``, your ``runtime.txt`` **will be ignored**.
``repo2docker`` support is best with Python 3.6, 3.5, and 2.7.
``requirements.txt``
~~~~~~~~~~~~~~~~~~~~
.. warning::
If you include a Python version in a ``runtime.txt`` file in addition to your
``environment.yml``, your ``runtime.txt`` **will be ignored**.
``requirements.txt`` - Install a Python environment
===================================================
This specifies a list of Python packages that should be installed in your
environment. Our
`requirements.txt example <https://github.com/binder-examples/requirements/blob/master/requirements.txt>`_
on GitHub shows a typical requirements file.
``REQUIRE``
~~~~~~~~~~~
This specifies a list of Julia packages. Repositories with a ``REQUIRE`` file
**must also contain an** ``environment.yml`` **file**. To see an example of a
Julia repository with ``REQUIRE`` and ``environment.yml``,
visit `binder-examples/julia-python <https://github.com/binder-examples/julia-python>`_.
``install.R``
~~~~~~~~~~~~~
This is used to install R libraries pinned to a specific snapshot on
`MRAN <https://mran.microsoft.com/documents/rro/reproducibility>`_.
To set the date of the snapshot add a runtime.txt_.
For an example ``install.R`` file, visit our `example install.R file <https://github.com/binder-examples/r/blob/master/install.R>`_.
``apt.txt``
~~~~~~~~~~~
A list of Debian packages that should be installed. The base image used is usually the latest released
version of Ubuntu.
We use ``apt.txt``, for example, to install LaTeX in our
`example apt.txt for LaTeX <https://github.com/binder-examples/latex/blob/master/apt.txt>`_.
``setup.py``
~~~~~~~~~~~~
``setup.py`` - Install Python packages
======================================
To install your repository like a Python package, you may include a
``setup.py`` file. repo2docker installs ``setup.py`` files by running
@ -109,10 +71,39 @@ While one can specify dependencies in ``setup.py``,
repo2docker **requires configuration files such as** ``environment.yml`` or
``requirements.txt`` to install dependencies during the build process.
``REQUIRE`` - Install a Julia environment
=========================================
This specifies a list of Julia packages. Repositories with a ``REQUIRE`` file
**must also contain an** ``environment.yml`` **file**. To see an example of a
Julia repository with ``REQUIRE`` and ``environment.yml``,
visit `binder-examples/julia-python <https://github.com/binder-examples/julia-python>`_.
``install.R`` - Install an R/RStudio environment
================================================
This is used to install R libraries pinned to a specific snapshot on
`MRAN <https://mran.microsoft.com/documents/rro/reproducibility>`_.
To set the date of the snapshot add a runtime.txt_.
For an example ``install.R`` file, visit our `example install.R file <https://github.com/binder-examples/r/blob/master/install.R>`_.
``apt.txt`` - Install packages with apt-get
===========================================
A list of Debian packages that should be installed. The base image used is usually the latest released
version of Ubuntu.
We use ``apt.txt``, for example, to install LaTeX in our
`example apt.txt for LaTeX <https://github.com/binder-examples/latex/blob/master/apt.txt>`_.
.. _postBuild:
``postBuild``
~~~~~~~~~~~~~
``postBuild`` - Run code after installing the environment
=========================================================
A script that can contain arbitrary commands to be run after the whole repository has been built. If you
want this to be a shell script, make sure the first line is ```#!/bin/bash``.
@ -121,10 +112,11 @@ An example use-case of ``postBuild`` file is JupyterLab's demo on mybinder.org.
It uses a ``postBuild`` file in a folder called ``binder`` to `prepare
their demo for binder <https://github.com/jupyterlab/jupyterlab-demo/blob/master/binder/postBuild>`_.
.. _start:
``start``
^^^^^^^^^
``start`` - Run code before the user sessions starts
====================================================
A script that can contain simple commands to be run at runtime (as an
`ENTRYPOINT <https://docs.docker.com/engine/reference/builder/#entrypoint>`
@ -138,13 +130,14 @@ should at most take a few seconds to run.
If you only need to run things once during the build phase use :ref:`postBuild`.
.. TODO: Discuss runtime limits, best practices, etc.
Also, point to an example.
.. _runtime.txt:
``runtime.txt``
~~~~~~~~~~~~~~~
``runtime.txt`` - Specifying runtimes
=====================================
This allows you to control the runtime of Python or R.
@ -163,3 +156,17 @@ used for installing libraries.
To see an example R repository, visit our `R
example in binder-examples <https://github.com/binder-examples/r/blob/master/runtime.txt>`_.
``Dockerfile`` - Advanced environments
======================================
In the majority of cases, providing your own Dockerfile is not necessary as the base
images provide core functionality, compact image sizes, and efficient builds. We recommend
trying the other configuration files before deciding to use your own Dockerfile.
With Dockerfiles, a regular Docker build will be performed.
**If a Dockerfile is present, all other configuration files will be ignored.**
See the `Binder Documentation <https://mybinder.readthedocs.io/en/latest/dockerfile.html>`_ for
best-practices with Dockerfiles.

Wyświetl plik

@ -0,0 +1,8 @@
.. _examples:
======================================
Example repositories using repo2docker
======================================
This page contains links to example repositories for each of the repo2docker
configuration files.

Wyświetl plik

@ -1,26 +1,38 @@
jupyter-repo2docker
===================
**jupyter-repo2docker** is a tool to build, run, and push Docker
images from source code repositories. repo2docker fetches a repository
``jupyter-repo2docker`` is a tool to **build, run, and push Docker
images from source code repositories** that run via a Jupyter server.
``repo2docker`` fetches a repository
(e.g., from GitHub or other locations) and builds a container image
based on the configuration files found in the repository. It can be
used to explore a repository locally by building and executing the
constructed image of the repository.
constructed image of the repository, or as a means of building images that
are pushed to a Docker registry.
Please report `Bugs <https://github.com/jupyter/repo2docker/issues>`_,
`ask questions <https://gitter.im/jupyterhub/binder>`_ or
`contribute to the project <https://github.com/jupyter/repo2docker/blob/master/CONTRIBUTING.md>`_.
Site Contents
-------------
.. toctree::
:maxdepth: 1
:maxdepth: 2
:caption: Getting started with repo2docker
install
usage
.. toctree::
:maxdepth: 2
:caption: Configuration files and examples
config_files
examples
.. toctree::
:maxdepth: 1
:caption: Advanced and developer information
faq
deploy
design

Wyświetl plik

@ -6,7 +6,7 @@ Installing ``repo2docker``
repo2docker requires Python 3.4 and above on Linux and macOS. See
:ref:`below <windows>` for more information about Windows support.
Prerequisite: docker
Prerequisite: Docker
--------------------
Install `Docker <https://www.docker.com>`_ as it is required
@ -56,28 +56,3 @@ Bash on Windows) provides additional information about Windows and docker.
.. _using Windows and the WSL: https://nickjanetakis.com/blog/setting-up-docker-for-windows-and-wsl-to-work-flawlessly
.. _jupyterhub_ready:
JupyterHub-ready images
-----------------------
JupyterHub_ allows multiple
users to collaborate on a shared Jupyter server. ``repo2docker`` can build
Docker images that can be shared within a JupyterHub deployment. For example,
`mybinder.org <https://mybinder.org>`_ uses JupyterHub and ``repo2docker``
to allow anyone to build a Docker image of a git repository online and
share an executable version of the repository with a URL to the built image.
To build JupyterHub_-ready Docker images with ``repo2docker``, the
version of your JupterHub deployment must be included in the
``environment.yml`` or ``requirements.txt`` of the git repositories you
build.
If your instance of JupyterHub uses ``DockerSpawner``, you will need to set its
command to run ``jupyterhub-singleuser`` by adding this line in your
configuration file::
c.DockerSpawner.cmd = ['jupyterhub-singleuser']
.. _JupyterHub: https://github.com/jupyterhub/jupyterhub

Wyświetl plik

@ -1,60 +1,50 @@
.. _usage:
=====================
Using ``repo2docker``
=====================
`Docker <https://docs.docker.com/>`_ **must be running** in
order to run ``repo2docker``. For more information on installing
``repo2docker``, see :ref:`install`.
.. note::
``repo2docker`` performs two steps:
`Docker <https://docs.docker.com/>`_ **must be running** in
order to run ``repo2docker``. For more information on installing
``repo2docker``, see :ref:`install`.
1. builds a Docker image from a git repo
2. runs a Jupyter server within the image to explore the repository
``repo2docker`` is called with a URL/path to a git repository. It then
performs these steps:
.. To ensure you can run the software in your repository, you must
1. Inspects the repository for :ref:`configuration files <config-files>`. These will be used to build
the environment needed to run the repository.
2. Builds a Docker image with an environment specified in these :ref:`configuration files <config-files>`.
3. Runs a Jupyter server within the image that lets you explore the
repository interactively (optional)
4. Pushes the images to a Docker registry so that it may be accessed remotely
(optional)
Calling repo2docker
===================
repo2docker is called with this command::
jupyter-repo2docker <URL-or-path to repository>
where ``<URL-or-path to repository>`` is a URL or path to the source repository.
where ``<URL-or-path to repository>`` is a URL or path to the source repository
for which you'd like to build an image.
For example, use the following to build an image of Peter Norvig's
Pytudes_::
For example, the following command will build an image of Peter Norvig's
Pytudes_ repository::
jupyter-repo2docker https://github.com/norvig/pytudes
To build a particular branch and commit, use the argument ``--ref`` to
specify the ``branch-name`` or ``commit-hash``::
jupyter-repo2docker https://github.com/norvig/pytudes --ref 9ced85dd9a84859d0767369e58f33912a214a3cf
.. tip::
For reproducible research, we recommend specifying a commit-hash to
deterministically build a fixed version of a repository. Not specifying a
commit-hash will result in the latest commit of the repository being built.
Building the image may take a few minutes.
During building, ``repo2docker``
clones the repository to obtain its contents and inspects the repository for
:ref:`configuration files <config-files>`.
By default, ``repo2docker`` will assume you are using
Python 3.6 unless you include the version of Python in your
:ref:`configuration files <config-files>`. ``repo2docker`` support is best with
Python 2.7, 3.5, and 3.6. In the case of this repository, a Python version is not
specified in their configuration files and Python 3.6 is installed.
Pytudes_
uses a `requirements.txt file <https://github.com/norvig/pytudes/blob/master/requirements.txt>`_
to specify its Python environment. ``repo2docker`` uses ``pip`` to install
dependencies listed in the ``requirement.txt`` in the image. To learn more about
to specify its Python environment. Because of this, ``repo2docker`` will use
``pip`` to install dependencies listed in this ``requirement.txt`` file, and
these will be present in the generated Docker image. To learn more about
configuration files in ``repo2docker`` visit :ref:`config-files`.
.. _Pytudes: https://github.com/norvig/pytudes
When the image is built, a message will be output to your terminal::
Copy/paste this URL into your browser when you connect for the first time,
@ -64,25 +54,82 @@ When the image is built, a message will be output to your terminal::
Pasting the URL into your browser will open Jupyter Notebook with the
dependencies and contents of the source repository in the built image.
Because JupyterLab is a server extension of the classic Jupyter Notebook server,
you can launch JupyterLab by opening Jupyter Notebook and visiting the
```/lab`` to the end of the URL:
Building a specific branch / commit / tag
=========================================
To build a particular branch and commit, use the argument ``--ref`` and
specify the ``branch-name`` or ``commit-hash``. For example::
jupyter-repo2docker https://github.com/norvig/pytudes --ref 9ced85dd9a84859d0767369e58f33912a214a3cf
.. tip::
For reproducible research, we recommend specifying a commit-hash to
deterministically build a fixed version of a repository. Not specifying a
commit-hash will result in the latest commit of the repository being built.
Where to put configuration files
================================
``repo2docker`` will look for configuration files in either:
* A folder named ``binder/`` in the root of the repository.
* The root directory of the repository.
If the folder ``binder/`` is located at the top level of the repository,
**only configuration files in the** ``binder/`` **folder will be considered**.
.. note::
``repo2docker`` builds an environment with Python 3.6 by default. If you'd
like a different version, you can specify this in your
:ref:`configuration files <config-files>`.
.. _user_interface:
Specifying the user interface
=============================
repo2docker can build several user interfaces into the resulting Docker image.
This is controlled with various :ref:`configuration files <config-files>`.
JupyterLab
----------
repo2docker does not need any extra configuration in order to allow the use
of the JupyterLab interface. You can launch JupyterLab from within a repo2docker
image by opening the Jupyter Notebook and appending ``/lab`` to the end of the URL
like so:
.. code-block:: none
http(s)://<server:port>/<lab-location>/lab
http(s)://<server:port>/lab
To switch back to the classic notebook, add ``/tree`` to the URL:
To switch back to the classic notebook, add ``/tree`` to the URL like so:
.. code-block:: none
http(s)://<server:port>/<lab-location>/tree
http(s)://<server:port>/tree
To learn more about URLs in JupyterLab and Jupyter Notebook, visit
`starting JupyterLab <http://jupyterlab.readthedocs.io/en/latest/getting_started/starting.html>`_.
``--debug`` and ``--no-build``
------------------------------
RStudio
-------
The RStudio user interface is enabled when repo2docker detects a configuration
for R (an R version specified in ``runtime.txt``). If this is detected,
RStudio will be accessible by appending ``/rstudio`` to the URL, like so:
.. code-block:: none
http(s)://<server:port>/rstudio
Debugging repo2docker with ``--debug`` and ``--no-build``
=========================================================
To debug the docker image being built, pass the ``--debug`` parameter:
@ -100,3 +147,28 @@ be used by docker directly.
.. code-block:: bash
jupyter-repo2docker --no-build --debug https://github.com/norvig/pytudes
Building JupyterHub-ready images
================================
JupyterHub_ allows multiple
users to collaborate on a shared Jupyter server. ``repo2docker`` can build
Docker images that can be shared within a JupyterHub deployment. For example,
`mybinder.org <https://mybinder.org>`_ uses JupyterHub and ``repo2docker``
to allow anyone to build a Docker image of a git repository online and
share an executable version of the repository with a URL to the built image.
To build JupyterHub_-ready Docker images with ``repo2docker``, the
version of your JupterHub deployment must be included in the
``environment.yml`` or ``requirements.txt`` of the git repositories you
build.
If your instance of JupyterHub uses ``DockerSpawner``, you will need to set its
command to run ``jupyterhub-singleuser`` by adding this line in your
configuration file::
c.DockerSpawner.cmd = ['jupyterhub-singleuser']
.. _JupyterHub: https://github.com/jupyterhub/jupyterhub
.. _Pytudes: https://github.com/norvig/pytudes