From a4063e3d8db99a4724105a2cb6de7e4c0489c7df Mon Sep 17 00:00:00 2001 From: Tim Head Date: Sat, 2 Mar 2019 08:40:35 +0100 Subject: [PATCH] Add FAQ entry and Howto on exporting conda envs --- docs/source/faq.rst | 30 +++++++++++++- docs/source/howto/export_environment.rst | 50 ++++++++++++++++++++++++ docs/source/index.rst | 1 + 3 files changed, 80 insertions(+), 1 deletion(-) create mode 100644 docs/source/howto/export_environment.rst diff --git a/docs/source/faq.rst b/docs/source/faq.rst index 9fbfcfbb..4d676415 100644 --- a/docs/source/faq.rst +++ b/docs/source/faq.rst @@ -55,6 +55,34 @@ R Only R 3.4.4 is currently supported, which is installed via ``apt`` from the `ubuntu bionic repository `_. + +Why is my repository is failing to build with ``ResolvePackageNotFound`` ? +-------------------------------------------------------------------------- + +If you used ``conda env export`` to generate your ``environment.yml`` it will +generate a list of packages and versions of packages that is pinned to platform +specific versions. These very specific versions are not available in the linux +docker image used by ``repo2docker``. A typical error message will look like +the following:: + + Step 39/44 : RUN conda env update -n root -f "environment.yml" && conda clean -tipsy && conda list -n root + ---> Running in ebe9a67762e4 + Solving environment: ...working... failed + + ResolvePackageNotFound: + - jsonschema==2.6.0=py36hb385e00_0 + - libedit==3.1.20181209=hb402a30_0 + - tornado==5.1.1=py36h1de35cc_0 + ... + +We recommend to use ``conda env export --no-builds -f environment.yml`` to export +your environment and then edit the file by hand to remove platform specific +packages like ``appnope``. + +See :ref:`export-environment` for a recipe on how to create strict exports of +your environment that will work with ``repo2docker``. + + Can I add executable files to the user's PATH? ---------------------------------------------- @@ -98,7 +126,7 @@ Yes: use the ``--editable`` or ``-E`` flag (don't confuse this with the ``-e`` flag for environment variables), and run repo2docker on a local repository:: - repo2docker -E my-repository/. + repo2docker -E my-repository/ This builds a Docker container from the files in that repository (using, for example, a ``requirements.txt`` or ``install.R`` file), diff --git a/docs/source/howto/export_environment.rst b/docs/source/howto/export_environment.rst new file mode 100644 index 00000000..b38a84c0 --- /dev/null +++ b/docs/source/howto/export_environment.rst @@ -0,0 +1,50 @@ +.. _export-environment: + +============================================================================= +How to automatically create a ``environment.yml`` that works with repo2docker +============================================================================= + +This how-to explains how to create a ``environment.yml`` that specifies all +installed packages and their precise versions from your environment. + + +The challenge +============= + +``conda env export -f environment.yml`` creates a strict export of all packages. +This is the most robust for reproducibility, but it does bake in potential +platform-specific packages, so you can only use an exported environment on the +same platform. + +``repo2docker`` uses a linux based image as the starting point for every docker +image it creates. However a lot of people use OSX or Windows as their day to +day operating system. This means that the ``environment.yml`` created by a strict +export will not work with error messages saying that certain packages can not +be resolved (``ResolvePackageNotFound``). + + +The solution +============ + +Follow this procedure to create a strict export of your environment that will +work with ``repo2docker`` and sites like `mybinder.org `_. + +We will launch a terminal inside a basic docker image, install the packages +you need and then perform a strict export of the environment. + +#. install repo2docker on your computer by following :ref:`install` +#. in a terminal launch a basic repository + ``repo2docker https://github.com/binder-examples/conda-freeze`` + inside repo2docker +#. open the URL printed at the end in a browser, the URL should look like + ``http://127.0.0.1:61037/?token=30e61ec80bda6dd0d14805ea76bb59e7b0cd78b5d6b436f0`` +#. open a terminal by clicking "New -> Terminal" next to the "Upload" button on the + right hand side of the webpage +#. install the packages your project requires with ``conda install `` +#. use ``conda env export -n root`` to print the environment +#. copy and paste the environment you just printed into a ``environment.yml`` in + your projects repository +#. close your browser tabs and exit the repo2docker session by pressing Ctrl-C. + +This will give you a strict export of your environment that precisely pins the +versions of packages in your environment based on a linux environment. diff --git a/docs/source/index.rst b/docs/source/index.rst index b3e598c6..f150f352 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -32,6 +32,7 @@ Please report `Bugs `_, howto/user_interface howto/languages + howto/export_environment howto/lab_workspaces howto/jupyterhub_images howto/deploy