mirror
/
repo2docker
kopia lustrzana https://github.com/jupyterhub/repo2docker
1
0
Forkuj 0
Kod Zgłoszenia Projekty Wydania Wiki Aktywność

Merge branch 'master' into change-julia-env-handling

pull/612/head
David Anthoff 2019-03-21 12:49:17 -07:00
rodzic d75333e00f e921993098
commit 7105a8a3ef
12 zmienionych plików z 435 dodań i 125 usunięć
Pokaż statystyki Ściągnij plik aktualizacji Ściągnij plik porównania Expand all files Collapse all files

2
README.md
Wyświetl plik

@ -9,6 +9,8 @@ the configuration files found in the repository.
See the [repo2docker documentation](http://repo2docker.readthedocs.io)
for more information on using repo2docker.
For support questions please search or post to https://discourse.jupyter.org/c/binder.
See the [contributing guide](CONTRIBUTING.md) for information on contributing to
repo2docker.

13
docs/issue_template.md 100644
Wyświetl plik

@ -0,0 +1,13 @@
<!--
Welcome!
Help us stay on top of things (and get faster replies) by submitting them to
the right place:
* for support questions please use https://discourse.jupyter.org/c/binder
* repo2docker development related issues https://github.com/jupyter/repo2docker/issues/new
* if the https://mybinder.org site is down or has inaccessible pages: https://github.com/jupyterhub/mybinder.org-deploy/issues/new
If you aren't sure where to go use https://discourse.jupyter.org/c/binder.
-->

2
docs/source/changelog.rst
Wyświetl plik

@ -17,6 +17,8 @@ API changes
Bug fixes
---------
- Install IJulia kernel into ${NB_PYTHON_PREFIX}/share/jupyter in :pr:`622` by
:user:`davidanthoff`.
Version 0.8.0

4
docs/source/config_files.rst
Wyświetl plik

@ -32,6 +32,10 @@ Below is a list of supported configuration files (roughly in the order of build
``environment.yml`` is the standard configuration file used by `conda <https://conda.io>`_
that lets you install any kind of package,
including Python, R, and C/C++ packages.
``repo2docker`` does not use ``environment.yml`` to create and activate a new conda environment.
Rather, it updates a base conda environment with the packages listed in ``environment.yml``.
This means that the environment will always have the same default name, not the name
specified in ``environment.yml``.
.. note::

16
docs/source/faq.rst
Wyświetl plik

@ -53,6 +53,8 @@ Only R 3.4.4 is currently supported, which is installed via ``apt`` from the
`ubuntu bionic repository <https://packages.ubuntu.com/bionic/r-base>`_.
Why is my repository is failing to build with ``ResolvePackageNotFound`` ?
--------------------------------------------------------------------------
@ -147,3 +149,17 @@ notebooks (among others).
the container, use the ``--volumes`` option instead. Similarly,
for a fully customized user Dockerfile, this option is not
guaranteed to work.
Why is my R shiny app not launching?
----------------------------------------------------------------------------------
If you are trying to run an R shiny app using the ``/shiny/folder_containing_shiny``
url option, but the launch returns "The application exited during initialization.",
there might be something wrong with the specification of the app. One way of debugging
the app in the container is by running the ``rstudio`` url, open either the ui or
server file for the app, and run the app in the container rstudio. This way you can
see the rstudio logs as it tries to initialise the shiny app. If you a missing a
package or other dependency for the container, this will be obvious at this stage.

6
docs/source/howto/languages.rst
Wyświetl plik

@ -73,10 +73,10 @@ Julia
To build an environment with Julia, include a configuration file called
``Project.toml``. The format of this file is documented at
`the Julia Pkg.jl documentation <https://julialang.github.io/Pkg.jl/stable/>`_.
`the Julia Pkg.jl documentation <https://julialang.github.io/Pkg.jl/v1/>`_.
To specify a specific version of Julia to install, put a Julia version in the
``Compat`` section of the ``Project.toml`` file, as described
here: https://julialang.github.io/Pkg.jl/stable/compatibility.
``[compat]`` section of the ``Project.toml`` file, as described
here: https://julialang.github.io/Pkg.jl/v1/compatibility/.
Languages not covered here
==========================

7
repo2docker/buildpacks/julia/julia_project.py
Wyświetl plik

@ -2,7 +2,7 @@
import os
import toml
from ..python import PythonBuildPack
from .julia_semver import find_semver_match
from .semver import find_semver_match
class JuliaProjectTomlBuildPack(PythonBuildPack):
"""
@ -61,7 +61,8 @@ class JuliaProjectTomlBuildPack(PythonBuildPack):
('JULIA_PATH', '${APP_BASE}/julia'),
('JULIA_DEPOT_PATH', '${JULIA_PATH}/pkg'),
('JULIA_VERSION', self.julia_version),
('JUPYTER', '${NB_PYTHON_PREFIX}/bin/jupyter')
('JUPYTER', '${NB_PYTHON_PREFIX}/bin/jupyter'),
('JUPYTER_DATA_DIR', '${NB_PYTHON_PREFIX}/share/jupyter'),
]
def get_env(self):
@ -122,7 +123,7 @@ class JuliaProjectTomlBuildPack(PythonBuildPack):
(
"${NB_USER}",
r"""
JULIA_PROJECT="" julia -e "using Pkg; Pkg.add(\"IJulia\"); using IJulia; installkernel(\"Julia\", env=Dict(\"JUPYTER_DATA_DIR\"=>\"${NB_PYTHON_PREFIX}/share/jupyter\"));" && \
JULIA_PROJECT="" julia -e "using Pkg; Pkg.add(\"IJulia\"); using IJulia; installkernel(\"Julia\", \"--project=${REPO_DIR}\");" && \
julia --project=${REPO_DIR} -e 'using Pkg; Pkg.instantiate(); pkg"precompile"'
"""
)

119
repo2docker/buildpacks/julia/julia_semver.py
Wyświetl plik

@ -1,119 +0,0 @@
# This file implements the julia-specific logic for handling SemVer (Semantic
# Versioning) strings in .toml files.
#
# It uses the python "semver" package to do most version string comparisons, but
# the places where julia's SemVer handling differs from the semver package have
# been implemented directly.
#
# Here, we use tuples to represent a Version, and functors as "matchers". The matcher
# functors take a version string and return true if it passes its constraints.
import re
import semver
# Main algorithm:
# Create an AbstractMatcher instance from the constraint string, and check it
# against each version in the versions_list, returning the first success.
def find_semver_match(constraint, versions_list):
matcher = create_semver_matcher(constraint)
for vstr in reversed(versions_list):
if matcher.match(str_to_version(vstr)):
return vstr
return None
def str_to_version(vstr):
return tuple([int(n) for n in vstr.split('.')])
# --- Matcher interface -------------------------------------------
class AbstractMatcher:
def match(self, v):
pass
class SemverMatcher(AbstractMatcher):
""" Match a version tuple to a given constraint_str using the `semver` package. """
def __init__(self, constraint_str):
self.constraint_str = constraint_str
def match(self, v):
while len(v) < 3:
v = v+(0,)
v_str = '.'.join(map(str, v))
return semver.match(v_str, self.constraint_str)
def __repr__(self):
return self.constraint_str
# --- Custom matcher for julia-specific SemVer handling: ---------
from enum import Enum
class Exclusivity(Enum):
EXCLUSIVE = 1
INCLUSIVE = 2
class VersionRange(AbstractMatcher):
""" Match a version tuple between lower and upper bounds. """
def __init__(self, lower, upper, upper_exclusivity):
self.lower = lower
self.upper = upper
self.upper_exclusivity = upper_exclusivity
def match(self, v):
if self.upper_exclusivity == Exclusivity.EXCLUSIVE:
return self.lower <= v < self.upper
else:
return self.lower <= v <= self.upper
def __repr__(self):
return ("["+".".join(map(str, self.lower)) +"-"+ ".".join(map(str, self.upper)) +
(")" if self.upper_exclusivity == Exclusivity.EXCLUSIVE else "]"))
# Helpers
def major(v): return v[0]
def minor(v): return v[1] if len(v) >= 2 else 0
def patch(v): return v[2] if len(v) >= 3 else 0
# --- main constraint parser function ------------------------------------
def create_semver_matcher(constraint_str):
"""
Returns a derived-class instance of AbstractMatcher that matches version
tuples against the provided constraint_str.
"""
constraint_str = constraint_str.strip()
first_digit = re.search(r"\d", constraint_str)
if not first_digit:
# Invalid version string (no numbers in it)
return ""
constraint = str_to_version(constraint_str[first_digit.start():])
comparison_symbol = constraint_str[0:first_digit.start()].strip()
# Default to "^" search if no matching mode specified (up to next major version)
if (first_digit.start() == 0) or (comparison_symbol == "^"):
if major(constraint) == 0:
# Also, julia treats pre-1.0 releases specially, as if the first
# non-zero number is actually a major number:
# https://docs.julialang.org/en/latest/stdlib/Pkg/#Caret-specifiers-1
# So we need to handle it separately by bumping the first non-zero number.
for i,n in enumerate(constraint):
if n != 0 or i == len(constraint)-1: # (using the last existing number handles situations like "^0.0" or "^0")
upper = constraint[0:i] + (n+1,)
break
return VersionRange(constraint, upper, Exclusivity.EXCLUSIVE)
else:
return VersionRange(constraint, (major(constraint)+1,), Exclusivity.EXCLUSIVE)
# '~' matching (only allowed to bump the last present number by one)
if (comparison_symbol == "~"):
return VersionRange(constraint, constraint[:-1] +(constraint[-1]+1,), Exclusivity.INCLUSIVE)
# Use semver package's comparisons for everything else:
# semver requires three version numbers
if len(constraint) < 3:
while len(constraint) < 3:
constraint = constraint+(0,)
constraint_str = constraint_str[0:first_digit.start()] + ".".join(map(str, constraint))
# Convert special comparison strings to format accepted by `semver` library.
constraint_str = constraint_str.replace("≥", ">=").replace("≤", "<=")
constraint_str = re.sub(r"(^|\b)=\b", "==", constraint_str)
return SemverMatcher(constraint_str)

165
repo2docker/buildpacks/julia/semver.py 100644
Wyświetl plik

@ -0,0 +1,165 @@
"""
Julia specific handling of SemVer strings
It uses the python "semver" package to do most version string comparisons, but
the places where julia's SemVer handling differs from the semver package have
been implemented directly.
We use tuples to represent a Version, and functors as "matchers". The
matcher functors take a version string and return True if it passes its
constraints.
"""
import re
import semver
def find_semver_match(constraint, versions_list):
matcher = create_semver_matcher(constraint)
for vstr in reversed(versions_list):
if matcher.match(str_to_version(vstr)):
return vstr
return None
def str_to_version(vstr):
return tuple([int(n) for n in vstr.split(".")])
# Helpers
def major(v):
return v[0]
def minor(v):
return v[1] if len(v) >= 2 else 0
def patch(v):
return v[2] if len(v) >= 3 else 0
def create_semver_matcher(constraint_str):
"""Create a matcher that can be used to match version tuples
Version tuples are matched against the provided regex `constraint_str`.
"""
constraint_str = constraint_str.strip()
first_digit = re.search(r"\d", constraint_str)
if not first_digit:
# Invalid version string (no numbers in it)
return ""
constraint = str_to_version(constraint_str[first_digit.start() :])
comparison_symbol = constraint_str[0 : first_digit.start()].strip()
# Default to "^" search if no matching mode specified (up to next major version)
if (first_digit.start() == 0) or (comparison_symbol == "^"):
if major(constraint) == 0:
# Also, julia treats pre-1.0 releases specially, as if the first
# non-zero number is actually a major number:
# https://docs.julialang.org/en/latest/stdlib/Pkg/#Caret-specifiers-1
# So we need to handle it separately by bumping the first non-zero
# enumber.
for i, n in enumerate(constraint):
if (
n != 0 or i == len(constraint) - 1
): # (using the last existing number handles situations like "^0.0" or "^0")
upper = constraint[0:i] + (n + 1,)
break
return VersionRange(constraint, upper, True)
else:
return VersionRange(
constraint, (major(constraint) + 1,), True
)
# '~' matching (only allowed to bump the last present number by one)
if comparison_symbol == "~":
return VersionRange(
constraint,
constraint[:-1] + (constraint[-1] + 1,),
exclusive=False
)
# Use semver package's comparisons for everything else:
# semver requires three version numbers
if len(constraint) < 3:
while len(constraint) < 3:
constraint = constraint + (0,)
constraint_str = constraint_str[0 : first_digit.start()] + ".".join(
map(str, constraint)
)
# Convert special comparison strings to format accepted by `semver` library.
constraint_str = constraint_str.replace("≥", ">=").replace("≤", "<=")
constraint_str = re.sub(r"(^|\b)=\b", "==", constraint_str)
return SemverMatcher(constraint_str)
class SemverMatcher:
"""Provides a utility for using `semver` package to do version matching.
The `SemverMatcher` takes a `constraint_str` to represent a regex to
determine if a version tuple matches the constraint.
The matching is handled via the `semver` package.
"""
def __init__(self, constraint_str):
self.constraint_str = constraint_str
def match(self, v):
"""Check if `v` matches the constraint"""
while len(v) < 3:
v = v + (0,)
v_str = ".".join(map(str, v))
return semver.match(v_str, self.constraint_str)
def __eq__(self, rhs):
return self.constraint_str == rhs.constraint_str
def __repr__(self):
return self.constraint_str
class VersionRange:
"""Represents a range of release versions.
A `VersionRange` contains versions from a `lower` to `upper` bound
which may be inclusive (default: `exclusive=False`) or exclusive (`exclusive=True`).
A release version (represented by a tuple) can be checked to see if it
falls within a `VersionRange`
"""
def __init__(self, lower, upper, exclusive=False):
self.lower = lower
self.upper = upper
self.exclusive = exclusive
def match(self, v):
"""Check if `v` falls into the version range"""
if self.exclusive:
return self.lower <= v < self.upper
else:
return self.lower <= v <= self.upper
def __eq__(self, rhs):
return (
self.lower == rhs.lower
and self.upper == rhs.upper
and self.exclusive == rhs.exclusive
)
def __repr__(self):
return (
"["
+ ".".join(map(str, self.lower))
+ "-"
+ ".".join(map(str, self.upper))
+ (")" if self.exclusive else "]")
)

8
tests/julia/julia_version-1.0.2/verify
Wyświetl plik

@ -14,4 +14,12 @@ catch
exit(1)
end
# Verify that kernels are not installed in home directory (issue #620)
try
using IJulia
@assert IJulia.kerneldir() == "/srv/conda/share/jupyter/kernels"
catch
exit(1)
end
exit(0)

8
tests/julia/julia_version-default/verify
Wyświetl plik

@ -14,4 +14,12 @@ catch
exit(1)
end
# Verify that kernels are not installed in home directory (issue #620)
try
using IJulia
@assert IJulia.kerneldir() == "/srv/conda/share/jupyter/kernels"
catch
exit(1)
end
exit(0)

210
tests/unit/test_semver.py 100644
Wyświetl plik

@ -0,0 +1,210 @@
from repo2docker.buildpacks.julia import semver
def test_str_to_version():
assert semver.str_to_version("1.5.2") == (1, 5, 2)
assert semver.str_to_version("1") == (1,)
def test_major_minor_patch():
V = (1, 2, 3)
assert (semver.major(V), semver.minor(V), semver.patch(V)) == (1, 2, 3)
assert semver.major((1,)) == 1
assert semver.minor((1,)) == 0
assert semver.patch((1,)) == 0
def test_simple_matches():
assert repr(semver.create_semver_matcher("1.2.5")) == "[1.2.5-2)"
assert repr(semver.create_semver_matcher("1.2.5")) == "[1.2.5-2)"
assert repr(semver.create_semver_matcher("^1.2.3")) == "[1.2.3-2)"
assert repr(semver.create_semver_matcher("^1.2")) == "[1.2-2)"
assert repr(semver.create_semver_matcher("^1")) == "[1-2)"
assert repr(semver.create_semver_matcher("^0.2.3")) == "[0.2.3-0.3)"
assert repr(semver.create_semver_matcher("^0.0.3")) == "[0.0.3-0.0.4)"
assert repr(semver.create_semver_matcher("^0.0")) == "[0.0-0.1)"
assert repr(semver.create_semver_matcher("^0")) == "[0-1)"
# This one seems wrong: `~1.2.3 = [1.2.3, 1.2.4)` but ~ is special in Julia
# from https://docs.julialang.org/en/latest/stdlib/Pkg/#Tilde-specifiers-1
assert repr(semver.create_semver_matcher("~1.2.3")) == "[1.2.3-1.2.4]"
assert repr(semver.create_semver_matcher("~1.3.5")) == "[1.3.5-1.3.6]"
assert repr(semver.create_semver_matcher("~1.2")) == "[1.2-1.3]"
assert repr(semver.create_semver_matcher("~1")) == "[1-2]"
def test_range_matches():
assert semver.create_semver_matcher(
"1.2.3"
) == semver.create_semver_matcher("^1.2.3")
assert semver.create_semver_matcher(
"1.2.3"
) == semver.create_semver_matcher("^1.2.3")
assert semver.create_semver_matcher("1.2") == semver.create_semver_matcher(
"^1.2"
)
assert semver.create_semver_matcher("1") == semver.create_semver_matcher(
"^1"
)
assert semver.create_semver_matcher(
"0.0.3"
) == semver.create_semver_matcher("^0.0.3")
assert semver.create_semver_matcher("0") == semver.create_semver_matcher(
"^0"
)
def test_match_particular_version():
assert semver.create_semver_matcher("1.2.3").match(
semver.str_to_version("1.5.2")
)
assert semver.create_semver_matcher("1.2.3").match(
semver.str_to_version("1.2.3")
)
assert (
semver.create_semver_matcher("1.2.3").match(
semver.str_to_version("2.0.0")
)
== False
)
assert (
semver.create_semver_matcher("1.2.3").match(
semver.str_to_version("1.2.2")
)
== False
)
assert semver.create_semver_matcher("~1.2.3").match(
semver.str_to_version("1.2.4")
)
assert semver.create_semver_matcher("~1.2.3").match(
semver.str_to_version("1.2.3")
)
assert (
semver.create_semver_matcher("~1.2.3").match(
semver.str_to_version("1.3")
)
== False
)
assert semver.create_semver_matcher("1.2").match(
semver.str_to_version("1.2.0")
)
assert semver.create_semver_matcher("1.2").match(
semver.str_to_version("1.9.9")
)
assert (
semver.create_semver_matcher("1.2").match(
semver.str_to_version("2.0.0")
)
== False
)
assert (
semver.create_semver_matcher("1.2").match(
semver.str_to_version("1.1.9")
)
== False
)
assert semver.create_semver_matcher("0.2.3").match(
semver.str_to_version("0.2.3")
)
assert (
semver.create_semver_matcher("0.2.3").match(
semver.str_to_version("0.3.0")
)
== False
)
assert (
semver.create_semver_matcher("0.2.3").match(
semver.str_to_version("0.2.2")
)
== False
)
assert semver.create_semver_matcher("0").match(
semver.str_to_version("0.0.0")
)
assert semver.create_semver_matcher("0").match(
semver.str_to_version("0.99.0")
)
assert (
semver.create_semver_matcher("0").match(semver.str_to_version("1.0.0"))
== False
)
assert semver.create_semver_matcher("0.0").match(
semver.str_to_version("0.0.0")
)
assert semver.create_semver_matcher("0.0").match(
semver.str_to_version("0.0.99")
)
assert (
semver.create_semver_matcher("0.0").match(
semver.str_to_version("0.1.0")
)
== False
)
def test_less_than_prefix():
assert repr(semver.create_semver_matcher("<1.2.3")) == "<1.2.3"
assert repr(semver.create_semver_matcher("<1")) == "<1.0.0"
assert repr(semver.create_semver_matcher("<0.2.3")) == "<0.2.3"
assert semver.create_semver_matcher("<2.0.3").match(
semver.str_to_version("2.0.2")
)
assert semver.create_semver_matcher("<2").match(
semver.str_to_version("0.0.1")
)
assert semver.create_semver_matcher("<2.0.3").match(
semver.str_to_version("0.2.3")
)
assert (
semver.create_semver_matcher("<0.2.4").match(
semver.str_to_version("0.2.4")
)
== False
)
def test_equal_prefix():
assert repr(semver.create_semver_matcher("=1.2.3")) == "==1.2.3"
assert repr(semver.create_semver_matcher("=1.2")) == "==1.2.0"
assert repr(semver.create_semver_matcher(" =1")) == "==1.0.0"
assert semver.create_semver_matcher("=1.2.3").match(
semver.str_to_version("1.2.3")
)
assert (
semver.create_semver_matcher("=1.2.3").match(
semver.str_to_version("1.2.4")
)
== False
)
assert (
semver.create_semver_matcher("=1.2.3").match(
semver.str_to_version("1.2.2")
)
== False
)
def test_fancy_unicode():
assert semver.create_semver_matcher(
"≥1.3.0"
) == semver.create_semver_matcher(">=1.3.0")
def test_largerthan_equal():
assert repr(semver.create_semver_matcher(">= 1.2.3")) == ">= 1.2.3"
assert repr(semver.create_semver_matcher(" >= 1")) == ">= 1.0.0"
assert semver.create_semver_matcher(">=1").match(
semver.str_to_version("1.0.0")
)
assert semver.create_semver_matcher(">=0").match(
semver.str_to_version("0.0.1")
)
assert semver.create_semver_matcher(">=1.2.3").match(
semver.str_to_version("1.2.3")
)
assert (
semver.create_semver_matcher(">=1.2.3").match(
semver.str_to_version("1.2.2")
)
== False
)