The Docker container executes a [`start-notebook.sh` script](./start-notebook.sh) script by default. The `start-notebook.sh` script handles the `NB_UID`, `NB_GID` and `GRANT_SUDO` features documented in the next section, and then executes the `jupyter notebook`.
The Docker container executes a `start-notebook.sh` script script by default. The `start-notebook.sh` script handles the `NB_UID`, `NB_GID` and `GRANT_SUDO` features documented in the next section, and then executes the `jupyter notebook`.
You can pass [Jupyter command line options](https://jupyter.readthedocs.io/en/latest/projects/jupyter-command.html) through the `start-notebook.sh` script when launching the container. For example, to secure the Notebook server with a custom password hashed using `IPython.lib.passwd()` instead of the default token, run the following:
You can pass [Jupyter command line options](https://jupyter.readthedocs.io/en/latest/projects/jupyter-command.html) through the `start-notebook.sh` script when launching the container. For example, to secure the Notebook server with a custom password hashed using `IPython.lib.passwd()` instead of the default token, run the following:
...
@@ -101,9 +101,9 @@ You can bypass the provided scripts and specify your an arbitrary start command.
...
@@ -101,9 +101,9 @@ You can bypass the provided scripts and specify your an arbitrary start command.
## Image Specifics
## Image Specifics
## Spark and PySpark
### Spark and PySpark
### Using Spark Local Mode
#### Using Spark Local Mode
This configuration is nice for using Spark on small, local data.
This configuration is nice for using Spark on small, local data.
...
@@ -166,9 +166,9 @@ To use Python 2 in the notebook and on the workers, change the `PYSPARK_PYTHON`
...
@@ -166,9 +166,9 @@ To use Python 2 in the notebook and on the workers, change the `PYSPARK_PYTHON`
Of course, all of this can be hidden in an [IPython kernel startup script](http://ipython.org/ipython-doc/stable/development/config.html?highlight=startup#startup-files), but "explicit is better than implicit." :)
Of course, all of this can be hidden in an [IPython kernel startup script](http://ipython.org/ipython-doc/stable/development/config.html?highlight=startup#startup-files), but "explicit is better than implicit." :)
## Connecting to a Spark Cluster on Standalone Mode
#### Connecting to a Spark Cluster in Standalone Mode
Connection to Spark Cluster on Standalone Mode requires the following set of steps:
Connection to Spark Cluster in Standalone Mode requires the following set of steps:
0. Verify that the docker image (check the Dockerfile) and the Spark Cluster which is being deployed, run the same version of Spark.
0. Verify that the docker image (check the Dockerfile) and the Spark Cluster which is being deployed, run the same version of Spark.
1.[Deploy Spark on Standalone Mode](http://spark.apache.org/docs/latest/spark-standalone.html).
1.[Deploy Spark on Standalone Mode](http://spark.apache.org/docs/latest/spark-standalone.html).
@@ -9,7 +9,7 @@ Jupyter Docker Stacks are a set of ready-to-run Docker images containing Jupyter
...
@@ -9,7 +9,7 @@ Jupyter Docker Stacks are a set of ready-to-run Docker images containing Jupyter
:maxdepth: 1
:maxdepth: 1
using
using
features
configuration
contributing
contributing
Quick Start
Quick Start
...
@@ -18,19 +18,19 @@ Quick Start
...
@@ -18,19 +18,19 @@ Quick Start
The examples below may help you get started if you have Docker installed, know which Docker image you want to use, and want to launch a single Jupyter Notebook server in a container. The other pages in this documentation describe additional uses and features in detail.::
The examples below may help you get started if you have Docker installed, know which Docker image you want to use, and want to launch a single Jupyter Notebook server in a container. The other pages in this documentation describe additional uses and features in detail.::
# Run a Jupyter Notebook server in a Docker container started
# Run a Jupyter Notebook server in a Docker container started
# from the jupyter/scipy-notebook image built from Git commit 27ba573.
# from the jupyter/scipy-notebook image built from Git commit 2c80cf3537ca.
# All files saved in the container are lost when the notebook server exits.
# All files saved in the container are lost when the notebook server exits.
# -ti: pseudo-TTY+STDIN open, so the logs appear in the terminal
# -ti: pseudo-TTY+STDIN open, so the logs appear in the terminal
# -rm: remove the container on exit
# -rm: remove the container on exit
# -p: publish the notebook port 8888 as port 8888 on the host
# -p: publish the notebook port 8888 as port 8888 on the host
docker run -ti --rm -p 8888:8888 jupyter/scipy-notebook:27ba573
docker run -ti --rm -p 8888:8888 jupyter/scipy-notebook:2c80cf3537ca
# Run a Jupyter Notebook server in a Docker container started from the
# Run a Jupyter Notebook server in a Docker container started from the
# jupyter/r-notebook image built from Git commit cf1a3aa.
# jupyter/r-notebook image built from Git commit e5c5a7d3e52d.
# All files written to ~/work in the container are saved to the
# All files written to ~/work in the container are saved to the
# current working on the host and persist even when the notebook server
# current working on the host and persist even when the notebook server
# exits.
# exits.
docker run -ti --rm -p 8888:8888 -v "$PWD":/home/jovyan/work jupyter/r-notebook:cf1a3aa
docker run -ti --rm -p 8888:8888 -v "$PWD":/home/jovyan/work jupyter/r-notebook:e5c5a7d3e52d
# Run a Jupyter Notebook server in a background Docker container started
# Run a Jupyter Notebook server in a background Docker container started
# from the latest jupyter/all-spark-notebook image available on the local
# from the latest jupyter/all-spark-notebook image available on the local
@@ -11,52 +11,132 @@ This section provides details about the available images and runtimes to inform
...
@@ -11,52 +11,132 @@ This section provides details about the available images and runtimes to inform
### Core Stacks
### Core Stacks
The Jupyter team maintains a set of Docker image definitions in the https://github.com/jupyter/docker-stacks GitHub repository. The following table describes these images, and links to their source on GitHub and their builds on Docker Cloud.
The Jupyter team maintains a set of Docker image definitions in the [https://github.com/jupyter/docker-stacks](https://github.com/jupyter/docker-stacks) GitHub repository. The following sections describe these images including their contents, relationships, and versioning strategy.
The following diagram depicts the build dependencies between the core images (aka the `FROM` statement in their Dockerfiles). Any image lower in the tree inherits
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/master/base-notebook)
`jupyter/base-notebook` is a small image supporting the [options common across all core stacks](configuration.html). It is the basis for all other stacks.
#### Versioning
* Minimally-functional Jupyter Notebook server (e.g., no [pandoc](https://pandoc.org/) for saving notebooks as PDFs)
* Unprivileged user `jovyan` (`uid=1000`, configurable, see options) in group `users` (`gid=100`) with ownership over the `/home/jovyan` and `/opt/conda` paths
*`tini` as the container entrypoint and a `start-notebook.sh` script as the default command
* A `start-singleuser.sh` script useful for launching containers in JupyterHub
* A `start.sh` script useful for running alternative commands in the container (e.g. `ipython`, `jupyter kernelgateway`, `jupyter lab`)
* Options for a self-signed HTTPS certificate and passwordless sudo
#### jupyter/minimal-notebook
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/master/minimal-notebook)
`jupyter/r-notebook` includes popular packages from the R ecosystem.
* Everything in `jupyter/minimal-notebook` and its ancestor images
* The [R](https://www.r-project.org/) interpreter and base environment
*[IRKernel](https://irkernel.github.io/) to support R code in Jupyter notebooks
*[tidyverse](https://www.tidyverse.org/) packages, including [ggplot2](http://ggplot2.org/), [dplyr](http://dplyr.tidyverse.org/), [tidyr](http://tidyr.tidyverse.org/), [readr](http://readr.tidyverse.org/), [purrr](http://purrr.tidyverse.org/), [tibble](http://tibble.tidyverse.org/), [stringr](http://stringr.tidyverse.org/), [lubridate](http://lubridate.tidyverse.org/), and [broom](https://cran.r-project.org/web/packages/broom/vignettes/broom.html) from [conda-forge](https://conda-forge.github.io/feedstocks)
*[plyr](https://cran.r-project.org/web/packages/plyr/index.html), [devtools](https://cran.r-project.org/web/packages/devtools/index.html), [shiny](https://shiny.rstudio.com/), [rmarkdown](http://rmarkdown.rstudio.com/), [forecast](https://cran.r-project.org/web/packages/forecast/forecast.pdf), [rsqlite](https://cran.r-project.org/web/packages/RSQLite/index.html), [reshape2](https://cran.r-project.org/web/packages/reshape2/reshape2.pdf), [nycflights13](https://cran.r-project.org/web/packages/nycflights13/index.html), [caret](http://topepo.github.io/caret/index.html), [rcurl](https://cran.r-project.org/web/packages/RCurl/index.html), and [randomforest](https://cran.r-project.org/web/packages/randomForest/randomForest.pdf) packages from [conda-forge](https://conda-forge.github.io/feedstocks)
#### jupyter/scipy-notebook
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/master/scipy-notebook)
`jupyter/tensorflow-notebook` includes popular Python deep learning libraries.
* Everything in `jupyter/scipy-notebook` and its ancestor images
*[tensorflow](https://www.tensorflow.org/) and [keras](https://keras.io/) machine learning libraries
[Click here for a commented build history of each image, with references to tag/SHA values.](https://github.com/jupyter/docker-stacks/wiki/Docker-build-history)
#### jupyter/datascience-notebook
The following are quick-links to READMEs about each image and their Docker image tags on Docker Cloud:
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/master/datascience-notebook)
* Everything in the `jupyter/scipy-notebook` and `jupyter/r-notebook` images, and their ancestor images
* The [Julia](https://julialang.org/) compiler and base environment
*[IJulia](https://github.com/JuliaLang/IJulia.jl) to support Julia code in Jupyter notebooks
*[HDF5](https://github.com/JuliaIO/HDF5.jl), [Gadfly](http://gadflyjl.org/stable/), and [RDatasets](https://github.com/johnmyleswhite/RDatasets.jl) packages
The latest tag in each Docker Hub repository tracks the master branch HEAD reference on GitHub. This is a moving target and will make backward-incompatible changes regularly.
#### jupyter/pyspark-notebook
Any 12-character image tag on Docker Hub refers to a git commit SHA here on GitHub. See the Docker build history wiki page for a table of build details.
Stack contents (e.g., new library versions) will be updated upon request via PRs against this project.
Users looking for reproducibility or stability should always refer to specific git SHA tagged images in their work, not latest.
For legacy reasons, there are two additional tags named 3.2 and 4.0 on Docker Hub which point to images prior to our versioning scheme switch.
If you haven't already, pin your image to a tag, e.g. FROM jupyter/scipy-notebook:7c45ec67c8e7. latest is a moving target which can change in backward-incompatible ways as packages and operating systems are updated.
## Community Stacks
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/master/pyspark-notebook)
`jupyter/all-spark-notebook` includes Python, R, and Scala support for Apache Spark, optionally on Mesos.
* Everything in `jupyter/pyspark-notebook` and its ancestor images
*[IRKernel](https://irkernel.github.io/) to support R code in Jupyter notebooks
*[Apache Toree](https://toree.apache.org/) and [spylon-kernel](https://github.com/maxpoint/spylon-kernel) to support Scala code in Jupyter notebooks
*[ggplot2](http://ggplot2.org/), [sparklyr](http://spark.rstudio.com/), and [rcurl](https://cran.r-project.org/web/packages/RCurl/index.html) packages
#### Image Relationships
The following diagram depicts the build dependency tree of the core images. (i.e., the `FROM` statements in their Dockerfiles). Any given image inherits the complete content of all ancestor images pointing to it.
Pull requests to the `jupyter/docker-stacks` repository trigger builds of all images on Travis CI. These images are for testing purposes only and are not saved for use. When pull requests merge to master, all images rebuild on Docker Cloud and become available to `docker pull` from Docker Hub.
#### Versioning
The `latest` tag in each Docker Hub repository tracks the master branch `HEAD` reference on GitHub. `latest` is a moving target, by definition, and will have backward-incompatible changes regularly.
Every image on Docker Hub also receives a 12-character tag which corresponds with the git commit SHA that triggered the image build. You can inspect the state of the `jupyter/docker-stacks` repository for that commit to review the definition of the image (e.g., images with tag 7c45ec67c8e7 were built from [https://github.com/jupyter/docker-stacks/tree/7c45ec67c8e7](https://github.com/jupyter/docker-stacks/tree/7c45ec67c8e7)).
You must refer to git-SHA image tags when stability and reproducibility are important in your work. (e.g. `FROM jupyter/scipy-notebook:7c45ec67c8e7`, `docker run -it --rm jupyter/scipy-notebook:7c45ec67c8e7`). You should only use `latest` when a one-off container instance is acceptable (e.g., you want to briefly try a new library in a notebook).
### Community Stacks
## Running a Container
## Running a Container
...
@@ -64,6 +144,4 @@ The Jupyter community maintains additional
...
@@ -64,6 +144,4 @@ The Jupyter community maintains additional
### Using JupyterHub
### Using JupyterHub
Every notebook stack is compatible with JupyterHub 0.5 or higher. When running with JupyterHub, you must override the Docker run command to point to the `start-singleuser.sh script, which starts a single-user instance of the Notebook server. See each stack's README for instructions on running with JupyterHub.
