Commit 5c2618f2 authored by Mladen Kolovic's avatar Mladen Kolovic

Add HDF5 support for Julia

* create `.juliarc.jl` start up script to point Julia to conda libraries, specifically the hdf5 libraries installed through the h5py package in the scipy stack
* add the HDF5 Julia package

Update README.md to document HDF5 addition

Add -F option to force config load

Add the -F flag to julia expression execution to load .juliarc.jl before execution.
parent 8dfd60b7
![docker pulls](https://img.shields.io/docker/pulls/jupyter/all-spark-notebook.svg) ![docker stars](https://img.shields.io/docker/stars/jupyter/all-spark-notebook.svg)
# Jupyter Notebook Python, Scala, R, Spark, Mesos Stack # Jupyter Notebook Python, Scala, R, Spark, Mesos Stack
## What it Gives You ## What it Gives You
...@@ -195,7 +197,7 @@ rdd.sum() ...@@ -195,7 +197,7 @@ rdd.sum()
Connection to Spark Cluster on Standalone Mode requires the following set of steps: Connection to Spark Cluster on Standalone Mode requires the following set of steps:
0. Verify that the docker image (check the Dockerfile) and the Spark Cluster which is being deployed, run the same version of Spark. 0. Verify that the docker image (check the Dockerfile) and the Spark Cluster which is being deployed, run the same version of Spark.
1. [Deploy Spark on Standalone Mode](http://spark.apache.org/docs/latest/spark-standalone.html). 1. [Deploy Spark on Standalone Mode](http://spark.apache.org/docs/latest/spark-standalone.html).
2. Run the Docker container with `--net=host` in a location that is network addressable by all of your Spark workers. (This is a [Spark networking requirement](http://spark.apache.org/docs/latest/cluster-overview.html#components).) 2. Run the Docker container with `--net=host` in a location that is network addressable by all of your Spark workers. (This is a [Spark networking requirement](http://spark.apache.org/docs/latest/cluster-overview.html#components).)
* NOTE: When using `--net=host`, you must also use the flags `--pid=host -e TINI_SUBREAPER=true`. See https://github.com/jupyter/docker-stacks/issues/64 for details. * NOTE: When using `--net=host`, you must also use the flags `--pid=host -e TINI_SUBREAPER=true`. See https://github.com/jupyter/docker-stacks/issues/64 for details.
......
...@@ -52,4 +52,8 @@ RUN julia -e 'Pkg.add("IJulia")' && \ ...@@ -52,4 +52,8 @@ RUN julia -e 'Pkg.add("IJulia")' && \
mv /home/$NB_USER/.local/share/jupyter/kernels/* $CONDA_DIR/share/jupyter/kernels/ && \ mv /home/$NB_USER/.local/share/jupyter/kernels/* $CONDA_DIR/share/jupyter/kernels/ && \
chmod -R go+rx $CONDA_DIR/share/jupyter && \ chmod -R go+rx $CONDA_DIR/share/jupyter && \
rm -rf /home/$NB_USER/.local/share rm -rf /home/$NB_USER/.local/share
RUN julia -e 'Pkg.add("Gadfly")' && julia -e 'Pkg.add("RDatasets")'
# Show Julia where conda libraries are
# Add essential packages
RUN echo 'push!(Sys.DL_LOAD_PATH, "/opt/conda/lib")' > /home/$NB_USER/.juliarc.jl && \
julia -e 'Pkg.add("Gadfly")' && julia -e 'Pkg.add("RDatasets")' && julia -F -e 'Pkg.add("HDF5")'
![docker pulls](https://img.shields.io/docker/pulls/jupyter/datascience-notebook.svg) ![docker stars](https://img.shields.io/docker/stars/jupyter/datascience-notebook.svg)
# Jupyter Notebook Data Science Stack # Jupyter Notebook Data Science Stack
## What it Gives You ## What it Gives You
...@@ -7,7 +9,7 @@ ...@@ -7,7 +9,7 @@
* pandas, matplotlib, scipy, seaborn, scikit-learn, scikit-image, sympy, cython, patsy, statsmodel, cloudpickle, dill, numba, bokeh pre-installed * pandas, matplotlib, scipy, seaborn, scikit-learn, scikit-image, sympy, cython, patsy, statsmodel, cloudpickle, dill, numba, bokeh pre-installed
* Conda R v3.2.x and channel * Conda R v3.2.x and channel
* plyr, devtools, dplyr, ggplot2, tidyr, shiny, rmarkdown, forecast, stringr, rsqlite, reshape2, nycflights13, caret, rcurl, and randomforest pre-installed * plyr, devtools, dplyr, ggplot2, tidyr, shiny, rmarkdown, forecast, stringr, rsqlite, reshape2, nycflights13, caret, rcurl, and randomforest pre-installed
* Julia v0.3.x with Gadfly and RDatasets pre-installed * Julia v0.3.x with Gadfly, RDatasets and HDF5 pre-installed
* Unprivileged user `jovyan` (uid=1000, configurable, see options) in group `users` (gid=100) with ownership over `/home/jovyan` and `/opt/conda` * Unprivileged user `jovyan` (uid=1000, configurable, see options) in group `users` (gid=100) with ownership over `/home/jovyan` and `/opt/conda`
* [tini](https://github.com/krallin/tini) as the container entrypoint and [start-notebook.sh](../minimal-notebook/start-notebook.sh) as the default command * [tini](https://github.com/krallin/tini) as the container entrypoint and [start-notebook.sh](../minimal-notebook/start-notebook.sh) as the default command
* Options for HTTPS, password auth, and passwordless `sudo` * Options for HTTPS, password auth, and passwordless `sudo`
......
![docker pulls](https://img.shields.io/docker/pulls/jupyter/minimal-kernel.svg) ![docker stars](https://img.shields.io/docker/stars/jupyter/minimal-kernel.svg)
# Kernel Gateway Stack # Kernel Gateway Stack
## What it Gives You ## What it Gives You
......
![docker pulls](https://img.shields.io/docker/pulls/jupyter/minimal-notebook.svg) ![docker stars](https://img.shields.io/docker/stars/jupyter/minimal-notebook.svg)
# Minimal Jupyter Notebook Stack # Minimal Jupyter Notebook Stack
## What it Gives You ## What it Gives You
......
![docker pulls](https://img.shields.io/docker/pulls/jupyter/pyspark-notebook.svg) ![docker stars](https://img.shields.io/docker/stars/jupyter/pyspark-notebook.svg)
# Jupyter Notebook Python, Spark, Mesos Stack # Jupyter Notebook Python, Spark, Mesos Stack
## What it Gives You ## What it Gives You
......
![docker pulls](https://img.shields.io/docker/pulls/jupyter/r-notebook.svg) ![docker stars](https://img.shields.io/docker/stars/jupyter/r-notebook.svg)
# Jupyter Notebook R Stack # Jupyter Notebook R Stack
## What it Gives You ## What it Gives You
......
![docker pulls](https://img.shields.io/docker/pulls/jupyter/scipy-notebook.svg) ![docker stars](https://img.shields.io/docker/stars/jupyter/scipy-notebook.svg)
# Jupyter Notebook Scientific Python Stack # Jupyter Notebook Scientific Python Stack
## What it Gives You ## What it Gives You
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment