Thank you for contributing to the Jupyter Docker Stacks! We review pull requests of new features (e.g., new packages, new scripts, new flags) to balance the value of the images to the Jupyter community with the cost of maintaining the images over time.
Thank you for contributing to the Jupyter Docker Stacks! We review pull requests of new features
(e.g., new packages, new scripts, new flags) to balance the value of the images to the Jupyter
community with the cost of maintaining the images over time.
## Suggesting a New Feature
Please follow the process below to suggest a new feature for inclusion in one of the core stacks:
1.[Open a GitHub issue](https://github.com/jupyter/docker-stacks/issues) describing the feature you'd like to contribute.
2. Discuss with the maintainers whether the addition makes sense in [one of the core stacks](../using/selecting.md#Core-Stacks), as a [recipe in the documentation](recipes.md), as a [community stack](stacks.md), or as something else entirely.
1.[Open a GitHub issue](https://github.com/jupyter/docker-stacks/issues) describing the feature
you'd like to contribute.
2. Discuss with the maintainers whether the addition makes sense in
[one of the core stacks](../using/selecting.md#Core-Stacks), as a
[recipe in the documentation](recipes.md), as a [community stack](stacks.md), or as something
else entirely.
## Selection Criteria
Roughly speaking, we evaluate new features based on the following criteria:
***Usefulness to Jupyter users**: Is the feature generally applicable across domains? Does it work with Jupyter Notebook, JupyterLab, JupyterHub, etc.?
***Fit with the image purpose**: Does the feature match the theme of the stack in which it will be added? Would it fit better in a new, community stack?
***Complexity of build / runtime configuration**: How many lines of code does the feature require in one of the Dockerfiles or startup scripts? Does it require new scripts entirely? Do users need to adjust how they use the images?
***Impact on image metrics**: How many bytes does the feature and its dependencies add to the image(s)? How many minutes do they add to the build time?
***Ability to support the addition**: Can existing maintainers answer user questions and address future build issues? Are the contributors interested in helping with long-term maintenance? Can we write tests to ensure the feature continues to work over time?
-**Usefulness to Jupyter users**: Is the feature generally applicable across domains? Does it work
with Jupyter Notebook, JupyterLab, JupyterHub, etc.?
-**Fit with the image purpose**: Does the feature match the theme of the stack in which it will be
added? Would it fit better in a new, community stack?
-**Complexity of build / runtime configuration**: How many lines of code does the feature require
in one of the Dockerfiles or startup scripts? Does it require new scripts entirely? Do users need
to adjust how they use the images?
-**Impact on image metrics**: How many bytes does the feature and its dependencies add to the
image(s)? How many minutes do they add to the build time?
-**Ability to support the addition**: Can existing maintainers answer user questions and address
future build issues? Are the contributors interested in helping with long-term maintenance? Can we
write tests to ensure the feature continues to work over time?
## Submitting a Pull Request
If there's agreement that the feature belongs in one or more of the core stacks:
1. Implement the feature in a local clone of the `jupyter/docker-stacks` project.
2. Please build the image locally before submitting a pull request. Building the image locally shortens the debugging cycle by taking some load off [Travis CI](http://travis-ci.org/), which graciously provides free build services for open source projects like this one. If you use `make`, call:
```bash
make build/somestack-notebook
```
3.[Submit a pull request](https://github.com/PointCloudLibrary/pcl/wiki/A-step-by-step-guide-on-preparing-and-submitting-a-pull-request)(PR) with your changes.
4. Watch for Travis to report a build success or failure for your PR on GitHub.
2. Please build the image locally before submitting a pull request. Building the image locally
shortens the debugging cycle by taking some load off GitHub Actions, which graciously provide
free build services for open source projects like this one. If you use `make`, call:
```bash
make build/somestack-notebook
```
3.[Submit a pull request](https://github.com/PointCloudLibrary/pcl/wiki/A-step-by-step-guide-on-preparing-and-submitting-a-pull-request)
(PR) with your changes.
4. Watch for GitHub to report a build success or failure for your PR on GitHub.
5. Discuss changes with the maintainers and address any build issues.
We actively seek pull requests which update packages already included in the project Dockerfiles. This is a great way for first-time contributors to participate in developing the Jupyter Docker Stacks.
We actively seek pull requests which update packages already included in the project Dockerfiles.
This is a great way for first-time contributors to participate in developing the Jupyter Docker
Stacks.
Please follow the process below to update a package version:
1. Locate the Dockerfile containing the library you wish to update (e.g., [base-notebook/Dockerfile](https://github.com/jupyter/docker-stacks/blob/master/base-notebook/Dockerfile), [scipy-notebook/Dockerfile](https://github.com/jupyter/docker-stacks/blob/master/scipy-notebook/Dockerfile))
2. Adjust the version number for the package. We prefer to pin the major and minor version number of packages so as to minimize rebuild side-effects when users submit pull requests (PRs). For example, you'll find the Jupyter Notebook package, `notebook`, installed using conda with `notebook=5.4.*`.
3. Please build the image locally before submitting a pull request. Building the image locally shortens the debugging cycle by taking some load off [Travis CI](http://travis-ci.org/), which graciously provides free build services for open source projects like this one. If you use `make`, call:
```bash
make build/somestack-notebook
```
4.[Submit a pull request](https://github.com/PointCloudLibrary/pcl/wiki/A-step-by-step-guide-on-preparing-and-submitting-a-pull-request)(PR) with your changes.
5. Watch for Travis to report a build success or failure for your PR on GitHub.
6. Discuss changes with the maintainers and address any build issues. Version conflicts are the most common problem. You may need to upgrade additional packages to fix build failures.
1. Locate the Dockerfile containing the library you wish to update (e.g.,
We greatly appreciate pull requests that extend the automated tests that vet the basic functionality of the Docker images.
We greatly appreciate pull requests that extend the automated tests that vet the basic functionality
of the Docker images.
## How the Tests Work
Travis executes `make build-test-all` against pull requests submitted to the `jupyter/docker-stacks` repository. This `make` command builds every docker image. After building each image, the `make` command executes `pytest` to run both image-specific tests like those in [base-notebook/test/](https://github.com/jupyter/docker-stacks/tree/master/base-notebook/test) and common tests defined in [test/](https://github.com/jupyter/docker-stacks/tree/master/test). Both kinds of tests make use of global [pytest fixtures](https://docs.pytest.org/en/latest/fixture.html) defined in the [conftest.py](https://github.com/jupyter/docker-stacks/blob/master/conftest.py) file at the root of the projects.
GitHub executes `make build-test-all` against pull requests submitted to the `jupyter/docker-stacks`
repository. This `make` command builds every docker image. After building each image, the `make`
command executes `pytest` to run both image-specific tests like those in
[base-notebook/test/](https://github.com/jupyter/docker-stacks/tree/master/base-notebook/test) and
common tests defined in [test/](https://github.com/jupyter/docker-stacks/tree/master/test). Both
kinds of tests make use of global [pytest fixtures](https://docs.pytest.org/en/latest/fixture.html)
defined in the [conftest.py](https://github.com/jupyter/docker-stacks/blob/master/conftest.py) file
at the root of the projects.
## Contributing New Tests
Please follow the process below to add new tests:
1. If the test should run against every image built, add your test code to one of the modules in [test/](https://github.com/jupyter/docker-stacks/tree/master/test) or create a new module.
2. If your test should run against a single image, add your test code to one of the modules in `some-notebook/test/` or create a new module.
1. If the test should run against every image built, add your test code to one of the modules in
[test/](https://github.com/jupyter/docker-stacks/tree/master/test) or create a new module.
2. If your test should run against a single image, add your test code to one of the modules in
`some-notebook/test/` or create a new module.
3. Build one or more images you intend to test and run the tests locally. If you use `make`, call:
```bash
make build/somestack-notebook
make test/somestack-notebook
```
4.[Submit a pull request](https://github.com/PointCloudLibrary/pcl/wiki/A-step-by-step-guide-on-preparing-and-submitting-a-pull-request)(PR) with your changes.
5. Watch for Travis to report a build success or failure for your PR on GitHub.
6. Discuss changes with the maintainers and address any issues running the tests on Travis.
```bash
make build/somestack-notebook
make test/somestack-notebook
```
4.[Submit a pull request](https://github.com/PointCloudLibrary/pcl/wiki/A-step-by-step-guide-on-preparing-and-submitting-a-pull-request)
(PR) with your changes.
5. Watch for GitHub to report a build success or failure for your PR on GitHub.
6. Discuss changes with the maintainers and address any issues running the tests on GitHub.
To build new images on Docker Cloud and publish them to the Docker Hub registry, do the following:
1. Make sure Travis is green for a PR.
1. Make sure GitHub Actions status checks pas for the PR.
2. Merge the PR.
3. Monitor the Docker Cloud build status for each of the stacks, starting with [jupyter/base-notebook](https://cloud.docker.com/app/jupyter/repository/docker/jupyter/base-notebook/general) and ending with [jupyter/all-spark-notebook](https://cloud.docker.com/app/jupyter/repository/docker/jupyter/all-spark-notebook/general). See the [stack hierarchy diagram](../using/selecting.html#image-relationships) for the current, complete build order.
4. Manually click the retry button next to any build that fails to resume that build and any dependent builds.
5. Try to avoid merging another PR to master until all outstanding builds complete. There's no way at present to propagate the git SHA to build through the Docker Cloud build trigger API. Every build trigger works off of master HEAD.
3. Monitor the Docker Cloud build status for each of the stacks, starting with
See the [stack hierarchy diagram](../using/selecting.html#image-relationships) for the current,
complete build order.
4. Manually click the retry button next to any build that fails to resume that build and any
dependent builds.
5. Try to avoid merging another PR to master until all outstanding builds complete. There's no way
at present to propagate the git SHA to build through the Docker Cloud build trigger API. Every
build trigger works off of master HEAD.
## Updating the Ubuntu Base Image
When there's a security fix in the Ubuntu base image or after some time passes, it's a good idea to update the pinned SHA in the [jupyter/base-notebook Dockerfile](https://github.com/jupyter/docker-stacks/blob/master/base-notebook/Dockerfile). Submit it as a regular PR and go through the build process. Expect the build to take a while to complete: every image layer will rebuild.
When there's a security fix in the Ubuntu base image or after some time passes, it's a good idea to
Submit it as a regular PR and go through the build process. Expect the build to take a while to
complete: every image layer will rebuild.
## Adding a New Core Image to Docker Cloud
When there's a new stack definition, do the following before merging the PR with the new stack:
1. Ensure the PR includes an update to the stack overview diagram [in the documentation](https://github.com/jupyter/docker-stacks/blob/master/docs/using/selecting.md#image-relationships). The image links to the [blockdiag source](http://interactive.blockdiag.com/) used to create it.
2. Ensure the PR updates the Makefile which is used to build the stacks in order on Travis CI.
3. Create a new repository in the `jupyter` org on Docker Cloud named after the stack folder in the git repo.
1. Ensure the PR includes an update to the stack overview diagram
[in the documentation](https://github.com/jupyter/docker-stacks/blob/master/docs/using/selecting.md#image-relationships).
The image links to the [blockdiag source](http://interactive.blockdiag.com/) used to create it.
2. Ensure the PR updates the Makefile which is used to build the stacks in order on GitHub Actions.
3. Create a new repository in the `jupyter` org on Docker Cloud named after the stack folder in the
git repo.
4. Grant the `stacks` team permission to write to the repo.
5. Click *Builds* and then *Configure Automated Builds* for the repository.
5. Click _Builds_ and then _Configure Automated Builds_ for the repository.
6. Select `jupyter/docker-stacks` as the source repository.
7. Choose *Build on Docker Cloud's infrastructure using a Small node* unless you have reason to believe a bigger host is required.
8. Update the *Build Context* in the default build rule to be `/<name-of-the-stack>`.
9. Toggle *Autobuild* to disabled unless the stack is a new root stack (e.g., like `jupyter/base-notebook`).
7. Choose _Build on Docker Cloud's infrastructure using a Small node_ unless you have reason to
believe a bigger host is required.
8. Update the _Build Context_ in the default build rule to be `/<name-of-the-stack>`.
9. Toggle _Autobuild_ to disabled unless the stack is a new root stack (e.g., like
`jupyter/base-notebook`).
10. If the new stack depends on the build of another stack in the hierarchy:
1. Hit *Save* and then click *Configure Automated Builds*.
2. At the very bottom, add a build trigger named *Stack hierarchy trigger*.
1. Hit _Save_ and then click _Configure Automated Builds_.
2. At the very bottom, add a build trigger named _Stack hierarchy trigger_.
3. Copy the build trigger URL.
4. Visit the parent repository *Builds* page and click *Configure Automated Builds*.
5. Add the URL you copied to the *NEXT_BUILD_TRIGGERS* environment variable comma separated list of URLs, creating that environment variable if it does not already exist.
6. Hit *Save*.
4. Visit the parent repository _Builds_ page and click _Configure Automated Builds_.
5. Add the URL you copied to the _NEXT_BUILD_TRIGGERS_ environment variable comma separated list
of URLs, creating that environment variable if it does not already exist.
6. Hit _Save_.
11. If the new stack should trigger other dependent builds:
1. Add an environment variable named *NEXT_BUILD_TRIGGERS*.
2. Copy the build trigger URLs from the dependent builds into the *NEXT_BUILD_TRIGGERS* comma separated list of URLs.
3. Hit *Save*.
12. Adjust other *NEXT_BUILD_TRIGGERS* values as needed so that the build order matches that in the stack hierarchy diagram.
1. Add an environment variable named _NEXT_BUILD_TRIGGERS_.
2. Copy the build trigger URLs from the dependent builds into the _NEXT_BUILD_TRIGGERS_ comma
separated list of URLs.
3. Hit _Save_.
12. Adjust other _NEXT_BUILD_TRIGGERS_ values as needed so that the build order matches that in the
stack hierarchy diagram.
## Adding a New Maintainer Account
...
...
@@ -61,22 +81,22 @@ If automated builds on Docker Cloud have got you down, do the following to push
