Skip to content

J
Jupyter Docker Stacks
Commit 8da0d49a authored by Peter Parente's avatar Peter Parente
Browse files
Options

Webhook to update docker builds page 

(c) Copyright IBM Corp. 2016
parent a2a2694f
Show whitespace changes
Inline Side-by-side
Showing with 407 additions and 0 deletions
internal/docker-stacks-webhook/conda_requirements.txt 0 → 100644
View file @ 8da0d49a
# Copyright (c) Jupyter Development Team.
# Distributed under the terms of the Modified BSD License.
# NOTE: jupyter_kernel_gateway uses the notebook package. We get a
# speed up if we pre-install the notebook package using conda since
# we'll get prebuilt binaries for all its dependencies like pyzmq.
notebook==4.1
internal/docker-stacks-webhook/docker-stacks-webhook.ipynb 0 → 100644
View file @ 8da0d49a
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# docker-stacks wiki webhook\n",
"\n",
"Listens for webhook callbacks from Docker Hub. Updates the [docker build history](https://github.com/jupyter/docker-stacks/wiki/Docker-build-history) wiki page in response to completed builds.\n",
"\n",
"References:\n",
"\n",
"* https://docs.docker.com/docker-hub/webhooks/\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import json\n",
"import datetime as dt\n",
"import requests\n",
"import os"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Read credentials from the environment."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"GH_USERNAME = os.getenv('GH_USERNAME')\n",
"GH_TOKEN = os.getenv('GH_TOKEN')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Configure git upfront."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"%%bash\n",
"git config --global user.email \"jupyter@googlegroups.com\"\n",
"git config --global user.name \"Jupyter Development Team\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Build the templates we need."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"wiki_git_tmpl = 'https://{GH_USERNAME}:{GH_TOKEN}@github.com/jupyter/docker-stacks.wiki.git'\n",
"commit_url_tmpl = 'https://github.com/jupyter/docker-stacks/commit/{sha}'\n",
"row_tmpl = '|{pushed_at}|[{sha}]({commit_url})|{commit_msg}|\\n'\n",
"api_commit_url_tmpl = 'https://api.github.com/repos/jupyter/docker-stacks/commits/{sha}'"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"REQUEST = json.dumps({\n",
" 'body' : {\n",
" \"push_data\": {\n",
" \"pushed_at\": 1449017033,\n",
" \"images\": [],\n",
" \"tag\": \"9f9907cf1df8\",\n",
" \"pusher\": \"biscarch\"\n",
" },\n",
" \"callback_url\": \"https://registry.hub.docker.com/u/biscarch/webhook-tester-repo/hook/2i5e3gj1bi354asb3f05gchi4ccjg0gas/\",\n",
" \"repository\": {\n",
" \"status\": \"Active\",\n",
" \"description\": \"\",\n",
" \"is_trusted\": False,\n",
" \"full_description\": None,\n",
" \"repo_url\": \"https://registry.hub.docker.com/u/biscarch/webhook-tester-repo/\",\n",
" \"owner\": \"biscarch\",\n",
" \"is_official\": False,\n",
" \"is_private\": False,\n",
" \"name\": \"webhook-tester-repo\",\n",
" \"namespace\": \"biscarch\",\n",
" \"star_count\": 0,\n",
" \"comment_count\": 0,\n",
" \"date_created\": 1449016916,\n",
" \"repo_name\": \"biscarch/webhook-tester-repo\"\n",
" }\n",
" }\n",
"})"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Read values we need out of the request body."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# POST /tag\n",
"body = json.loads(REQUEST)['body']\n",
"\n",
"tag = body['push_data']['tag']\n",
"pushed_at_ts = body['push_data']['pushed_at']\n",
"callback_url = body['callback_url']"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Validate the request by seeing if the tag is a valid SHA in the docker-stacks repo."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# POST /tag\n",
"commit_resp = requests.get(api_commit_url_tmpl.format(sha=tag))\n",
"try:\n",
" commit_resp.raise_for_status()\n",
"except Exception as ex:\n",
" requests.post(callback_url, json={\n",
" 'state': 'failure',\n",
" 'description': 'request does not contain a valid sha',\n",
" 'context' : 'docker-stacks-webhook',\n",
" 'target_url' : 'https://github.com/jupyter/docker-stacks/wiki/Docker-build-history'\n",
" })\n",
" raise ex"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Get a fresh clone of the wiki git repo."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# POST /tag\n",
"wiki_git = wiki_git_tmpl.format(GH_USERNAME=GH_USERNAME, GH_TOKEN=GH_TOKEN)\n",
"\n",
"!rm -rf docker-stacks.wiki\n",
"!git clone $wiki_git"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Read the build page markdown."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# POST /tag\n",
"with open('docker-stacks.wiki/Docker-build-history.md') as f:\n",
" lines = f.readlines()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Find the start of the table."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# POST /tag\n",
"for table_top_i, line in enumerate(lines):\n",
" if line.startswith('|--'):\n",
" break\n",
"else:\n",
" requests.post(callback_url, json={\n",
" 'state': 'failure',\n",
" 'description': 'could not locate table on wiki page',\n",
" 'context' : 'docker-stacks-webhook',\n",
" 'target_url' : 'https://github.com/jupyter/docker-stacks/wiki/Docker-build-history'\n",
" })\n",
" raise RuntimeError('wiki table missing')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Format the text we want to put into the wiki table row."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# POST /tag\n",
"pushed_at_dt = dt.datetime.fromtimestamp(pushed_at_ts)\n",
"pushed_at = pushed_at_dt.strftime('%b. %d, %Y')\n",
"commit_url = commit_url_tmpl.format(sha=tag)\n",
"commit_msg = commit_resp.json()['commit']['message'].replace('\\n', ' ')\n",
"row = row_tmpl.format(pushed_at=pushed_at, sha=tag, commit_url=commit_url, commit_msg=commit_msg)\n",
"row"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Insert the table row."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# POST /tag\n",
"lines.insert(table_top_i+1, row)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Write the file back out."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# POST /tag\n",
"with open('docker-stacks.wiki/Docker-build-history.md', 'w') as f:\n",
" f.writelines(lines)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Commit and push."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# POST /tag\n",
"!cd docker-stacks.wiki/ && \\\n",
" git add -A && \\\n",
" git commit -m 'Add build $tag' && \\\n",
" git push origin master"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Tell Docker Hub we succeeded."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# POST /tag\n",
"resp = requests.post(callback_url, json={\n",
" 'state': 'success',\n",
" 'description': 'updated docker-stacks wiki build page',\n",
" 'context' : 'docker-stacks-webhook',\n",
" 'target_url' : 'https://github.com/jupyter/docker-stacks/wiki/Docker-build-history'\n",
"})\n",
"\n",
"print(resp.status_code)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.1"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
internal/docker-stacks-webhook/manifest.yml 0 → 100644
View file @ 8da0d49a
# Copyright (c) Jupyter Development Team.
# Distributed under the terms of the Modified BSD License.
---
applications:
- name: docker-stacks-webhook
memory: 128M
instances: 1
path: .
buildpack: https://github.com/ihuston/python-conda-buildpack
command: >
jupyter-kernelgateway --KernelGatewayApp.port=$PORT
--KernelGatewayApp.ip=0.0.0.0
--KernelGatewayApp.api=notebook-http
--KernelGatewayApp.seed_uri='./docker-stacks-webhook.ipynb'
internal/docker-stacks-webhook/requirements.txt 0 → 100644
View file @ 8da0d49a
# Copyright (c) Jupyter Development Team.
# Distributed under the terms of the Modified BSD License.
git+https://github.com/parente/kernel_gateway@concatenate-cells
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment