docker/pod-webhooks
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Compare commits

base: docker:master
Branches Tags
docker:master
docker:gh-pages
docker:docs-dir
docker:fix-submodules
..
compare: docker:docs-dir
Branches Tags
docker:master
docker:gh-pages
docker:docs-dir
docker:fix-submodules

No commits in common. "master" and "docs-dir" have entirely different histories.
master ... docs-dir

28 changed files with 176 additions and 1275 deletions
Show Stats Expand all files Collapse all files

10
.gitignore vendored
View File

@ -1,12 +1,2 @@
#mkdocs #mkdocs
site/ site/
docker-compose.yml
# ignore scripts generated from templates
scripts/pages_init_setup.py
scripts/subdomains_init_setup.py
scripts/captain-hook-canary.service
scripts/captain_hook_canary.sh
scripts/captain_hook_pull_host.py
scripts/pod-webhooks.service
scripts/output/

22
README.md
View File

@ -7,34 +7,16 @@ This docker pod runs two services:
* Static content server for subdomain pages (nginx) * Static content server for subdomain pages (nginx)
* [d-nginx-subdomains](https://git.charlesreid1.com/docker/d-nginx-subdomains) * [d-nginx-subdomains](https://git.charlesreid1.com/docker/d-nginx-subdomains)
These two services are in this repo as submodules. These two services are in this repo as submodules.
## Links ## Links
See documentation page here: <https://pages.charlesreid1.com/pod-webhooks> See documentation page here: <https://pages.charlesreid1.com/pod-webhooks>
Or visit [docs/index.md](/docs/index.md) Or view it locally in this repository here: [docs/index.md](/docs/index.md)
Source code on git.charlesreid1.com: <https://git.charlesreid1.com/docker/pod-webhooks> Source code on git.charlesreid1.com: <https://git.charlesreid1.com/docker/pod-webhooks>
Source code on github.com: <https://github.com/charlesreid1-docker/pod-webhooks> Source code on github.com: <https://github.com/charlesreid1-docker/pod-webhooks>
## If you get a 403
If you visit the IP of the page running things and you get a 403,
it's probably because you haven't created the `/www/` directory
structure yet.
To do this, you need a folder for each subdomain at `/www/`, along
with subdirectories for `htdocs` and for the `.git` directory
(see `scripts/` directory for scripts to do this automatically):
```
/www/pages.charlesreid1.com/
/www/bots.charlesreid1.com/
/www/hooks.charlesreid1.com/
```

2
b-captain-hook

@ -1 +1 @@
Subproject commit 361f59c21a733a484f48e9bd60bce2d94dbf7b1b Subproject commit f084dce02c9a943a585e192d1e601196dfec5b14

2
d-nginx-subdomains

@ -1 +1 @@
Subproject commit a87dcd5f3d997596c87deec03d396f0911b4b03f Subproject commit 3d82d7728b7e58e101138d8645a98dbb6cff266a

34
docker-compose.yml
View File

@ -1,4 +1,4 @@
version: "3.3" version: "3.1"
services: services:
stormy_nginx_subs: stormy_nginx_subs:
@ -7,36 +7,22 @@ services:
command: /bin/bash -c "nginx -g 'daemon off;'" command: /bin/bash -c "nginx -g 'daemon off;'"
volumes: volumes:
- "./d-nginx-subdomains/conf.d:/etc/nginx/conf.d" - "./d-nginx-subdomains/conf.d:/etc/nginx/conf.d"
- "/www/pages.charlesreid1.red/htdocs:/www/pages.charlesreid1.red/htdocs:ro" - "/www/pages.charlesreid1.com/htdocs:/www/pages.charlesreid1.com/htdocs:ro"
- "/www/hooks.charlesreid1.red/htdocs:/www/hooks.charlesreid1.red/htdocs:ro" - "/www/hooks.charlesreid1.com/htdocs:/www/hooks.charlesreid1.com/htdocs:ro"
- "/www/bots.charlesreid1.red/htdocs:/www/bots.charlesreid1.red/htdocs:ro" - "/www/bots.charlesreid1.com/htdocs:/www/bots.charlesreid1.com/htdocs:ro"
logging:
driver: "json-file"
options:
max-size: "1m"
max-file: "10"
ports: ports:
- "7777:7777" - "7777:7777"
- "7778:7778" - "7778:7778"
- "7779:7779" - "7779:7779"
stormy_captain_hook: stormy_captain_hook:
restart: always
build: b-captain-hook build: b-captain-hook
volumes:
- "/www/pages.charlesreid1.red:/www/pages.charlesreid1.red"
- "/www/hooks.charlesreid1.red:/www/hooks.charlesreid1.red"
- "/www/bots.charlesreid1.red:/www/bots.charlesreid1.red"
- "/tmp/triggers:/tmp/triggers"
- "./b-captain-hook/config.json:/app/config.json"
- "./b-captain-hook/hooks:/app/hooks"
logging:
driver: "json-file"
options:
max-size: "1m"
max-file: "10"
ports: ports:
- "5000:5000" - "5000:5000"
depends_on: volumes:
- stormy_nginx_subs - "/www/pages.charlesreid1.com/htdocs:/www/pages.charlesreid1.com/htdocs"
- "/www/hooks.charlesreid1.com/htdocs:/www/hooks.charlesreid1.com/htdocs"
- "/www/bots.charlesreid1.com/htdocs:/www/bots.charlesreid1.com/htdocs"
- "/tmp/triggers:/tmp/triggers"
- "./b-captain-hook/hooks:/app/hooks"

42
docker-compose.yml.j2
View File

@ -1,42 +0,0 @@
version: "3.3"
services:
stormy_nginx_subs:
restart: always
image: nginx
command: /bin/bash -c "nginx -g 'daemon off;'"
volumes:
- "./d-nginx-subdomains/conf.d:/etc/nginx/conf.d"
- "/www/pages.{{ server_name_default }}/htdocs:/www/pages.{{ server_name_default }}/htdocs:ro"
- "/www/hooks.{{ server_name_default }}/htdocs:/www/hooks.{{ server_name_default }}/htdocs:ro"
- "/www/bots.{{ server_name_default }}/htdocs:/www/bots.{{ server_name_default }}/htdocs:ro"
logging:
driver: "json-file"
options:
max-size: "1m"
max-file: "10"
ports:
- "7777:7777"
- "7778:7778"
- "7779:7779"
stormy_captain_hook:
restart: always
build: b-captain-hook
volumes:
- "/www/pages.{{ server_name_default }}:/www/pages.{{ server_name_default }}"
- "/www/hooks.{{ server_name_default }}:/www/hooks.{{ server_name_default }}"
- "/www/bots.{{ server_name_default }}:/www/bots.{{ server_name_default }}"
- "/tmp/triggers:/tmp/triggers"
- "./b-captain-hook/config.json:/app/config.json"
- "./b-captain-hook/hooks:/app/hooks"
logging:
driver: "json-file"
options:
max-size: "1m"
max-file: "10"
ports:
- "5000:5000"
depends_on:
- stormy_nginx_subs

28
docs/adding.md → docs/Adding.md
View File

@ -1,4 +1,4 @@
## Adding New Hooks ## Adding Hooks
To add a hook to Captain Hook: To add a hook to Captain Hook:
@ -17,28 +17,18 @@ To add a hook to Captain Hook:
Settings > Webhooks page, and add a Gitea webhook. Settings > Webhooks page, and add a Gitea webhook.
5. Enter info: 5. Enter info:
a. Payload URL is the Captain Hook server, which is `https://hooks.charlesreid1.com/webhook`. a. Payload URL is the Captain Hook server, which is `https://hooks.charlesreid1.com/webhook`.
b. Content type is `application/json` b. Content type is application/json
c. Secret is (that's my little secret) c. Secret is my little secret
d. Pick what events you would like to trigger webhooks, usually "just the push event" d. Pick what you'd like, I usually go with "just the push event"
6. Save the webhook, then click on the webhook again to open it back up. 6. Save the webhook, then click on the webhook again to open it back up.
Scroll down to the bottom right and click "Test Delivery". Scroll down to the bottom right and click "Test Delivery".
You should see a green success sign. You should see a green success sign. If you see a red warning sign:
* ensure webhooks-subdomain pods are running
* ensure port 5000 open in captain hook container and on blackbeard and on aws
* ensure hook has been added to b-captain-hook repository's hooks folder
## Debugging Failed Hooks
If you see a red warning sign:
* Ensure the webhooks docker pod is actually running okay (`docker ps` on the host machine)
* Ensure port 5000 is open in the Captain Hook container, and on the host machine
* Ensure you can see port 5000 of the `pod-webhooks` host machine from the `pod-charlesreid1` host machine
* Ensure there is actually a hook in the `hooks/` directory of the Captain Hook repo
Captain Hook repo: <https://git.charlesreid1.com/bots/b-captain-hook>
Captain Hook repo (Github mirror): <https://github.com/charlesreid1-docker/b-captain-hook>

58
docs/canary.md → docs/Canary.md
View File

@ -1,33 +1,26 @@
# Captain Hook's Canary # Captain Hook's Canary
First things first: Captain Hook is the webhook server that Captain Hook's Canary is a mechanism by which a docker pod
is part of the webhooks docker pod. It receives webhooks running a webhook server can send a signal to the host to
from Github and Gitea and uses them to trigger scrips. restart the docker pod.
Links to [documentation](https://pages.charlesreid1.com/b-captain-hook)
and [code](https://git.charlesreid1.com/bots/b-captain-hook)
for Captain Hook.
Captain Hook's Canary is a mechanism by which the Captain Hook
webhooks server (running in a docker container) can trigger an action
on the host machine (running the pod). In this case the action is to
update Captain Hook and restart the docker pod anytime a webhook is
received indicating the Captain Hook repo has changed.
This is done by bind-mounting a host directory at `/tmp/triggers/` This is done by bind-mounting a host directory at `/tmp/triggers/`
inside the Captain Hook docker container. When Captain Hook receives inside the docker container, and when a webhook is received
a webhook from Github or Gitea that indicates the Captain Hook from git.charlesreid1.com that indicates there was a change
repo (<https://git.charlesreid1.com/bots/b-captain-hook> or to Captain Hook, the docker pod creates a trigger file.
<https://github.com/charlesreid1/captain-hook>), it creates a
trigger file.
Meanwhile, on the host that is running the docker pod, a service The canary bash script, meanwhile, is a bash script that runs
script is running continuously to check for that trigger file forever and checks for a trigger file from the docker pod
every 10 seconds. If the trigger file is seen, it updates the every 10 seconds.
Captain Hook git repository on the host machine and then restarts
the docker pod.
Sections below cover the following scripts, all run on the host: The pull host Captain Hook script is a script that updates the
Captain Hook git repo on the host machine.
On top of all of that, we also need a startup service that will
actually run the captain hook canary script, and keep it running,
and stop it when we ask it to stop.
Sections below cover:
* The canary bash script * The canary bash script
* The docker host pull script * The docker host pull script
* The canary statup service * The canary statup service
@ -63,18 +56,15 @@ and this script processes each trigger differently.
```bash ```bash
#!/bin/bash #!/bin/bash
TRIGGER="/tmp/triggers/push-b-captain-hook-master"
UPDATE_SCRIPT="${HOME}/pod-webhooks/scripts/captain_hook_pull_host.py"
while true while true
do do
# bootstrap-pull captain hook # bootstrap-pull captain hook
if [ -f "$TRIGGER" ]; then if [ -f "/tmp/triggers/push-b-captain-hook-master" ]; then
echo "CAPTAIN HOOK'S CANARY:" echo "CAPTAIN HOOK'S CANARY:"
echo "Running trigger to update Captain Hook on the host machine (user charles)" echo "Running trigger to update Captain Hook on the host machine (user charles)"
sudo -H -u charles python $UPDATE_SCRIPT sudo -H -u charles python /home/charles/blackbeard_scripts/captain_hook_pull_host.py
echo "All done." echo "All done."
rm -f ${TRIGGER} rm -f "/tmp/triggers/push-b-captain-hook-master"
fi fi
sleep 10; sleep 10;
@ -107,7 +97,7 @@ of Captain Hook running on the host, and
restart the container pod. restart the container pod.
""" """
work_dir = os.path.join('/home','charles','pod-webhooks','b-captain-hook') work_dir = os.path.join('/home','charles','codes','bots','b-captain-hook')
# Step 1: # Step 1:
# Update Captain Hook # Update Captain Hook
@ -131,18 +121,18 @@ The stop directive uses pgrep to find the process id and stops any PIDs returned
``` ```
[Unit] [Unit]
Description=captain hook canary script Description=captain hook canary script
Requires=pod-webhooks.service Requires=dockerpod-captainhook.service
After=pod-webhooks.service After=dockerpod-captainhook.service
[Service] [Service]
Restart=always Restart=always
ExecStart=/home/charles/pod-webhooks/scripts/captain_hook_canary.sh ExecStart=/home/charles/blackbeard_scripts/captain_hook_canary.sh
ExecStop=/usr/bin/pgrep -f captain_hook_canary | /usr/bin/xargs /bin/kill ExecStop=/usr/bin/pgrep -f captain_hook_canary | /usr/bin/xargs /bin/kill
[Install] [Install]
WantedBy=default.target WantedBy=default.target
``` ```
See [Services](services.md) for more info on what to do with this file. See [Services](Services.md) for more info on what to do with this file.

15
docs/running.md → docs/Running.md
View File

@ -1,13 +1,12 @@
## The Docker Compose File ## The Docker Compose File
The `docker-compose.yml` file contains everything needed to The `docker-compose.yml` file contains all the directives needed
run the webhooks docker pod: a Captain Hook webhook server, and to run a docker pod for captain hook and the subdomains nginx server.
an nginx server to serve up static pages for each subdomain.
Why use docker-compose instead of docker? Why use docker-compose instead of docker?
docker-compose is the preferred way to run multiple containers. docker-compose is the preferred way to run multiple containers.
## Running Webhooks Docker Pod from Command Line ## Running Webhooks-Subdomains Docker Pod from Command Line
Run the pod in the foreground or background by running these Run the pod in the foreground or background by running these
commands from the directory containing `docker-compose.yml`: commands from the directory containing `docker-compose.yml`:
@ -17,15 +16,13 @@ docker-compose up # interactive
docker-compose up -d # detached docker-compose up -d # detached
``` ```
If you want to rebuild all the containers before bringing If you want to rebuild all the containers before running up,
the pod up, add the `--build` flag:
``` ```
docker-compose up --build docker-compose up --build
``` ```
If you just want to rebuild the containers without bringing If you just want to rebuild the containers,
them up,
``` ```
docker-compose build docker-compose build
@ -38,7 +35,7 @@ docker-compose build --no-cache
``` ```
***WARNING:*** this will re-download all aptitude packages, ***WARNING:*** this will re-download all aptitude packages,
which can be extremely slow. Use with caution. which can be extremely slow. Use with caution.)
You can restart all containers in a pod using the restart command: You can restart all containers in a pod using the restart command:

86
docs/Services.md Normal file
View File

@ -0,0 +1,86 @@
## Running Hooks-Subdomains Docker Pod as Startup Service
The webhooks-subdomains docker pod requires two startup
services - one keeps the docker pod running, the other
watches a folder shared between the host and container
for signals from the container, and uses that to trigger
updates to the subdomains web content.
Also see the `services/` folder of the
[dotfiles/debian repository](https://git.charlesreid1.com/dotfiles/debian),
repository for the systemd services.
### Service 1: Webhooks Docker Pod
This service keeps the webhooks docker pod service running
continuously. If the pod stops, this service will restart it.
(This service should not be running if you are troubleshooting
the docker pod.)
**`dockerpod-webhooks.service:`**
```
[Unit]
Description=webhooks and subdomains docker pod
Requires=docker.service
After=docker.service
[Service]
Restart=always
ExecStart=/usr/local/bin/docker-compose -f /home/charles/codes/docker/pod-webhooks/docker-compose.yml up
ExecStop=/usr/local/bin/docker-compose -f /home/charles/codes/docker/pod-webhooks/docker-compose.yml down
[Install]
WantedBy=default.target
```
**`captain-hook-canary.service:`**
```
[Unit]
Description=captain hook canary script
Requires=dockerpod-captainhook.service
After=dockerpod-captainhook.service
[Service]
Restart=always
ExecStart=/home/charles/blackbeard_scripts/captain_hook_canary.sh
ExecStop=/usr/bin/pgrep -f captain_hook_canary | /usr/bin/xargs /bin/kill
[Install]
WantedBy=default.target
```
Now install the service to `/etc/systemd/system/dockerpod-webhooks.servce`,
and/or `/etc/systemd/system/captain-hook-canary.servce`,
and activate it:
```
sudo systemctl enable dockerpod-webhooks.service
sudo systemctl enable captain-hook-canary.service
```
Now you can start/stop the service with:
```
sudo systemctl (start|stop) dockerpod-webhooks.service
sudo systemctl (start|stop) captain-hook-canary.service
```
As mentioned above, these services should be stopped before
doing a `docker-compose stop` or a `docker-compose up --build`
to keep the pod from respawning in the middle of the task.
Stop using:
```
sudo systemctl stop dockerpod-webhooks.service
sudo systemctl stop captain-hook-canary.service
```

11
docs/css/custom.css
View File

@ -1,11 +0,0 @@
.md-typeset h1 { font-weight: 600; }
.md-typeset h2 { font-weight: 600; }
.md-typeset h3 { font-weight: 600; }
.md-typeset h4 { font-weight: 600; }
body {
background-color: #FAFAFA;
}
div.body {
background-color: #FAFAFA;
}

100
docs/index.md
View File

@ -9,49 +9,22 @@ This docker pod runs two services:
These two services are in this repo as submodules. These two services are in this repo as submodules.
## Links
See documentation page here: <https://pages.charlesreid1.com/pod-webhooks>
Source code on git.charlesreid1.com: <https://git.charlesreid1.com/docker/pod-webhooks>
Source code on github.com: <https://github.com/charlesreid1-docker/pod-webhooks>
## Initial Setup of Pages Subdomain
The pages.charlesreid1.com subdomain is served from content
in the directory `/www/pages.charlesreid1.com`.
To set this up with pages and sites that should exist already,
use the set up script at `scripts/pages_init_setup.py` and
run it on the pages.charlesreid1.com server:
```
python scripts/pages_init_setup.py
```
This will create the `/www/pages.charlesreid1.com/` directory
structure and will clone several repositories to populate it
with content.
## Adding Hooks ## Adding Hooks
Since this is probably the only thing you'll care about once everything Since this is probably the only thing you'll care about once everything
is actually running... until it breaks. is actually running... until it breaks.
[How To Add A Hook](adding.md) [How To Add A Hook](Adding.md)
## How It Works ## How It Works
See [Running.md](running.md) for info about running this docker pod. See [Running.md](Running.md) for info about running this docker pod.
* Running the Docker Pod from Comand Line * Running the Docker Pod from Comand Line
* Workflow for Docker Pod Updates * Workflow for Docker Pod Updates
See [Services.md](services.md) for info about running startup services. See [Services.md](Services.md) for info about running startup services.
* Running the Docker Pod as a Startup Service * Running the Docker Pod as a Startup Service
* Running Captain Hook's Canary (Script) * Running Captain Hook's Canary (Script)
@ -59,18 +32,18 @@ See [Services.md](services.md) for info about running startup services.
Enable/disable service (installs/uninstalls, but does not start): Enable/disable service (installs/uninstalls, but does not start):
``` ```
sudo systemctl (enable|disable) pod-webhooks.service sudo systemctl (enable|disable) dockerpod-webhooks.service
sudo systemctl (enable|disable) captain-hook-canary.service sudo systemctl (enable|disable) captain-hook-canary.service
``` ```
Start/stop: Start/stop:
``` ```
sudo systemctl (start|stop) pod-webhooks.service sudo systemctl (start|stop) dockerpod-webhooks.service
sudo systemctl (start|stop) captain-hook-canary.service sudo systemctl (start|stop) captain-hook-canary.service
``` ```
See [Captain Hook's Canary](canary.md) for details on the canary script that allows the See [Captain Hook's Canary (Canary.md)](Canary.md) for details on the canary script that allows the
webhooks docker pod to trigger itself to be re-loaded when there are new hooks webhooks docker pod to trigger itself to be re-loaded when there are new hooks
added to captain hook. added to captain hook.
@ -98,34 +71,32 @@ on the contents of `/www/pages.charlesreid1.com/my-page`.
### Captain Hook's Canary ### Captain Hook's Canary
Captain Hook presents a bit of a paradox: what happens when Captain Hook installs Captain Hook presents a bit of a paradox: the webhook docker pod needs to be
a hook for itself, and detects that Captain Hook itself has changed? able to tell the host to restart the webhook docker pod when changes are pushed
to Captain Hook itself.
In this case, the webhooks docker pod needs to be able to tell the host machine This is done by Captain Hook's Canary. This is a script that checks every 10 seconds
to restart the webhooks docker pod. for a trigger file in a directory mounted between the host and container. If the
trigger file is present, the host will update its copy of Captain Hook,
This is done by Captain Hook's Canary. This is a service script that checks every
10 seconds for a trigger file in a directory mounted between the host and container.
If the trigger file is present, the host will update its copy of Captain Hook,
then restart the webhooks-subdomains docker pod. then restart the webhooks-subdomains docker pod.
As per the [dotfiles/debian](https://git.charlesreid1.com/dotfiles/debian) repo, As per the [`blackbeard_scripts`](https://git.charlesreid1.com/dotfiles/debian/src/branch/master/dotfiles/blackbeard_scripts)
directory of [dotfiles/debian](https://git.charlesreid1.com/dotfiles/debian),
the `captain_hook_canary.sh` canary will restart the webhooks docker pod if it the `captain_hook_canary.sh` canary will restart the webhooks docker pod if it
detects the presence following file: detects the file:
``` ```
/tmp/triggers/push-b-captain-hook-master /tmp/triggers/push-b-captain-hook-master
``` ```
(The canary script will remove this file once it has restarted the webhooks docker pod.) (The canary script will clean up this file.)
Now, a hook can be added to Captain Hook that will be run when there is a push event Now a hook can be added to Captain Hook that will be run when there is a push event
on the master branch of [bots/b-captain-hook](https://git.charlesreid1.com/bots/b-captain-hook). on the master branch of [bots/b-captain-hook](https://git.charlesreid1.com/bots/b-captain-hook).
By creating a hook named `push-b-captain-hook-master` in the By creating a hook named `push-b-captain-hook-master` in the
`hooks/` directory of Captain Hook, and having it simply run `hooks/` directory of captain hook that runs a simple `touch` command,
`touch /tmp/triggers/push-b-captain-hook-master`, this webhook can trigger the script which triggers a restart of the
this webhook can trigger the Captain Hook's Canary service script, docker pod.
which triggers a restart of the webhooks docker pod.
Code: <https://git.charlesreid1.com/bots/b-captain-hook/src/branch/master/hooks/push-b-captain-hook-master> Code: <https://git.charlesreid1.com/bots/b-captain-hook/src/branch/master/hooks/push-b-captain-hook-master>
@ -147,30 +118,15 @@ the correct secret or the trigger will be ignored.
## Servers ## Servers
[pod-webhooks](https://pages.charlesreid1.com/pod-webhooks/) This pod runs on blackbeard.
runs on a virtual server somewhere.
This pod's nginx service provides a backend that is The nginx service is reverse-proxied HTTP with krash,
reverse-proxied by the machine running and accessible at ports 7777+ and up.
[pod-charlesreid1](https://pages.charlesreid1.com/pod-charlesreid1)
(and the whole <https://charlesreid1.com> frontend).
It opens ports 7777+ and up (one per subdomain).
The [pod-charlesreid1](https://pages.charlesreid1.com/pod-charlesreid1) The Captain Hook webhook server is also reverse-proxied HTTP.
docker pod (actually the The krash nginx server will handle all traffic to
[d-nginx-charlesreid1](https://pages.charlesreid1.com/d-nginx-charlesreid1/) <https://hooks.charlesreid1.com> except URLs prefixed
submodule in that pod) contains the nginx with `webhook`, which are forwarded on to Captain Hook
config files that control the reverse proxy on port 5000.
behavior of <https://charlesreid1.com>.
Like the nginx service in this webhooks pod, the Captain Hook
webhook service is also reverse-proxied by the main nginx frontend
running <https://charlesreid1.com> with the
[pod-charlesreid1](https://pages.charlesreid1.com/pod-charlesreid1)
docker pod. If a URL request for <https://hooks.charlesreid1.com>
is received by the <https://charlesreid1.com> server,
it is forwarded to the machine running the [pod-webhooks](https://git.charlesreid1.com/pod-webhooks)
docker pod, which decides whether it is a POST request
(that should be sent to this pod's Captain Hook) or a GET request
(that should be sent to this pod's nginx).

93
docs/services.md
View File

@ -1,93 +0,0 @@
## Running Hooks-Subdomains Docker Pod as Startup Service
The webhooks-subdomains docker pod requires two startup
services:
* Docker pod service - this startup service keeps the pod running,
and will restart it if it crashes
* Captain Hook's canary - this watches a folder that is shared between
the docker pod host and the Captain Hook container, which allows the
pod to send triggers to the host.
Also see the `scripts/` folder of this repo,
[pod-webhooks](https://git.charlesreid1.com/docker/pod-webhooks)
([Github mirror](https://github.com/charlesreid1-docker/pod-webhooks)).
### Docker Pod Startup Service
This service keeps the webhooks docker pod service running
continuously. If the pod stops, this service will restart it.
(This service should not be running if you are troubleshooting
the docker pod, otherwise every time you try and stop the pod
it will respawn.)
**`pod-webhooks.service:`**
```
[Unit]
Description=webhooks and subdomains docker pod
Requires=docker.service
After=docker.service
[Service]
Restart=always
ExecStart=/usr/local/bin/docker-compose -f /home/charles/codes/docker/pod-webhooks/docker-compose.yml up
ExecStop=/usr/local/bin/docker-compose -f /home/charles/codes/docker/pod-webhooks/docker-compose.yml down
[Install]
WantedBy=default.target
```
### Captain Hook's Canary Startup Service
This service just watches a folder for a particular
watchfile, and runs a script if it sees the watchfile
appear.
**`captain-hook-canary.service:`**
```
[Unit]
Description=captain hook canary script
Requires=pod-webhooks.service
After=pod-webhooks.service
[Service]
Restart=always
ExecStart=/home/charles/pod-webhooks/scripts/captain_hook_canary.sh
ExecStop=/usr/bin/pgrep -f captain_hook_canary | /usr/bin/xargs /bin/kill
[Install]
WantedBy=default.target
```
### Installing Services
Install the services by copying the `*.service` files
to `/etc/systemd/system/dockerpod-webhooks.servce`
and `/etc/systemd/system/captain-hook-canary.servce`,
and activate the startup services:
```
sudo systemctl enable pod-webhooks.service
sudo systemctl enable captain-hook-canary.service
```
Now you can start/stop the services with:
```
sudo systemctl (start|stop) pod-webhooks.service
sudo systemctl (start|stop) captain-hook-canary.service
```
As mentioned above, these services should be stopped before
doing a `docker-compose stop` or a `docker-compose up --build`
to keep the pod from respawning in the middle of the task.

2
mkdocs-material

@ -1 +1 @@
Subproject commit b0c6890853aa9138baf5f9749862b927518ab656 Subproject commit 6569122bb1e9d6bd73fd964639c33c9a3a4f46d3

17
mkdocs.yml
View File

@ -13,19 +13,19 @@ theme:
name: null name: null
custom_dir: 'mkdocs-material/material' custom_dir: 'mkdocs-material/material'
palette: palette:
primary: 'blue' primary: 'pink'
accent: 'blue' accent: 'pink'
logo: logo:
icon: 'dns' icon: 'dns'
font: font:
text: 'Roboto' text: 'Roboto'
code: 'Roboto Mono' code: 'Roboto Mono'
nav: pages:
- "Home": "index.md" - Home: 'index.md'
- "Adding Webhooks": "adding.md" - Adding Webhooks: 'Adding.md'
- "Running Captain Hook": "running.md" - Running Captain Hook: 'Running.md'
- "Captain Hook Startup Services": "services.md" - Captain Hook Startup Services: 'Services.md'
- "Captain Hook's Canary": "canary.md" - Captain Hook's Canary: 'Canary.md'
# Extensions # Extensions
markdown_extensions: markdown_extensions:
@ -35,4 +35,3 @@ markdown_extensions:
- toc: - toc:
permalink: true permalink: true
strict: true

24
scripts/Readme.md
View File

@ -1,24 +0,0 @@
# pod-webhooks scripts
Copy this `pod-webhooks.service` service script to
`/etc/systemd/system/pod-webhooks.service`:
```
sudo cp pod-webhooks.service /etc/systemd/system/pod-webhooks.service
```
Enable/disable the service:
```
sudo systemctl enable pod-webhooks
sudo systemctl disable pod-webhooks
```
Start/restart/stop the service:
```
sudo systemctl start pod-webhooks
sudo systemctl restart pod-webhooks
sudo systemctl stop pod-webhooks
```

89
scripts/apply_templates_captainhook.py
View File

@ -1,89 +0,0 @@
import os, re, sys
import glob
from jinja2 import Environment, FileSystemLoader, select_autoescape
"""
Apply Default Values to
Captain Hook Jinja Templates
This script applies default values to
templates in this folder.
The templates are used by Ansible,
but this script uses the same template
engine as Ansible to apply template
variable values to the template files
and make real files.
variables are:
- `webhooks_install_dir` - location of pod-webhooks repo
- `username` - user/group name to change ownership to
- `server_name_default` - name of server
(e.g., charlesreid1.com or charlesreid1.red)
"""
# Where templates live
TEMPLATEDIR = '.'
# Where rendered templates will go
OUTDIR = 'output'
# Should existing (destination) files
# be overwritten if they exist?
OVERWRITE = True
# Template variables
TV = {
'server_name_default': 'charlesreid1.red',
'username': 'charles',
'webhooks_install_dir' : '/home/charles/pod-webhooks'
}
def apply_templates(template_dir, output_dir, template_vars, overwrite=False):
"""Apply the template variables
to the template files.
"""
if not os.path.exists(output_dir):
msg = "Error: output dir %s does not exist!"%(output_dir)
raise Exception(msg)
if not os.path.exists(template_dir):
msg = "Error: template dir %s does not exist!"%(output_dir)
raise Exception(msg)
# Jinja env
env = Environment(loader=FileSystemLoader('.'))
# Render templates
template_files = glob.glob('captain*.j2')
render_files = [re.sub('\.j2','',s) for s in template_files]
for rfile,tfile in zip(render_files,template_files):
# Get rendered template content
content = env.get_template(tfile).render(**template_vars)
# Write to file
dest = os.path.join(output_dir,rfile)
if os.path.exists(dest) and overwrite is False:
msg = "Error: template rendering destination %s already exists!"%(dest)
raise Exception(msg)
with open(dest,'w') as f:
f.write(content)
print("Rendered the following templates:%s\nOutput files:%s\n"%(
"".join(["\n- "+os.path.join(template_dir,j) for j in template_files]),
"".join(["\n- "+os.path.join(output_dir,j) for j in render_files])
))
if __name__=="__main__":
apply_templates(TEMPLATEDIR,OUTDIR,TV,OVERWRITE)

84
scripts/apply_templates_pod.py
View File

@ -1,84 +0,0 @@
import os, re, sys
import glob
from jinja2 import Environment, FileSystemLoader, select_autoescape
"""
Apply Default Values to
pod-webhooks Jinja Templates
This script applies default values to
templates in this folder.
The templates are used by Ansible,
but this script uses the same template
engine as Ansible to apply template
variable values to the template files
and make real files.
variables are:
- `webhooks_install_dir` - location of pod-webhooks repo
(i.e., location of pod-webhooks docker-compose file)
"""
# Where templates live
TEMPLATEDIR = '.'
# Where rendered templates will go
OUTDIR = 'output'
# Should existing (destination) files
# be overwritten if they exist?
OVERWRITE = True
# Template variables
TV = {
'webhooks_install_dir' : '/home/charles/pod-webhooks'
}
def apply_templates(template_dir, output_dir, template_vars, overwrite=False):
"""Apply the template variables
to the template files.
"""
if not os.path.exists(output_dir):
msg = "Error: output dir %s does not exist!"%(output_dir)
raise Exception(msg)
if not os.path.exists(template_dir):
msg = "Error: template dir %s does not exist!"%(output_dir)
raise Exception(msg)
# Jinja env
env = Environment(loader=FileSystemLoader('.'))
# Render templates
template_files = glob.glob('*.service.j2')
render_files = [re.sub('\.j2','',s) for s in template_files]
for rfile,tfile in zip(render_files,template_files):
# Get rendered template content
content = env.get_template(tfile).render(**template_vars)
# Write to file
dest = os.path.join(output_dir,rfile)
if os.path.exists(dest) and overwrite is False:
msg = "Error: template rendering destination %s already exists!"%(dest)
raise Exception(msg)
with open(dest,'w') as f:
f.write(content)
print("Rendered the following templates:%s\nOutput files:%s\n"%(
"".join(["\n- "+os.path.join(template_dir,j) for j in template_files]),
"".join(["\n- "+os.path.join(output_dir,j) for j in render_files])
))
if __name__=="__main__":
apply_templates(TEMPLATEDIR,OUTDIR,TV,OVERWRITE)

90
scripts/apply_templates_subdomains.py
View File

@ -1,90 +0,0 @@
import os, re, sys
import glob
import subprocess
from jinja2 import Environment, FileSystemLoader, select_autoescape
"""
Apply Default Values to
SUBDOMAINS Jinja Templates
This script applies default values to
templates in this folder.
The templates are used by Ansible,
but this script uses the same template
engine as Ansible to apply template
variable values to the template files
and make real files.
only variables are:
- `username` - user/group name to change ownership to
- `server_name_default` - name of server
(e.g., charlesreid1.com or charlesreid1.red)
"""
# Where templates live
TEMPLATEDIR = '.'
# Where rendered templates will go
OUTDIR = 'output'
# Should existing (destination) files
# be overwritten if they exist?
OVERWRITE = True
# Template variables
TV = {
'server_name_default': 'charlesreid1.red',
'username': 'charles'
}
def apply_templates(template_dir, output_dir, template_vars, overwrite=False):
"""Apply the template variables
to the template files.
"""
if not os.path.exists(output_dir):
msg = "Error: output dir %s does not exist!"%(output_dir)
raise Exception(msg)
if not os.path.exists(template_dir):
msg = "Error: template dir %s does not exist!"%(output_dir)
raise Exception(msg)
# Jinja env
env = Environment(loader=FileSystemLoader('.'))
# Render templates
template_files = glob.glob('*_setup.py.j2') + glob.glob('*_pull.py.j2')
render_files = [re.sub('\.j2','',s) for s in template_files]
for rfile,tfile in zip(render_files,template_files):
# Get rendered template content
content = env.get_template(tfile).render(**template_vars)
# Write to file
dest = os.path.join(output_dir,rfile)
if os.path.exists(dest) and overwrite is False:
msg = "Error: template rendering destination %s already exists!"%(dest)
raise Exception(msg)
with open(dest,'w') as f:
f.write(content)
x = 'executioner.py'
subprocess.call(['cp',x,os.path.join(output_dir,x)])
print("Rendered the following templates:%s\nOutput files:%s\n"%(
"".join(["\n- "+os.path.join(template_dir,j) for j in template_files]),
"".join(["\n- "+os.path.join(output_dir,j) for j in render_files])
))
if __name__=="__main__":
apply_templates(TEMPLATEDIR,OUTDIR,TV,OVERWRITE)

12
scripts/captain-hook-canary.service.j2
View File

@ -1,12 +0,0 @@
[Unit]
Description=captain hook canary script
Requires=pod-webhooks.service
After=pod-webhooks.service
[Service]
Restart=always
ExecStart={{ webhooks_install_dir }}/scripts/captain_hook_canary.sh
ExecStop=/usr/bin/pgrep -f captain_hook_canary | /usr/bin/xargs /bin/kill
[Install]
WantedBy=default.target

47
scripts/captain_hook_canary.sh.j2
View File

@ -1,47 +0,0 @@
#!/bin/bash
: '
Captain Hook Canary Script (Template)
Note: this needs an associated systemd service.
See the services directory of the dotfiles repo.
This is a canary script for connecting
the Captain Hook container to the host
machine, and triggering tasks on the
host machine with webhooks.
The Captain Hook container mounts the
following host directory inside the
container (same location for host/container):
/tmp/triggers/
When a webhook in Captain Hook wants to
trigger an event on the host (blackbeard),
it puts a file in /tmp/triggers/.
Meanwhile, on the host, this script checks
every 10 seconds for trigger files.
Each webhook can create its own trigger file,
and this script processes each trigger differently.
'
while true
do
# bootstrap-pull captain hook
if [ -f "/tmp/triggers/push-b-captain-hook-master" ]; then
echo "CAPTAIN HOOK'S CANARY:"
echo "Running trigger to update Captain Hook on the host machine (user charles)"
sudo -H -u charles python {{ webhooks_install_dir }}/scripts/captain_hook_pull_host.py
echo "All done."
rm -f "/tmp/triggers/push-b-captain-hook-master"
touch /tmp/canary-yup-host-works
fi
sleep 10;
done

110
scripts/captain_hook_pull_host.py.j2
View File

@ -1,110 +0,0 @@
#!/usr/bin/env python3
import subprocess
import os
import time
"""
Captain Hook: Pull Captain Hook on the Host
This script is called by the host machine
(blackbeard) running the Captain Hook container.
This is triggered by push actions to the
master branch of b-captain-hook.
The action is to update (git pull) the copy
of Captain Hook running on the host, and
restart the container pod.
"""
pod_dir = '{{ webhooks_install_dir }}'
work_dir = os.path.join(pod_dir,'b-captain-hook')
from datetime import datetime
d = datetime.now().strftime('%Y-m-%d')
with open('/tmp/captain_hook_pull_host_%s.log'%(d),'w') as f:
# Step 1:
# Update Captain Hook
co_cmd = ['git','checkout','master']
proc = subprocess.call(co_cmd, cwd=work_dir, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = proc.communicate()
o = stdout.decode('utf-8')
e = stderr.decode('utf-8')
f.write(" ".join(co_cmd))
f.write("\n")
f.write("-"*40)
f.write("\n")
f.write(o)
f.write("\n")
f.write(e)
f.write("\n\n")
f_cmd = ['git','fetch','--all']
proc = subprocess.call(f_cmd, cwd=work_dir, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = proc.communicate()
o = stdout.decode('utf-8')
e = stderr.decode('utf-8')
f.write(" ".join(f_cmd))
f.write("\n")
f.write("-"*40)
f.write("\n")
f.write(o)
f.write("\n")
f.write(e)
f.write("\n\n")
time.sleep(5)
pull_cmd = ['git','pull','origin','master']
proc = subprocess.call(pull_cmd, cwd=work_dir, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = proc.communicate()
o = stdout.decode('utf-8')
e = stderr.decode('utf-8')
f.write(" ".join(pull_cmd))
f.write("\n")
f.write("-"*40)
f.write("\n")
f.write(o)
f.write("\n")
f.write(e)
f.write("\n\n")
time.sleep(10)
submod_cmd = ['git','submodule','update','--remote']
proc = subprocess.call(submod_cmd, cwd=work_dir, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = proc.communicate()
o = stdout.decode('utf-8')
e = stderr.decode('utf-8')
f.write(" ".join(pull_cmd))
f.write("\n")
f.write("-"*40)
f.write("\n")
f.write(o)
f.write("\n")
f.write(e)
f.write("\n\n")
time.sleep(10)
# Step 2:
# Restart Captain Hook pod
pod_restart = ['systemctl','restart','pod-webhooks']
#pod_restart = ['docker-compose','restart']
proc = subprocess.call(pod_restart, cwd=pod_dir, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = proc.communicate()
o = stdout.decode('utf-8')
e = stderr.decode('utf-8')
f.write(" ".join(pod_restart))
f.write("\n")
f.write("-"*40)
f.write("\n")
f.write(o)
f.write("\n")
f.write(e)
f.write("\n\n")

21
scripts/executioner.py
View File

@ -1,21 +0,0 @@
#!/usr/bin/env python3
import subprocess
import os
def execute(cmd):
"""
A function to run a command and return the
lines of output as they are generated,
allowing the calling function to "stream"
the output of the command to print() or etc.
"""
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True)
for stdout_line in iter(p.stdout.readline, ""):
yield stdout_line
p.stdout.close()
err = "".join([j for j in iter(p.stderr.readline,"")])
return_code = p.wait()
if return_code:
yield err
raise subprocess.CalledProcessError(return_code, cmd)

125
scripts/pages_init_setup.py.j2
View File

@ -1,125 +0,0 @@
#!/usr/bin/env python3
import subprocess
import os
from executioner import execute
"""
Pages Initial Setup
This script sets up the initial pages.charlesreid1.XYZ
folder structure and checks out all pages that should
be present initially at pages.charlesreid1.XYZ.
This clones the page for a given project to
/www/pages.charlesreid1.XYZ/htdocs/project-name
(where project = repo)
"""
SERVER_NAME_DEFAULT = '{{ server_name_default }}'
USERNAME = '{{ username }}'
# Repos must have a gh-pages branch
repo_names = '''bots/b-apollo
bots/b-captain-hook
bots/b-ginsberg
bots/b-milton
bots/uncle-archie
bots/boring-mind-machine
docker/d-gitea
docker/d-mediawiki
docker/d-mysql
docker/d-nginx-charlesreid1
docker/d-nginx-subdomains
docker/d-phpmyadmin
docker/d-python-files
docker/d-python-helium
charlesreid1/dont-sudo-pip
bots/embarcadero-mind-machine
charlesreid1/git-commit-ectomy
charlesreid1/git-subway-maps
charlesreid1/github-heroku-attack-rabbits
charlesreid1/how-do-i-heroku
charlesreid1/how-do-i-pandoc
charlesreid1/how-do-i-pelican
charlesreid1/how-do-i-pyenv
charlesreid1/how-do-i-snakemake
docker/pod-bots
docker/pod-charlesreid1
docker/pod-webhooks
bots/russian-rainbow-mind-machine
charlesreid1/scurvy-knave-theme
charlesreid1/translate-yer-docs
charlesreid1/wisko-manual'''.split('\n')
repo_urls = ['https://git.charlesreid1.com/%s.git'%(j) for j in repo_names]
root = '/www'
pages = 'pages.%s'%(SERVER_NAME_DEFAULT)
basedir = os.path.join(root,pages)
if not os.path.exists(basedir):
raise Exception("Error: base directory %s does not exist!"%(basedir))
for name,url in zip(repo_names,repo_urls):
# for a hypothetical repo "project":
#
# base dir: /www/pages.charlesreid1.com
# .git dir: /www/pages.charlesreid1.com/git.project
# htdocs dir: /www/pages.charlesreid1.com/htdocs/project
# Step 1: clone
# Construct the clone command to set up
# live htdocs directory for this project
repo_owner, repo_name = name.split("/")
clonepth = os.path.join(basedir,"htdocs",repo_name)
dotgitpth = os.path.join(basedir,"git.%s"%(repo_name))
if os.path.exists(clonepth):
print("Git repository has already been cloned, %s already exists on disk!"%(clonepth))
else:
clonecmd = ["git","clone",
"--separate-git-dir=%s"%(dotgitpth),
"-b","gh-pages",
url,clonepth]
print("About to clone gh-pages branch for %s using command:\n"%(name))
print(" $ " + " ".join(clonecmd))
print("\n")
try:
for loo in execute(clonecmd):
print(loo)
print("\n")
print("Done.")
except subprocess.CalledProcessError:
print("Failed on clone command for project %s"%(name))
print("Continuing...")
# Step 2: chown everybody
# Construct chown command
chowncmd = ['chown','-R',
USERNAME+":"+USERNAME,
basedir]
print("About to change owner of %s using command:\n"%(basedir))
print(" $ " + " ".join(chowncmd))
print("\n")
for loo in execute(chowncmd):
print(loo)
print("\n")
print("Done.")

129
scripts/pages_pull.py.j2
View File

@ -1,129 +0,0 @@
#!/usr/bin/env python3
import subprocess
import os
from executioner import execute
"""
Pages Pull Script
This script pulls chnages to all repos in the
pages.charlesreid1.XYZ folder structure
This clones the page for a given project to
/www/pages.charlesreid1.XYZ/htdocs/project-name
(where project = repo)
"""
SERVER_NAME_DEFAULT = '{{ server_name_default }}'
USERNAME = '{{ username }}'
# Repos must have a gh-pages branch
repo_names = '''bots/b-apollo
bots/b-captain-hook
bots/b-ginsberg
bots/b-milton
bots/uncle-archie
bots/boring-mind-machine
docker/d-gitea
docker/d-mediawiki
docker/d-mysql
docker/d-nginx-charlesreid1
docker/d-nginx-subdomains
docker/d-phpmyadmin
docker/d-python-files
docker/d-python-helium
charlesreid1/dont-sudo-pip
bots/embarcadero-mind-machine
charlesreid1/git-commit-ectomy
charlesreid1/git-subway-maps
charlesreid1/github-heroku-attack-rabbits
charlesreid1/how-do-i-heroku
charlesreid1/how-do-i-pandoc
charlesreid1/how-do-i-pelican
charlesreid1/how-do-i-pyenv
charlesreid1/how-do-i-snakemake
docker/pod-bots
docker/pod-charlesreid1
docker/pod-webhooks
bots/russian-rainbow-mind-machine
charlesreid1/scurvy-knave-theme
charlesreid1/translate-yer-docs
charlesreid1/wisko-manual'''.split('\n')
repo_urls = ['https://git.charlesreid1.com/%s.git'%(j) for j in repo_names]
root = '/www'
pages = 'pages.%s'%(SERVER_NAME_DEFAULT)
basedir = os.path.join(root,pages)
htdocsdir = os.path.join(basedir,"htdocs")
if not os.path.exists(basedir):
raise Exception("Error: base directory %s does not exist!"%(basedir))
if not os.path.exists(htdocsdir):
raise Exception("Error: htdocs directory %s does not exist!"%(htdocsdir))
for name,url in zip(repo_names,repo_urls):
# for a hypothetical repo "project":
#
# base dir: /www/pages.charlesreid1.com
# .git dir: /www/pages.charlesreid1.com/git.project
# htdocs dir: /www/pages.charlesreid1.com/htdocs/project
# Step 1: pull
# Construct the git pull command to update
# the live htdocs directory for this project
repo_owner, repo_name = name.split("/")
workdir = os.path.join(basedir,"htdocs",repo_name)
dotgitdir = os.path.join(basedir,"git.%s"%(repo_name))
if not os.path.exists(workdir):
print("Git repository has not yet been cloned, %s does not exist on disk!"%(workdir))
else:
pullcmd = ['git',
'-C', basedir,
'--git-dir=%s'%(dotgitdir),
'--work-tree=%s'%(workdir),
'pull','origin','gh-pages']
print("About to pull gh-pages branch for %s using command:\n"%(name))
print(" $ " + " ".join(pullcmd))
print("\n")
try:
for loo in execute(pullcmd):
print(loo)
print("\n")
print("Done.")
except subprocess.CalledProcessError:
print("Failed on pull command for project %s"%(name))
print("Continuing...")
# Step 2: chown everybody
# Construct chown command
chowncmd = ['chown','-R',
USERNAME+":"+USERNAME,
basedir]
print("About to change owner of %s using command:\n"%(basedir))
print(" $ " + " ".join(chowncmd))
print("\n")
for loo in execute(chowncmd):
print(loo)
print("\n")
print("Done.")

14
scripts/pod-webhooks.service.j2
View File

@ -1,14 +0,0 @@
# Hopefully this comment works
[Unit]
Description=webhooks and subdomains docker pod
Requires=docker.service
After=docker.service
[Service]
Restart=always
ExecStart=/usr/local/bin/docker-compose -f {{ webhooks_install_dir }}/docker-compose.yml up
ExecStop=/usr/local/bin/docker-compose -f {{ webhooks_install_dir }}/docker-compose.yml down
[Install]
WantedBy=default.target

89
scripts/subdomains_init_setup.py.j2
View File

@ -1,89 +0,0 @@
#!/usr/bin/env python3
import subprocess
import os
from executioner import execute
"""
Subdomains Initial Setup
This script sets up the initial folder structure
for the charlesreid1.com subdomains and their content.
"""
SERVER_NAME_DEFAULT = '{{ server_name_default }}'
USERNAME = '{{ username }}'
# Each subdomain page has an htdocs dir at
#
# /www/SUBDOMAIN.charlesreid1.XYZ/htdocs
#
# and a .git directory at
#
# /www/SUBDOMAIN.charlesreid1.XYZ/git.SUBDOMAIN.charlesreid1.XYZ
# Name of all subdomains to set up
sub_names = ['bots','pages','hooks']
subs = [n+'.'+SERVER_NAME_DEFAULT for n in sub_names]
# Use git.charlesreid1.com
# (Note we have to hard-code the suffix
# charlesreid1.com b/c that is the name
# of the repo for ALL subdomains)
urls = ['https://git.charlesreid1.com/charlesreid1/%s.charlesreid1.com'%(n) for n in sub_names]
#
# Use github.com (alternative)
# (Note we have to hard-code the suffix charlesreid1.com)
#urls = ['https://github.com/charlesreid1-docker/%s.charlesreid1.com'%(n) for n in sub_names]
# Base directory for all htdocs folders
pth = '/www'
# Create the base directory
subprocess.call(['mkdir','-p',pth])
# Step 1: clone htdocs dir for each subdomain
for subname, sub, url in zip(sub_names,subs,urls):
# Construct the clone command for the live
# htdocs directory for this subdomain
clonepth = os.path.join(pth,sub,'htdocs')
dotgitpth = os.path.join(pth,sub,'git.%s'%(subname))
if os.path.exists(clonepth):
print("Git repository has already been cloned, %s already exists on disk!"%(clonepth))
else:
clonecmd = ['git','clone',
'--separate-git-dir=%s'%(dotgitpth),
'-b','gh-pages',
url,clonepth]
print("About to clone htdocs dir for %s using command:\n"%(sub))
print(" $ " + " ".join(clonecmd))
print("\n")
for loo in execute(clonecmd):
print(loo)
print("\n")
print("Done.")
# Step 2: chown
# Construct chown command to chown everybody
chowncmd = ['chown','-R',USERNAME+":"+USERNAME,pth]
print("About to change owner of %s using command:\n"%(pth))
print(" $ " + " ".join(chowncmd))
print("\n")
for loo in execute(chowncmd):
print(loo)
print("\n")
print("Done.")

95
scripts/subdomains_pull.py.j2
View File

@ -1,95 +0,0 @@
#!/usr/bin/env python3
import subprocess
import os
from executioner import execute
"""
Subdomains Pull Script
This script sets up the initial folder structure
for the charlesreid1.com subdomains and their content.
"""
SERVER_NAME_DEFAULT = '{{ server_name_default }}'
USERNAME = '{{ username }}'
# Each subdomain page has an htdocs dir at
#
# /www/SUBDOMAIN.charlesreid1.XYZ/htdocs
#
# and a .git directory at
#
# /www/SUBDOMAIN.charlesreid1.XYZ/git.SUBDOMAIN.charlesreid1.XYZ
# Name of all subdomains to set up
sub_names = ['bots','pages','hooks']
subs = [n+'.'+SERVER_NAME_DEFAULT for n in sub_names]
# Use git.charlesreid1.com
# (Note we have to hard-code the suffix
# charlesreid1.com b/c that is the name
# of the repo for ALL subdomains)
urls = ['https://git.charlesreid1.com/charlesreid1/%s.charlesreid1.com'%(n) for n in sub_names]
#
# Use github.com (alternative)
# (Note we have to hard-code the suffix charlesreid1.com)
#urls = ['https://github.com/charlesreid1-docker/%s.charlesreid1.com'%(n) for n in sub_names]
# Base directory for all htdocs folders
pth = '/www'
# Create the base directory
subprocess.call(['mkdir','-p',pth])
# Step 1: clone htdocs dir for each subdomain
for subname, sub, url in zip(sub_names,subs,urls):
# Step 1: pull
# Construct the git pull command to update
# the live htdocs directory for this project
basedir = os.path.join(pth,sub)
htdocsdir = os.path.join(basedir,"htdocs")
dotgitdir = os.path.join(basedir,"git.%s"%(subname))
if not os.path.exists(htdocsdir):
print("Git repository has not been cloned, %s does not exist on disk!"%(htdocsdir))
else:
# git pull, respecting the
# unusual .git/repo folder layout
pullcmd = ['git',
'-C', basedir,
'--git-dir=%s'%(dotgitdir),
'--work-tree=%s'%(htdocsdir),
'pull','origin','gh-pages']
print("About to pull htdocs dir for %s using command:\n"%(sub))
print(" $ " + " ".join(pullcmd))
print("\n")
for loo in execute(pullcmd):
print(loo)
print("\n")
print("Done.")
# Step 2: chown
# Construct chown command to chown everybody
chowncmd = ['chown','-R',USERNAME+":"+USERNAME,basedir]
print("About to change owner of %s using command:\n"%(basedir))
print(" $ " + " ".join(chowncmd))
print("\n")
for loo in execute(chowncmd):
print(loo)
print("\n")
print("Done.")