Browse Source
* 'master' of ssh://git.charlesreid1.com:222/docker/pod-charlesreid1: (22 commits) update gitea submodule redundant in the final stretch, wrapping up gitea, only thing left is backup script utilities another index update add info about python files service update port information group services into TOC and add as index section nginx + ssl finished add nginx ssl info updates to domain control section - top level domain and subdomain info update doomain info on nginx service page update config files in d-nginx-charlesreid1 update index page with what is finished update phpmyadmin port info update port info update with port info nearly-complete nginx service page add phpMyAdmin page update mysql page with links add link to apache page ...spew-fix
Charles Reid
6 years ago
17 changed files with 639 additions and 127 deletions
@ -1 +1 @@
@@ -1 +1 @@
|
||||
Subproject commit 31cc1713145a6f0c4101f966612670349e7649f3 |
||||
Subproject commit 8c2bd1a55ec89c74601c958693912ad9fc62c720 |
@ -1 +1 @@
@@ -1 +1 @@
|
||||
Subproject commit 922d09e2be611295a59924c3c3063f85b12f44ac |
||||
Subproject commit 1591083a8e2c5a4e39bc87e5a9dfd108e759a47c |
@ -1 +1 @@
@@ -1 +1 @@
|
||||
Subproject commit 28f90eb21df82871fe7923617566fb7bb06eb7a8 |
||||
Subproject commit 4bd88e74c169bfe62a1b3709b3b65a6405c7de24 |
@ -1 +1 @@
@@ -1 +1 @@
|
||||
Subproject commit f0629fa74961cde670a09f7f04df88d25386483b |
||||
Subproject commit f9f611a1503484ba230a1ffe2d5cbec1d449835f |
@ -1 +1 @@
@@ -1 +1 @@
|
||||
Subproject commit ea4696c0cdd2a947259bfaa378b764165df03309 |
||||
Subproject commit e84de96196618787219c9a77135fa1726153ddec |
@ -1 +1 @@
@@ -1 +1 @@
|
||||
Subproject commit ff7846b6398e7ebbeb17c49be5909726858b9085 |
||||
Subproject commit 473b497e2851692c808e6d2d9ea660be64930c20 |
@ -0,0 +1,103 @@
@@ -0,0 +1,103 @@
|
||||
# Gitea |
||||
|
||||
Gitea is a self-hosted Github clone. It is written in Go, and |
||||
provides a web interface and an API to interact with an instance |
||||
of a git server. |
||||
|
||||
Gitea manages its own database, so to get data in and out of |
||||
Gitea, use its dump and load functionality (more below). |
||||
|
||||
This page describes how the Gitea container is configured. |
||||
|
||||
## Configuring Docker Container |
||||
|
||||
To run gitea, we use a stock Gitea container image. We set |
||||
several options in the docker configuration: |
||||
|
||||
* `USER_UID` and `USER_GID` are set to `1000` (this avoids some |
||||
problems with files that would otherwise be owned by root) |
||||
|
||||
* Set `restart: always` to restart the container when there |
||||
is a failure |
||||
|
||||
``` |
||||
stormy_gitea: |
||||
image: gitea/gitea:latest |
||||
environment: |
||||
- USER_UID=1000 |
||||
- USER_GID=1000 |
||||
restart: always |
||||
``` |
||||
|
||||
## Gitea Volumes |
||||
|
||||
The Gitea container stores all of its data in `/data/` inside |
||||
the container. |
||||
|
||||
When the container is launched, the `custom/` directory in |
||||
the [docker/d-gitea](https://git.charlesreid1.com/docker/d-gitea) |
||||
repository is mounted to `/data/gitea/`, which is the directory |
||||
that contains the files that are used to control the way that |
||||
Gitea pages look. These contain HTML templates used to render |
||||
different views in Gitea and templates in `custom/` will override |
||||
the default Gitea page templates. |
||||
|
||||
A docker volume named `stormy_gitea_data` is also created and |
||||
mounted at `/data/`. This is a persistent volume that will |
||||
survive even if the container is shut down. |
||||
|
||||
``` |
||||
volumes: |
||||
- "stormy_gitea_data:/data" |
||||
- "./d-gitea/custom/conf/app.ini:/data/gitea/conf/app.ini" |
||||
- "./d-gitea/custom/public:/data/gitea/public" |
||||
- "./d-gitea/custom/templates:/data/gitea/templates" |
||||
``` |
||||
|
||||
## Gitea Ports |
||||
|
||||
Gitea provides both SSH and HTTPS interfaces, as it has its own |
||||
built-in web server and SSH server, as well as git server. |
||||
|
||||
The server that is hosting the Gitea container and this Docker |
||||
pod already has an SSH server listening on port 22, so Gitea |
||||
listens for SSH connections _externally_ on port 222. |
||||
|
||||
``` |
||||
ports: |
||||
- "222:22" |
||||
``` |
||||
|
||||
Note that this _bypasses_ our d-nginx-charlesreid1 nginx container |
||||
entirely and allows clients to connect to Gitea directly. |
||||
|
||||
The Gitea server listens for HTTP/HTTPS connections on port |
||||
3000, but that is by default only listening on the internal |
||||
Docker network, which is exactly how we want it. We want all |
||||
HTTP and HTTPS traffic to be handled by the front-end d-nginx-charlesreid1 |
||||
container, and it will reverse-proxy HTTP/HTTPS requests to |
||||
the Gitea container. |
||||
|
||||
## Gitea Configuration Files |
||||
|
||||
`app.ini` is the name of the configuration file used by Gitea. |
||||
An [example `app.ini` configuration file](https://git.charlesreid1.com/docker/d-gitea/src/branch/master/app.ini.sample) |
||||
is contained in the [docker/d-gitea](https://git.charlesreid1.com/docker/d-gitea) |
||||
repository, as well as a script to |
||||
[make a configuration file](https://git.charlesreid1.com/docker/d-gitea/src/branch/master/make_app_ini.sh). |
||||
|
||||
``` |
||||
volumes: |
||||
- "stormy_gitea_data:/data" |
||||
- "./d-gitea/custom/conf/app.ini:/data/gitea/conf/app.ini" |
||||
``` |
||||
|
||||
|
||||
## Backups |
||||
|
||||
### Backing Up Gitea |
||||
|
||||
### Restoring Gitea |
||||
|
||||
|
||||
|
@ -0,0 +1,348 @@
@@ -0,0 +1,348 @@
|
||||
# Nginx |
||||
|
||||
This describes the configuration of the main nginx container |
||||
for the charlesreid1.com pod. |
||||
|
||||
|
||||
## Configuration Files |
||||
|
||||
The nginx container is critical to routing operations on |
||||
charlesreid1.com, so we cover how the configuration files |
||||
are split up and organized on this page, and include |
||||
a summary of each file below. |
||||
|
||||
|
||||
### Where are the nginx config files? |
||||
|
||||
Nginx configuration files are located in the `d-nginx-charlesreid1` |
||||
submodule of this repository, which points to the |
||||
[docker/d-nginx-charlesreid1](https://git.charlesreid1.com/docker/d-nginx-charlesreid1) |
||||
repository. |
||||
|
||||
Within that repository is the [`conf.d/`](https://git.charlesreid1.com/docker/d-nginx-charlesreid1/src/branch/master/conf.d) |
||||
folder, which contains several `.conf` files. |
||||
|
||||
|
||||
### What Files Are There? |
||||
|
||||
We have three sets of files present: |
||||
|
||||
HTTP config files that redirect HTTP traffic on port 80 to |
||||
be HTTPS traffic on port 443, one for each top-level domain |
||||
(charlesreid1.com, charlesreid1.blue, and charlesreid1.red): |
||||
|
||||
* `http.com.charlesreid1.conf` |
||||
* `http.red.charlesreid1.conf` |
||||
* `http.blue.charlesreid1.conf` |
||||
|
||||
HTTPS rules for charlesreid1.com endpoints (the wiki, |
||||
phpMyAdmin, and ignoring `.git` directories): |
||||
|
||||
* `https.com.charlesreid1.conf` |
||||
* `https.red.charlesreid1.conf` |
||||
* `https.blue.charlesreid1.conf` |
||||
|
||||
HTTPS rules for subdomains (pages.charlesreid1.com, |
||||
hooks.charlesreid1.com, bots.charlesreid1.com): |
||||
|
||||
* `https.com.charlesreid1.subdomains.conf` |
||||
* `https.red.charlesreid1.subdomains.conf` |
||||
* `https.blue.charlesreid1.subdomains.conf` |
||||
|
||||
|
||||
### HTTP Config Files |
||||
|
||||
The HTTP config files are listed below: |
||||
|
||||
* `http.com.charlesreid1.conf` |
||||
* `http.red.charlesreid1.conf` |
||||
* `http.blue.charlesreid1.conf` |
||||
|
||||
The HTTP config files do the following: |
||||
|
||||
* Requests for port 80 (for a domain or subdomain) are always redirected |
||||
to port 443 for the same domain/subdomain |
||||
|
||||
|
||||
### HTTPS Config Files |
||||
|
||||
The HTTPS config files are listed below: |
||||
|
||||
* `https.com.charlesreid1.conf` |
||||
* `https.red.charlesreid1.conf` |
||||
* `https.blue.charlesreid1.conf` |
||||
|
||||
The HTTPS config files (without "subdomain" in their name) do the following: |
||||
|
||||
* Requests for `/` are redirected to the htdocs folders mentioned above |
||||
|
||||
* Requests for `/w/` or `/wiki/` are reverse-proxied to the local |
||||
[Apache+PHP container](Service_apachephp.md) on port 8989 |
||||
|
||||
* (Optional) requests for `/phpMyAdmin/` are reverse-proxied to the local |
||||
[phpMyAdmin container](Service_phpmyadmin.md) on port 80 |
||||
|
||||
* Requests for `git.charlesreid1.com` are reverse-proxied to the local |
||||
[Gitea container](Service_gitea.md) on port 3000 |
||||
|
||||
* Requests for `files.charlesreid1.com` are reverse-proxied to the local |
||||
[Python files](Service_pythonfiles.md) on port 8081 |
||||
|
||||
|
||||
### HTTPS Subdomain Config Files |
||||
|
||||
The HTTPS subdomain config files are listed below: |
||||
|
||||
* `https.com.charlesreid1.subdomains.conf` |
||||
* `https.red.charlesreid1.subdomains.conf` |
||||
* `https.blue.charlesreid1.subdomains.conf` |
||||
|
||||
The subdomains config files redirect requests for a set of subdomains |
||||
on charlesreid1.com, namely: |
||||
|
||||
* `pages.charlesreid1.com` |
||||
* `hooks.charlesreid1.com` |
||||
* `bots.charlesreid1.com` |
||||
|
||||
This charlesreid1.com docker pod will redirect requests for these subdomains |
||||
to another server running another docker pod, called the webhook docker pod, |
||||
available at [docker/pod-webhooks](https://git.charlesreid1.com/docker/pod-webhooks)). |
||||
|
||||
The webhook docker pod runs an nginx server, both to serve up static sites that live |
||||
under the <https://pages.charlesreid1.com> domain, and to handle webhook traffic - the |
||||
container also runs a Python webhook server that receives webhooks from git.charlesreid1.com, |
||||
which enables push-to-deploy functionality similar to Github Pages. |
||||
|
||||
Also see <https://pages.charlesreid1.com/pod-webhooks/>. |
||||
|
||||
|
||||
### Why All The Config Files? |
||||
|
||||
If everything is stuffed into a smaller number of |
||||
nginx config files, they become long and unwieldy, |
||||
and more prone to mistakes. |
||||
|
||||
* HTTP config files only contain redirects |
||||
* HTTPS (no subdomain) config files handle |
||||
|
||||
|
||||
### Getting Configuration Files Into Container |
||||
|
||||
The configuration files are mounted into the container |
||||
by bind-mounting the `conf.d` folder of the `d-nginx-charlesreid1` |
||||
submodule at `/et/nginx/conf.d` in the container. |
||||
|
||||
This is done in the [charlesreid1 pod docker-compose |
||||
file](https://git.charlesreid1.com/docker/pod-charlesreid1/src/branch/master/docker-compose.fixme.yml#L57). |
||||
|
||||
|
||||
## Static Content |
||||
|
||||
The nginx container hosts static content, in addition to serving |
||||
as a reverse-proxy for several other services. This section covers |
||||
how static content is treated by the charlesreid1.com nginx |
||||
container |
||||
|
||||
### Outside the Container: `/www/` |
||||
|
||||
Static content hosted by the container is stored on the host |
||||
machine in `/www/`. |
||||
|
||||
Each site (e.g., charlesreid1.com) has its own folder, containing |
||||
the source for the static site and an htdocs folder containing the |
||||
static content actually being hosted. |
||||
|
||||
Additionally, because the static content for the site is actually |
||||
contained on the `gh-pages` branch of [charlesreid1/charlesreid1.com](https://git.charlesreid1.com/charlesreid1/charlesreid1.com), |
||||
the `htodcs` folder is actually a git repository. When it is |
||||
cloned, it is cloned such that the git repo's contents go into |
||||
`/www/charlesreid1.com/htdocs/` and the contents of the `.git` |
||||
folder go into `/www/charlesreid1.com/git`. |
||||
|
||||
This allows the site static content to be updated to reflect the |
||||
contents of the `gh-pages` branch by pulling the latest upstream |
||||
changes into htdocs. |
||||
|
||||
The directory structure used is as follows: |
||||
|
||||
``` |
||||
/www |
||||
/charlesreid1.com |
||||
/charlesreid1-src |
||||
...source for pelican site... |
||||
/htdocs |
||||
...static content... |
||||
/git |
||||
...dot git folder... |
||||
|
||||
/charlesreid1.blue |
||||
/charlesreid1-blue-src |
||||
...source for pelican site... |
||||
/htodcs |
||||
...static content... |
||||
/git |
||||
...dot git folder... |
||||
|
||||
/charlesreid1.red |
||||
/charlesreid1-red-src |
||||
...source for pelican site... |
||||
/htodcs |
||||
...static content... |
||||
/git |
||||
...dot git folder... |
||||
|
||||
``` |
||||
|
||||
This directory structure can be achieved using the following |
||||
bash command: |
||||
|
||||
``` |
||||
REPOURL="https://git.charlesreid1.com/charlesreid1/charlesreid1.com.git" |
||||
|
||||
git -C /www/example.com \ |
||||
clone \ |
||||
--separate-git-dir=git \ |
||||
-b gh-pages \ |
||||
$REPOURL htdocs |
||||
``` |
||||
|
||||
The [dotfiles/debian](https://git.charlesreid1.com/debian/dotfiles) |
||||
repository contains scripts for krash, which runs the charlesreid1.com |
||||
pod, to set up the directory structure as above, as well as scripts |
||||
to pull the latest changes from upstream for each of the live |
||||
web directories above. |
||||
|
||||
|
||||
### Inside the Container: `/usr/share/nginx/html` |
||||
|
||||
Once the contents of the `/www/` directory have been set up, |
||||
the content can be made avialable inside the container by |
||||
bind-mounting the htdocs directories into the `/www/` directory |
||||
inside the container. This is done with the following |
||||
volume directives in the [`docker-compose.yml` file](https://git.charlesreid1.com/docker/pod-charlesreid1/src/branch/master/docker-compose.fixme.yml#L60-L62): |
||||
|
||||
``` |
||||
volumes: |
||||
- "/www/charlesreid1.blue/htdocs:/www/charlesreid1.blue/htdocs:ro" |
||||
- "/www/charlesreid1.red/htdocs:/www/charlesreid1.red/htdocs:ro" |
||||
- "/www/charlesreid1.com/htdocs:/www/charlesreid1.com/htdocs:ro" |
||||
... |
||||
``` |
||||
|
||||
### Utility Scripts: Updating Site Contents |
||||
|
||||
In the [`krash_scripts/` folder](https://git.charlesreid1.com/dotfiles/debian/src/branch/master/dotfiles/krash_scripts) |
||||
of the debian/dotfiles](https://git.charlesreid1.com/dotfiles/debian) |
||||
repository, there are several utility scripts to help with |
||||
setting up and updating this directory structure. |
||||
|
||||
To set up the directory structure, use the [`git_clone_www.sh` script](https://git.charlesreid1.com/dotfiles/debian/src/branch/master/dotfiles/krash_scripts/git_clone_www.sh). |
||||
|
||||
``` |
||||
#!/bin/bash |
||||
|
||||
REPOURL="https://git.charlesreid1.com/charlesreid1/charlesreid1.com.git" |
||||
|
||||
git -C /www/example.com \ |
||||
clone \ |
||||
--separate-git-dir=git \ |
||||
-b gh-pages \ |
||||
$REPOURL htdocs |
||||
``` |
||||
|
||||
To update the contents of the `htdocs/` folder using the latest changes |
||||
on the `gh-pages` branch, use the [`git_pull_www.sh` script](https://git.charlesreid1.com/dotfiles/debian/src/branch/master/dotfiles/krash_scripts/git_pull_www.sh). |
||||
|
||||
``` |
||||
#!/bin/bash |
||||
|
||||
git -C /www/example.com \ |
||||
--git-dir=git --work-tree=htdocs \ |
||||
pull origin gh-pages |
||||
``` |
||||
|
||||
|
||||
|
||||
## SSL Certificates |
||||
|
||||
We utilize Let's Encrypt to issue SSL certificates for our domain. |
||||
This is a pain because of the need to automatically renew certificates |
||||
every 90 days, and because of missing information in their documentation. |
||||
|
||||
|
||||
### Where To Put Certificates |
||||
|
||||
We leave the certificates where Let's Encrypt puts them, namely, in `/etc/letsencrypt/live`. |
||||
|
||||
The certificates are bind-mounted into the container at the same location, |
||||
`/etc/letsencrypt/live`. The [`docker-compose.yml` file](#) |
||||
then contains the following: |
||||
|
||||
``` |
||||
volumes: |
||||
- "/etc/letsencrypt:/etc/letsencrypt" |
||||
... |
||||
``` |
||||
|
||||
|
||||
### Which Server Needs Certificates |
||||
|
||||
Because the [d-nginx-charlesreid1 docker container](#) |
||||
is serving as the front-end nginx server handling all |
||||
incoming charlesreid1.com web requests, it is also the |
||||
one negotiating HTTPS sessions with clients, so it is |
||||
the container that needs the Let's Encrypt certificates. |
||||
|
||||
This means that, for example, the SSL certificate for |
||||
`pages.charlesreid1.com` is on krash, even though the |
||||
content served up by the site is on blackbeard. |
||||
|
||||
|
||||
### Getting and Renewing Certificates |
||||
|
||||
Once you have worked everything out and gotten auto-renew cron scripts |
||||
running smoothly, everything is unicorns and rainbows. But when you hit |
||||
the Let's Encrypt rate limit because they did not explain it properly |
||||
in the documentation, you may want everyone to die. |
||||
|
||||
Let's Encrypt provides a command line utility called certbot that can be |
||||
used to automate the process of creating and renewing certificates. |
||||
|
||||
See the [charlesreid1/certbot](https://git.charlesreid1.com/charlesreid1/certbot) |
||||
repository for scripts. The basic setup is as follows: |
||||
|
||||
* Use the [`krash_make_cert.sh`](https://git.charlesreid1.com/charlesreid1/certbot/src/branch/master/krash_make_cert.sh) |
||||
script (which wraps the [`make_cert.sh`](https://git.charlesreid1.com/charlesreid1/certbot/src/branch/master/make_cert.sh) |
||||
script) to create SSL certificates for each domain and subdomain |
||||
|
||||
* Use the [`krash_renew.sh`](https://git.charlesreid1.com/charlesreid1/certbot/src/branch/master/krash_renew.sh) |
||||
script (which wraps the [`renew_cert.sh`](https://git.charlesreid1.com/charlesreid1/certbot/src/branch/master/renew_cert.sh) |
||||
script) to renew SSL certificates for each domain and subdomain |
||||
|
||||
* Certificates must be renewed every 90 days. I run the certificate renewal |
||||
script in the root acount's crontab every 15 days so the certs never expire. |
||||
|
||||
|
||||
## Domain Control |
||||
|
||||
There are three top-level domains controlled by pod-charlesreid1: |
||||
|
||||
* <https://charlesreid1.com> |
||||
* <https://charlesreid1.blue> |
||||
* <https://charlesreid1.red> |
||||
|
||||
There are several subdomains available on charlesreid1.com. |
||||
|
||||
Hosted on krash: |
||||
|
||||
* <https://git.charlesreid1.com> - gitea service |
||||
* <https://files.charlesreid1.com> - static file hosting |
||||
|
||||
Hosted on blackbeard: |
||||
|
||||
* <https://pages.charlesreid1.com> - push-to-deploy static pages |
||||
* <https://hooks.charlesreid1.com> - webhook server |
||||
* <https://bots.charlesreid1.com> - info about bots |
||||
|
||||
|
||||
|
@ -0,0 +1,32 @@
@@ -0,0 +1,32 @@
|
||||
# phpMyAdmin |
||||
|
||||
This page describes the container-specific details |
||||
of the phpMyAdmin container. |
||||
|
||||
phpMyAdmin provides a web interface for interacting with MySQL |
||||
databases and can be connected to the MySQL container to ensure |
||||
the backup/restore process proceeds smoothly. |
||||
|
||||
This is run as a stock phpMyAdmin container - run script |
||||
is [here](https://git.charlesreid1.com/docker/d-phpmyadmin/src/branch/master/run_stock_phpmyadmin.sh) |
||||
in the [docker/d-phpmyadmin](https://git.charlesreid1.com/docker/d-phpmyadmin) |
||||
repository on git.charlesreid1.com. |
||||
|
||||
The phpMyAdmin service is a web interface that is available |
||||
on port 80. The container should only be bound to the |
||||
Docker container network (default behavior). Then any |
||||
container on the network can reach the container's |
||||
phpMyAdmin service on port 80. |
||||
|
||||
This allows the phpMyAdmin service to be made available at |
||||
a URL like `/phpMyAdmin` and have all requests reverse-proxied |
||||
by the nginx container and passed to port 80 on the back end. |
||||
|
||||
The phpMyAdmin service can also be disabled/enabled by |
||||
commenting it out of the nginx configuration files containing |
||||
HTTPS rules for the charlesreid1.com domains. |
||||
|
||||
See the configuration section of the [Nginx](Service_nginx.md) |
||||
container page for more information about the nginx configuration |
||||
files. |
||||
|
@ -0,0 +1,22 @@
@@ -0,0 +1,22 @@
|
||||
# Python File Server |
||||
|
||||
This page describes how pod-charlesreid1 provides a lightweight |
||||
HTTP file server that is reverse-proxied by nginx to provide |
||||
a dead-simple file hosting service at `files.charlesreid1.com`. |
||||
|
||||
We use an alpine container with Python 3 for a minimal image |
||||
size. Python comes with a simple HTTP server built-in that |
||||
will do the job for us, available through the `http.server` |
||||
module (or `SimpleHTTPServer` in Python 2): |
||||
|
||||
``` |
||||
python3 -m http.server 8081 |
||||
``` |
||||
|
||||
We expose this to port 8081 locally, making the service available |
||||
on the Docker network and therefore available to be reverse-proxied |
||||
by the nginx container. |
||||
|
||||
Files to be served up are located in `/www/files/` on the host, |
||||
which is bind-mounted to `/files` in the container. |
||||
|
Loading…
Reference in new issue