Browse Source

Merge branch 'master' of ssh://git.charlesreid1.com:222/docker/pod-charlesreid1

* 'master' of ssh://git.charlesreid1.com:222/docker/pod-charlesreid1: (22 commits)
  update gitea submodule
  redundant
  in the final stretch, wrapping up gitea, only thing left is backup script utilities
  another index update
  add info about python files service
  update port information
  group services into TOC and add as index section
  nginx + ssl finished
  add nginx ssl info
  updates to domain control section - top level domain and subdomain info
  update doomain info on nginx service page
  update config files in d-nginx-charlesreid1
  update index page with what is finished
  update phpmyadmin port info
  update port info
  update with port info
  nearly-complete nginx service page
  add phpMyAdmin page
  update mysql page with links
  add link to apache page
  ...
spew-fix
Charles Reid 6 years ago
parent
commit
e331439911
  1. 13
      README.md
  2. 2
      d-gitea
  3. 2
      d-mediawiki
  4. 2
      d-mysql
  5. 2
      d-nginx-charlesreid1
  6. 2
      d-phpmyadmin
  7. 2
      d-python-files
  8. 87
      docs/Ports.md
  9. 3
      docs/Service_apachephp.md
  10. 103
      docs/Service_gitea.md
  11. 17
      docs/Service_mediawiki.md
  12. 26
      docs/Service_mysql.md
  13. 348
      docs/Service_nginx.md
  14. 32
      docs/Service_phpmyadmin.md
  15. 22
      docs/Service_pythonfiles.md
  16. 82
      docs/index.md
  17. 21
      mkdocs.yml

13
README.md

@ -3,10 +3,6 @@ @@ -3,10 +3,6 @@
This repo contains a docker compose file
for running the charlesreid1.com site.
See the documentation site here: <https://pages.charlesreid1.com/pod-charlesreid1>
Or visit [docs/index.md](/docs/index.md)
The services are:
* mediawiki
@ -17,6 +13,15 @@ The services are: @@ -17,6 +13,15 @@ The services are:
* python
* gitea
## Links
See the documentation site here: <https://pages.charlesreid1.com/pod-charlesreid1>
Or visit [docs/index.md](/docs/index.md)
Source code on git.charlesreid1.com: <https://git.charlesreid1.com/docker/pod-charlesreid1>
Source code on github.com: <https://github.com/charlesreid1-docker/pod-charlesreid1>
## Running

2
d-gitea

@ -1 +1 @@ @@ -1 +1 @@
Subproject commit 31cc1713145a6f0c4101f966612670349e7649f3
Subproject commit 8c2bd1a55ec89c74601c958693912ad9fc62c720

2
d-mediawiki

@ -1 +1 @@ @@ -1 +1 @@
Subproject commit 922d09e2be611295a59924c3c3063f85b12f44ac
Subproject commit 1591083a8e2c5a4e39bc87e5a9dfd108e759a47c

2
d-mysql

@ -1 +1 @@ @@ -1 +1 @@
Subproject commit 28f90eb21df82871fe7923617566fb7bb06eb7a8
Subproject commit 4bd88e74c169bfe62a1b3709b3b65a6405c7de24

2
d-nginx-charlesreid1

@ -1 +1 @@ @@ -1 +1 @@
Subproject commit f0629fa74961cde670a09f7f04df88d25386483b
Subproject commit f9f611a1503484ba230a1ffe2d5cbec1d449835f

2
d-phpmyadmin

@ -1 +1 @@ @@ -1 +1 @@
Subproject commit ea4696c0cdd2a947259bfaa378b764165df03309
Subproject commit e84de96196618787219c9a77135fa1726153ddec

2
d-python-files

@ -1 +1 @@ @@ -1 +1 @@
Subproject commit ff7846b6398e7ebbeb17c49be5909726858b9085
Subproject commit 473b497e2851692c808e6d2d9ea660be64930c20

87
docs/Ports.md

@ -59,6 +59,32 @@ but all http urls are redirected to https urls. @@ -59,6 +59,32 @@ but all http urls are redirected to https urls.
## Ports
### overview
The apache-mediawiki combination is running an apache service listening on port 8989.
This can be adjusted, but should be adjusted in the Dockerfile, `ports.conf`, and `wiki.conf`.
The apache service listens on all interfaces (hence `*:8989` in the apache conf file),
but there is no port mapping specified in `docker-compose.yml` so it does not listen
on any public interfaces.
Thus, the wiki is not publicly accessible via port 8989, but the wiki is available via port 8989
to any container linked to, or connected to the same network as, the mediawiki apache container.
Meanwhile, the nginx container has a public interface listening on port 80
and another listening on port 443. nginx listens for requests going to
the wiki, detected via the url resource prefix being `/w/` or `/wiki/`,
and acts as a reverse proxy, forwarding the requests to Apache.
The user transparently sees everything happening via port 80 or (preferrably) 443,
but on the backend nginx is passing along the URL request and returning the result.
Subdomains are served via reverse proxy on port 7777+.
The webhook server is a flask server listening on port 50000.
### nginx ports
Also see [nginx service](Service_nginx.md).
@ -84,7 +110,6 @@ and proxies those to the correct container. @@ -84,7 +110,6 @@ and proxies those to the correct container.
### mediawiki/apache ports
Also see [mediawiki service](Service_mediawiki.md)
@ -151,17 +176,20 @@ phpMyAdmin provides a web interface for MySQL databases. @@ -151,17 +176,20 @@ phpMyAdmin provides a web interface for MySQL databases.
This follows a similar pattern to the MediaWiki Apache container:
* The phpMyAdmin container is connected to the MySQL container
via the docker network created by the `docker-compose` command
(no container links needed)
* The phpMyAdmin container runs an HTTP web interface,
and listens only for incoming requests from the local
network. Requests to phpMyAdmin are reverse-proxied
by the nginx container in this pod.
* Because phpMyAdmin is not a heavily-used tool in
daily tasks, and because it provides access to
sensitive data and operations, it should be
completely disabled from public access unless
needed.
via the docker network created by the `docker-compose` command
(no container links needed)
* The phpMyAdmin container runs an HTTP web interface available inside
the container on port 80. This service is exposed on port 80 on the
internal docker network only.
* Since phpMyAdmin only listens on the Docker pod network for incoming
requests, all requests to phpMyAdmin must come through nginx via
reverse proxy. These are forwarded to port 80 of the phpMyAdmin container
on the back end.
* We keep phpMyAdmin disabled on a regular basis, as it is not
heavily used and provides access to sensitive data and operations.
To control access to phpMyAdmin,
configure the [nginx service](Service_nginx.md)
@ -202,12 +230,12 @@ the same reverse proxy pattern: @@ -202,12 +230,12 @@ the same reverse proxy pattern:
Also see [python files service](Service_pythonfiles.md).
We have a simple, lightweight Python HTTP server
that's run in a Docker container via the following
command:
We have a simple, lightweight Python HTTP server on port 8081
on the Docker network. This container runs the following Python
command to start the server:
```
python -m http.server -b <bind-address> 8080
python -m http.server -b <bind-address> 8081
```
This works because Python provides a built-in HTTP server
@ -218,28 +246,33 @@ as far as file servers go. @@ -218,28 +246,33 @@ as far as file servers go.
This follows the same reverse proxy pattern:
* Python HTTP server listens for incoming requests
on the Docker network only. Client requests are
reverse proxied by nginx on the front end.
on port 8081 on the Docker network only. Client
requests are reverse proxied by [d-nginx-charlesreid1](https://git.charlesreid1.com/docker/d-nginx-charlesreid1)
on the front end.
* The server does not handle HTTPS, this is also
handled by the nginx container on the frontend.
handled by the nginx container on the frontend.
* The bind address and port of the Python HTTP server
are set in the command line. The `<bind-address>`
should be set to the name of the docker container image
(`stormy_files`).
are set in the command line. The `<bind-address>`
should be set to the name of the docker container image
(`stormy_files`).
The command
```
python -m http.server -b stormy_files 8080
python -m http.server -b stormy_files 8081
```
This listens on port 8080 inside the
python file server container `stormy_files`.
listens on port 8081 inside the python file server
container `stormy_files` (the container itself).
The nginx server reverse-proxies requests for
[https://files.charlesreid1.com](https://files.charlesreid1.com)
<https://files.charlesreid1.com>
and forwards them to the container.
Note: this container can be expanded to a container
that serves multiple directories on multilpe ports
that serves multiple directories on multiple ports
by using twisted. See the
[d-python-helium](https://git.charlesreid1.com/docker/d-python-helium)
repository for an example.

3
docs/Service_apachephp.md

@ -40,7 +40,8 @@ Apache `*.conf` files @@ -40,7 +40,8 @@ Apache `*.conf` files
into the container at
`/etc/nginx/conf.d`
via the following line
in the docker-compose file:
in the [pod-charlesreid1
docker-compose file](https://git.charlesreid1.com/docker/pod-charlesreid1/src/branch/master/docker-compose.fixme.yml):
```
services:

103
docs/Service_gitea.md

@ -0,0 +1,103 @@ @@ -0,0 +1,103 @@
# Gitea
Gitea is a self-hosted Github clone. It is written in Go, and
provides a web interface and an API to interact with an instance
of a git server.
Gitea manages its own database, so to get data in and out of
Gitea, use its dump and load functionality (more below).
This page describes how the Gitea container is configured.
## Configuring Docker Container
To run gitea, we use a stock Gitea container image. We set
several options in the docker configuration:
* `USER_UID` and `USER_GID` are set to `1000` (this avoids some
problems with files that would otherwise be owned by root)
* Set `restart: always` to restart the container when there
is a failure
```
stormy_gitea:
image: gitea/gitea:latest
environment:
- USER_UID=1000
- USER_GID=1000
restart: always
```
## Gitea Volumes
The Gitea container stores all of its data in `/data/` inside
the container.
When the container is launched, the `custom/` directory in
the [docker/d-gitea](https://git.charlesreid1.com/docker/d-gitea)
repository is mounted to `/data/gitea/`, which is the directory
that contains the files that are used to control the way that
Gitea pages look. These contain HTML templates used to render
different views in Gitea and templates in `custom/` will override
the default Gitea page templates.
A docker volume named `stormy_gitea_data` is also created and
mounted at `/data/`. This is a persistent volume that will
survive even if the container is shut down.
```
volumes:
- "stormy_gitea_data:/data"
- "./d-gitea/custom/conf/app.ini:/data/gitea/conf/app.ini"
- "./d-gitea/custom/public:/data/gitea/public"
- "./d-gitea/custom/templates:/data/gitea/templates"
```
## Gitea Ports
Gitea provides both SSH and HTTPS interfaces, as it has its own
built-in web server and SSH server, as well as git server.
The server that is hosting the Gitea container and this Docker
pod already has an SSH server listening on port 22, so Gitea
listens for SSH connections _externally_ on port 222.
```
ports:
- "222:22"
```
Note that this _bypasses_ our d-nginx-charlesreid1 nginx container
entirely and allows clients to connect to Gitea directly.
The Gitea server listens for HTTP/HTTPS connections on port
3000, but that is by default only listening on the internal
Docker network, which is exactly how we want it. We want all
HTTP and HTTPS traffic to be handled by the front-end d-nginx-charlesreid1
container, and it will reverse-proxy HTTP/HTTPS requests to
the Gitea container.
## Gitea Configuration Files
`app.ini` is the name of the configuration file used by Gitea.
An [example `app.ini` configuration file](https://git.charlesreid1.com/docker/d-gitea/src/branch/master/app.ini.sample)
is contained in the [docker/d-gitea](https://git.charlesreid1.com/docker/d-gitea)
repository, as well as a script to
[make a configuration file](https://git.charlesreid1.com/docker/d-gitea/src/branch/master/make_app_ini.sh).
```
volumes:
- "stormy_gitea_data:/data"
- "./d-gitea/custom/conf/app.ini:/data/gitea/conf/app.ini"
```
## Backups
### Backing Up Gitea
### Restoring Gitea

17
docs/Service_mediawiki.md

@ -45,8 +45,9 @@ in the `mediawiki/` directory. @@ -45,8 +45,9 @@ in the `mediawiki/` directory.
That's also where the `extensions/`
directory goes. There is also
a script there called `build_extensions_dir.sh`
to clone copies of each MediaWiki extension.
a script in the [wiki/charlesreid1-config](https://git.charlesreid1.com/wiki/charlesreid1-config)
repo called [`build_extensions_dir.sh`](https://git.charlesreid1.com/wiki/charlesreid1-config/src/branch/master/mediawiki/build_extensions_dir.sh)
that clone copies of each MediaWiki extension.
Inside the MediaWiki container,
the live HTML directory is at
@ -66,7 +67,7 @@ on git.charlesreid1.com. @@ -66,7 +67,7 @@ on git.charlesreid1.com.
The configuration files mentioned above
(LocalSettings, skins, and extensions)
must be coiped into the container at build time.
must be copied into the container at build time.
This is done in the MediaWiki Dockerfile -
see [d-mediawiki](https://git.charlesreid1.com/docker/d-mediawiki).
@ -152,7 +153,7 @@ CMD cd /var/www/html/extensions/Math/math && make && apache2-foreground @@ -152,7 +153,7 @@ CMD cd /var/www/html/extensions/Math/math && make && apache2-foreground
## Updating Skin or LocalSettings.php
Note that if you update the MediaWiki skin
or the LocalSettings.php file,
or the `LocalSettings.php` file,
you will need to rebuild the container
and restart it.
@ -178,7 +179,7 @@ git push or git pull. @@ -178,7 +179,7 @@ git push or git pull.
(This is not currently how
it is structured, as the
skin and LocalSettings.php
skin and `LocalSettings.php`
files are not under version
control in the container.
This would be difficult for
@ -220,7 +221,7 @@ A couple of things have to happen: @@ -220,7 +221,7 @@ A couple of things have to happen:
There are utilities for MediaWiki in `utils-mw`:
* `backup_wikifiles.sh` - back up wiki image files to a tarball
* `restore_wikifiles.sh` - restore backed up image files from a tarball
* `update_wikidb.sh` - one-time script to update the wiki database after a version bump
* [`backup_wikifiles.sh`](https://git.charlesreid1.com/docker/pod-charlesreid1/src/branch/master/utils-mw/backup_wikifiles.sh) - back up wiki image files to a tarball from the `story_mw` container
* [`restore_wikifiles.sh`](https://git.charlesreid1.com/docker/pod-charlesreid1/src/branch/master/utils-mw/restore_wikifiles.sh) - restore backed up image files from a tarball into the `story_mw` container
* [`update_wikidb.sh`](https://git.charlesreid1.com/docker/pod-charlesreid1/src/branch/master/utils-mw/update_wikidb.sh) - one-time script to update the wiki database after a version bump

26
docs/Service_mysql.md

@ -17,10 +17,15 @@ We don't have an extensive MySQL configuration. @@ -17,10 +17,15 @@ We don't have an extensive MySQL configuration.
The container demostrates how to mount a configuration
file into the container, but this is optional.
See [this line](https://git.charlesreid1.com/docker/d-mysql/src/branch/master/run_super_mysql.sh#L17)
of the run script in the [docker/d-mysql](https://git.charlesreid1.com/docker/d-mysql)
repository.
## Getting Stuff Into The Container (How To Seed MySQL?)
This section refers to scripts contained in
the `utils-mysql` directory.
the [`utils-mysql/`](https://git.charlesreid1.com/docker/pod-charlesreid1/src/branch/master/utils-mw)
directory.
The MySQL data must come from a seed
(what we call a krash seed). This seed
@ -29,23 +34,22 @@ MySQL database, from which the database @@ -29,23 +34,22 @@ MySQL database, from which the database
can be restored.
There are both backup and restore scripts
in the repo under `utils-mysql`.
in the repo under [`utils-mysql/`](https://git.charlesreid1.com/docker/pod-charlesreid1/src/branch/master/utils-mysql).
Running the `mysqldump` tool will dump
database backup files in `.sql` format.
These can be created using the `dump_database.sh`
script.
The [`dump_database.sh`](https://git.charlesreid1.com/docker/pod-charlesreid1/src/branch/master/utils-mysql/dump_database.sh)
script will run the `mysqldump` tool to back up
all the databases in the container into a file
in `.sql` format.
These `.sql` files can be used to restore a
MySQL database using the `restore_database.sh`.
MySQL database using the [`restore_database.sh`](https://git.charlesreid1.com/docker/pod-charlesreid1/src/branch/master/utils-mysql/restore_database.sh)
script.
## Utilities
There are utilities for MySQL in `utils-mysql`:
* `dump_databases.sh` - create an `..sql` dump file from a database
* `restore_database.sh` - restore a database from an `.sql` dump file
* [`dump_databases.sh`](https://git.charlesreid1.com/docker/pod-charlesreid1/src/branch/master/utils-mysql/dump_database.sh) - create an `.sql` dump file from a database
* [`restore_database.sh`](https://git.charlesreid1.com/docker/pod-charlesreid1/src/branch/master/utils-mysql/restore_database.sh) - restore a database from an `.sql` dump file

348
docs/Service_nginx.md

@ -0,0 +1,348 @@ @@ -0,0 +1,348 @@
# Nginx
This describes the configuration of the main nginx container
for the charlesreid1.com pod.
## Configuration Files
The nginx container is critical to routing operations on
charlesreid1.com, so we cover how the configuration files
are split up and organized on this page, and include
a summary of each file below.
### Where are the nginx config files?
Nginx configuration files are located in the `d-nginx-charlesreid1`
submodule of this repository, which points to the
[docker/d-nginx-charlesreid1](https://git.charlesreid1.com/docker/d-nginx-charlesreid1)
repository.
Within that repository is the [`conf.d/`](https://git.charlesreid1.com/docker/d-nginx-charlesreid1/src/branch/master/conf.d)
folder, which contains several `.conf` files.
### What Files Are There?
We have three sets of files present:
HTTP config files that redirect HTTP traffic on port 80 to
be HTTPS traffic on port 443, one for each top-level domain
(charlesreid1.com, charlesreid1.blue, and charlesreid1.red):
* `http.com.charlesreid1.conf`
* `http.red.charlesreid1.conf`
* `http.blue.charlesreid1.conf`
HTTPS rules for charlesreid1.com endpoints (the wiki,
phpMyAdmin, and ignoring `.git` directories):
* `https.com.charlesreid1.conf`
* `https.red.charlesreid1.conf`
* `https.blue.charlesreid1.conf`
HTTPS rules for subdomains (pages.charlesreid1.com,
hooks.charlesreid1.com, bots.charlesreid1.com):
* `https.com.charlesreid1.subdomains.conf`
* `https.red.charlesreid1.subdomains.conf`
* `https.blue.charlesreid1.subdomains.conf`
### HTTP Config Files
The HTTP config files are listed below:
* `http.com.charlesreid1.conf`
* `http.red.charlesreid1.conf`
* `http.blue.charlesreid1.conf`
The HTTP config files do the following:
* Requests for port 80 (for a domain or subdomain) are always redirected
to port 443 for the same domain/subdomain
### HTTPS Config Files
The HTTPS config files are listed below:
* `https.com.charlesreid1.conf`
* `https.red.charlesreid1.conf`
* `https.blue.charlesreid1.conf`
The HTTPS config files (without "subdomain" in their name) do the following:
* Requests for `/` are redirected to the htdocs folders mentioned above
* Requests for `/w/` or `/wiki/` are reverse-proxied to the local
[Apache+PHP container](Service_apachephp.md) on port 8989
* (Optional) requests for `/phpMyAdmin/` are reverse-proxied to the local
[phpMyAdmin container](Service_phpmyadmin.md) on port 80
* Requests for `git.charlesreid1.com` are reverse-proxied to the local
[Gitea container](Service_gitea.md) on port 3000
* Requests for `files.charlesreid1.com` are reverse-proxied to the local
[Python files](Service_pythonfiles.md) on port 8081
### HTTPS Subdomain Config Files
The HTTPS subdomain config files are listed below:
* `https.com.charlesreid1.subdomains.conf`
* `https.red.charlesreid1.subdomains.conf`
* `https.blue.charlesreid1.subdomains.conf`
The subdomains config files redirect requests for a set of subdomains
on charlesreid1.com, namely:
* `pages.charlesreid1.com`
* `hooks.charlesreid1.com`
* `bots.charlesreid1.com`
This charlesreid1.com docker pod will redirect requests for these subdomains
to another server running another docker pod, called the webhook docker pod,
available at [docker/pod-webhooks](https://git.charlesreid1.com/docker/pod-webhooks)).
The webhook docker pod runs an nginx server, both to serve up static sites that live
under the <https://pages.charlesreid1.com> domain, and to handle webhook traffic - the
container also runs a Python webhook server that receives webhooks from git.charlesreid1.com,
which enables push-to-deploy functionality similar to Github Pages.
Also see <https://pages.charlesreid1.com/pod-webhooks/>.
### Why All The Config Files?
If everything is stuffed into a smaller number of
nginx config files, they become long and unwieldy,
and more prone to mistakes.
* HTTP config files only contain redirects
* HTTPS (no subdomain) config files handle
### Getting Configuration Files Into Container
The configuration files are mounted into the container
by bind-mounting the `conf.d` folder of the `d-nginx-charlesreid1`
submodule at `/et/nginx/conf.d` in the container.
This is done in the [charlesreid1 pod docker-compose
file](https://git.charlesreid1.com/docker/pod-charlesreid1/src/branch/master/docker-compose.fixme.yml#L57).
## Static Content
The nginx container hosts static content, in addition to serving
as a reverse-proxy for several other services. This section covers
how static content is treated by the charlesreid1.com nginx
container
### Outside the Container: `/www/`
Static content hosted by the container is stored on the host
machine in `/www/`.
Each site (e.g., charlesreid1.com) has its own folder, containing
the source for the static site and an htdocs folder containing the
static content actually being hosted.
Additionally, because the static content for the site is actually
contained on the `gh-pages` branch of [charlesreid1/charlesreid1.com](https://git.charlesreid1.com/charlesreid1/charlesreid1.com),
the `htodcs` folder is actually a git repository. When it is
cloned, it is cloned such that the git repo's contents go into
`/www/charlesreid1.com/htdocs/` and the contents of the `.git`
folder go into `/www/charlesreid1.com/git`.
This allows the site static content to be updated to reflect the
contents of the `gh-pages` branch by pulling the latest upstream
changes into htdocs.
The directory structure used is as follows:
```
/www
/charlesreid1.com
/charlesreid1-src
...source for pelican site...
/htdocs
...static content...
/git
...dot git folder...
/charlesreid1.blue
/charlesreid1-blue-src
...source for pelican site...
/htodcs
...static content...
/git
...dot git folder...
/charlesreid1.red
/charlesreid1-red-src
...source for pelican site...
/htodcs
...static content...
/git
...dot git folder...
```
This directory structure can be achieved using the following
bash command:
```
REPOURL="https://git.charlesreid1.com/charlesreid1/charlesreid1.com.git"
git -C /www/example.com \
clone \
--separate-git-dir=git \
-b gh-pages \
$REPOURL htdocs
```
The [dotfiles/debian](https://git.charlesreid1.com/debian/dotfiles)
repository contains scripts for krash, which runs the charlesreid1.com
pod, to set up the directory structure as above, as well as scripts
to pull the latest changes from upstream for each of the live
web directories above.
### Inside the Container: `/usr/share/nginx/html`
Once the contents of the `/www/` directory have been set up,
the content can be made avialable inside the container by
bind-mounting the htdocs directories into the `/www/` directory
inside the container. This is done with the following
volume directives in the [`docker-compose.yml` file](https://git.charlesreid1.com/docker/pod-charlesreid1/src/branch/master/docker-compose.fixme.yml#L60-L62):
```
volumes:
- "/www/charlesreid1.blue/htdocs:/www/charlesreid1.blue/htdocs:ro"
- "/www/charlesreid1.red/htdocs:/www/charlesreid1.red/htdocs:ro"
- "/www/charlesreid1.com/htdocs:/www/charlesreid1.com/htdocs:ro"
...
```
### Utility Scripts: Updating Site Contents
In the [`krash_scripts/` folder](https://git.charlesreid1.com/dotfiles/debian/src/branch/master/dotfiles/krash_scripts)
of the debian/dotfiles](https://git.charlesreid1.com/dotfiles/debian)
repository, there are several utility scripts to help with
setting up and updating this directory structure.
To set up the directory structure, use the [`git_clone_www.sh` script](https://git.charlesreid1.com/dotfiles/debian/src/branch/master/dotfiles/krash_scripts/git_clone_www.sh).
```
#!/bin/bash
REPOURL="https://git.charlesreid1.com/charlesreid1/charlesreid1.com.git"
git -C /www/example.com \
clone \
--separate-git-dir=git \
-b gh-pages \
$REPOURL htdocs
```
To update the contents of the `htdocs/` folder using the latest changes
on the `gh-pages` branch, use the [`git_pull_www.sh` script](https://git.charlesreid1.com/dotfiles/debian/src/branch/master/dotfiles/krash_scripts/git_pull_www.sh).
```
#!/bin/bash
git -C /www/example.com \
--git-dir=git --work-tree=htdocs \
pull origin gh-pages
```
## SSL Certificates
We utilize Let's Encrypt to issue SSL certificates for our domain.
This is a pain because of the need to automatically renew certificates
every 90 days, and because of missing information in their documentation.
### Where To Put Certificates
We leave the certificates where Let's Encrypt puts them, namely, in `/etc/letsencrypt/live`.
The certificates are bind-mounted into the container at the same location,
`/etc/letsencrypt/live`. The [`docker-compose.yml` file](#)
then contains the following:
```
volumes:
- "/etc/letsencrypt:/etc/letsencrypt"
...
```
### Which Server Needs Certificates
Because the [d-nginx-charlesreid1 docker container](#)
is serving as the front-end nginx server handling all
incoming charlesreid1.com web requests, it is also the
one negotiating HTTPS sessions with clients, so it is
the container that needs the Let's Encrypt certificates.
This means that, for example, the SSL certificate for
`pages.charlesreid1.com` is on krash, even though the
content served up by the site is on blackbeard.
### Getting and Renewing Certificates
Once you have worked everything out and gotten auto-renew cron scripts
running smoothly, everything is unicorns and rainbows. But when you hit
the Let's Encrypt rate limit because they did not explain it properly
in the documentation, you may want everyone to die.
Let's Encrypt provides a command line utility called certbot that can be
used to automate the process of creating and renewing certificates.
See the [charlesreid1/certbot](https://git.charlesreid1.com/charlesreid1/certbot)
repository for scripts. The basic setup is as follows:
* Use the [`krash_make_cert.sh`](https://git.charlesreid1.com/charlesreid1/certbot/src/branch/master/krash_make_cert.sh)
script (which wraps the [`make_cert.sh`](https://git.charlesreid1.com/charlesreid1/certbot/src/branch/master/make_cert.sh)
script) to create SSL certificates for each domain and subdomain
* Use the [`krash_renew.sh`](https://git.charlesreid1.com/charlesreid1/certbot/src/branch/master/krash_renew.sh)
script (which wraps the [`renew_cert.sh`](https://git.charlesreid1.com/charlesreid1/certbot/src/branch/master/renew_cert.sh)
script) to renew SSL certificates for each domain and subdomain
* Certificates must be renewed every 90 days. I run the certificate renewal
script in the root acount's crontab every 15 days so the certs never expire.
## Domain Control
There are three top-level domains controlled by pod-charlesreid1:
* <https://charlesreid1.com>
* <https://charlesreid1.blue>
* <https://charlesreid1.red>
There are several subdomains available on charlesreid1.com.
Hosted on krash:
* <https://git.charlesreid1.com> - gitea service
* <https://files.charlesreid1.com> - static file hosting
Hosted on blackbeard:
* <https://pages.charlesreid1.com> - push-to-deploy static pages
* <https://hooks.charlesreid1.com> - webhook server
* <https://bots.charlesreid1.com> - info about bots

32
docs/Service_phpmyadmin.md

@ -0,0 +1,32 @@ @@ -0,0 +1,32 @@
# phpMyAdmin
This page describes the container-specific details
of the phpMyAdmin container.
phpMyAdmin provides a web interface for interacting with MySQL
databases and can be connected to the MySQL container to ensure
the backup/restore process proceeds smoothly.
This is run as a stock phpMyAdmin container - run script
is [here](https://git.charlesreid1.com/docker/d-phpmyadmin/src/branch/master/run_stock_phpmyadmin.sh)
in the [docker/d-phpmyadmin](https://git.charlesreid1.com/docker/d-phpmyadmin)
repository on git.charlesreid1.com.
The phpMyAdmin service is a web interface that is available
on port 80. The container should only be bound to the
Docker container network (default behavior). Then any
container on the network can reach the container's
phpMyAdmin service on port 80.
This allows the phpMyAdmin service to be made available at
a URL like `/phpMyAdmin` and have all requests reverse-proxied
by the nginx container and passed to port 80 on the back end.
The phpMyAdmin service can also be disabled/enabled by
commenting it out of the nginx configuration files containing
HTTPS rules for the charlesreid1.com domains.
See the configuration section of the [Nginx](Service_nginx.md)
container page for more information about the nginx configuration
files.

22
docs/Service_pythonfiles.md

@ -0,0 +1,22 @@ @@ -0,0 +1,22 @@
# Python File Server
This page describes how pod-charlesreid1 provides a lightweight
HTTP file server that is reverse-proxied by nginx to provide
a dead-simple file hosting service at `files.charlesreid1.com`.
We use an alpine container with Python 3 for a minimal image
size. Python comes with a simple HTTP server built-in that
will do the job for us, available through the `http.server`
module (or `SimpleHTTPServer` in Python 2):
```
python3 -m http.server 8081
```
We expose this to port 8081 locally, making the service available
on the Docker network and therefore available to be reverse-proxied
by the nginx container.
Files to be served up are located in `/www/files/` on the host,
which is bind-mounted to `/files` in the container.

82
docs/index.md

@ -3,13 +3,15 @@ @@ -3,13 +3,15 @@
This repo contains a docker compose file
for running the charlesreid1.com site.
The services are:
## Services
The services available through pod-charlesreid1 are:
* [mediawiki](Service_mediawiki.md)
* [apache + php](Service_apachephp.md)
* [mysql](Service_mysql.md)
* [phpmyadmin](Service_phpmyadmin.md) (in progress)
* [nginx + ssl](Service_nginx.md) (in progress)
* [phpmyadmin](Service_phpmyadmin.md)
* [nginx + ssl](Service_nginx.md)
* [python](Service_pythonfiles.md) (in progress)
* [gitea](Service_gitea.md) (in progress)
@ -23,10 +25,24 @@ See **[Running.md](/Running.md)** for info about running this docker pod: @@ -23,10 +25,24 @@ See **[Running.md](/Running.md)** for info about running this docker pod:
* Workflow for Charlesreid1 Docker Pod Updates
* Restoring the Docker Pod from Backups
The quickstart: run
```
$ docker-compose up
```
If you want to rebuild the images (if you changed the Dockerfile),
use the `--build` flag:
```
$ docker-compose up --build
```
## Volumes
See **[Volumes.md](/Volumes.md)** for info about data and volumes
used by this docker pod:
used by this docker pod and each of its services:
* Persistent Data Volumes
* nginx
@ -50,7 +66,7 @@ dedicated to creating backups of any non-version-controlled data inside the cont @@ -50,7 +66,7 @@ dedicated to creating backups of any non-version-controlled data inside the cont
See **[Backups.md](Backups.md)** for coverage of backup and utility scripts.
`utils-backups` - backup utilities (use the scripts below; good for cron jobs)
`utils-backups` - backup utilities (for cron jobs)
`utils-mw` - mediawiki backup utilities
@ -58,7 +74,7 @@ See **[Backups.md](Backups.md)** for coverage of backup and utility scripts. @@ -58,7 +74,7 @@ See **[Backups.md](Backups.md)** for coverage of backup and utility scripts.
## Domains and Ports
See **[Ports.md](Ports.md)** for info about top-level domain names
See **[Domains and Ports.md](Ports.md)** for info about top-level domain names
and ports used by this docker pod.
The domains ports document covers:
@ -73,30 +89,6 @@ The domains ports document covers: @@ -73,30 +89,6 @@ The domains ports document covers:
* gitea ports
* python file server ports
### Additional Port Info
The apache-mediawiki combination is running an apache service listening on port 8989.
This can be adjusted, but should be adjusted in the Dockerfile, `ports.conf`, and `wiki.conf`.
The apache service listens on all interfaces (hence `*:8989` in the apache conf file),
but there is no port mapping specified in `docker-compose.yml` so it does not listen
on any public interfaces.
Thus, the wiki is not publicly accessible via port 8989, but the wiki is available via port 8989
to any container linked to, or connected to the same network as, the mediawiki apache container.
Meanwhile, the nginx container has a public interface listening on port 80
and another listening on port 443. nginx listens for requests going to
the wiki, detected via the url resource prefix being `/w/` or `/wiki/`,
and acts as a reverse proxy, forwarding the requests to Apache.
The user transparently sees everything happening via port 80 or (preferrably) 443,
but on the backend nginx is passing along the URL request and returning the result.
Subdomains are served via reverse proxy on port 7777+.
The webhook server is a flask server listening on port 50000.
## Secrets
@ -109,21 +101,6 @@ without leaking out the information. @@ -109,21 +101,6 @@ without leaking out the information.
* gitea secret key and session id
* nginx ssl certificates
## Container-Specific Configuration Details
Each container has a different way of getting
configuration files into the container.
In the following documents we cover
the specifics of each container.
* [mediawiki](Service_mediawiki.md)
* [apache + php](Service_apachephp.md)
* [mysql](Service_mysql.md)
* [phpmyadmin](Service_phpmyadmin.md)
* [nginx + ssl](Service_nginx.md)
* [python](Service_pythonfiles.md)
* [gitea](Service_gitea.md)
## Backups
@ -133,21 +110,6 @@ See `utils-mw` for mediawiki utilities. @@ -133,21 +110,6 @@ See `utils-mw` for mediawiki utilities.
See `utils-mysql` for mysql utilities.
## Running
From your project directory, start up your application by running:
```
$ docker-compose up
```
If you want to rebuild the images (if you changed the Dockerfile),
use the `--build` flag:
```
$ docker-compose up --build
```
## Links
docker compose documentation:

21
mkdocs.yml

@ -22,19 +22,20 @@ theme: @@ -22,19 +22,20 @@ theme:
text: 'Roboto'
code: 'Roboto Mono'
nav:
- 'Index' : 'index.md'
- 'Running the Charlesreid1 Docker Pod' : 'Running.md'
- 'Index': 'index.md'
- 'Services':
- 'Pod Service: MediaWiki' : 'Service_mediawiki.md'
- 'Pod Service: Apache/PHP' : 'Service_apachephp.md'
- 'Pod Service: MySQL' : 'Service_mysql.md'
- 'Pod Service: phpMyAdmin' : 'Service_phpmyadmin.md'
- 'Pod Service: nginx' : 'Service_nginx.md'
- 'Pod Service: Python File Server' : 'Service_pythonfiles.md'
- 'Pod Service: Gitea' : 'Service_gitea.md'
- 'Running the Pod' : 'Running.md'
- 'Volumes' : 'Volumes.md'
- 'Backups (TODO)' : 'Backups.md'
- 'Backups' : 'Backups.md'
- 'Domains and Ports' : 'Ports.md'
- 'Secrets' : 'Secrets.md'
- 'Pod Service: MediaWiki' : 'Service_mediawiki.md'
- 'Pod Service: Apache/PHP' : 'Service_apachephp.md'
- 'Pod Service: MySQL' : 'Service_mysql.md'
- 'Pod Service: phpMyAdmin' : 'Service_phpmyadmin.md'
- 'Pod Service: nginx' : 'Service_nginx.md'
- 'Pod Service: Python File Server' : 'Service_pythonfiles.md'
- 'Pod Service: Gitea' : 'Service_gitea.md'
# Extensions
markdown_extensions:

Loading…
Cancel
Save