17 Commits

Author SHA1 Message Date
95cbc571d7 try again? 2021-09-08 15:37:49 -07:00
d145d4f783 remove unused files 2021-09-08 15:36:45 -07:00
524e02a136 gitea home -> gitea explore 2020-05-01 23:36:15 -07:00
aedecbb306 add arc-green theme 2020-04-28 00:16:37 -07:00
983cd1bf18 update .gitignore file to ignore custom/ entires 2020-04-27 12:32:57 -07:00
49358ba97e bind-mount d-gitea/data at /app/gitea/data 2020-04-27 11:36:58 -07:00
8149a70815 fix file location in jinja render script 2020-04-27 11:36:38 -07:00
e76c729458 add a lot of (messy) documentation and notes 2020-04-27 11:36:20 -07:00
95430633d4 add more entries to .gitignore to account for different bind mount paths 2020-04-26 21:26:33 -07:00
57ed0ae90b update docs about custom dir and backups 2020-04-26 21:25:53 -07:00
ef006686e9 change activity bkg to match gitea arc-green themme 2019-11-16 16:13:10 -08:00
f73c8c8258 Merge branch 'master' of github.com:charlesreid1-docker/d-gitea
* 'master' of github.com:charlesreid1-docker/d-gitea:
  modify app.ini to give https clone addresses
2019-07-24 09:52:36 -07:00
a7cc86ee0f add todo 2019-07-24 09:51:10 -07:00
0199b5de8d modify app.ini to give https clone addresses 2019-07-23 18:28:18 -07:00
49ff55c039 add script to apply jinja templates - for testing 2019-07-23 17:07:26 -07:00
d637b69ed7 fix variable names in app.ini template 2019-07-23 17:07:01 -07:00
f5149ebece Merge branch 'new-app-ini'
* new-app-ini:
  update gitignore with new custom configuration file location
  moving app.ini template to custom/conf/
2019-07-23 16:57:45 -07:00
8 changed files with 276 additions and 336 deletions

6
.gitignore vendored
View File

@@ -1,3 +1,9 @@
site
*.secret
data/
custom/conf/app.ini
custom/gitea.db
custom/avatars
custom/log/
custom/public/
custom/queues/

View File

@@ -5,13 +5,65 @@ This is the gitea docker container used to run gitea on charlesreid1.com.
You should not run this container by itself (see
[pod-charlesreid1](https://git.charlesreid1.com/docker/pod-charlesreid1.git)).
See the documentation here: <https://pages.charlesreid1.com/d-gitea/>
## TODO
Organize the documentation better, there's a lot going on
* Setup
* Repo organization
* Jinja/ansible
* Gitea files
* New instance
* Restoring instance
* Backing up instance
## Quick Start
### Before you begin
Check `docker-compose.yml` and ensure the bind mounting of folders is
set up to match what you want. Currently:
* `d-gitea/custom/` directory maps to `/data/gitea` in the container
* `d-gitea/data/` directory maps to `/app/gitea/data`
To change this, modify the `docker-compose.yml` file in this repo
or the `docker-compose.yml.j2` file in the pod-charlesreid1 repo.
### Create app.ini
The `app.ini` file is not stored directly in this repo, only a template file
`app.ini.j2` is stored. You need to create an `app.ini` file from the template
before beginning.
To create an `app.ini` file from the template, populate the secret files at:
* `internal_token.secret`
* `secret_key.secret`
Then run the `make_app_ini.sh` script:
```
./make_app_ini.sh
```
This will create an `app.ini` file from the template at `custom/conf/app.ini.j2`,
and will put the new file in `custom/conf/app.ini`.
### Running
Start the container with `docker-compose up` if running standalone, or by starting
the docker pod if running in a pod.
## Links
[documentation: d-gitea container](https://pages.charlesreid1.com/d-gitea/) (you are here)
[documentation: d-gitea container](https://pages.charlesreid1.com/d-gitea/)
[source code on git.charlesreid1.com: docker/d-gitea](https://git.charlesreid1.com/docker/d-gitea)
[source code on github.com: charlesreid1-docker/d-gitea](https://github.com/charlesreid1-docker/d-gitea)
[source code on github.com: charlesreid1-docker/d-gitea](https://github.com/charlesreid1-docker/d-gitea) (you are here)
[gitea documentation - running gitea with docker](https://docs.gitea.io/en-us/install-with-docker/)

View File

@@ -5,10 +5,14 @@
;; march 2017
;; https://github.com/go-gitea/gitea/blob/master/conf/app.ini
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
APP_NAME = {{ gitea_app_name }}
RUN_USER = git
RUN_MODE = prod
[ui]
DEFAULT_THEME = arc-green
[database]
DB_TYPE = sqlite3
HOST =
@@ -23,18 +27,18 @@ ROOT = /data/git/repositories
PREFERRED_LICENSES = MIT License
; This gets rid of the HTTP option to check out repos...
DISABLE_HTTP_GIT = true
DISABLE_HTTP_GIT = false
[server]
PROTOCOL = http
DOMAIN = {{ gitea_domain }}
DOMAIN = git.{{ server_name_default }}
#CERT_FILE = /www/gitea/certs/cert.pem
#KEY_FILE = /www/gitea/certs/key.pem
SSH_DOMAIN = {{ gitea_domain }}
SSH_DOMAIN = git.{{ server_name_default }}
HTTP_PORT = 3000
HTTP_ADDR = 0.0.0.0
;ROOT_URL = https://{{ gitea_domain }}
ROOT_URL = %(PROTOCOL)s://%(DOMAIN)s:%(HTTP_PORT)s/
ROOT_URL = https://git.{{ server_name_default }}
;ROOT_URL = %(PROTOCOL)s://%(DOMAIN)s:%(HTTP_PORT)s/
DISABLE_SSH = false
; port to display in clone url:
SSH_PORT = 222

View File

@@ -1,97 +0,0 @@
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; My Amazing Gitea Config File
;;
;; charles reid
;; march 2017
;; https://github.com/go-gitea/gitea/blob/master/conf/app.ini
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
APP_NAME = charlesreid1.blue code hub
RUN_USER = git
RUN_MODE = prod
[database]
DB_TYPE = sqlite3
HOST =
NAME =
USER =
PASSWD =
SSL_MODE = disable
PATH = /data/gitea/gitea.db
[repository]
ROOT = /data/git/repositories
PREFERRED_LICENSES = MIT License
; This gets rid of the HTTP option to check out repos...
;DISABLE_HTTP_GIT = true
[server]
PROTOCOL = http
DOMAIN = git.charlesreid1.blue
#CERT_FILE = /www/gitea/certs/cert.pem
#KEY_FILE = /www/gitea/certs/key.pem
SSH_DOMAIN = git.charlesreid1.blue
HTTP_PORT = 3000
HTTP_ADDR = 0.0.0.0
;ROOT_URL = http://git.charlesreid1.blue
ROOT_URL = %(PROTOCOL)s://%(DOMAIN)s:%(HTTP_PORT)s/
DISABLE_SSH = false
SSH_PORT = 22
OFFLINE_MODE = false
; Upper level of template and static file path
; default is the path where Gitea is executed
;;;;STATIC_ROOT_PATH = /www/gitea/static
[mailer]
ENABLED = false
[service]
REGISTER_EMAIL_CONFIRM = false
ENABLE_NOTIFY_MAIL = false
DISABLE_REGISTRATION = true
ENABLE_CAPTCHA = false
REQUIRE_SIGNIN_VIEW = false
[picture]
DISABLE_GRAVATAR = true
ENABLE_FEDERATED_AVATAR = false
AVATAR_UPLOAD_PATH = /data/gitea/avatars
[session]
PROVIDER = file
[log]
MODE = file
; Either "Trace", "Debug", "Info", "Warn", "Error", "Critical", default is "Trace"
LEVEL = Info
ROOT_PATH = /data/gitea/log
; For "file" mode only
[log.file]
LEVEL =
; This enables automated log rotate(switch of following options), default is true
LOG_ROTATE = true
; Max line number of single file, default is 1000000
MAX_LINES = 1000000
; Max size shift of single file, default is 28 means 1 << 28, 256MB
MAX_SIZE_SHIFT = 28
; Segment log daily, default is true
DAILY_ROTATE = true
; Expired days of log file(delete after max days), default is 7
MAX_DAYS = 7
[cron.archive_cleanup]
ENABLED = false
[security]
INSTALL_LOCK = true
SECRET_KEY = REPLACEME_SECRETKEY_SECRET
MIN_PASSWORD_LENGTH = 6
INTERNAL_TOKEN = REPLACEME_INTERNALTOKEN_SECRET
[other]
SHOW_FOOTER_BRANDING = false
; Show version information about Gitea and Go in the footer
SHOW_FOOTER_VERSION = false
; Show time of template execution in the footer
SHOW_FOOTER_TEMPLATE_LOAD_TIME = false

View File

@@ -1,158 +1,3 @@
{{template "base/head" .}}
<div class="home">
<div class="ui stackable middle very relaxed page grid">
<div class="sixteen wide center aligned centered column">
</div>
<div class="hero">
<h1 class="ui icon header title">
git.charlesreid1.com
</h1>
<!--
<h2>{{.i18n.Tr "app_desc"}}</h2>
-->
</div>
<div>
<!--
<img class="logo" src="{{AppSubUrl}}/img/gitea-lg.png" />
-->
<h3 class="ui icon header title"><b>@charlesreid1 activity graph:</b></h3>
<div id="calendar"></div>
</div>
</div>
</div>
<!-- gitea source: templates/base/head -->
<script src="https://d3js.org/d3.v4.min.js"></script>
<script>
//<script type="text/javascript">
var width = 700,
height = 90,
cellSize = 12;
// big integers
var formatStuff = d3.format(",");
/*
TEH COLORRRZZZZ
*/
var realBackgroundColor = "#fff";//"#272b30";
var tileBackgroundColor = realBackgroundColor;//"#3a3a3a";
var tileStrokeColor = "#ccc";
var monthStrokeColor = "#aaa";
var color = d3.scaleQuantize()
.domain([0, 60])
.range([ "#ffeda0", "#fed976", "#feb24c", "#fd8d3c", "#fc4e2a", "#e31a1c", "#bd0026", "#800026" ]);
// red and yeller
//.range(["#4d2b4b","#5a3961","#684777","#77558f","#8463a5","#cc9189","#fba25c","#f78e43","#f47b2b","#f16913"]);
// purple and orange
//.range(["#004a4a","#0b6f6f","#169494","#29a8a8","#4c9090","#717777","#945f5f","#b84747","#db2f2f","#ff1717"]);
// seafoam green blue to candy red
/*
Make the canvas
*/
var svg = d3.select("div#calendar")
.selectAll("svg")
.data(d3.range(2014, 2020).reverse())
.enter().append("svg")
.attr("width", width)
.attr("height", height)
.append("g")
.attr("transform", "translate(" + ((width - cellSize * 53) / 2) + "," + (height - cellSize * 7 - 1) + ")");
/*
Write the years
*/
svg.append("text")
.attr("transform", "translate(-6," + cellSize * 3.5 + ")rotate(-90)")
.attr("font-family", "sans-serif")
.attr("font-size", 10)
.attr("fill", "#bbb")
.attr("text-anchor", "middle")
.text(function(d) { return d; });
/*
Draw the tiles representing days of the year
(also draw tile outlines)
*/
var rect = svg.append("g")
.attr("fill", tileBackgroundColor)
.attr("stroke", tileStrokeColor)
.selectAll("rect")
.data(function(d) { return d3.timeDays(new Date(d, 0, 1), new Date(d + 1, 0, 1)); })
.enter().append("rect")
.attr("width", cellSize)
.attr("height", cellSize)
.attr("x", function(d) { return d3.timeWeek.count(d3.timeYear(d), d) * cellSize; })
.attr("y", function(d) { return d.getDay() * cellSize; })
.datum(d3.timeFormat("%Y-%m-%d"));
/*
Draw outlines of groups representing months
*/
svg.append("g")
.attr("fill", "none")
.attr("stroke", monthStrokeColor)
.selectAll("path")
.data(function(d) { return d3.timeMonths(new Date(d, 0, 1), new Date(d + 1, 0, 1)); })
.enter().append("path")
.attr("d", pathMonth);
/*
Load up the csv file
*/
d3.csv("/data/charlesreid1-data/raw/branch/master/commit_counts.csv", function(error, csv) {
if (error) throw error;
/*
This is where you decide what values to plot
*/
var data = d3.nest()
.key(function(d) { return d.date; })
.rollup(function(d) {
return d[0].commits;
})
.object(csv);
rect.filter(function(d) { return d in data; })
.attr("fill", function(d) { return color(data[d]); })
.append("title")
.text(function(d) { return d + ": " + formatStuff(data[d]); });
});
function pathMonth(t0) {
var t1 = new Date(t0.getFullYear(), t0.getMonth() + 1, 0),
d0 = t0.getDay(), w0 = d3.timeWeek.count(d3.timeYear(t0), t0),
d1 = t1.getDay(), w1 = d3.timeWeek.count(d3.timeYear(t1), t1);
return "M" + (w0 + 1) * cellSize + "," + d0 * cellSize
+ "H" + w0 * cellSize + "V" + 7 * cellSize
+ "H" + w1 * cellSize + "V" + (d1 + 1) * cellSize
+ "H" + (w1 + 1) * cellSize + "V" + 0
+ "H" + (w0 + 1) * cellSize + "Z";
}
</script>
<meta http-equiv="Refresh" content="0; url=/explore/repos" />
{{template "base/footer" .}}

View File

@@ -15,7 +15,8 @@ services:
- gitea
volumes:
- "stormy_gitea_data:/data"
- "./custom:/data/gitea"
- "./custom:/data/gitea"
- "./data:/app/gitea/data"
ports:
- "3000:3000"
- "222:22"

View File

@@ -14,30 +14,53 @@ Docker files for running gitea
## Table of Contents
* secrets
* app.ini file
* jinja templates
* container directory structure
* container data volume
* git repositories
* files mounted into container
* using the `docker-compose.yml` file
* notes on custom files
* configuring gitea with `app.ini`
* customizing gitea with custom files
* backing up and restoring gitea
## Secrets
There are two secrets to set in the `app.ini` before running gitea:
There are two secrets that must be set in the Gitea configuration file:
the internal token and the secret key.
These can be set in `*.secret` files:
The two secret values can be placed in two `*.secret` files:
```plain
internal_token.secret
secret_key.secret
```
The contents should be the value of the variable
The contents should be the value of the secret variable
you wish to use in `custom/conf/app.ini`.
These files are not tracked by git.
## Jinja Templates
The `app.ini` script is not stored directly in this repo, since it contains
sensitive information. Instead, we store a Jinja template, `app.ini.j2`, which
is rendered into a real `app.ini` file after variable substitutions, etc.
Normally, we use this repo with Ansible, so we don't deal with the Jinja template
ourselves.
If you want to render the Jinja template into a real config file without using
Ansible, use the `make_app_ini.sh` script:
```
$ ./make_app_ini.sh
```
(This requires the two secret files mentioned above to be present.)
## Container Directory Structure
The `custom/` dir in this folder maps to the `/data/gitea` volume
@@ -56,6 +79,8 @@ On the host machine, you can access named data volumes at
`/var/lib/docker/volumes/gitea_gitea/_data`
or copy files in and out of the container using `docker cp`.
## Git Repositories
### directory structure before adding repos to gitea
Directory structure for host-mounted gitea directory
@@ -149,54 +174,38 @@ gitea
25 directories, 29 files
```
### Backing up git repositories
Backing up git repositories associated with the gitea instance
is a separate step from gitea dump (see above).
Before backing up repositories, perform a gitea dump without the
repositories included.
All git repositories are stored in `/data/git/repositories/`. They are stored
in the subdirectory `org-name/repo-name`.
To back up all repositories, copy the folder `/data/git/repositories` from the
gitea container.
To restore all repositories, copy the folder `/data/git/repositories` into the
gitea container.
## Files Mounted Into Container
### `custom/conf` configuration file
The `d-gitea/custom/` directory is mounted into the container at `/data/gitea`.
The conf dir contains configuration files to configure how gitea works.
The `d-gitea/data/` directory is mounted into the container at `/app/gitea/data`.
The `app.ini` file needs to contain two secrets,
which are scrubbed in `app.ini.sample`.
To make the custom configuration file, follow the instructions mentioned in
the "Secrets" section.
The two secrets that are needed are:
* an "internal token" secret, contained in `internal_token.secret`
* a "secret key" secret, contained in `secret_key.secret`
Use the `make_app_ini.sh` script to add the two secrets to the document.
This will use sed to find/replace instances of the scrubbed secret,
and will output the result to `custom/conf/app.ini`.
```plain
./make_app_ini.sh
```
This generates `custom/conf/app.ini`.
When the container is run, this file will be at `/data/gitea/conf/app.ini`.
## `custom/templates` template files
The templates directory contains template files. These are gitea templates that
control how particular kinds of gitea pages look. For example, a template can
be used to modify how the user page looks, or modify the layout of repository
main pages.
In the container, this will be at `/data/gitea/templates/`.
## `custom/pages` gitea pages
The pages directory contains one-off pages or static content that is
hosted by gitea at the same domain (git.charlesreid1.com) but
not necessarily incorporated into the gitea site.
For example, a custom "about me" page could be added as a static .html file,
and it would be hosted at `git.charlesreid1.com/about`.
In the container, this will be at `/data/gitea/pages/`.
## Using the `docker-compose.yml` File
### Standalone
This directory contains a docker-compose file that can be used to run
a gitea server on port 3000.
@@ -212,43 +221,172 @@ structure. See below for more info.
Use this as a project seed to add gitea containers to other docker pods.
### pod-charlesreid1
The main use of this repo is as a submodule in
[pod-charlesreid1](https://github.com/charlesreid1-docker/pod-charlesreid1.git).
This pod is set up by Ansible, which integrates well with the Jinja template approach.
## Notes on Custom Files
The settings in the app.ini file are documented [here](https://docs.gitea.io/en-us/config-cheat-sheet/).
The settings in the `app.ini` file are documented [here](https://docs.gitea.io/en-us/config-cheat-sheet/).
An extensive sample app.ini file is [here](https://github.com/go-gitea/gitea/blob/master/custom/conf/app.ini.sample) (WARNING: this broke gitea).
An extensive sample `app.ini` file is [here](https://github.com/go-gitea/gitea/blob/master/custom/conf/app.ini.sample) (WARNING: this broke gitea).
The existing gitea templates are in the gitea repo under [templates/](https://github.com/go-gitea/gitea/tree/master/templates).
These can be modified as needed and placed in the `custom/templates/` directory.
## Backing Up and Restoring Gitea
### custom templates directory
Fortunately, gitea provides dump functionality.
The templates directory `d-gitea/custom/templates` contains template files.
These are gitea templates that control how particular kinds of gitea pages look.
For example, a template can be used to modify how the user page looks, or modify
the layout of repository main pages.
Unfortunately, there is no restore functionality.
In the container, this will be at `/data/gitea/templates/`.
See [pod-charlesreid1/utils-gitea](https://git.charlesreid1.com/docker/pod-charlesreid1/src/branch/master/utils-gitea)
for proper backup/restore scripts.
### custom gitea pages
### Executive Summary
The custom pages directory `d-gitea/custom/pages` contains one-off pages or static
content that is hosted by the gitea instance, but not necessarily incorporated into
the gitea site.
Backup:
For example, a custom "about me" page could be added as a static .html file,
and it would be hosted at `git.charlesreid1.com/about`.
* create a backup target directory in the container
* create a gitea dump zip file using `gitea dump` command
* create a gitea avatars zip file
* copy everything in the backup target directory out of the container
* remove the backup target directory
In the container, this will be at `/data/gitea/pages/`.
Restore:
* create a restore target directory in the container
* copy gitea dump and gitea avatars zip files into restore target dir
* unpack dump zip, unpack avatars zip
* unzip repositories zip (contained in dump zip)
* restore `repositories/` folder to `/data/git/repositories/`
* (skip restoring `custom/` files, version control takes care of that)
* restore sqlite database using sql database dump
* restore avatars
* remove the restore target directory
## Backing up and restoring gitea
Running `gitea dump` command will dump files required for restoring an existing
Gitea instance. Unfortunately, gitea's backup and restore functionality is an
absolute dumpster fire.
We do our best to walk through the process, but here is a summary:
* On the old system:
* Create a backup file `gitea-dump.zip` using the very specific `gitea dump` incantation that works
* Move the backup file out of the gitea container
* On the new system:
* Unzip the backup file `gitea-dump.zip`
* Put appropriate files in appropriate location
* Use the table below to determine where in the repo the gitea dump files should go,
and where they will be available inside the gitea container
### Creating a Backup
To create a gitea dump, connect to the docker container and get a bash shell as the user
`git` via the docker exec command:
```
# connect to gitea container
docker exec -it --user git name_of_gitea_container /bin/bash
```
This will give you a bash shell as the user `git`. Now create a gitea dump file
(the gitea dump command requires you to be in `/app/gitea`, this assumes that the
gitea executable is at the default location of `/app/gitea/gitea`):
```
# necessary
cd /app/gitea
# create gitea dump
/app/gitea/gitea dump --file gitea-dump.zip --skip-repository
```
**IMPORTANT: The `--skip-repositories` flag means we are making the gitea dump
way, way, way smaller, but it also means we need to back up and restore the
repositories folder ourselves! (See below for instructions.)**
Now copy the file out of the container, then remove it from the container:
```
# copy gitea dump out of container
docker cp name_of_gitea_container:/app/gitea/gitea-dump.zip .
# remove gitea dump
docker exec -it name_of_gitea_container rm /app/gitea/gitea-dump.zip
```
### Contents of Dump File
When the gitea dump file is unzipped, it will create the following files:
* `app.ini`
* `custom/` directory
* `log/` directory (useless duplicate, already contained in `custom/` dir)
* `data/` directory
These files should map to the following locations in the docker container
running the live gitea instance:
```
gitea dump file: gitea live instance:
---------------- --------------------
app.ini /data/gitea/conf/app.ini
custom/ /data/gitea/
log/ (useless duplicate of custom/log/)
data /app/gitea/data
```
### Restoring a Backup
To restore a backup, copy the following files from the gitea dump
to the following locations inside this repository:
```
gitea dump file: d-gitea repo location:
---------------- ----------------------
app.ini d-gitea/custom/conf/app.ini
custom/* d-gitea/custom/*
data d-gitea/data
```
(If you're running pod-charlesreid1, put these files in the
specified location in the `d-gitea` submodule.)
### Restoring repositories directory
Note that when we created the gitea dump, we excluded the repositories themselves.
This is because these will greatly inflate the size of our gitea dump and will make
it much more difficult to store our backup files.
Repository contents can be backed up separately as follows:
* Log in to the old server
* Back up the `/data/git/repositories` directory (copy and compress)
* Copy the backup to the new server
* Log in to the new server
* Mount the `/data/git/repositories` folder
Optionally, if you want to keep the repositories folder in its own location,
modify `docker-compose.yml` to add the following line to the `gitea` container's
`volumes` configuration:
```
services:
server:
image: gitea/gitea:latest
...
volumes:
- "/path/to/repositories:/data/git/repositories"
...
```
This should make it easier to manage, back up, and restore the repositories folder.
### Database backups
We opt for the SQLite backend for gitea, which means the database
is kept in a flat file on disk called `/data/gitea/gitea.db`.
The location of this file and the format of the database are specified
in the config file in `d-gitea/custom/conf/app.ini`.
This file should not be edited, instead change the Jinja template
`d-gitea/custom/conf/app.ini.j2` and remake `app.ini` from the template.

View File

@@ -1,9 +0,0 @@
#!/bin/bash
set -x
# secret_key and internal_token should be SIMPLE or this
# gets screwy, and I don't want to do any more debugging.
#
cat app.ini.sample \
| sed 's/REPLACEME_INTERNALTOKEN_SECRET/'$(cat internal_token.secret)'/g' \
| sed 's/REPLACEME_SECRETKEY_SECRET/'$(cat secret_key.secret)'/g' > custom/conf/app.ini