Self-hosted Git Server

Host your code on your or Forgejo instance behind Cloudflare.

Self-hosted Git Server

Despite the click-bait subheading, I like Github. However, I've been trying to avoid storing my most important data (code being a big subset of that) on somebody else's servers (including cloud-based VPS).

There are many reasons one might want to host their own Git server, such as costs, discomfort with AI policies, privacy for sensitive projects, etc., but, if I'm being honest, the biggest reason for me is that it's fun (ok... well, I think it's fun...)!

Of course, there are also some good reasons NOT to host your own Git server.

  1. Github has a huge team of infrastructure and security folks making sure that everything stays up and running, and secure. It's hard to compete with that.
  2. Hosting your code on a private instance like this, especially if your disable user signups like I have, eliminates/complicates the social side of coding. No helpful drive-by pull requests. Nobody else can raise bugs, etc.
๐Ÿ’ฝ
The full code for this tutorial can be found in this git repo.

For this project, I'll run all of this under Docker on my home server and use Cloudflare Tunnels to expose it to the Internet. This has some advantages for me:

  1. My home servers have ample resources (RAM, CPU, and disk). Forgejo is fairly lightweight on its own, but as soon as you start using the GitHub Action-style build tools, it starts to push the limits of any reasonably priced VPS.
  2. It avoids me exposing my home IP or punching holes in the firewall.
  3. I can use Cloudflare's defenses if/when I need them.

So, let's get started and set up Forgejo (a more open fork of Gitea) for my domain git.onewheelgeek.ca.

Setup a Cloudflare Tunnel

First, you'll need your domain's DNS to be hosted with Cloudflare to use Cloudflare Tunnels. If you bought your domain from Cloudflare, this is probably already the case. If you purchased your domain from a different registrar, you must switch your authoritative nameservers to Cloudflare.

Once that's done, click on the "Zero Trust" tab and then "Networks" and "Tunnels." From here, you can create a new Cloudflare "Cloudflared" Tunnel.

Giving our Tunnel a good name is important. You may have only one now, but they are addicting. Trust me!

On the next screen, copy the example code that looks like:

sudo cloudflared service install eyJhIjoiMGQwNjMxMTgzN2M1ZWJlYzVkOWMxMWNkOWY0MWZmMGMiLCJ0IjoiZjRjN2JiNmEtYjg5Ni00OTgyLWExMDMtNTUyOTkwMWQxNjhmIiwicyI6Ik1tRTFORFprWlRFdE9XVmxZaTAwTWpWaUxXRXhOR1l0WlROallqWmpaV00xT0RneiJ9
๐Ÿ—’๏ธ
The big code after "install" is your Cloudflare Tunnel token, which you'll need later.

Next, you'll add a public hostname for your git server, such as git.onewheelgeek.ca. Your name will vary here, but make sure to point it to server:3000 to match the configuration in the docker scripts.

At this point, we're ready to start spinning up our service!

The Docker Service

We will use Docker Compose here, allowing us to spin up multiple services in one go.

Additionally, the intent is that this will be entirely self-contained. This means my docker-compose.yml will contain everything required (Cloudflared, Forgejo, Postgres, Forgejo runners, ...) with no external dependencies. To this end, instead of using Docker Volumes, I use bind-mounts throughout and store all the application data under ./data. This has the HUGE advantage of portability and simplifying backups/restores.

I will build this up in pieces and try to explain it as we go.

First, make sure you've got Docker installed on your server machine. The machine itself can be a VPS or a machine in your homelab. Thanks to the magic of Cloudflare Tunnels, it doesn't matter. Even more magical is I can move this service at any time, anywhere, and it will continue to work.

Next, let's create a folder for this service. For this example, I'll use /docker/git.

๐Ÿ—’๏ธ
Incidentally, always make sure you have backups! My current favorite recipe is to store my docker stuff under /docker and then use Autorestic to back that content up (along with /etc and /root and /home) every few hours to Backblaze B2 or Borgbase.

In this folder, we'll create two files: docker-compose.yml (for the common stuff) and .env (which contains the installation-specific stuff). In the future, if you want another git server, you can copy these two files, adjust .env and you are off to the races!

The Cloudflare Tunnel

Now, let's create our Cloudflare Tunnel container. This container will connect to Cloudflare and shuttle traffic from Cloudflare to your underlying services.

In docker-compose.yml let's start with:

services:

  tunnel:
    image: cloudflare/cloudflared
    command: tunnel --no-autoupdate run
    restart: unless-stopped
    environment:
      - TUNNEL_TOKEN=${TUNNEL_TOKEN}

Notice that we've got a variable ${TUNNEL_TOKEN} referenced here. This is the credential we got when creating our tunnel earlier. Let's add this to our .env file (your token will differ from mine).

TUNNEL_TOKEN=eyJhIjoiMGQwNjMxMTgzN2M1ZWJlYzVkOWMxMWNkOWY0MWZmMGMiLCJ0IjoiZjRjN2JiNmEtYjg5Ni00OTgyLWExMDMtNTUyOTkwMWQxNjhmIiwicyI6Ik1tRTFORFprWlRFdE9XVmxZaTAwTWpWaUxXRXhOR1l0WlROallqWmpaV00xT0RneiJ9

Now, if we start this up by running docker compose up we'll see a bunch of stuff that looks like this:

tunnel-1  | 2024-11-02T18:01:30Z INF Initial protocol quic
tunnel-1  | 2024-11-02T18:01:30Z INF ICMP proxy will use 172.80.1.5 as source for IPv4
tunnel-1  | 2024-11-02T18:01:30Z INF ICMP proxy will use ::1 in zone lo as source for IPv6
tunnel-1  | 2024-11-02T18:01:30Z INF Starting metrics server on 127.0.0.1:34821/metrics
tunnel-1  | 2024-11-02T18:01:30Z INF Registered tunnel connection connIndex=0 connection=a826eaba-0ce9-4b1b-90f3-aae75a13b6dc event=0 ip=198.41.200.33 location=sea09 protocol=quic
tunnel-1  | 2024-11-02T18:01:31Z INF Registered tunnel connection connIndex=1 connection=d3acef82-abe9-4a71-86b6-fcd8f7b3e07b event=0 ip=198.41.192.167 location=yvr01 protocol=quic
tunnel-1  | 2024-11-02T18:01:32Z INF Registered tunnel connection connIndex=2 connection=0503088f-cbf4-423d-9c2c-b8edaec705bd event=0 ip=198.41.192.227 location=yvr01 protocol=quic
tunnel-1  | 2024-11-02T18:01:33Z INF Registered tunnel connection connIndex=3 connection=81e36f92-8631-455e-b8cd-4018baf62be4 event=0 ip=198.41.200.73 location=sea08 protocol=quic

Perfect. But it's not going to work yet. Let's CTRL+C out of this and go on to setting up Forgejo!

Setting up Forgejo

We will set up Forgejo backed by Postgres so we must add a couple of containers for that.

First, in docker-compose.yml add the following:

  server:
    image: codeberg.org/forgejo/forgejo:${FORGEJO_TAG}
    environment:
      - RUN_MODE=prod
      - USER_UID=1000
      - USER_GID=1000
      - FORGEJO__database__DB_TYPE=postgres
      - FORGEJO__database__HOST=db:5432
      - FORGEJO__database__NAME=gitea
      - FORGEJO__database__USER=gitea
      - FORGEJO__database__PASSWD=${FORGEJO_DB_PASSWORD}
    restart: always
    volumes:
      - ./data/data:/data
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    depends_on:
      - db

  db:
    image: postgres:13
    restart: always
    environment:
      - POSTGRES_USER=gitea
      - POSTGRES_PASSWORD=${FORGEJO_DB_PASSWORD}
      - POSTGRES_DB=gitea
    volumes:
      - ./data/postgres:/var/lib/postgresql/data

And we'll add our new variables to our .env with a great randomly generated password and the version of Forgejo we want to use.

FORGEJO_DB_PASSWORD=great_random_password
FORGEJO_TAG=8

Notice the volumes are defined as ./data/... for Postgres and Forgejo. This keeps all the important data for these containers in a subfolder alongside our docker-compose.yml and .env files. This is handy because you can move this one folder to a new machine, and it'll work. It also makes backups/restores much easier (for personal projects, easy is good).

Let's start it up again with docker compose up -d.

At this point, you should be able to visit https://git.onewheelgeek.ca in your browser, you'll see the Forgejo setup screen asking you to update the site name, confirm database settings, set up email, initial admin account, etc.

You can do this now and then log in to your new Forgejo server.

Adding Runners (optional)

โ—
Note that Forgejo recommends against using their runners in production for reliability and security reasons. Since the runners themselves are not exposed to the Internet, and the users of my Forgejo servers are all trusted, this is not a concern for my use-case, and I find them very helpful.

Forgejo supports Github-style Actions, which allow you to run automated builds/deploys upon pushes, releases, etc. This requires a bit more setup.

First, we need the Runner registration token from your running Forgejo server. Navigate to "Site Administration" under your user menu (top right). Select "Actions" and "Runners" in the menu on the left. Now click the "Create New Runner" button and copy the "Registration Token." We'll need that in a moment to allow the Runner to connect to Gitea.

Next, we'll add some new containers to our docker-compose.yml.

  runner-register:
    image: code.forgejo.org/forgejo/runner:${FORGEJO_RUNNER_TAG}
    links:
      - docker-in-docker
      - server
    environment:
      DOCKER_HOST: tcp://docker-in-docker:2376
    volumes:
      - ./data/runner-data:/data
    user: 0:0
    command: >-
      bash -ec '
      while : ; do
        forgejo-runner create-runner-file --connect --instance http://server:3000 --name ${RUNNER_NAME} --secret ${SHARED_SECRET} && break ;
        sleep 1 ;
      done ;
      sed -i -e "s|\"labels\": null|\"labels\": ${RUNNER_LABELS}|" .runner ;
      forgejo-runner generate-config > config.yml ;
      sed -i -e "s|network: .*|network: host|" config.yml ;
      sed -i -e "s|^  labels: \[\]$$|  labels: ${RUNNER_LABELS}|" config.yml ;
      sed -i -e "s|^  envs:$$|  envs:\n    DOCKER_HOST: tcp://docker:2376\n    DOCKER_TLS_VERIFY: 1\n    DOCKER_CERT_PATH: /certs/client|" config.yml ;
      sed -i -e "s|^  options:|  options: -v /certs/client:/certs/client|" config.yml ;
      sed -i -e "s|  valid_volumes: \[\]$$|  valid_volumes:\n    - /certs/client|" config.yml ;
      chown -R 1000:1000 /data
      '      

  runner-daemon:
    image: code.forgejo.org/forgejo/runner:4.0.1
    links:
      - docker-in-docker
      - server
    environment:
      DOCKER_HOST: tcp://docker:2376
      DOCKER_CERT_PATH: /certs/client
      DOCKER_TLS_VERIFY: "1"
    volumes:
      - ./data/runner-data:/data
      - ./data/docker_certs:/certs
    command: >-
      bash -c '
      while : ; do test -w .runner && forgejo-runner --config config.yml daemon ; sleep 1 ; done
      '      

  docker-in-docker:
    image: docker:dind
    hostname: docker  # Must set hostname as TLS certificates are only valid for docker or localhost
    privileged: true
    environment:
      DOCKER_TLS_CERTDIR: /certs
      DOCKER_HOST: docker-in-docker
    volumes:
      - ./data/docker_certs:/certs

And define our new variables in .env

RUNNER_NAME=runner
RUNNER_LABELS='[\"docker:docker://code.forgejo.org/oci/node:20-bookworm\", \"ubuntu-22.04:docker://catthehacker/ubuntu:act-22.04\"]'
SHARED_SECRET=##REGISTRATION TOKEN GOES HERE##
FORGEJO_RUNNER_TAG=4.0.1

Now, stop your Forgejo instance (CTRL+C, if it's running in the foreground) and re-run docker compose up -d.

At this point, you should see that your runner has successfully registered.

And you can test it out by creating a new repo and adding a file .forgejo/workflows/demo.yml: (for now, it's probably easiest to do this from the New File button in the Web UI)

on: [push]
jobs:
  test:
    runs-on: docker
    steps:
      - run: echo All Good

When you commit your changes, you'll see a new action running on the "Actions" tab. The first time you do this, it will take a while as it's downloading container images, but eventually, you should see something like this:

Actions are an awesome way to automate builds and deployments when your repositories are updated. I routinely use it to build and deploy static websites, for example.

Now, the only thing we're missing is SSH access. This is a bit tricky because of limitations around Cloudflare Tunnels.

Using SSH

One cool feature of Cloudflare's Tunnels is that you can use them to tunnel non-HTTP traffic, such as SSH. Unfortunately, they aren't quite as seamless to use as they are for HTTP.

  1. Your clients have to use cloudflared as a proxy to access those services. This means you'll need to have it installed on any machine wanting to use the SSH service.
  2. Non-HTTP services can't share the same subdomain as your HTTP server. At least, I'm pretty sure this is a limitation.

So, let's add another "Public Hostname" for our tunnel that points to SSH.

Note that here, I'm using git-ssh.onewheelgeek.ca. This isn't just automatic. Normally, Forgejo's web and SSH services are expected to be on the same hostname, but fortunately, you can override this with the FORGEJO__server__SSH_DOMAIN variable in the configuration in our docker-compose.yml.

So, updating the server container in docker-compose.yml:

  server:
    image: codeberg.org/forgejo/forgejo:${FORGEJO_TAG}
    environment:
      - RUN_MODE=prod
      - USER_UID=1000
      - USER_GID=1000
      - FORGEJO__server__SSH_DOMAIN=${SSH_DOMAIN}
      - FORGEJO__database__DB_TYPE=postgres
      - FORGEJO__database__HOST=db:5432
      - FORGEJO__database__NAME=gitea
      - FORGEJO__database__USER=gitea
      - FORGEJO__database__PASSWD=${FORGEJO_DB_PASSWORD}
    restart: always
    volumes:
      - ./data/data:/data
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    depends_on:
      - db

And a new variable in .env:

SSH_DOMAIN=git-ssh.onewheelgeek.ca

And, of course, restarting everything again.

We then need to add that new service to Cloudflare. Find your existing Tunnel and add a new Public Hostname to it.

Unlike HTTP, when we want to use our SSH service, we have to use cloudflared on our client machine to proxy traffic through to the underlying SSH service. Fortunately, this is fairly easy and doesn't require us to set tokens/credentials on the client-side cloudflared.

The easiest option is to add an entry to your SSH configuration (~/.ssh/config) instructing it to use cloudflared when connecting to your new host.

Host git-ssh.onewheelgeek.ca
    ProxyCommand /opt/homebrew/bin/cloudflared access ssh --hostname %h

Now, when you ssh to git-ssh.onewheelgeek.ca, SSH will fire up a Cloudflare Tunnel to make it happen. Like so!

โžœ  docker-samples git:(main) โœ— ssh [email protected]
PTY allocation request failed on channel 0
Hi there, jeff! You've successfully authenticated with the key named Yubikey, but Forgejo does not provide shell access.
If this is unexpected, please log in with password and setup Forgejo under another user.
Connection to git-ssh.onewheelgeek.ca closed.

Success!

Bonus: Gists?

I routinely need to share snippets of code with others, or hang on to handy examples for myself. Historically, I used Github Gists for this. Unfortunately, Forgejo doesn't have an equivalent. Fortunately, OpenGist exists; it supports SSO through Forgejo, and it even supports embedding code on other pages (like so):

To add this to our setup, we can add another container to our docker-compose.yml file:

  opengist:
    image: ghcr.io/thomiceli/opengist:1.8
    restart: unless-stopped
    environment:
     - OG_SSH_GIT_ENABLED=false
     - OG_HTTP_PORT=80
     - OG_GITEA_NAME=Forgejo
     - OG_GITEA_URL=${ROOT_URL}
     - OG_GITEA_CLIENT_KEY=${OPENGIST_GITEA_CLIENT_ID}
     - OG_GITEA_SECRET=${OPENGIST_GITEA_SECRET}
    volumes:
      - "./data/opengist:/opengist"

And then in our .env define two new variables. These will come from our Forgejo instance (once it's running), and federated sign-on will be allowed with our Forgejo instance.

OPENGIST_GITEA_CLIENT_ID=
OPENGIST_GITEA_SECRET=

To get the values, log in to Forgejo as an admin and navigate to "Site administration" (under your user menu). Then, navigate to "Integration" > "Applications" in the sidebar.

Create a new OAuth2 application:

This will generate a new Client ID and Secret which must be pasted in to you those new variables in your .env file.

Finally, make sure a new public hostname in Cloudflare for gist.yourname.com that points to http://opengist.

And, docker compose up -d.

Now you should be able to connect to gist.yourname.com and you'll see a "Continue with Forgejo account" button that will let your Forgejo users log in all seamlessly.

Polishing it

The above content is somewhat simplified to make it easier to follow in blog form. In the full example in the git repo, I've gone a little bit further and tried to define the majority of the configuration in my docker-compose.yml and .env to eliminate some of those manual steps and the need to use the installation wizard and manually set up the runner.

Resources

Forgejo has this great page listing the different configuration parameters available to be set via. environment variables (from docker-compose.yml).

Configuration Cheat Sheet | Forgejo โ€“ Beyond coding. We forge.

I also heavily relied on the documentation here for the Runner setup.

runner
Forgejo runner - alpha release, should not be considered secure enough to deploy in production
Tags
Mastodon