pi_mcps/docs/wiki/pages/CannaManage-09-Deployment.md

# 09 — Deployment Guide

**Project:** CannaManage — B2B SaaS for German Cannabis Social Clubs  
**Version:** 0.1.0-PLAN  
**Date:** 2026-04-06  
**Target environment:** Hetzner VPS — Ubuntu 22.04 LTS — Docker Compose (Release) | TrueNAS.local — Docker (Build/CI)

---

## 1. Prerequisites

### Hetzner VPS Specification

| Resource | Value | Monthly Cost |
|----------|-------|-------------|
| Server type | CX21 | ~€5.88/month |
| vCPU | 2 | — |
| RAM | 4 GB | — |
| SSD | 40 GB | — |
| Network | 20 TB transfer | — |
| OS | Ubuntu 22.04 LTS | — |

> **Scale-up trigger:** Upgrade to CX31 (8GB RAM) when concurrent active clubs exceeds 20. PostgreSQL is the memory consumer — headroom is consumed by connection pools, not application heap.

### DNS Setup

| Record | Type | Value |
|--------|------|-------|
| `cannamanage.de` | A | `<VPS-IP>` |
| `app.cannamanage.de` | A | `<VPS-IP>` |
| `*.cannamanage.de` | A | `<VPS-IP>` |

Wildcard A record enables future per-club subdomains (`clubname.cannamanage.de`) without additional DNS changes.

### Required Software

- Docker Engine 24+ (`docker.io` or Docker CE)
- Docker Compose v2 (`docker compose` — not `docker-compose`)
- Certbot with Nginx plugin (`python3-certbot-nginx`)
- OpenSSH server (enabled by default on Ubuntu)

---

## 2. Infrastructure Architecture

```mermaid
graph TB
    Dev["👨‍💻 Dev Workstation\n(Fedora, 192.168.188.x)"]
    Gitea["🏠 Gitea\n(truenas.local:30008)"]
    TrueNAS["🖧 TrueNAS.local Docker\n(192.168.188.119)\nBuild + Staging"]
    Hetzner["☁️ Hetzner VPS CX21\nProduction Release"]

    Dev -->|"git push"| Gitea
    Gitea -->|"Gitea Actions runner\n(on TrueNAS.local)"| TrueNAS
    TrueNAS -->|"mvn package + docker build"| TrueNAS
    TrueNAS -->|"docker save | scp\n(on merge to main)"| Hetzner

    subgraph TrueNAS ["TrueNAS.local — CI/CD Build Environment"]
        GiteaRunner["Gitea Actions Runner"]
        BuildCache["Maven .m2 cache\n(persistent volume)"]
        StagingDB["PostgreSQL staging\n(ephemeral)"]
    end

    subgraph Hetzner ["Hetzner VPS — Production Release Environment"]
        Nginx["Nginx (reverse proxy + TLS)"]
        App["cannamanage-app\n(Spring Boot 3.x)"]
        DB["PostgreSQL 16\n(persistent pgdata volume)"]
        Nginx -->|"proxy_pass :8080"| App
        App -->|"JDBC :5432"| DB
    end

    Internet["🌍 Internet HTTPS"] -->|"port 443"| Nginx
```

### Environment Roles

| Environment | Host | Purpose |
|---|---|---|
| **Development** | Dev workstation (Fedora) | Local feature development, unit tests |
| **Build / CI** | TrueNAS.local Docker | Gitea Actions runner; Maven build; integration tests (Testcontainers); Docker image build |
| **Production / Release** | Hetzner VPS CX21 | Live clubs, real data; Hetzner = our release environment |

All three services on Hetzner run on an internal Docker bridge network (`cannamanage_net`). Only Nginx is exposed to the public internet. PostgreSQL has no external port binding.

---

## 3. Docker Compose Setup

**File:** `/opt/cannamanage/docker-compose.yml`

```yaml
version: '3.9'

networks:
  cannamanage_net:
    driver: bridge

volumes:
  pgdata:
    driver: local
  nginx_certs:
    driver: local

services:
  nginx:
    image: nginx:1.25-alpine
    container_name: cannamanage-nginx
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
      - ./nginx/conf.d:/etc/nginx/conf.d:ro
      - nginx_certs:/etc/letsencrypt:ro
      - /var/log/nginx:/var/log/nginx
    depends_on:
      app:
        condition: service_healthy
    networks:
      - cannamanage_net
    restart: unless-stopped

  app:
    image: cannamanage:${VERSION:-latest}
    container_name: cannamanage-app
    environment:
      - SPRING_DATASOURCE_URL=jdbc:postgresql://db:5432/cannamanage
      - SPRING_DATASOURCE_USERNAME=${DB_USERNAME}
      - SPRING_DATASOURCE_PASSWORD=${DB_PASSWORD}
      - APP_JWT_SECRET=${JWT_SECRET}
      - STRIPE_SECRET_KEY=${STRIPE_SECRET_KEY}
      - STRIPE_WEBHOOK_SECRET=${STRIPE_WEBHOOK_SECRET}
      - SPRING_MAIL_HOST=${MAIL_HOST}
      - SPRING_MAIL_USERNAME=${MAIL_USERNAME}
      - SPRING_MAIL_PASSWORD=${MAIL_PASSWORD}
      - SENTRY_DSN=${SENTRY_DSN}
      - SPRING_PROFILES_ACTIVE=production
    depends_on:
      db:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/actuator/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 60s
    networks:
      - cannamanage_net
    restart: unless-stopped

  db:
    image: postgres:16-alpine
    container_name: cannamanage-db
    environment:
      - POSTGRES_DB=cannamanage
      - POSTGRES_USER=${DB_USERNAME}
      - POSTGRES_PASSWORD=${DB_PASSWORD}
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${DB_USERNAME} -d cannamanage"]
      interval: 10s
      timeout: 5s
      retries: 5
    networks:
      - cannamanage_net
    restart: unless-stopped
    # PostgreSQL port intentionally NOT exposed externally
```

**Nginx site config** (`/opt/cannamanage/nginx/conf.d/cannamanage.conf`):

```nginx
server {
    listen 80;
    server_name app.cannamanage.de;
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl http2;
    server_name app.cannamanage.de;

    ssl_certificate /etc/letsencrypt/live/app.cannamanage.de/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/app.cannamanage.de/privkey.pem;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;

    # Security headers
    add_header Strict-Transport-Security "max-age=31536000" always;
    add_header X-Frame-Options DENY;
    add_header X-Content-Type-Options nosniff;

    location / {
        proxy_pass http://app:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_read_timeout 60s;
    }

    # Stripe webhook — allow larger body
    location /api/v1/billing/webhook {
        proxy_pass http://app:8080;
        proxy_set_header Host $host;
        client_max_body_size 1m;
    }
}
```

---

## 4. Environment Variables

**File:** `/opt/cannamanage/.env` (never committed to git — add to `.gitignore`)

```bash
# Database
DB_USERNAME=cannamanage_user
DB_PASSWORD=<strong-random-password-min-32-chars>

# JWT signing key (256-bit minimum — generate with: openssl rand -hex 32)
JWT_SECRET=<256-bit-random-hex>

# Stripe (use sk_live_ for production, sk_test_ for staging)
STRIPE_SECRET_KEY=sk_live_...
STRIPE_WEBHOOK_SECRET=whsec_...

# Email (SMTP)
MAIL_HOST=smtp.example.com
MAIL_PORT=587
MAIL_USERNAME=noreply@cannamanage.de
MAIL_PASSWORD=<mail-password>
MAIL_FROM=CannaManage <noreply@cannamanage.de>

# Error tracking
SENTRY_DSN=https://<key>@<org>.ingest.sentry.io/<project-id>

# Application version (set by CI during deploy)
VERSION=latest
```

> **Security:** Never store `.env` in version control. Use Gitea repository secrets for CI and inject at deploy time. On the VPS, set file permissions: `chmod 600 /opt/cannamanage/.env`.

---

## 5. First-Time Deployment

### Step 1 — Create Hetzner VPS

1. Log into [console.hetzner.cloud](https://console.hetzner.cloud)
2. Create server: **CX21**, Ubuntu 22.04, Nuremberg or Frankfurt datacenter
3. Add your SSH public key during setup (`cat ~/.ssh/id_ed25519.pub`)
4. Note the assigned IPv4 address — update DNS A records

### Step 2 — Install Docker + Docker Compose

```bash
ssh root@<VPS-IP>

# Update system
apt update && apt upgrade -y

# Install Docker
curl -fsSL https://get.docker.com | sh

# Add deploy user (never run production as root)
adduser deploy
usermod -aG docker deploy
usermod -aG sudo deploy

# Install Certbot
apt install -y python3-certbot-nginx certbot
```

### Step 3 — Clone Repository

```bash
su - deploy
mkdir -p /opt/cannamanage
cd /opt/cannamanage
git clone http://192.168.188.119:30008/pplate/cannamanage.git .
# Or from public mirror when available
```

### Step 4 — Create Production `.env`

```bash
cd /opt/cannamanage
cp .env.example .env
nano .env  # Fill in all production secrets
chmod 600 .env
```

### Step 5 — Obtain SSL Certificate

```bash
# Stop anything on port 80 first (nothing should be running yet)
certbot certonly --standalone \
  -d app.cannamanage.de \
  --non-interactive \
  --agree-tos \
  -m ssl@cannamanage.de

# Symlink certs into nginx_certs volume location
# Certbot places certs at /etc/letsencrypt/live/app.cannamanage.de/
```

### Step 6 — Build Docker Image

```bash
# On the VPS (or build locally and push to registry)
./mvnw package -DskipTests -P production
docker build -t cannamanage:latest .
```

### Step 7 — Start Services

```bash
cd /opt/cannamanage
docker compose up -d
```

### Step 8 — Verify Health

```bash
# All containers should be 'healthy' or 'running'
docker compose ps

# Check application logs
docker compose logs -f app --tail=100

# Test health endpoint
curl -f http://localhost:8080/actuator/health
# Expected: {"status":"UP","components":{"db":{"status":"UP"},"diskSpace":{"status":"UP"}}}
```

### Step 9 — Flyway Migrations

Flyway runs automatically on Spring Boot startup. Verify migration log:

```bash
docker compose logs app | grep -i flyway
# Expected: Successfully applied N migrations to schema "public"
```

### Step 10 — Create First Admin User

```bash
# Option A: via REST API (recommended)
curl -X POST https://app.cannamanage.de/api/v1/admin/bootstrap \
  -H "Content-Type: application/json" \
  -d '{
    "adminEmail": "admin@yourclub.de",
    "adminPassword": "<strong-password>",
    "clubName": "Your Club e.V.",
    "clubRegistrationNumber": "VR 12345"
  }'

# The bootstrap endpoint is disabled after first use (one-time setup flag in DB)
```

### Step 11 — Verify Production Access

```bash
# Web UI
open https://app.cannamanage.de

# API health check
curl https://app.cannamanage.de/actuator/health
```

---

## 6. CI/CD Pipeline (Gitea Actions on TrueNAS.local)

The Gitea Actions runner runs **on TrueNAS.local** — this is our homelab build machine. It has Docker, a persistent Maven `.m2` cache volume, and direct SSH access to the Hetzner VPS. Builds happen locally; only the final artifact (Docker image tarball) is shipped to Hetzner.

**File:** `.gitea/workflows/deploy.yml`

```yaml
name: Build and Deploy to Production

on:
  push:
    branches:
      - main

jobs:
  test:
    runs-on: self-hosted   # <-- TrueNAS.local Gitea runner
    steps:
      - uses: actions/checkout@v4

      - name: Set up JDK 21
        uses: actions/setup-java@v4
        with:
          java-version: '21'
          distribution: 'temurin'
          cache: maven

      - name: Run unit tests
        run: ./mvnw test -pl cannamanage-service

      - name: Run integration tests
        run: ./mvnw verify -P integration-tests
        # Testcontainers starts PostgreSQL via Docker on the TrueNAS runner

      - name: Coverage gate check
        run: ./mvnw verify -P coverage-check

  build-and-deploy:
    needs: test
    runs-on: self-hosted   # <-- TrueNAS.local Gitea runner
    steps:
      - uses: actions/checkout@v4

      - name: Build JAR (production profile)
        run: ./mvnw package -DskipTests -P production

      - name: Build Docker image
        run: |
          docker build \
            -t cannamanage:${{ github.sha }} \
            -t cannamanage:latest \
            .

      - name: Save Docker image
        run: docker save cannamanage:${{ github.sha }} | gzip > /tmp/cannamanage.tar.gz

      - name: Copy image to Hetzner VPS
        run: |
          scp -o StrictHostKeyChecking=no \
            /tmp/cannamanage.tar.gz \
            deploy@${{ secrets.HETZNER_IP }}:/tmp/cannamanage.tar.gz

      - name: Deploy via SSH to Hetzner (Production Release)
        run: |
          ssh -o StrictHostKeyChecking=no deploy@${{ secrets.HETZNER_IP }} "
            set -e
            cd /opt/cannamanage

            # Load new image
            docker load < /tmp/cannamanage.tar.gz
            rm /tmp/cannamanage.tar.gz

            # Rolling restart app only (DB stays up)
            VERSION=${{ github.sha }} docker compose up -d app

            # Wait for health
            sleep 10
            docker compose ps app | grep 'healthy' || (docker compose logs app --tail=50 && exit 1)

            # Prune old images (keep last 3 SHAs)
            docker image prune -f
          "

      - name: Cleanup local build artifact
        run: rm -f /tmp/cannamanage.tar.gz
```

### Gitea Actions Runner on TrueNAS.local

The self-hosted runner is a Docker container on TrueNAS.local:

```bash
# On TrueNAS.local — install Gitea Actions runner
docker run -d \
  --name gitea-runner-cannamanage \
  --restart unless-stopped \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /opt/gitea-runner/cannamanage:/data \
  -v /opt/gitea-runner/.m2:/root/.m2 \    # Maven cache persisted across builds
  -e GITEA_INSTANCE_URL=http://192.168.188.119:30008 \
  -e GITEA_RUNNER_REGISTRATION_TOKEN=<token-from-gitea-settings> \
  gitea/act_runner:latest
```

### Required Gitea Repository Secrets

| Secret | Where set | Value |
|--------|-----------|-------|
| `HETZNER_IP` | Gitea repo secrets | Hetzner VPS IPv4 address |
| `SSH_PRIVATE_KEY` | Gitea repo secrets | Private key for `deploy` user on Hetzner |

```bash
# On Hetzner VPS — add TrueNAS runner's public key
# (generate keypair on TrueNAS.local: ssh-keygen -t ed25519 -f ~/.ssh/gitea_runner_deploy)
mkdir -p ~/.ssh && chmod 700 ~/.ssh
echo "<truenas-runner-public-key>" >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
```

---

## 7. Database Backup

### Automated Daily Backup

Add to root crontab (`crontab -e`):

```bash
# Daily backup at 03:00 UTC — keep 14 days
0 3 * * * docker exec cannamanage-db pg_dump \
  -U cannamanage_user \
  --format=custom \
  cannamanage | gzip > /opt/backups/cannamanage_$(date +\%Y\%m\%d).sql.gz

# Cleanup backups older than 14 days
5 3 * * * find /opt/backups -name "cannamanage_*.sql.gz" -mtime +14 -delete
```

Create backup directory:
```bash
mkdir -p /opt/backups
chown deploy:deploy /opt/backups
```

### Restore from Backup

```bash
# Restore (caution: this overwrites existing data)
gunzip -c /opt/backups/cannamanage_20260406.sql.gz | \
  docker exec -i cannamanage-db pg_restore \
    -U cannamanage_user \
    --clean \
    --dbname=cannamanage

# Verify restore
docker exec cannamanage-db psql \
  -U cannamanage_user \
  -d cannamanage \
  -c "SELECT COUNT(*) FROM clubs;"
```

### Offsite Backup (Optional)

For additional redundancy, sync backups to Hetzner Object Storage:

```bash
# Install s3cmd and configure with Hetzner S3-compatible endpoint
s3cmd sync /opt/backups/ s3://cannamanage-backups/
```

---

## 8. Monitoring & Health Checks

### Spring Boot Actuator

The application exposes health endpoints via `spring-boot-actuator`:

```bash
# Full health detail (requires ROLE_ADMIN JWT)
GET /actuator/health

# Example response
{
  "status": "UP",
  "components": {
    "db": { "status": "UP", "details": { "database": "PostgreSQL", "validationQuery": "isValid()" } },
    "diskSpace": { "status": "UP", "details": { "total": 42GB, "free": 30GB } },
    "ping": { "status": "UP" }
  }
}
```

Expose only `health` and `info` publicly in `application-production.yml`:
```yaml
management:
  endpoints:
    web:
      exposure:
        include: health,info
  endpoint:
    health:
      show-details: when-authorized
```

### Log Locations

| Source | Location |
|--------|----------|
| Application logs | `docker compose logs -f app` |
| Nginx access logs | `/var/log/nginx/access.log` |
| Nginx error logs | `/var/log/nginx/error.log` |
| PostgreSQL logs | `docker compose logs db` |
| Sentry (errors) | `https://sentry.io/organizations/<org>/` |

### Alerting

Configure Sentry to email on new errors:
1. Set `SENTRY_DSN` in `.env`
2. Add `io.sentry:sentry-spring-boot-starter-jakarta:7.x` to POM
3. Sentry auto-captures all unhandled exceptions with full stack trace

Simple uptime check via `cron` + email:

```bash
# Health check every 5 minutes — email on 3 consecutive failures
*/5 * * * * /opt/cannamanage/scripts/health_check.sh
```

```bash
#!/bin/bash
# /opt/cannamanage/scripts/health_check.sh
HEALTH_URL="https://app.cannamanage.de/actuator/health"
FAIL_COUNT_FILE="/tmp/cannamanage_health_fails"

HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" "$HEALTH_URL")
if [ "$HTTP_STATUS" != "200" ]; then
    FAILS=$(cat "$FAIL_COUNT_FILE" 2>/dev/null || echo 0)
    FAILS=$((FAILS + 1))
    echo "$FAILS" > "$FAIL_COUNT_FILE"
    if [ "$FAILS" -ge 3 ]; then
        echo "CannaManage health check failed $FAILS times" | \
          mail -s "ALERT: CannaManage DOWN" admin@cannamanage.de
    fi
else
    echo 0 > "$FAIL_COUNT_FILE"
fi
```

---

## 9. SSL Certificate Renewal

Let's Encrypt certificates expire after 90 days. Certbot handles renewal automatically:

```bash
# Test renewal (dry run — no actual renewal)
certbot renew --dry-run

# Manual renewal
certbot renew --nginx

# Reload Nginx after renewal
docker exec cannamanage-nginx nginx -s reload
```

### Auto-Renewal via Cron

```bash
# Renew at 02:00 UTC on the 1st and 15th of each month
0 2 1,15 * * certbot renew --quiet --nginx && \
  docker exec cannamanage-nginx nginx -s reload
```

Certbot only renews when the certificate is less than 30 days from expiry — safe to run frequently.

---

## 10. Rollback Procedure

If a deployment causes issues:

```bash
# On VPS — list recent images
docker images cannamanage --format "table {{.Tag}}\t{{.CreatedAt}}"

# Roll back to previous SHA
cd /opt/cannamanage
VERSION=<previous-sha> docker compose up -d app

# Verify health after rollback
docker compose ps app
curl https://app.cannamanage.de/actuator/health
```

If database migrations were applied and rollback is needed:
1. Restore from last backup (see Section 7)
2. Redeploy the previous image version
3. Flyway baseline the schema at previous version

> **Note:** Flyway migrations are append-only and forward-only. Design migrations to be reversible where possible (add columns before removing old ones, etc.).