vps/vps1-state-25012026.md

# VPS1 State - 2026-01-25

**Host:** vps1.phiiiil.de (152.53.119.222)
**Date:** 2026-01-25
**Status:** ✅ Production Ready

## Executive Summary

VPS1 is a secure Docker-based hosting platform running on Debian 12 (ARM64) with 1TB storage and 15GB RAM. The system hosts three main services: Netbird VPN management, Immich photo management (37GB of photos), and Gitea Git hosting.

**Key Achievements:**
- ✅ All services running and healthy (12 containers)
- ✅ Security hardened (UFW firewall + Fail2ban)
- ✅ Complete automated daily backups (databases + 39GB photo library)
- ✅ SSL certificates for all domains via Caddy
- ✅ Single sign-on ready via Netbird Zitadel
- ✅ rsync installed for efficient backups

## Services

### Netbird VPN Management
**URL:** https://nb.phiiiil.de
**Purpose:** VPN management, user authentication, reverse proxy

**Components (8 containers):**
- `netbird-caddy-1` - Reverse proxy (ports 80, 443)
- `netbird-dashboard-1` - Web UI
- `netbird-management-1` - Management API
- `netbird-zitadel-1` - Identity provider (OIDC/SSO)
- `netbird-zdb-1` - Zitadel PostgreSQL database
- `netbird-signal-1` - Signal service
- `netbird-relay-1` - Relay service
- `netbird-coturn-1` - TURN/STUN relay (host networking)

**Features:**
- Centralized user management
- OAuth2/OIDC authentication
- Automatic SSL certificate management via Caddy
- Routes for all services

### Immich Photo Management
**URL:** https://immich.phiiiil.de
**Purpose:** Photo backup, management, and AI-powered features

**Components (3 containers):**
- `immich_server` - Main application (port 2283)
- `immich_postgres` - PostgreSQL database with vector extension
- `immich_redis` - Caching layer
- `immich_machine_learning` - ML/AI features

**Data:**
- 34,694 files (photos, videos, thumbnails) stored in `/home/phil/docker/immich-app/library/`
- 39.34GB total library size
- PostgreSQL database with indexed metadata

**Backup:** ✅ Fully automated daily backup at 04:00
- Photo library: rsync to `/mnt/backup/immich-photos/` (39GB, 30-day retention)
- Database: PostgreSQL dump to `/mnt/backup/latest/` (52MB, 30-day retention)
- Both backups include detailed logging and restore procedures

### Gitea Git Hosting
**URL:** https://git.phiiiil.de
**Purpose:** Private Git hosting with integrated SSO

**Components (2 containers):**
- `gitea` - Git application (port 3000, SSH on 2222)
- `gitea-db` - PostgreSQL 16 database

**Configuration:**
- **Admin:** User `phil` with secure password
- **Registration:** Disabled (private instance)
- **SSH:** Port 2222
- **Database:** PostgreSQL 16 with strong password
- **OIDC:** Enabled (ready for Netbird integration)

## Security Configuration

### Firewall (UFW)
**Status:** ✅ Active

**Allowed Inbound Ports:**
- 22/tcp (SSH)
- 80/tcp (HTTP → Caddy)
- 443/tcp (HTTPS → Caddy)
- 443/udp (HTTP3/QUIC → Caddy)
- 2283/tcp (Immich direct access)
- 2222/tcp (Gitea SSH)

**Policy:** Deny all other incoming traffic

### Fail2ban
**Status:** ✅ Active protecting SSH

**Configuration:**
- Ban time: 1 hour
- Max retries: 3
- Find time: 10 minutes
- Currently banned: 5 IPs (SSH brute force attempts)

### Security Headers (Caddy)
All domains have:
- HSTS (3600s, includeSubDomains, preload)
- X-Content-Type-Options: nosniff
- X-Frame-Options: SAMEORIGIN
- X-XSS-Protection: 1; mode=block
- Referrer-Policy: strict-origin-when-cross-origin

## Network & DNS

### DNS Configuration
**All domains correctly pointing to 152.53.119.222:**
- ✅ nb.phiiiil.de → 152.53.119.222
- ✅ git.phiiiil.de → 152.53.119.222
- ✅ immich.phiiiil.de → 152.53.119.222

### SSL Certificates
**All domains have valid Let's Encrypt certificates:**
- ✅ nb.phiiiil.de
- ✅ git.phiiiil.de
- ✅ immich.phiiiil.de

Certificates obtained and renewed automatically by Caddy.

### Docker Networks
- `netbird_netbird` - Shared network for all services (Netbird, Gitea, Immich, Caddy)
- `immich_default` - Immich internal network
- `bridge` - Default Docker network
- `host` - For Coturn TURN server

## Backup Strategy

### Automated Backups

**Schedule 1: Database Backup (02:00)**
- **Script:** `/home/phil/docker/backup/backup.sh`
- **Location:** `/mnt/backup/`
- **Retention:** 30 days
- **Log:** `/var/log/vps-backup.log`

**What Gets Backed Up:**
1. Immich PostgreSQL database (52MB)
2. Immich library manifest (file count, size)
3. Netbird management data (31MB)
4. Netbird Zitadel database (77KB)
5. Gitea PostgreSQL database (24KB)
6. Gitea data volume (9.4KB)

**Latest backup symlink:** `/mnt/backup/latest`

---

**Schedule 2: Immich Photo Backup (04:00)** ✨ NEW
- **Script:** `/home/phil/docker/backup/backup-immich-photos.sh`
- **Location:** `/mnt/backup/immich-photos/`
- **Retention:** 30 days
- **Log:** `/var/log/immich-photo-backup.log`

**What Gets Backed Up:**
- 34,694 files (photos, videos, thumbnails, encoded videos)
- 39.34GB total library size
- Incremental backup using rsync with hardlinking
- Space-efficient (unchanged files use hardlinks)
- Automatic cleanup of backups older than 30 days

**Latest backup symlink:** `/mnt/backup/immich-photos/latest`

### Backup Performance

**Immich Photo Backup:**
- First backup: ~4 minutes (164 MB/s)
- Subsequent backups: ~1-2 minutes (incremental)
- Storage overhead: Minimal (hardlinks for unchanged files)

### Storage Requirements

**Current Usage:**
- Photo backups: ~39GB (with hardlinks, additional days minimal)
- Database backups: ~2.5GB (30 days × 83MB)
- Total: ~42GB for 30-day retention

**Available Space:** 863GB free (91% of 1TB)

## File Structure

```
/home/phil/docker/
├── netbird/              # Netbird VPN + Caddy reverse proxy
│   ├── docker-compose.yml
│   ├── Caddyfile
│   ├── dashboard.env
│   ├── management.json
│   ├── relay.env
│   ├── turnserver.conf
│   ├── zitadel.env
│   ├── zdb.env
│   └── machinekey/
├── immich-app/           # Immich Photo Management
│   ├── docker-compose.yml
│   ├── .env
│   ├── library/          # 39GB of photos (34,694 files)
│   └── postgres/         # Database files
├── gitea/                # Git Hosting
│   ├── docker-compose.yml
│   └── .env              # Contains DB password
└── backup/               # Backup scripts
    ├── backup.sh                 # Database backup (02:00)
    └── backup-immich-photos.sh   # Photo backup (04:00)

/mnt/backup/              # Backup storage mount point
├── latest/               # Symlink to latest database backup
└── immich-photos/        # Photo backups
    └── latest/           # Symlink to latest photo backup
```

## Container Status

| Container | Status | Health | Purpose |
|-----------|--------|--------|---------|
| gitea | ✅ Running | Healthy | Git hosting |
| gitea-db | ✅ Running | Healthy | Gitea database |
| immich_server | ✅ Running | Healthy | Photo management |
| immich_postgres | ✅ Running | Healthy | Immich database |
| immich_redis | ✅ Running | Healthy | Immich cache |
| immich_machine_learning | ✅ Running | Healthy | Immich ML/AI |
| netbird-caddy-1 | ✅ Running | - | Reverse proxy |
| netbird-dashboard-1 | ✅ Running | - | Netbird UI |
| netbird-management-1 | ✅ Running | - | Netbird API |
| netbird-zitadel-1 | ✅ Running | - | Identity provider |
| netbird-zdb-1 | ✅ Running | Healthy | Zitadel DB |
| netbird-signal-1 | ✅ Running | - | Signal service |
| netbird-relay-1 | ✅ Running | - | Relay service |
| netbird-coturn-1 | ✅ Running | - | TURN/STUN relay |

**Total:** 12 containers
**Healthy:** 6
**All running:** Yes

## System Resources

**Disk Usage:**
- Total: 1TB (1006GB)
- Used: 92GB (10%)
- Available: 863GB
- Change: +37GB (photo backup)

**RAM:**
- Total: 15GB
- Used: ~2GB
- Available: ~13GB

**OS:**
- Distribution: Debian GNU/Linux 12 (bookworm)
- Kernel: Linux 6.1.0-40-arm64
- Architecture: ARM64
- rsync: ✅ Installed

## Access Credentials

### SSH
**Host:** vps1.phiiiil.de (152.53.119.222)
**User:** phil
**Key:** ~/.ssh/id_rsa (RSA)
**Config:** `~/.ssh/config` (Host vps1)

### Gitea
**URL:** https://git.phiiiil.de
**Admin User:** `phil`
**Password:** `j8bKvIl3AtIp5aTG`
**SSH:** git@152.53.119.222:2222

### Netbird
**URL:** https://nb.phiiiil.de
**Admin:** Configured via Netbird dashboard

### Immich
**URL:** https://immich.phiiiil.de
**Admin:** Created during initial setup

## Maintenance Commands

### Daily Tasks
```bash
# Check all containers
docker ps

# Check system resources
df -h
free -h

# Check database backup log
tail -f /var/log/vps-backup.log

# Check photo backup log
tail -f /var/log/immich-photo-backup.log

# Verify photo backup completed
ls -lah /mnt/backup/immich-photos/latest/
cat /mnt/backup/immich-photos/latest/backup-info.txt
```

### Weekly Tasks
```bash
# Check latest database backup
ls -lah /mnt/backup/latest/

# Check latest photo backup
ls -lah /mnt/backup/immich-photos/latest/

# Review fail2ban bans
sudo fail2ban-client status sshd

# Check container resource usage
docker stats
```

### Monthly Tasks
```bash
# Update container images
cd /home/phil/docker/gitea && docker compose pull && docker compose up -d
cd /home/phil/docker/netbird && docker compose pull && docker compose up -d
cd /home/phil/docker/immich-app && docker compose pull && docker compose up -d

# Security updates
sudo apt update && sudo apt upgrade

# Review banned IPs
sudo fail2ban-client status sshd
```

### Manual Backup
```bash
# Run database backup manually
/home/phil/docker/backup/backup.sh

# Run photo backup manually
/home/phil/docker/backup/backup-immich-photos.sh

# Check photo backup info
cat /mnt/backup/immich-photos/latest/backup-info.txt
```

## Service Management

### Start/Stop Services
```bash
# Netbird (includes Caddy)
cd /home/phil/docker/netbird
docker compose up -d    # Start
docker compose down    # Stop

# Gitea
cd /home/phil/docker/gitea
docker compose up -d
docker compose down

# Immich
cd /home/phil/docker/immich-app
docker compose up -d
docker compose down
```

### View Logs
```bash
# Caddy (reverse proxy)
docker logs -f netbird-caddy-1

# Netbird
docker logs -f netbird-management-1

# Gitea
docker logs -f gitea

# Immich
docker logs -f immich_server

# Database backups
tail -f /var/log/vps-backup.log

# Photo backups
tail -f /var/log/immich-photo-backup.log
```

## Disaster Recovery

### Restore Immich Photos
```bash
# Stop Immich first
cd /home/phil/docker/immich-app
docker compose stop

# Restore all photos from latest backup
rsync -av /mnt/backup/immich-photos/latest/ /home/phil/docker/immich-app/library/

# Or restore specific date
rsync -av /mnt/backup/immich-photos/20260125_160319/ /home/phil/docker/immich-app/library/

# Restart Immich
docker compose start
```

### Restore Immich Database
```bash
# From latest backup
gunzip < /mnt/backup/latest/immich-db.sql.gz | docker exec -i immich_postgres psql -U postgres immich
```

### Restore Gitea
```bash
# Database
gunzip < /mnt/backup/latest/gitea-db.sql.gz | docker exec -i gitea-db psql -U gitea gitea

# Data volume
docker run --rm -v gitea_gitea_data:/data -v /mnt/backup/latest:/backup alpine tar xzf /backup/gitea-data.tar.gz -C /data
```

### Restore Netbird
```bash
# Management data
docker run --rm -v netbird_netbird_management:/data -v /mnt/backup/latest:/backup alpine tar xzf /backup/netbird-management.tar.gz -C /data
```

## Pending Tasks

### 1. Remote Backup Storage ⚠️
**Action:** Mount remote storage to `/mnt/backup`

**Options:**
- S3 bucket via s3fs
- NFS mount
- SSHFS
- WebDAV

**Benefit:** Off-site backup for disaster recovery

### 2. OIDC SSO Integration ⏳
**Optional:** Integrate Gitea and Immich with Netbird Zitadel for SSO

**See:**
- `docs/sso-integration-plan.md` - Complete implementation guide
- `docs/oidc-integration-guide.md` - Original setup guide

**Benefits:**
- Unified authentication across all services
- Centralized user management
- Single sign-on

## Important Notes

### BACKUP STATUS ✅
- ✅ **Immich Photos:** Automated daily backup (04:00, 39GB, 30-day retention)
- ✅ **Immich Database:** Automated daily backup (02:00, 52MB)
- ✅ **Gitea:** Automated daily backup (02:00, database + data)
- ✅ **Netbird:** Automated daily backup (02:00, management + database)

### DO NOT DELETE
- `/home/phil/docker/immich-app/library/` - Contains 39GB of irreplaceable photos (now backed up)
- `/home/phil/docker/immich-app/postgres/` - Immich database with metadata
- Docker volumes: `gitea_gitea_*`, `immich_*`, `netbird_netbird_*`

### Security Reminders
- All SSH access via key authentication only
- UFW firewall blocks all non-essential ports
- Fail2ban actively blocks brute force attempts
- Daily backups of all databases (30-day retention)
- SSL/TLS enforced for all services

### System Upgrades
When upgrading containers:
1. Always pull images first: `docker compose pull`
2. Then restart: `docker compose up -d`
3. Check logs: `docker logs <container>`
4. Have backup ready before major upgrades

## Contact & Support

**System Admin:** phil@phiiiil.de
**Documentation Location:** `/home/phil/Schreibtisch/Privat/vps/docs/`

**Documentation Files:**
- `vps1-state-25012026.md` (this file) - Current system state
- `vps1-startpoint.md` - Initial analysis and architecture
- `vps1-todo.md` - Action items and maintenance tasks
- `docs/deployment-summary.md` - Deployment details
- `docs/oidc-integration-guide.md` - SSO setup guide
- `docs/sso-integration-plan.md` - Complete SSO implementation plan
- `docs/sso-integration-diagram.txt` - SSO architecture diagrams
- `docs/immich-backup-implementation.md` - Photo backup implementation

## Changelog

### 2026-01-25 (Later)
- ✅ **Immich photo backup implemented** (34,694 files, 39GB)
- ✅ Automated daily backup at 04:00 using rsync
- ✅ 30-day retention with hardlinking for space efficiency
- ✅ rsync installed for backup operations
- ✅ Complete backup coverage (databases + photos)
- ✅ SSO integration plan created (ready to implement)

### 2026-01-25 (Earlier)
- ✅ Security hardening (UFW + Fail2ban)
- ✅ Gitea deployment with PostgreSQL
- ✅ Automated database backup system implemented
- ✅ DNS configured for all domains
- ✅ SSL certificates obtained via Caddy
- ✅ Nextcloud removed (FastCGI configuration issues)
- ✅ System production ready

---

**Last Updated:** 2026-01-25
**System Status:** ✅ All services operational