Claude_Homelab/114_koillection_deployment.md

509 lines
20 KiB
Markdown
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# 114 — Koillection Deployment Guide
> Status: **IN PROGRESS** — CT created 2026-08-11, app + nginx + SSL + DNS live, Authelia bypassed by decision, admin login done, image upload defect found + fixed. Remaining: create the 5 real collections (Phase 11).
> CT ID: 114 · IP: 192.168.1.114
> Domain: `collections.spendlik.sk`
> Last updated: 2026-08-11
---
## Overview
Koillection is a self-hosted collection manager for tracking physical collections of any kind. No pre-built metadata scrapers — metadata is added freely per item, with custom fields via templates. MIT licensed.
**Collections planned for this instance:**
- 🚗 Hot Wheels (series, year, colour, condition, variants)
- 🧱 LEGO (set number, theme, piece count, minifigures, completion status)
- 🦇 Batmobiles (source media, scale, manufacturer, condition)
- 📚 Comics (title, issue, publisher, language, condition)
- 📄 Paper Models (designer, scale, subject, format — physical printed copies)
Stack: `koillection/koillection` (PHP/Symfony + Vue.js, served via FrankenPHP/Caddy) + PostgreSQL 16. Uploads (item photos) bind-mounted to NAS for data safety.
---
## Resource Allocation
| Resource | Allocation |
|---|---|
| **CT ID** | 114 |
| **IP** | 192.168.1.114 |
| **CPUs** | 1 |
| **RAM** | 512 MB |
| **Disk** | 8 GB (app + DB only; photos on NAS) |
| **Template** | Debian 13 (trixie) — created from `13.1-2` (current template at time of creation; `13.6-1` is now the default for new deployments, but this doesn't matter post-creation — see `Proxmox LXC Templates.md`) |
| **Privileged** | Yes (Docker requires it) |
| **Nesting** | Enabled (`features: nesting=1`) |
---
## NAS Mount Planning
Photos uploaded to Koillection land in `/uploads` inside the container. This will be bind-mounted from the NAS at:
```
/volume1/proxmox/data/koillection/uploads
```
Create this directory on the NAS before deployment:
```bash
# On the Synology NAS (SSH or File Station)
mkdir -p /volume1/proxmox/data/koillection/uploads
```
> ⚠️ The NAS path follows the same convention as Paperless (`/volume1/proxmox/data/<service>`). Be consistent.
>
> **Host-side mount**: verified 2026-08-11 — the Proxmox storage ID is `spendlik-nas`, mounted at `/mnt/pve/spendlik-nas` on the host (confirmed via live `ls /mnt/pve/`). CT 111 (Paperless) uses `/mnt/pve/spendlik-nas/data/paperless` as its exact host-side bind-mount path — Koillection follows the identical pattern in Phase 4 below.
>
> ⚠️ **Also create a `.phptmp` subfolder under `/uploads` at the same time** — required by the image-upload fix documented after Phase 5. Doing this now, during initial NAS prep, avoids a second round-trip.
---
## Phase 1 — Create LXC Container ✅ DONE (2026-08-11)
```bash
pct create 114 local:vztmpl/debian-13-standard_13.1-2_amd64.tar.zst \
--hostname koillection \
--cores 1 \
--memory 512 \
--swap 512 \
--rootfs local-lvm:8 \
--net0 name=eth0,bridge=vmbr0,ip=192.168.1.114/24,gw=192.168.1.1 \
--unprivileged 0 \
--features nesting=1 \
--ostype debian \
--start 1
```
Enter the container:
```bash
pct enter 114
```
---
## Phase 2 — Base Setup ✅ DONE (2026-08-11)
```bash
apt update && apt upgrade -y
apt install -y nano curl ca-certificates gnupg lsb-release
```
---
## Phase 3 — Install Docker ✅ DONE (2026-08-11)
```bash
install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg
chmod a+r /etc/apt/keyrings/docker.gpg
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
https://download.docker.com/linux/debian \
$(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null
apt update
apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin
```
Verify:
```bash
docker run --rm hello-world
```
---
## Phase 4 — NAS Bind Mount ✅ DONE (2026-08-11)
Add the NAS uploads path as a Proxmox bind mount. Exit the container first:
```bash
exit
```
On the Proxmox host:
```bash
pct set 114 --mp0 /mnt/pve/spendlik-nas/data/koillection/uploads,mp=/uploads,shared=1
```
> ✅ Verified 2026-08-11 against live `ls /mnt/pve/` (only `spendlik-nas` present) and cross-checked against CT 111 (Paperless)'s actual documented host mount (`/mnt/pve/spendlik-nas/data/paperless`) — this replaces an earlier placeholder path in this guide that would have failed (wrong storage name).
Re-enter the container and verify the mount is visible:
```bash
pct enter 114
ls -la /uploads
```
Confirmed: `/uploads` mounted, `777 nobody:nogroup`, empty — ready for the app to write to.
**Also create the PHP temp-file directory here** (needed for the fix in Phase 5 below — for a fresh deployment following this guide top-to-bottom, do this now rather than discovering it's missing later):
```bash
mkdir -p /uploads/.phptmp
chmod 1777 /uploads/.phptmp
```
---
## Phase 5 — Deploy Koillection ✅ DONE (2026-08-11, includes image-upload fix)
```bash
mkdir -p /opt/koillection
cd /opt/koillection
nano .env
```
Paste (fill in a strong password for `DB_PASSWORD`):
```env
DB_DRIVER=pdo_pgsql
DB_NAME=koillection
DB_HOST=db
DB_PORT=5432
DB_USER=koillection
DB_PASSWORD=CHANGE_ME
DB_VERSION=16
APP_ENV=prod
APP_DEBUG=0
APP_SECRET=CHANGE_ME_32CHAR_RANDOM_STRING
PHP_TZ=Europe/Bratislava
HTTPS_ENABLED=0
```
> Generate `APP_SECRET` with: `openssl rand -hex 16`
Create a PHP ini override for the temp directory (belt-and-suspenders alongside the entrypoint fix below — some code paths do respect this, even though the specific bug fixed here does not):
```bash
cat > /opt/koillection/upload-tmp.ini << 'EOF'
upload_tmp_dir = /uploads/.phptmp
sys_temp_dir = /uploads/.phptmp
EOF
```
```bash
nano docker-compose.yml
```
Paste **this exact version, which includes the image-upload fix baked in from the start** (see "Known Issue" section below for why):
```yaml
services:
koillection:
image: koillection/koillection:latest
container_name: koillection
restart: unless-stopped
entrypoint: ["sh", "-c", "rm -rf /tmp && ln -s /uploads/.phptmp /tmp && exec sh /app/public/docker/entrypoint.sh"]
ports:
- "8080:80"
env_file:
- .env
volumes:
- /uploads:/uploads
- ./upload-tmp.ini:/usr/local/etc/php/conf.d/zz-upload-tmp.ini:ro
depends_on:
db:
condition: service_healthy
db:
image: postgres:16
container_name: koillection-db
restart: unless-stopped
env_file:
- .env
environment:
- POSTGRES_DB=${DB_NAME}
- POSTGRES_USER=${DB_USER}
- POSTGRES_PASSWORD=${DB_PASSWORD}
volumes:
- ./volumes/postgresql:/var/lib/postgresql/data
healthcheck:
test: ["CMD", "pg_isready", "-q", "-d", "koillection", "-U", "koillection"]
timeout: 45s
interval: 10s
retries: 10
```
Start:
```bash
docker compose up -d
docker compose logs -f
```
Wait until the koillection container logs settle (Symfony/FrankenPHP startup — you'll see deprecation.INFO notices about API Platform `#[ApiResource]` shortName deduplication and symfony/form; these are harmless upstream framework warnings on this image version, not errors). Then verify with a proper status check, **not** a text grep against the homepage (the homepage is just a redirect, so grepping for "koillection" in it will always come back empty and looks like a false failure):
```bash
docker compose ps
curl -sv http://localhost:8080
```
Expect both containers `healthy`, and the curl to show `HTTP/1.1 302 Found` with `Location: /first-connection` — that's Koillection's normal first-run redirect, confirming the app is up and reachable.
Also verify the entrypoint fix actually took effect:
```bash
docker compose exec koillection ls -la / | grep tmp
```
Should show `tmp -> /uploads/.phptmp` as a symlink.
> `chown: Invalid argument` lines for `/uploads` (and now `/uploads/.phptmp`) in the startup logs are expected and harmless — same NAS-bind-mount ownership limitation already known from Paperless. Doesn't affect functionality.
---
## 🐛 Known Issue: Image/Photo Uploads Fail with "rename(): Invalid argument"
**Symptom**: Uploading any image (profile picture, collection photo, item photo) fails with a generic "critical error" in the UI. Everything else — creating collections, text fields, login — works fine.
**Root cause**: Koillection's image-upload handler (used for profile pictures and collection/item photos) writes its temp file to a **hardcoded `/tmp` path**, then calls PHP's `rename()` to move it into `/uploads`. `/tmp` lives on the container's local overlay filesystem, while `/uploads` is NFS-mounted from the NAS — two different filesystems. Linux's `rename()` syscall cannot move a file across filesystem boundaries (this is `EXDEV` normally, but surfaces here as a generic "Invalid argument" through PHP/Symfony's wrapper). This is **not** a permissions or NAS-connectivity issue — the mount itself is healthy throughout.
**What didn't work** (documented so this isn't re-attempted blind on a future upgrade):
- Setting `TMPDIR` env var — Koillection's upload handler doesn't consult it (hardcoded path, not `sys_get_temp_dir()`)
- Setting `upload_tmp_dir`/`sys_temp_dir` in `php.ini` — confirmed via `php -i` that the values loaded correctly, but the specific code path still ignored them
- Bind-mounting a second NFS path directly onto `/tmp` (`volumes: - /uploads/.tmp:/tmp`) — `stat` showed matching device IDs, but `rename()` still failed with the same `Invalid argument` error. Two separate mounts of the same NFS export are not treated as one filesystem by `rename()` on this NAS, even though they report identical device numbers.
**What actually works**: replace `/tmp` with a **symlink** into a folder inside the already-mounted `/uploads` NFS export, so there's only ever one real mount involved and no ambiguity for `rename()`:
```bash
rm -rf /tmp && ln -s /uploads/.phptmp /tmp
```
This is baked into the `entrypoint:` override in Phase 5's `docker-compose.yml` above, so it runs fresh on every container start/recreate — it does **not** persist through a plain `docker compose restart` alone if done manually outside the entrypoint, which is exactly why it needed to be wrapped into the entrypoint rather than run once by hand.
**Verified working** 2026-08-11: created a test collection with an uploaded photo, then deleted it — full round trip succeeded with no errors.
---
## Phase 6 — nginx Reverse Proxy (CT 101) ✅ DONE (2026-08-11)
Enter CT 101 — this must be done **on CT 101**, not CT 114. nginx does not and should not exist on CT 114 itself.
```bash
pct enter 101
nano /etc/nginx/sites-available/collections.spendlik.sk
```
> ✅ Verified 2026-08-11 against live `ls /etc/nginx/sites-available/` on CT 101 — every existing vhost is named by full domain (`paperless.spendlik.sk`, `jellyfin.spendlik.sk`, `vault.spendlik.sk`, etc.), not by short service name. Use `collections.spendlik.sk` as the filename here, not `koillection`.
Paste:
```nginx
server {
listen 80;
server_name collections.spendlik.sk;
location / {
proxy_pass http://192.168.1.114:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
client_max_body_size 20M;
}
}
```
> `client_max_body_size 20M` — item photos can be large. Adjust upward if needed.
Enable and reload:
```bash
ln -s /etc/nginx/sites-available/collections.spendlik.sk /etc/nginx/sites-enabled/
nginx -t && systemctl reload nginx
```
Confirmed: `nginx -t` → syntax ok, config test successful, reload applied cleanly.
---
## Phase 7 — SSL Certificate ✅ DONE (2026-08-11)
Still in CT 101:
```bash
certbot --nginx -d collections.spendlik.sk
```
Certificate issued successfully, expires 2026-11-09, auto-renewal scheduled by certbot.
Config inspected after issuance — **no corruption this time**: two `server_name collections.spendlik.sk;` lines are expected (one in the port-80 redirect block, one in the port-443 SSL block, matching the verified-good structure already used by CT 111 Paperless). Brace counts balance correctly in both blocks.
Also set `HTTPS_ENABLED=1` in `/opt/koillection/.env` in CT 114, then restart:
```bash
# In CT 114
cd /opt/koillection
nano .env # set HTTPS_ENABLED=1
docker compose restart koillection
```
> ⚠️ Required — without this, Koillection generates internal links as `http://`, which combined with the nginx 80→443 redirect causes a redirect loop.
---
## Phase 8 — DNS Record ✅ DONE (2026-08-11)
> **Updated 2026-08-10**: All `*.spendlik.sk` subdomains are now **CNAME** records pointing at the root `spendlik.sk`. Only the root `spendlik.sk` A record holds an IP — WebSupport rejects any duplicate IP value elsewhere in the zone. Do **not** create an A record for this subdomain.
CNAME record created: `collections.spendlik.sk``spendlik.sk`, TTL 600, **record ID `340219244`**. Added to `00_index.md` DNS table.
Verified resolving via `nslookup collections.spendlik.sk` on CT 101 (canonical name → `spendlik.sk``95.102.127.184`).
> ✅ No DDNS updater step needed for this subdomain. `ddns-update.sh` on CT 108 only updates the root A record on IP change; this CNAME resolves through automatically.
>
> ⚠️ **Note for future deployments**: certbot's HTTP-01 challenge (Phase 7) needs DNS to already resolve publicly — do DNS *before* attempting SSL if it hasn't propagated yet. In this deployment, DNS (Phase 8) was done before Phase 7 for exactly this reason, even though the guide lists them in this numeric order for documentation clarity.
---
## Phase 9 — Authelia Protection (CT 102) — ❌ SKIPPED by decision (2026-08-11)
**Decision: bypass Authelia, rely on Koillection's own login only.** Personal single-user instance — Koillection's built-in authentication is sufficient, and no Authelia middleware was ever added to the CT 101 nginx vhost (Phase 6), so there is nothing to add or remove. No `/etc/authelia/configuration.yml` changes were made for this domain.
<details>
<summary>Reference: steps to add Authelia later if this decision changes</summary>
Enter CT 102, edit `/etc/authelia/configuration.yml`. Add to `access_control.rules`:
```yaml
- domain: collections.spendlik.sk
policy: two_factor
```
Restart Authelia after editing:
```bash
docker compose restart
```
Add the Authelia middleware to the nginx vhost in CT 101 (follow the pattern from other protected services).
</details>
---
## Phase 10 — First Login & Initial Setup ✅ DONE (2026-08-11)
Opened `https://collections.spendlik.sk` from mobile data (hairpin NAT — never test from LAN). Admin account created successfully at `/first-connection`.
Recommended, still worth confirming in profile settings if not already done:
1. Set your timezone to `Europe/Bratislava` in profile settings
2. Set the display currency if tracking purchase values
3. Set visibility defaults (private by default is fine for a personal instance)
---
## Phase 11 — Collection Setup
**Smoke test passed 2026-08-11**: created a "Test" collection with an uploaded photo, confirmed it worked end-to-end (this is what caught and validated the fix for the image-upload defect above), then deleted it. The actual 5 planned collections below are not yet created.
Recommended collection structure. Create each as a top-level Collection:
### 🚗 Hot Wheels
Suggested item fields (via Template):
- Series / Line
- Year of release
- Colour
- Casting name
- Country of manufacture
- Condition (Mint / Good / Played)
- Treasure Hunt (yes/no)
### 🧱 LEGO
Suggested item fields:
- Set number
- Theme
- Sub-theme
- Piece count
- Minifigure count
- Year
- Completion status (Sealed / Built / Parts only)
- Instruction booklet present (yes/no)
> 💡 The set number field makes cross-referencing with kocka-novinky.sk and Brickset API straightforward.
### 🦇 Batmobiles
Suggested item fields:
- Source (Film / TV / Comics / Game)
- Year of appearance
- Manufacturer (Hot Wheels / Corgi / LEGO / custom)
- Scale
- Condition
### 📚 Comics
Suggested item fields:
- Title / Series
- Issue number
- Publisher
- Language
- Year
- Condition (Mint / Very Good / Good / Fair)
- Story arc
### 📄 Paper Models (physical hardcopy)
Suggested item fields:
- Designer / Publisher
- Subject (aircraft, ship, building…)
- Scale
- Format (magazine supplement / standalone / kit)
- Build status (Unbuilt / Built / Display)
> 💡 Tags are cross-collection in Koillection — tag items with `#display`, `#wishlist`, `#for-sale` etc. to group across all five collections at once.
---
## Backup
The only things that need backing up:
1. **PostgreSQL database** — contains all collection metadata
2. **NAS uploads directory** — contains all item photos (already on NAS, covered by NAS backup)
Add a daily DB dump to cron in CT 114:
```bash
crontab -e
```
Add:
```cron
0 3 * * * docker exec koillection-db pg_dump -U koillection koillection > /opt/koillection/backups/koillection-$(date +\%Y\%m\%d).sql 2>/dev/null
```
```bash
mkdir -p /opt/koillection/backups
```
> ⚠️ Always back up the database before upgrading Koillection — the developer notes that data migrations can occasionally have edge cases.
>
> ⚠️ **Note re: image-upload fix above** — if Koillection is ever upgraded to a new image version, re-verify the `entrypoint:` symlink workaround is still needed (a future upstream release may fix the hardcoded `/tmp` path) and re-test an image upload after any upgrade, before assuming it still works.
---
## Gotchas
| Issue | Fix |
|---|---|
| Photos not saving | Verify `/uploads` bind mount is writable inside the container |
| App not starting | Check `docker compose logs koillection` — usually a DB connection issue on first boot |
| certbot corrupts nginx config | Always inspect after issuance |
| Large photo uploads rejected | Increase `client_max_body_size` in nginx vhost |
| HTTPS redirect loop | Set `HTTPS_ENABLED=1` in `.env` and restart the koillection container after SSL is in place |
| DNS record type | Use CNAME → `spendlik.sk`, never a per-subdomain A record (see Phase 8) |
| Wrong template filename | Verify exact template string with `pveam list local` before `pct create` — versions bump periodically |
| Wrong NAS host mount path | Storage ID is `spendlik-nas`, mounted at `/mnt/pve/spendlik-nas` — verify with `ls /mnt/pve/` before trusting any guide's hardcoded path |
| `curl \| grep koillection` shows nothing | Not a failure — the app root just 302-redirects to `/first-connection`, whose HTML doesn't contain the word "koillection". Use `docker compose ps` (expect `healthy`) and `curl -sv` (expect `302` + `Location: /first-connection`) instead |
| `deprecation.INFO` API Platform / symfony-form log spam on startup | Harmless upstream framework warnings, not errors — ignore |
| `nginx: command not found` when following Phase 6 | You're inside CT 114 (koillection), not CT 101 (reverse-proxy). Check the shell prompt — nginx work always happens on CT 101, never on the app container itself |
| Wrong nginx vhost filename | Use the full domain as the filename (`collections.spendlik.sk`), matching every other vhost on CT 101 — not the short service name |
| certbot fails domain validation | DNS (Phase 8) must resolve publicly before certbot's HTTP-01 challenge (Phase 7) will succeed — do DNS first if it hasn't propagated yet |
| No 2FA on collections.spendlik.sk | Intentional — Authelia was bypassed by decision (Phase 9). Security relies solely on Koillection's own login. Revisit if this ever becomes multi-user or exposed beyond personal use |
| Image uploads fail with "critical error" / `rename(): Invalid argument` | Hardcoded `/tmp` path in Koillection's upload handler colliding with NFS-mounted `/uploads` being a different filesystem — see "Known Issue" section above for full root cause and the entrypoint-symlink fix (already baked into this guide's Phase 5 compose file) |