State: volumes, secrets & the backup pipeline

All of a Talkyard site's durable state lives in a handful of Docker named volumes, while its configuration and secrets come in as bind mounts and Docker secrets. This topic maps which container mounts what (and read-only vs read-write), then walks through the stock talkyard-backup image and the caveats worth knowing before you rely on it. Everything below is as of v1.2026.003.

Volumes, config & secrets

The named volumes hold everything you can't afford to lose; the config files and the Postgres password are supplied from the host at read-only. Note that the same container often mounts a volume at a different access level than its neighbours — pub-files is rw for app but ro for web.

flowchart TB
  subgraph hostfs["Host filesystem — your deployment dir"]
    playconf["conf/app/play-framework.conf"]
    sitesconf["conf/web/sites-enabled/"]
    pgsecret["secrets/postgres_password.txt"]
  end

  web["web"]
  app["app"]
  cache["cache"]
  rdb["rdb"]
  search["search"]

  subgraph vols["Named volumes (<project>_*)"]
    pgdata[("pg-data")]
    esdata[("es-data + es-logs")]
    redisdata[("redis-data")]
    pubfiles[("pub-files<br/>public uploads")]
    privfiles[("priv-files<br/>private uploads")]
    webgen[("web-generated<br/>auto-ssl ACME acct key")]
  end

  sitesconf -->|"ro"| web
  playconf -->|"ro"| app
  pgsecret -->|"secret"| app
  pgsecret -->|"secret"| rdb

  web ---|"ro"| pubfiles
  web --- webgen
  app ---|"rw"| pubfiles
  app ---|"rw"| privfiles
  cache --- redisdata
  rdb --- pgdata
  search --- esdata

  classDef vol fill:#4a2c5e,stroke:#ab7fd1,color:#fff
  classDef ctr fill:#1e3a5f,stroke:#4a90d9,color:#fff
  classDef fs fill:#7a4a1e,stroke:#e0a060,color:#fff
  class pgdata,esdata,redisdata,pubfiles,privfiles,webgen vol
  class web,app,cache,rdb,search ctr
  class playconf,sitesconf,pgsecret fs

• pg-data (rdb) — the PostgreSQL cluster; the talkyard database is your entire forum: users, pages, posts, votes, settings. This is the single most important volume.
• es-data + es-logs (search) — the Elasticsearch full-text index. Derived data: it can be rebuilt by re-indexing from Postgres, so it isn't part of the backup set.
• redis-data (cache) — Redis persistence. Holds ephemeral-ish state (presence, watchbar, one-time login secrets, link-preview cache), not authoritative forum content.
• pub-files — public uploads (avatars, attachments). Mounted rw by app (which writes them) and ro by web (which serves the bytes directly).
• priv-files — private uploads. Mounted rw by app only. Note the backup caveat below.
• web-generated (web) — storage for the built-in lua-resty-auto-ssl ACME account key. Dormant if you terminate TLS in a reverse proxy in front of web; still present.
• Bind mounts (ro): conf/web/sites-enabled/ into web, and conf/app/play-framework.conf into app. These are your config, read once at container start.
• Docker secret: postgres_password.txt is delivered as a file-based secret to app and rdb (and to the backup job) rather than an environment variable.

The backup pipeline

Talkyard ships a stock backup service defined under profiles: [backup], so it never auto-starts — it's a one-shot container you invoke on demand (docker compose run --rm). Point your own scheduler at it: a cron entry or a systemd timer, on whatever schedule you like. Each run produces a self-contained set of archives in a bind-mounted output directory.

flowchart LR
  sched["Your scheduler<br/>cron / systemd timer<br/>(any schedule you like)"]
  backup["backup container<br/>docker compose run --rm<br/>profiles: [backup] one-shot"]
  rdb[("rdb")]
  pubfiles[("pub-files")]
  privfiles[("priv-files")]
  redisdata[("redis-data")]
  projdir["project dir<br/>conf + .env + compose + secrets/"]
  bksecret["secrets/backup_password.txt"]
  bkdir["backup archives dir<br/>(rw bind mount)"]

  sched ==>|"/ty/backup.sh"| backup
  rdb ==>|"pg_dumpall + gzip<br/>(all dbs + roles)"| backup
  pubfiles ==>|"ro — only uploads/<br/>subdir archived"| backup
  redisdata ==>|"ro — dump.rdb"| backup
  projdir ==>|"ro — whole dir tarred into config<br/>archive incl. secrets: why GPG matters"| backup
  bksecret -->|"secret — GPG passphrase"| backup
  privfiles -.->|"mounted ro but NEVER archived<br/>(known gap)"| backup
  backup ==>|"GPG AES256 only while backup_password<br/>non-empty, else SILENT plaintext"| bkdir

  classDef vol fill:#4a2c5e,stroke:#ab7fd1,color:#fff
  classDef fs fill:#7a4a1e,stroke:#e0a060,color:#fff
  classDef bk fill:#2d5016,stroke:#7cb342,color:#fff
  classDef warn fill:#5e2c2c,stroke:#d17f7f,color:#fff
  class rdb,pubfiles,redisdata vol
  class privfiles warn
  class projdir,bksecret fs
  class backup,sched,bkdir bk

• What the script dumps. /ty/backup.sh runs pg_dumpall against rdb (piped through gzip) — that captures all databases plus roles, not a single-database pg_dump. Redis is captured from its dump.rdb.
• Uploads. From pub-files, only the uploads/ subdirectory is archived (read-only mount).
• The config archive contains your secrets. The whole project directory — conf/, .env, the compose files, and the secrets/ directory — is tarred (read-only) into a config archive. That means the archive includes your Postgres password and any other secrets on disk. This is the reason encryption matters.
• Optional GPG-at-rest. If the backup_password secret (secrets/backup_password.txt) is non-empty, the output is encrypted with GPG AES256. Caveat: if that secret is empty, the script does not fail — it silently falls back to writing the archives (including the secrets-bearing config archive) in plaintext. Set a passphrase, or your at-rest backups expose everything.
• Output. Archives land in a read-write bind-mounted directory on the host. Retention/pruning of old archives, and copying archives off-box, are things you arrange yourself around this one-shot job.

Known caveat: priv-files is mounted but not archived

priv-files (private uploads) is mounted into the backup container read-only, but the stock backup.sh never reads it — nothing from priv-files ends up in any archive. If your site stores private uploads, the default backup does not protect them; you'll want to archive that volume separately until this gap is closed upstream.

Summary

Volume	Owner	In stock backup?
pg-data	rdb	Yes — via pg_dumpall
pub-files (uploads/)	app rw / web ro	Yes
redis-data	cache	Yes — dump.rdb
priv-files	app	No — known gap
es-data / es-logs	search	No — rebuildable index
project dir (config + secrets)	host bind	Yes — GPG-encrypted only if a passphrase is set