From e9fbbe3373eb66a345f5e3829e2563b94dc92051 Mon Sep 17 00:00:00 2001 From: Gab Virebent Date: Mon, 24 Aug 2026 17:34:45 +0200 Subject: Harden transport and preserve profile identities --- README.md | 75 +++++++--- deploy/n2usenet-gateway-watch | 74 ++++++++++ deploy/n2usenet-gateway-watch.service | 18 +++ deploy/n2usenet-gateway-watch.timer | 11 ++ docs/architecture.md | 52 ++++++- internal/assets/web/templates/index.html | 230 +++++++++++++++++++++++-------- internal/config/config.go | 64 +++++++++ internal/config/config_test.go | 151 ++++++++++++++++++++ internal/smtpclient/client.go | 141 ++++++++++++++----- internal/smtpclient/client_test.go | 99 +++++++++++++ internal/submit/message.go | 14 +- internal/submit/message_test.go | 41 ++++++ internal/submit/types.go | 91 ++++++++++-- internal/submit/types_test.go | 125 ++++++++++++++++- internal/submit/validation_test.go | 2 +- internal/transporthealth/monitor.go | 83 +++++++++++ internal/transporthealth/monitor_test.go | 30 ++++ 17 files changed, 1169 insertions(+), 132 deletions(-) create mode 100644 deploy/n2usenet-gateway-watch create mode 100644 deploy/n2usenet-gateway-watch.service create mode 100644 deploy/n2usenet-gateway-watch.timer create mode 100644 internal/config/config_test.go create mode 100644 internal/submit/message_test.go create mode 100644 internal/transporthealth/monitor.go create mode 100644 internal/transporthealth/monitor_test.go diff --git a/README.md b/README.md index 35acc73..460c50f 100644 --- a/README.md +++ b/README.md @@ -15,10 +15,14 @@ Browser -> N2Usenet Go backend -> local nym-socks5-client -> Nym network requester - -> SMTP mail2news.tcpreset.net:25 STARTTLS + -> SMTP submission 51.178.55.22:587 STARTTLS + AUTH + (TLS name mail.virebent.art) + -> Postfix with SPF/DKIM/DMARC -> mail2news@mail2news.tcpreset.net -> Mail2News gateway - -> news.tcpreset.net NNTP + -> Tor SOCKS + -> peannyjkqwqfynd24p6dszvtchkq7hfkwymi5by5y332wmosy5dwfaqd.onion:119 + -> news.tcpreset.net INN ``` ## Build @@ -37,18 +41,20 @@ go build -o n2u-check ./cmd/n2u-check - **Ed25519 signing**: the message is signed locally in the browser with the self-hosted TweetNaCl (`nacl`) library. The secret key never leaves the device; the server only verifies the signature against the submitted text. -- **VFACE identicon / `Face:` header**: the public key deterministically - generates a visual identicon (`identicons-cli` engine, same backend as +- **VFACE identicon / `Face:` header**: the complete + `username|email|public-key` identity deterministically generates a visual + identicon (`identicons-cli` engine, same backend as `identicons.virebent.art`), embedded as a folded `Face:` header alongside `X-Ed25519-Pub`/`X-Ed25519-Sig` so any reader can re-verify authorship. - **Optional persistent identity (save/load)**: after generating a keypair, "Save Identity" downloads a JSON file with the public/secret key and - username/email. Loading that file later reuses the same keypair, so the - same identicon keeps appearing across posts and proves later messages - come from the same author. This is entirely optional: skip it for a - fresh, unlinkable throwaway identity on every post. The file contains the - secret key in clear, so it must be kept private, anyone holding it can - sign as that identity. + username/email. Loading that file later restores the complete identity, so + the same identicon keeps appearing across posts and lets readers associate + later signatures with the same public key. Username, email, and keypair are + locked together while that identity is active. This is entirely optional: + skip it for a fresh, unlinkable throwaway identity on every post. The file + contains the secret key in clear, so it must be kept private, anyone holding + it can sign as that identity. - **Usenet-style line wrapping**: before signing, the message body is automatically word-wrapped to 72 columns (RFC 1855 Netiquette convention), for compatibility with classic terminal-based newsreaders. @@ -58,9 +64,11 @@ go build -o n2u-check ./cmd/n2u-check - **CSRF protection and per-IP rate limiting**, both server-side. - **No access logs, ephemeral state**: identity and form data are kept only in browser tab memory and wiped after a successful send. -- **Delivery over the Nym mixnet**, not Tor: SOCKS5 through a local - `nym-socks5-client`/`nym-network-requester` path to the Mail2News relay, - then into `news.tcpreset.net` via NNTP. +- **Nym followed by Tor for separate hops**: SOCKS5 through a local + `nym-socks5-client`/`nym-network-requester` carries authenticated SMTP/TLS + submission to the Virebent MTA. Postfix signs and relays the message to + Mail2News. Mail2News then uses Tor to post to the TCPReset onion NNTP + endpoint. The browser-to-N2Usenet hop remains HTTPS. ## Required Configuration @@ -70,14 +78,19 @@ Set these in an environment file or systemd unit: N2U_LISTEN=127.0.0.1:8095 N2U_PUBLIC_BASE_URL=https://n2usenet.virebent.art -N2U_SMTP_HOST=mail2news.tcpreset.net -N2U_SMTP_PORT=25 -N2U_SMTP_TLS_SERVER_NAME=mail.tcpreset.net +N2U_SMTP_HOST=51.178.55.22 +N2U_SMTP_PORT=587 +N2U_SMTP_TLS_SERVER_NAME=mail.virebent.art N2U_SMTP_IMPLICIT_TLS=false N2U_SMTP_RECIPIENT=mail2news@mail2news.tcpreset.net N2U_SMTP_ENVELOPE_FROM=n2usenet@virebent.art +N2U_SMTP_SENDER=n2usenet@virebent.art +N2U_SMTP_USERNAME=n2usenet@virebent.art +N2U_SMTP_PASSWORD_FILE=/etc/n2usenet/smtp-password N2U_SMTP_HELO=n2usenet.virebent.art +N2U_TRANSPORT_PROBE_INTERVAL=15m N2U_MESSAGE_ID_DOMAIN=n2usenet.virebent.art +N2U_CSRF_KEY_FILE=/etc/n2usenet/csrf-key N2U_IDENTICONS_CLI=/usr/local/bin/identicons-cli N2U_REQUIRE_FACE=true @@ -107,9 +120,32 @@ secrets. The provider address is deployment configuration. - SMTP uses a fixed envelope sender by default, `n2usenet@virebent.art`, so the Mail2News relay does not depend on the user's pseudonymous email passing remote sender policy checks. +- The RFC 5322 `From:` header publishes the complete profile identity, + `username `. `Sender:` identifies `n2usenet@virebent.art` as the + submitting agent, while SMTP AUTH and the envelope sender remain fixed on + that service account. The profile email is a signed pseudonymous identity + claim, not proof that the user controls its domain. +- SMTP credentials are read from a root-managed file and AUTH is rejected by + the application configuration unless TLS is enabled. +- Production uses the MTA's global IPv4 address as the SOCKS destination while + still verifying `mail.virebent.art` in TLS. This keeps the Nym requester's + non-global-address filter strict when the MTA host resolves its own hostname + to local interface addresses. - `/submit` returns `Cache-Control: no-store` and the browser path uses fetch so successful submissions do not leave the full form state in the rendered response page. +- The CSRF cookie is session-scoped without a fixed elapsed-time expiry. The + browser refreshes the signed token immediately before submission. Production + uses a root-managed signing key so restarts do not invalidate open tabs, and + valid session tokens are reused so multiple tabs cannot rotate each other's + cookie. +- `/healthz` reports only process liveness. `/readyz` reports the cached result + of a background Nym/STARTTLS/SMTP AUTH probe and never creates transport + traffic in response to an HTTP request. +- `n2usenet-gateway-watch.timer` checks the active Nym gateway registration + daily. It rotates only within 24 hours of expiry, verifies the complete SMTP + path without sending mail, and restores the previous gateway database if the + replacement fails. - The app does not add a clearnet fallback to the M2Usenet onion PHP sender. - The service now prefers the `N2U_*` environment variable prefix and still accepts legacy `M2U_*` variables during the rollout. @@ -125,10 +161,11 @@ Working deployment shape: - `victor` and `rasmus` are SSH aliases for the same host, `51.178.55.22`. - `victor` runs `n2usenet-network-requester.service`. - `pietro` runs `n2usenet-socks.service` on `127.0.0.1:11080`. -- `pietro` runs `n2usenet.service` with SMTP target `mail2news.tcpreset.net:25`. +- `pietro` runs `n2usenet.service` with authenticated SMTP submission to + `51.178.55.22:587` through Nym and verifies `mail.virebent.art` in TLS. - `pietro` uses `n2usenet@virebent.art` as the SMTP envelope sender. -- `cmd/n2u-check` completed SMTP greeting, EHLO, STARTTLS, and TLS 1.3 through - the live Nym path on 2026-07-13. +- `victor` Postfix relays to `mail2news@mail2news.tcpreset.net` with the + domain's SPF, DKIM, and DMARC policy. `identicons-cli` is already available on `victor` at `/usr/local/bin/identicons-cli`. Its output was checked against the `Ch1ffr3punk/identicons` engine using diff --git a/deploy/n2usenet-gateway-watch b/deploy/n2usenet-gateway-watch new file mode 100644 index 0000000..6390197 --- /dev/null +++ b/deploy/n2usenet-gateway-watch @@ -0,0 +1,74 @@ +#!/usr/bin/env bash +set -euo pipefail + +client_home=/var/lib/n2usenet/nym +client_bin=/opt/n2usenet/bin/nym-socks5-client +check_bin=/opt/n2usenet/bin/n2u-check +gateway_db="$client_home/.nym/socks5-clients/n2usenet/data/gateways_registrations.sqlite" +backup_root=/var/lib/n2usenet-gateway-watch/backups +renew_before_seconds=86400 +lock_file=/run/n2usenet-gateway-watch/lock + +exec 9>"$lock_file" +if ! flock -n 9; then + exit 0 +fi + +for required in "$client_bin" "$check_bin" "$gateway_db" /etc/n2usenet/smtp-password; do + if [[ ! -e "$required" ]]; then + logger -t n2usenet-gateway-watch "required path is missing: $required" + exit 1 + fi +done + +expiry=$(sqlite3 "$gateway_db" "select d.expiration_timestamp from active_gateway a join remote_gateway_details d on d.gateway_id_bs58=a.active_gateway_id_bs58;") +if [[ -n "$expiry" ]]; then + expiry_epoch=$(date -u -d "$expiry" +%s) + now_epoch=$(date -u +%s) + if (( expiry_epoch - now_epoch > renew_before_seconds )); then + exit 0 + fi +fi + +stamp=$(date -u +%Y%m%dT%H%M%SZ) +backup_dir="$backup_root/$stamp" +install -d -m 0700 "$backup_dir" + +restart_services() { + systemctl start n2usenet-socks.service || true + systemctl start n2usenet.service || true +} +trap restart_services EXIT + +systemctl stop n2usenet.service n2usenet-socks.service +cp -a "$gateway_db" "$backup_dir/gateways_registrations.sqlite.before" + +if ! runuser -u n2usenet -- "$client_bin" add-gateway --id n2usenet --set-active -o json; then + cp -a "$backup_dir/gateways_registrations.sqlite.before" "$gateway_db" + chown n2usenet:n2usenet "$gateway_db" + logger -t n2usenet-gateway-watch "gateway registration failed; previous state restored" + exit 1 +fi + +chown -R n2usenet:n2usenet "$client_home/.nym" +systemctl start n2usenet-socks.service +sleep 5 + +if ! "$check_bin" \ + -socks 127.0.0.1:11080 \ + -smtp 51.178.55.22:587 \ + -tls-name mail.virebent.art \ + -helo n2usenet.virebent.art \ + -auth-user n2usenet@virebent.art \ + -auth-password-file /etc/n2usenet/smtp-password \ + -timeout 180s; then + systemctl stop n2usenet-socks.service + cp -a "$backup_dir/gateways_registrations.sqlite.before" "$gateway_db" + chown n2usenet:n2usenet "$gateway_db" + logger -t n2usenet-gateway-watch "new gateway failed transport verification; previous state restored" + exit 1 +fi + +systemctl start n2usenet.service +trap - EXIT +logger -t n2usenet-gateway-watch "gateway renewed and transport verification passed" diff --git a/deploy/n2usenet-gateway-watch.service b/deploy/n2usenet-gateway-watch.service new file mode 100644 index 0000000..9854ae0 --- /dev/null +++ b/deploy/n2usenet-gateway-watch.service @@ -0,0 +1,18 @@ +[Unit] +Description=Renew and verify the N2Usenet Nym gateway registration +After=network-online.target +Wants=network-online.target + +[Service] +Type=oneshot +ExecStart=/usr/local/sbin/n2usenet-gateway-watch +TimeoutStartSec=10min +NoNewPrivileges=true +PrivateTmp=true +ProtectHome=true +ProtectSystem=strict +RuntimeDirectory=n2usenet-gateway-watch +RuntimeDirectoryMode=0750 +StateDirectory=n2usenet-gateway-watch +StateDirectoryMode=0700 +ReadWritePaths=/var/lib/n2usenet/nym diff --git a/deploy/n2usenet-gateway-watch.timer b/deploy/n2usenet-gateway-watch.timer new file mode 100644 index 0000000..d169872 --- /dev/null +++ b/deploy/n2usenet-gateway-watch.timer @@ -0,0 +1,11 @@ +[Unit] +Description=Daily N2Usenet Nym gateway expiry check + +[Timer] +OnCalendar=*-*-* 03:17:00 UTC +RandomizedDelaySec=15m +Persistent=true +Unit=n2usenet-gateway-watch.service + +[Install] +WantedBy=timers.target diff --git a/docs/architecture.md b/docs/architecture.md index 501903e..12a83da 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -8,6 +8,8 @@ - `internal/nymclient`: optional managed `nym-socks5-client` lifecycle. - `internal/socks5`: minimal SOCKS5 dialer used for Nym transport. - `internal/smtpclient`: SMTP + STARTTLS sender over an injected dialer. +- `internal/transporthealth`: cached background readiness state for the complete + SOCKS, SMTP, TLS, and AUTH path. - `internal/submit`: HTTP handlers, validation, message construction. - `internal/storage`: replay cache for Hashcash tokens. - `internal/assets/web`: embedded template and browser assets. @@ -23,20 +25,37 @@ kept only in tab memory and cleared after successful send. ## Transport Boundary -The app never dials `mail2news.tcpreset.net` directly when `N2U_NYM_ENABLED=true`. -All SMTP traffic goes through `N2U_NYM_SOCKS`, normally provided by -`nym-socks5-client`. +The app never dials an SMTP target directly when `N2U_NYM_ENABLED=true`. All +SMTP traffic goes through `N2U_NYM_SOCKS`, normally provided by +`nym-socks5-client`. The MTA boundary uses STARTTLS plus SMTP AUTH. Credentials +are loaded from a root-managed file, not embedded in source or command-line +arguments. SMTP AUTH, the SMTP envelope sender, and the RFC 5322 `Sender:` +header use the fixed N2Usenet service account. The public RFC 5322 `From:` +header instead contains the complete profile identity, `username `, so +Mail2News and newsreaders do not replace it with the transport account. This +profile address is a pseudonymous claim bound to the VFACE tuple and public +key, not verification that the author controls the address domain. The intended posting chain is: ```text Browser -> HTTPS n2usenet.virebent.art N2Usenet -> Nym SOCKS5/requester -Nym requester -> mail2news.tcpreset.net:25 SMTP with STARTTLS +Nym requester -> 51.178.55.22:587 SMTP with STARTTLS + AUTH + (TLS name mail.virebent.art) +mail.virebent.art Postfix -> SPF/DKIM/DMARC signing and relay mail2news@mail2news.tcpreset.net -> Mail2News gateway -Mail2News gateway -> news.tcpreset.net NNTP +Mail2News -> Tor SOCKS +Mail2News -> peannyjkqwqfynd24p6dszvtchkq7hfkwymi5by5y332wmosy5dwfaqd.onion:119 NNTP +TCPReset INN -> news.tcpreset.net ``` +Nym and Tor protect different transport segments. Nym carries the +N2Usenet-to-MTA SMTP connection, with STARTTLS providing authenticated TLS +inside that path. After Postfix relays the message to Mail2News, Mail2News uses +Tor for the final connection to the onion NNTP endpoint. The final Usenet +article is public. + ## Deployment Boundary The current onion PHP app remains deployed under Onion Newsreader `/compose/`. @@ -46,5 +65,26 @@ in the current production deployment. Current production split: -- `victor` runs the dedicated Nym network requester. +- `victor` runs the dedicated Nym network requester and the authenticated + Postfix submission endpoint. - `pietro` runs the public HTTPS app and the local `nym-socks5-client`. + +The SOCKS destination is the MTA's global IPv4 address because local hostname +resolution on `victor` also returns interface-local addresses that the Nym +requester correctly rejects. TLS identity verification still uses +`mail.virebent.art`. + +## Operational Readiness + +`/healthz` is a liveness endpoint and does not imply that Nym can deliver. +`/readyz` exposes only the cached result of a periodic transport probe. The +probe performs SOCKS connection, SMTP greeting, STARTTLS, TLS verification, +post-TLS EHLO, and SMTP AUTH, then quits before `MAIL FROM`; it never sends a +message. Keeping the probe out of the request path prevents public health +requests from consuming Nym bandwidth. + +Nym gateway registrations are time-limited. The deployment timer runs once a +day and renews the active gateway only when it is within 24 hours of expiry. +The rotation is performed with the application and SOCKS services stopped, +backed up first, and accepted only after the same no-mail transport probe +succeeds. diff --git a/internal/assets/web/templates/index.html b/internal/assets/web/templates/index.html index 60df055..4595421 100644 --- a/internal/assets/web/templates/index.html +++ b/internal/assets/web/templates/index.html @@ -3,7 +3,7 @@ - N2Usenet Gateway v2.7 Nym + N2Usenet Gateway v2.7.4 Nym