diff options
Diffstat (limited to 'docs/FOG-ARCHITECTURE.md')
| -rw-r--r-- | docs/FOG-ARCHITECTURE.md | 1122 |
1 files changed, 1122 insertions, 0 deletions
diff --git a/docs/FOG-ARCHITECTURE.md b/docs/FOG-ARCHITECTURE.md new file mode 100644 index 0000000..9060c7c --- /dev/null +++ b/docs/FOG-ARCHITECTURE.md @@ -0,0 +1,1122 @@ +# FOG Architecture + +Status: Draft 0.1 + +Date: 2026-08-08 + +## 1. Purpose + +This document defines the normative component architecture, trust boundaries, +information flows, state ownership, deployment units, and dependency rules for +the FOG autonomous mix network. + +It refines `FOG-THREAT-MODEL.md`. It does not select final cryptographic +algorithms, packet geometry, delay distributions, or active numeric message, +storage, and Composer suites. Those decisions belong to narrower protocol +specifications and MUST satisfy the boundaries defined here. + +FOG is not implemented yet. Statements in this document are design +requirements, not claims about deployed software. + +The key words MUST, MUST NOT, REQUIRED, SHOULD, SHOULD NOT, and MAY describe +normative requirements in the sense of BCP 14 when they appear in uppercase. + +## 2. Architectural Scenario + +FOG is one product and one coordinated protocol suite with multiple +independently deployable security roles. It is not a single daemon, a shared +database application, or a collection of unrelated public microservices. + +The baseline implementation shape is: + +- one source repository and release process; +- one separately built executable for each online security role; +- a networkless Composer executable and image profile; +- small protocol libraries with strict dependency direction; +- a public client SDK with a deliberately narrow surface; +- a simulator and conformance corpus in the same repository; +- local Podman deployment only for the functional PoC; +- independent hosts and operators where a deployment makes anonymity claims. + +Separate processes are justified by independent deployment, key ownership, +network policy, compromise containment, and operator ownership. Application +features that do not need a server-side trust boundary MUST remain modules, +not daemons. + +## 3. Core Architectural Invariants + +The following invariants apply to every profile. + +### ARC-001: No universal trust domain + +No process, host, operator account, database, management system, or private key +MAY be required to observe or control the complete user-to-storage path. + +### ARC-002: The Composer is networkless + +`fog-compose` MUST have no network interface. All external input and output +cross explicit bounded transfer interfaces. A network-capable convenience +mode is not a conforming Composer profile. + +### ARC-003: No data-plane bypass + +Native user operations MUST traverse an entry, all configured mix layers, and +a terminal courier or service endpoint. Failure handling MUST NOT introduce a +direct client-to-courier, client-to-storage, entry-to-storage, or +mix-to-clearnet path. + +### ARC-004: Three distinct cryptographic layers + +Message end-to-end protection, KEMSphinx routing protection, and adjacent-node +Noise transport MUST use purpose-separated protocols and keys. Success or +failure of one layer MUST NOT be interpreted as validation of another. + +### ARC-005: Complete authenticated network view + +Route construction, entry eligibility, node roles, packet profiles, security +parameters, and key validity MUST derive from a complete threshold-authenticated +consensus accepted by the Composer. A relay-supplied partial view MUST NOT +control route construction. + +### ARC-006: Uniform external behavior + +Native applications MUST share consensus-authorized packet classes, routing +rules, retry classes, and cover processing. Core nodes MUST NOT vary packet +geometry or parser selection according to `fog-drop`, `fog-mailbox`, or +`fog-im`. + +### ARC-007: One private-key owner + +Each private key MUST have one owning role and one purpose. Separate roles MUST +NOT share private key files, secret volumes, writable state directories, or +backup credentials. + +### ARC-008: Bounded state and parsing + +Every queue, cache, request, response, parser, retry sequence, and storage +operation MUST have explicit size, time, and resource bounds. Unknown-critical, +non-canonical, expired, or unauthenticated input MUST fail closed. + +### ARC-009: Privacy-aware failure + +Availability recovery MUST preserve the declared layer count, packet class, +authenticated profile, entry constraints, cover policy, and consensus +freshness. If it cannot, the affected operation MUST stop and report a local +coarse failure. + +### ARC-010: No inherited external guarantee + +Tor, Nym, Katzenpost, YAMN, SMTP, NNTP, and foreign Sphinx networks are not +runtime dependencies of the FOG core. An edge bridge terminates the FOG trust +claim and MUST publish a separate threat model. + +## 4. System Planes + +FOG separates four planes: + +1. **Offline user plane**: plaintext, identities, contacts, message state, + storage capabilities, route construction, and validation inside the + Composer. +2. **Anonymous data plane**: entry, stratified mixes, courier, native service + endpoints, and storage replicas carrying fixed-class opaque traffic. +3. **Consensus control plane**: node descriptors, authority voting, + threshold-signed consensus, revocation, profile authorization, and + equivocation evidence. +4. **Operations and release plane**: local administration, signed builds, + configuration generation, coarse metrics, backups, incident response, and + updates. + +These planes MAY share public code and formats. They MUST NOT share universal +credentials or writable runtime state. + +```text + consensus control plane + node descriptors -> authorities -> signed consensus + | + v +offline user plane anonymous data plane + +Composer -> transfer -> blind relay -> entry -> L1 -> L2 -> L3 -> courier + ^ | | + | | v + +---- opaque input <-+ storage replicas + + operations and release plane + signed artifacts, local administration, coarse aggregates +``` + +## 5. Data Classification + +Architecture and protocol documents MUST label fields and state using these +classes or a stricter derived scheme. + +| Class | Meaning | Examples | +| --- | --- | --- | +| `PUBLIC-AUTH` | Public but authenticity or freshness matters | consensus, descriptors, release metadata, protocol profiles | +| `OPAQUE-META` | Encrypted content with sensitive timing or relationship metadata | KEMSphinx packets, transfer bundles, reply envelopes, relay queues | +| `CAPABILITY` | Possession grants a read, write, reply, or recovery operation | mailbox capabilities, reply blocks, rendezvous handles | +| `SECRET-ROLE` | Private material owned by one online role | Noise, KEMSphinx, authority, replica, queue-sealing keys | +| `SECRET-USER` | User secret or conversation state | identity keys, ratchet state, contact state, backup keys | +| `PLAINTEXT-USER` | Decrypted user content | drafts, messages, rendered attachment content | +| `AGGREGATE` | Delayed and privacy-reviewed operational data | thresholded loss or availability measurements | + +`CAPABILITY`, `SECRET-ROLE`, `SECRET-USER`, and `PLAINTEXT-USER` MUST NOT enter +logs, metrics, command lines, crash reports, descriptors, consensus, or support +bundles. + +## 6. Trust Domains and Deployment Units + +Each row is a distinct security domain even when the PoC places containers on +one host. + +| Domain | Baseline deployable | Primary state owner | Network position | +| --- | --- | --- | --- | +| Offline user | `fog-compose` | user identity, contacts, message and capability state | no network | +| Transfer receiver | restricted adapter owned by `fog-client-relay` | bounded incomplete transfer state | local physical input only | +| Online user relay | `fog-client-relay` | opaque queues, schedules, temporary entry and return state | client edge | +| Directory authority | `fog-authority` | descriptors, votes, consensus and revocation state | control plane only | +| Entry | `fog-entry` | bounded ingress and temporary return state | edge of data plane | +| Mix layer | `fog-mix` with fixed layer assignment | delay queues, replay state, epoch keys | one stratified layer | +| Courier | `fog-courier` | bounded request deduplication and replica dispatch state | terminal mix service | +| Storage | `fog-store` | opaque boxes, tombstones, replica metadata | behind courier | +| Optional native service | `fog-service-*` | minimum service-specific opaque state | terminal mix service | +| Observer | `fog-observer` | delayed coarse aggregates | operations plane | +| Release | offline release tooling | release signing and provenance state | outside runtime network | +| External bridge | `fog-bridge-*` | bridge-specific state | outside core guarantee | + +A conforming executable MUST activate exactly one online role. Multi-role +configuration flags are forbidden outside explicitly labeled local simulation +fixtures. + +## 7. Role Contracts + +### 7.1 `fog-compose` + +The Composer is the only core component authorized to handle user plaintext, +user identity private keys, contact secrets, message state, and mailbox +capabilities. + +It MUST: + +- verify release, consensus, contact, recovery, and message authenticity; +- persist the highest accepted consensus epoch and rollback state; +- build message-level ciphertext before export; +- select internal routes from the authenticated consensus; +- construct KEMSphinx packets, reply material, and storage operations; +- create fixed-class opaque work items for the blind relay; +- consume inbound bundles only after complete bounded parsing and + cryptographic verification; +- store mutable secrets in an authenticated encrypted state store; +- minimize plaintext lifetime and exclude plaintext from generic desktop + services, swap, previews, indexing, and crash dumps. + +It MUST NOT: + +- open a network socket or expose an HTTP, RPC, shell, or plugin server; +- delegate message encryption, route selection, capability derivation, or + contact authentication to the blind relay; +- import generic archives, office documents, HTML, scripts, executable files, + or unbounded media; +- accept a lower consensus epoch without an explicit authenticated recovery + procedure. + +`fog-drop`, `fog-mailbox`, and `fog-im` are initially Composer-side application +modules over shared messaging and storage protocol libraries. They are not +independent network daemons in the initial architecture. + +Threats addressed: `TM-ENDPOINT-01`, `TM-ENDPOINT-02`, `TM-ENDPOINT-03`, +`TM-APP-01`, `TM-CRYPTO-01`, `TM-PKI-02`. + +### 7.2 Transfer receiver + +The transfer receiver is a minimal adapter at the blind-relay boundary. It MAY +decode QR or one explicitly configured FOG-SX physical backend. It MUST output +only a bounded opaque FOG bundle to the relay queue. + +It MUST run without access to relay network credentials, queue decryption +keys, user identity keys, or general filesystems. FEC and transport checksums +MUST be completed before the inner bundle enters the relay, but they MUST NOT +be treated as authenticity. + +The high-assurance offline-to-online path MUST have no automatic reverse data +channel. Online-to-offline input uses a separately controlled visual or +receive-only import path and is not the reverse channel of FOG-SX. + +Threats addressed: `TM-ENDPOINT-02`, `TM-AVAIL-01`. + +### 7.3 `fog-client-relay` + +The blind relay is the online scheduling and transport agent for one local +Composer profile. It is allowed to know that its local user is using FOG and +to observe local submission and import times. It MUST remain blind to +plaintext, contacts, capabilities, application type, and the internal mix +route. + +It MUST: + +- maintain bounded encrypted-at-rest queues of opaque work items; +- fetch complete consensus documents and pass them unmodified to the Composer; +- verify enough public consensus metadata to reject obvious network misuse, + while treating the Composer as the final consensus authority for user work; +- choose an entry only from the Composer-authorized temporary candidate set; +- follow consensus-authorized transmission, retrieval, cover, retry, and + failure classes; +- generate client cover and loop traffic without using user identity or + contact secrets; +- maintain only short-lived, opaque return rendezvous state; +- transfer received opaque bundles to the controlled Composer import path; +- erase expired work, return state, and incomplete transfers according to + explicit limits. + +It MUST NOT: + +- parse message or service payloads; +- construct or rewrite the Composer's internal route; +- hold message identity keys, mailbox capabilities, contact vouchers, ratchet + state, or release signing keys; +- bypass an entry after failure; +- expose a general-purpose proxy, SOCKS, VPN, TUN, SMTP, NNTP, or web API; +- use a stable remote account identity as the default entry authentication + mechanism. + +The relay MAY select among opaque entry-specific submission variants prepared +by the Composer. The exact entry-capsule and fallback construction is defined +by `FOG-WIRE` and `FOG-SPHINX-PROFILES`; the relay MUST NOT learn the first +internal mix hop from that construction. + +Threats addressed: `TM-NET-01`, `TM-NET-02`, `TM-NET-03`, `TM-NET-06`, +`TM-ROLE-01`, `TM-ENDPOINT-01`, `TM-OPS-01`. + +### 7.4 `fog-authority` + +Authorities collectively define the authenticated network view. They do not +carry user data-plane packets. + +Each authority MUST: + +- authenticate and validate descriptors for admitted role identities; +- record declared operator families and deployment attributes; +- validate role, layer, key, profile, address, epoch, and revocation rules; +- exchange signed votes with peer authorities; +- produce canonical consensus only at the configured threshold; +- archive sufficient signed material to diagnose rollback and equivocation; +- publish current and next public keys with bounded overlap; +- separate long-term authority identity and recovery material from online + voting state; +- fail deterministically when a quorum or time condition is not met. + +An authority MUST NOT: + +- inject data-plane traffic through privileged paths; +- assign one process to multiple data-plane roles; +- unilaterally create a valid consensus; +- automatically trust a replacement authority or node identity; +- receive user capabilities, packet identifiers, or fine-grained flow data. + +Consensus distribution MAY use multiple mirrors, but mirrors are untrusted +transport. Signatures, epoch monotonicity, validity, and profile authorization +are verified at every consumer. + +Threats addressed: `TM-PKI-01`, `TM-PKI-02`, `TM-PKI-03`, `TM-NET-06`, +`TM-SUPPLY-01`. + +### 7.5 `fog-entry` + +The entry is an access gateway distinct from the three privacy-relevant mix +layers. It accepts relay sessions, normalizes ingress handling, and dispatches +opaque KEMSphinx packets to layer 1. + +It MUST: + +- authenticate itself through the fixed entry link profile; +- authorize submission using short-lived, consensus-bound material rather + than a stable user account by default; +- accept only fixed-class opaque submissions for the active epoch; +- decrypt or validate only the entry capsule needed to learn the authorized + first internal hop and packet binding; +- forward only to consensus-authorized layer-1 nodes; +- maintain bounded queues and non-amplifying failure behavior; +- process real and cover submissions through the same path; +- support bounded, opaque, short-lived return rendezvous where required by the + reply protocol; +- perform exactly one terminal KEMSphinx unwrap on an authorized reply route + when the return profile requires it, using a dedicated entry-return key and + never a mix-layer or entry-capsule key. + +It MUST NOT: + +- process an internal KEMSphinx mix layer or any forward KEMSphinx hop merely + because it is an entry; +- perform a reply-terminal unwrap outside an authenticated return-rendezvous + context or forward the result as if it were another mix hop; +- learn the complete route, final service, mailbox capability, contact, or + plaintext; +- select or rewrite the internal route; +- forward directly to layer 2, layer 3, courier, storage, bridge, or Internet; +- retain a durable per-user mailbox or account in the core profile; +- produce application-specific errors or timing classes. + +The exact entry capsule and return rendezvous are protocol decisions. They +MUST preserve Composer route authority and relay blindness. An implementation +convenience MUST NOT silently make the entry the first of only three total +hops. + +Threats addressed: `TM-NET-01`, `TM-NET-03`, `TM-NET-06`, `TM-ROLE-01`, +`TM-AVAIL-01`. + +### 7.6 `fog-mix` + +Every mix executable has one fixed layer assignment from the current +consensus. It is a cryptographic router, not an application server. + +It MUST: + +- authenticate adjacent eligible nodes through the fixed node link profile; +- accept packets only from roles permitted for its layer; +- perform exactly one KEMSphinx hop transformation; +- validate the packet profile, epoch, replay tag, routing command, and delay + bounds before enqueueing; +- persist replay state for the required maximum packet lifetime; +- delay and schedule packets according to the authenticated profile; +- process real, loop, decoy, forwarded, and reply traffic uniformly within + their authorized packet class; +- forward only to the next authorized layer or terminal role; +- enforce bounded queues, connection counts, cryptographic work, and retries; +- emit only coarse local health aggregates. + +It MUST NOT: + +- expose application plugins, storage APIs, user accounts, or general proxy + functions; +- select an arbitrary next layer or skip a layer; +- parse message, capability, courier, storage, or bridge payloads; +- share KEMSphinx private keys or replay databases with another role; +- log packets, replay tags, routes, per-packet delay, or fine-grained timing. + +Loss of valid replay state places the node outside the active privacy profile. +It MUST stop packet processing until safe state is restored or a new epoch +with fresh keys begins according to the replay specification. + +Threats addressed: `TM-NET-01`, `TM-NET-03`, `TM-NET-04`, `TM-NET-05`, +`TM-NET-06`, `TM-ROLE-02`, `TM-OPS-01`, `TM-AVAIL-01`. + +### 7.7 `fog-courier` + +The courier is the terminal data-plane mediator between anonymous requests and +storage or a separately declared native service. It terminates only the final +KEMSphinx service envelope and MUST not learn the client network origin. + +It MUST: + +- accept terminal packets only after all configured mix layers; +- parse one bounded, versioned courier command set; +- dispatch opaque encrypted operations to eligible replicas; +- keep replica selection knowledge separate from final capability-derived + record location where the storage construction requires it; +- keep only bounded, expiring request-deduplication and reply state; +- use single-use reply material or an equivalently reviewed construction; +- return replies through a consensus-authorized anonymous reply route; +- apply identical outer behavior to reads, writes, misses, retries, expected + outcomes, and cover operations as defined by `FOG-STORAGE`; +- enforce non-amplifying limits before expensive or fan-out work. + +A courier acceptance reply is not replica durability or message delivery. The +courier MUST preserve opaque final-replica receipt material without creating +or interpreting it. + +It MUST NOT: + +- receive user identity keys, message plaintext, contact state, or long-term + mailbox capabilities; +- connect directly to a blind relay or Composer; +- expose replicas directly to clients or mixes; +- become a durable mailbox database; +- load arbitrary third-party plugins in the core process; +- send data to the Internet or an external bridge under the core profile. + +Threats addressed: `TM-NET-01`, `TM-NET-02`, `TM-NET-05`, `TM-ROLE-03`, +`TM-APP-01`, `TM-AVAIL-01`. + +### 7.8 `fog-store` + +Storage replicas hold authenticated encrypted records addressed through +capability-derived, rotating, pseudorandom locations. They are not public +mailbox servers. + +Each replica MUST: + +- authenticate couriers and eligible replica peers through a fixed node link + profile; +- accept only bounded courier or replica protocol operations; +- validate record authentication before committing state; +- issue purpose-separated authenticated receipts only after durable local + commit under the exact active storage manifest; +- implement idempotent writes, explicit expiry, tombstones, quotas, bounded + retention, and garbage collection; +- keep replica and storage-envelope keys separate from node link keys; +- return fixed-class authenticated encrypted results; +- support the replica-independence and intermediate/final separation required + by the selected storage profile; +- encrypt storage media and backups as defense in depth without treating disk + encryption as end-to-end protection. + +It MUST NOT: + +- accept connections from Composers, blind relays, entries, or ordinary mixes; +- receive user plaintext or message identity private keys; +- expose record existence through a public unauthenticated lookup interface; +- share a writable database, database credentials, or backup key with another + replica operator; +- publish per-record or fine-grained access metrics. + +The intended privacy profile requires at least four independently operated +replicas. A smaller local fixture is functional testing only and MUST disable +the corresponding unlinkability claim. + +Threats addressed: `TM-NET-02`, `TM-ROLE-03`, `TM-ENDPOINT-03`, +`TM-CRYPTO-01`, `TM-AVAIL-01`. + +### 7.9 Native application modules and optional services + +The initial `fog-drop`, `fog-mailbox`, and `fog-im` state machines run inside +the Composer and produce a common bounded message/storage envelope. Their +network traffic MUST be indistinguishable within the declared packet class. + +A future feature MAY require a server-side `fog-service-*` executable. Such a +service MUST: + +- be a separately keyed terminal role behind all mix layers; +- declare its exact request fields and information exposure; +- accept one fixed bounded protocol, not arbitrary code or generic HTTP; +- use anonymous reply routes; +- receive no privilege to call unrelated services or public Internet targets; +- add a threat-model extension and independent conformance tests. + +No service is permitted to weaken the common packet profile or make the +courier parse application plaintext. + +Threats addressed: `TM-NET-06`, `TM-APP-01`, `TM-ROLE-03`. + +### 7.10 `fog-observer` + +The observer receives delayed, coarse, thresholded aggregates. It is not a +packet-flow collector. + +It MUST: + +- accept only a versioned aggregate schema; +- enforce release delay, minimum population, and suppression rules; +- separate operator health views from public views; +- expire raw signed aggregate submissions after bounded processing; +- publish the aggregation and differencing-risk policy. + +It MUST NOT: + +- receive packet IDs, replay tags, capabilities, user IDs, full routes, + connection logs, queue contents, or event-level timestamps; +- require universal read access to node logs or databases; +- instruct nodes to enable debug logging; +- be required for packet forwarding or consensus validity. + +Threats addressed: `TM-NET-03`, `TM-OPS-01`. + +### 7.11 Release system + +Release signing is outside every runtime role. Release keys MUST NOT exist on +mix, authority, entry, courier, storage, observer, relay, or general CI +workers. + +The release process MUST produce canonical signed metadata, artifact hashes, +version and compatibility information, rollback constraints, and provenance. +Composer and node update verification MUST fail closed. Emergency revocation +MUST use an authenticated path distinct from ordinary online administration. + +Threats addressed: `TM-SUPPLY-01`, `TM-ENDPOINT-01`, `TM-CRYPTO-01`, +`TM-CRYPTO-02`. + +### 7.12 `fog-bridge-*` + +An external bridge is never part of the core trust claim. It MUST be a +separate executable, identity, service descriptor, process, host policy, data +store, log policy, and threat-model appendix. + +It MUST NOT share a process with courier, storage, entry, or mix roles. Its +output is governed by the external protocol, and FOG cannot conceal metadata +that the external endpoint reveals. + +## 8. Trust-Boundary Interfaces + +Every interface MUST have one owning specification, exact framing, maximum +size, authentication rule, replay rule, timeout, failure class, and test +vectors. + +| ID | Interface | Producer -> consumer | Data class | Owning specification | +| --- | --- | --- | --- | --- | +| `IF-01` | Composer export | Composer -> transfer receiver -> relay | `OPAQUE-META` | `FOG-COMPOSER`, `FOG-SX` | +| `IF-02` | Composer import | relay or controlled medium -> Composer | `PUBLIC-AUTH`, `OPAQUE-META`, `CAPABILITY` | `FOG-COMPOSER` | +| `IF-03` | Descriptor upload | node -> authorities | `PUBLIC-AUTH` | `FOG-PKI`, `FOG-WIRE` | +| `IF-04` | Authority vote | authority -> authority | `PUBLIC-AUTH` | `FOG-PKI`, `FOG-WIRE` | +| `IF-05` | Consensus distribution | authorities or mirrors -> all roles | `PUBLIC-AUTH` | `FOG-PKI` | +| `IF-06` | Relay submission | relay -> entry | `OPAQUE-META` | `FOG-WIRE`, `FOG-SPHINX-PROFILES` | +| `IF-07` | Layer forwarding | entry/L1/L2/L3 -> next role | `OPAQUE-META` | `FOG-WIRE`, `FOG-SPHINX-PROFILES` | +| `IF-08` | Terminal request | L3 -> courier or service | `OPAQUE-META` | `FOG-WIRE`, `FOG-SPHINX-PROFILES`, service contract | +| `IF-09` | Replica operation | courier <-> replicas; replica <-> replica | `OPAQUE-META` | `FOG-STORAGE`, `FOG-WIRE` | +| `IF-10` | Anonymous reply | courier/service -> mix route -> entry/relay | `OPAQUE-META`, `CAPABILITY` | `FOG-SPHINX-PROFILES`, `FOG-WIRE` | +| `IF-11` | Aggregate submission | role -> observer | `AGGREGATE` | `FOG-OBSERVABILITY`, `FOG-WIRE` | +| `IF-12` | Signed update | release distribution -> role | `PUBLIC-AUTH` | `FOG-UPDATE` | +| `IF-13` | Local administration | operator -> one role | role-local | deployment profile | + +No generic RPC bus, shared event bus, shared SQL database, shared Redis, or +service mesh identity MAY span these trust boundaries in the core profile. + +## 9. Control-Plane Flows + +### 9.1 Node admission and descriptor publication + +1. An operator creates separate role and node identities using an offline or + controlled enrollment procedure. +2. The operator submits a signed admission request with role, family, layer, + address, public keys, supported profiles, and declared infrastructure. +3. Authorities validate policy and operator-family conflicts. +4. An admitted node creates an epoch descriptor and signs canonical bytes. +5. The node submits the descriptor independently to the authority set. +6. Authorities validate and include eligible descriptors in their votes. + +Admission does not imply health, honesty, independence, or permanent +eligibility. These are separately governed and measured. + +### 9.2 Consensus production + +1. Authorities exchange authenticated votes and commitments. +2. Each authority derives canonical topology and network parameters. +3. A consensus becomes valid only with the configured threshold of signatures + over identical canonical bytes. +4. Current and next key material overlap only for the specified window. +5. Authorities and mirrors publish signed votes, consensus, and revocations. + +FOG-PKI defines deterministic responses to split votes, missing quorum, clock +skew, stale epochs, rollback, freeze, and equivocation. Consumers MUST NOT +merge partial views locally. + +### 9.3 Consensus consumption by an offline Composer + +1. The relay obtains complete consensus from more than one retrieval path + where practical. +2. The relay passes the bytes and available consistency evidence through the + controlled import boundary. +3. The Composer performs canonical parsing, threshold signature validation, + epoch and rollback checks, profile checks, and trust-anchor checks. +4. Only the accepted consensus may drive routes, entry candidates, packet + profiles, and traffic parameters. +5. The Composer persists the highest accepted epoch before exporting work + based on it. + +An untrusted relay can withhold or replay data and cause denial of service, but +it MUST NOT be able to make a forged or rolled-back view valid. + +## 10. Outbound User Flow + +The logical outbound flow is: + +1. A Composer application module creates a bounded semantic message. +2. The Composer stages one authenticated message transition, encrypts the + message for the recipient, and atomically persists the advanced state with + the exact immutable envelope before that envelope becomes exportable. +3. The Composer atomically derives the required storage box and capability + transition, persists its immutable storage request generation, and only + then makes the operation exportable. +4. The Composer selects a consensus-valid terminal role and one node from each + mix layer, obeying operator-family constraints. +5. The Composer creates the KEMSphinx packet, reply material, and entry-bound + opaque submission variants. +6. The Composer exports a fixed-class work bundle over `IF-01`. +7. The relay validates only the outer work-bundle contract, queues it, and + selects one Composer-authorized entry variant under the current schedule. +8. The relay sends the opaque variant to the selected entry over `IF-06`. +9. The entry validates the entry capsule and forwards the bound packet to the + authorized layer-1 node. +10. Each mix performs exactly one transformation, delay, replay check, and + forwarding decision. +11. Layer 3 delivers the terminal packet to the selected courier or native + service. +12. The courier performs the bounded opaque operation and, where applicable, + contacts storage replicas through `IF-09`. +13. A result returns only through the supplied anonymous reply mechanism. + +At no point may the relay reconstruct the internal route, the entry change it, +or a mix interpret the user message. + +The exact choice between a complete packet per entry variant and a smaller +entry capsule bound to a shared packet remains an explicit protocol decision. +It MUST be resolved with packet-size, replay, fallback, and correlation +analysis before implementation. + +## 11. Retrieval and Reply Flow + +FOG does not require direct user-to-user sessions. Retrieval is a Composer- +constructed anonymous service operation. + +1. The relay obtains short-lived return-rendezvous material from eligible + entries or maintains an eligible live session under the active profile. +2. The relay transfers the opaque public or capability-bound rendezvous + material to the Composer through `IF-02`. +3. The Composer validates it against accepted consensus and binds a single-use + reply route to a retrieval or write operation. +4. The outbound operation traverses entry, all mix layers, and the courier. +5. The courier or service returns one fixed-class response using the supplied + single-use reply material. +6. The reply traverses the consensus-authorized mix route and terminates at a + bounded entry rendezvous or eligible live relay session. +7. The relay stores only the opaque response until expiry or controlled + Composer import. +8. The Composer stages authentication, decryption, deduplication, and message + state advancement, then atomically commits all effects before rendering + content or scheduling an acknowledgment. + +An entry return rendezvous MUST be random, short-lived, bounded, and unrelated +to a stable global username. It MUST NOT become a durable provider mailbox. +Exact queueing, polling, retransmission, acknowledgment, and SURB behavior is +defined by the packet, wire, messaging, and storage specifications. + +Empty reads, hits, misses, replies, retries, and acknowledgments MUST fit the +same declared external traffic classes. A relay MUST continue its configured +cover and retrieval schedule independently of whether the Composer has a real +operation pending. + +## 12. Cover and Loop Traffic Architecture + +Cover generation is a protocol subsystem, not an optional application feature. + +- The Composer prepares user-protocol decoys when they require message or + storage semantics unavailable to the relay. +- The blind relay generates network cover and loop traffic using only public + consensus and ephemeral local state. +- Entries process cover submissions identically to real submissions. +- Mixes generate and process consensus-authorized loop or decoy packets. +- Couriers, services, and replicas implement indistinguishable bounded cover + outcomes defined by their protocol. +- Nodes export only delayed aggregate loss and health measurements. + +The simulator determines cover rate, destination selection, delay, loop, +retry, polling, and shutdown parameters. Operators MUST NOT tune privacy- +critical distributions independently outside an authenticated profile. + +`FOG-SIMULATION.md` records the first deterministic comparison matrix. It +confirms that sparse use and the functional PoC do not support anonymity +claims, and that cover volume, local pool overlap, latency, and long-term +observation proxies trade different resources. It deliberately selects no +numeric profile. Loop, polling, retry, queue, degraded-mode, formal observer, +and trace-driven extensions remain required before parameters are frozen. + +If the minimum traffic or cover conditions attached to a claim disappear, +nodes follow the specified degraded-mode or shutdown policy. They MUST NOT +silently continue under the stronger claim. + +## 13. State Ownership + +| Role | Permitted durable state | Forbidden durable state | +| --- | --- | --- | +| Composer | encrypted identity, contacts, message state, capabilities, drafts, trust anchors, highest epoch | online session credentials for other roles, node private keys | +| Relay | encrypted opaque queues, schedule state, temporary entries, return handles, highest relay-checked epoch | plaintext, contact state, mailbox capabilities, internal routes | +| Authority | descriptors, votes, consensus history, admission and revocation records | user packets, capabilities, data-plane queues | +| Entry | node configuration, bounded replay-independent ingress state, short-lived opaque return state | durable user accounts, plaintext, internal routes, mailboxes | +| Mix | epoch keys, replay state, bounded delay queues, local aggregate counters | message state, application data, full routes, user identities | +| Courier | bounded deduplication, retry and reply state | durable mailboxes, user identity state, plaintext | +| Store | opaque records, tombstones, expiry and replica state | user identities, plaintext, courier dedup state | +| Service | explicitly specified minimum opaque service state | unrelated application state, client network identity | +| Observer | delayed aggregate submissions and published aggregates | event streams, packet identifiers, role secrets | + +State schemas MUST include version, ownership, integrity, maximum size, +retention, migration, backup, restore, corruption, and deletion behavior. +Copying a database between roles or replicas is not a recovery mechanism. + +## 14. Key Ownership + +Each narrower protocol specification MUST refine this table into exact key +lifecycle entries. + +| Key class | Sole owning role | Authorized use | Baseline persistence | +| --- | --- | --- | --- | +| Pairwise contact roots and handshake identities | Composer | private contact authentication and session establishment | long-term encrypted user state, with profile-specific recovery rules | +| Ratchet, outbox, deduplication, and conversation state | Composer | atomic message send, retry, receive, and acknowledgment transitions | encrypted mutable user state, non-resumable after stale restore | +| Directional stream capability and storage outbox state | Composer | rotating box derivation, immutable request generations, reads, writes, tombstones, and receipt validation | encrypted mutable user state, non-resumable after stale restore | +| Composer state-encryption key | Composer | local authenticated encryption | profile-specific protected storage | +| Backup or recovery key | user offline recovery domain | restore Composer state | separate from backup ciphertext | +| Relay queue-sealing key | blind relay | local opaque queue protection | local service-protected storage | +| Relay ephemeral cover state | blind relay | cover and loop construction | bounded or ephemeral | +| Node identity key | one online node | descriptor and role authentication | role-local protected storage | +| Noise link key | one online node | fixed adjacent-link profile | role-local, rotated by profile | +| Entry capsule key | one entry | open entry-bound submission capsule | epoch-bounded role-local state | +| Entry return KEMSphinx key | one entry | exactly one terminal reply unwrap for a short-lived rendezvous | epoch-bounded role-local state | +| Mix KEMSphinx key | one mix | exactly one layer transformation | epoch-bounded role-local state | +| Terminal KEMSphinx key | one courier or service | open terminal service envelope | epoch-bounded role-local state | +| Replica envelope key | one storage replica | replica request and response protection | epoch or storage-profile bounded | +| Replica receipt key | one storage replica | authenticate local durable storage results to the Composer | storage-manifest bounded, separate from envelope, identity, and Noise keys | +| Storage-at-rest key | one replica operator | defense-in-depth disk or database encryption | deployment-specific | +| Authority identity key | one authority | authority identity and authenticated recovery | preferably offline or hardware-protected | +| Authority online vote key | one authority | epoch vote and consensus participation | short-lived or tightly controlled online state | +| Authority wire key | one authority wire service | authenticated descriptor and authority-peer transport | root-certified bounded online state | +| Aggregate signing key | one reporting role | authenticate coarse metrics | role-local | +| Release signing key | offline release domain | sign canonical releases and metadata | offline, never on runtime hosts | + +Public consensus contains only public keys and authenticated parameters. +Private keys MUST NOT be copied through consensus, container images, shared +volumes, environment templates, logs, or support artifacts. + +No role may use a long-term user identity key as a network account, storage +capability, node identity, transport key, release key, or backup key. + +## 15. Network Reachability Policy + +Default-deny reachability is part of the architecture. + +| Source | Permitted destinations | Explicitly forbidden destinations | +| --- | --- | --- | +| Composer | none | every network target | +| Relay | consensus mirrors, eligible entries, explicitly configured local transfer interface | mixes, courier, storage, external bridges, Internet proxy targets | +| Authority | peer authorities, descriptor submitters, consensus publication endpoints, local administration | user data plane, storage records | +| Entry | eligible relays, layer-1 nodes, bounded reply-route peers required by profile | layer 2, layer 3, storage, public Internet | +| Layer 1 mix | entries and layer-2 nodes | relay, layer 3, storage, public Internet | +| Layer 2 mix | layer-1 and layer-3 nodes | relay, entry, storage, public Internet | +| Layer 3 mix | layer-2 and terminal courier/service nodes | relay, entry, storage, public Internet | +| Courier | layer-3 or authorized reply-route mix peers, eligible replicas | relay, Composer, public Internet | +| Store | eligible couriers and replica peers | Composer, relay, entry, ordinary mixes, public Internet | +| Observer | aggregate-reporting roles and publication endpoint | packet interfaces, role databases | +| Bridge | explicitly declared FOG terminal and external endpoint | all undeclared core interfaces | + +The reply protocol MAY require a terminal role to connect to a +consensus-authorized first reply hop that differs from the forward table. That +exception MUST be explicit in `FOG-SPHINX-PROFILES`, constrained by role and +epoch, and tested as part of the firewall policy. + +Management access is a separate operator-local boundary. No central +management credential may administer all authorities, all mix layers, the +courier, and the replica set in a claim-bearing deployment. + +## 16. Co-Location and Operator Separation + +### 16.1 Functional PoC + +The local PoC MAY place separate containers on one host and MAY use simulated +authority or storage fixtures. It MUST still use separate executables, service +users, state paths, key files, ports, and default-deny container networks. + +PoC co-location invalidates operator-independence, infrastructure-diversity, +and production anonymity claims. + +`FOG-LOCAL-POC.md` and `../deploy/podman/topology.json` refine this into the +first deployment contract. The Composer and authority fixtures are +networkless. Relay, entry, one mix in each fixed layer, courier, and four +storage fixtures use only internal pairwise networks for `IF-06` through +`IF-09`. The contract forbids host networking, published ports, shared +writable volumes, shared secret scopes, and missing containment controls. No +Compose manifest is generated until role fixtures exist. + +### 16.2 Alpha + +An alpha profile MUST use: + +- at least three authorities with a 2-of-3 quorum; +- three mix layers with at least two independently operated mixes per layer; +- at least four independently operated storage replicas; +- separate entry, mix, courier, and storage role identities; +- documented operator-family, provider, ASN, location, and administrative + relationships; +- no route containing the same operator family twice. + +One operator MAY run more than one role only when the relationship is declared, +route policy accounts for it, and the relevant security claim explicitly +allows it. One process or private key MUST NOT implement more than one role. + +### 16.3 Claim-bearing production profile + +The preferred initial topology is three mix nodes per layer. Authority, +entry, mix, courier, storage, observer, build, and release administration +SHOULD be organizationally separated where practical. + +No provider account, orchestration control plane, backup service, monitoring +credential, or CI system SHOULD control enough nominal operators to defeat the +claim. Concealed common control remains a documented residual risk. + +## 17. Failure and Degraded Modes + +| Condition | Required architectural response | +| --- | --- | +| Consensus signature or canonical parse failure | reject and retain last safely usable state only within its validity | +| Stale or rolled-back consensus | stop new route construction; require authenticated recovery | +| Authority quorum failure | no locally synthesized consensus; continue only under explicitly valid prior epoch rules | +| Entry failure | choose only another pre-authorized opaque variant under randomized bounded retry policy | +| Mix connection failure | bounded backoff; no layer skip or deterministic emergency route | +| Replay state unavailable | stop affected mix processing until safe recovery or specified fresh epoch | +| Cover process failure | leave the affected unobservability profile and follow explicit shutdown/degraded policy | +| Queue saturation | bounded shedding without detailed remote oracle or amplification | +| Courier or replica timeout | bounded randomized retry through the defined protocol; no direct client fallback | +| Replica loss | follow specified quorum or erasure behavior; never fabricate successful durability | +| Clock uncertainty outside bound | reject time-sensitive new state and surface coarse local fault | +| Update verification failure | retain last verified non-revoked version or stop if policy requires it | +| State corruption | quarantine state, avoid secret-bearing diagnostics, and use authenticated recovery | + +Detailed error codes may exist inside a trusted local process boundary for +testing. Remote peers receive only bounded protocol outcomes that do not expose +parsing, capability, record-existence, or route oracles. + +## 18. Configuration, Build, and Update Boundaries + +Configuration MUST be role-specific and schema-validated. A deployment tool +MAY generate multiple role configurations, but the generated runtime artifacts +MUST contain only the public information and secrets required by that role. + +Runtime services MUST use: + +- a dedicated unprivileged account; +- a role-specific read-only executable and configuration; +- one role-specific writable state directory; +- no shared writable source or configuration checkout; +- explicit network allowlists; +- bounded resource controls; +- disabled core dumps and privacy-unsafe debug modes; +- local secret injection that does not place secrets in command arguments or + container images. + +Build, release signing, consensus signing, node operation, and user identity +management are separate authorities. A successful CI build does not authorize +a release, a release does not authorize a network epoch, and an epoch does not +authorize user messages. + +### 18.1 Cost-minimal build and packaging profile + +One release build MUST compile all required role executables once per supported +operating-system and architecture target. Nodes and operators MUST reuse those +verified artifacts; they MUST NOT require per-node or per-operator compilation. +The build SHOULD share dependency, object, and module caches across role +executables. + +The functional fixture and local alpha laboratory MAY use one immutable +digest-pinned image containing multiple fixture executables, provided every +container activates exactly one role and preserves separate configuration, +service identity, state, secrets, and network policy. This packaging exception +does not permit a multi-role process. + +A distributed alpha or claim-bearing deployment SHOULD assemble one minimal +image per runtime role from the already compiled artifact set. Role images +SHOULD share identical OCI base layers so registries and hosts deduplicate +storage and transfer. Image assembly MUST NOT trigger a separate compilation +for every image or node. + +The supported build matrix MUST contain only platforms required by current +deployment or evidence gates. Independent reproducible rebuilds are release +verification evidence and MAY reuse the same source and pinned toolchain; they +are not required for routine startup of each node. + +## 19. Repository and Module Shape + +The future implementation repository SHOULD begin with this logical layout. +Exact language-specific names MAY vary without changing dependency direction. + +```text +fog/ + cmd/ + fog-compose/ + fog-sx-send/ + fog-sx-receive/ + fog-client-relay/ + fog-authority/ + fog-entry/ + fog-mix/ + fog-courier/ + fog-store/ + fog-observer/ + protocol/ + encoding/ + pki/ + wire/ + sphinx/ + messaging/ + storage/ + composer/ + sx/ + roles/ + authority/ + entry/ + mix/ + courier/ + store/ + observer/ + clientrelay/ + sdk/ + client/ + composer/ + sim/ + specs/ + testvectors/ + tests/ + conformance/ + integration/ + fault/ + topology/ + deploy/ + podman/ + ops/ +``` + +Dependency direction MUST be: + +```text +cmd -> one role -> protocol packages +sdk ------------> protocol packages +sim ------------> protocol models and independent simulation models +tests ----------> public contracts and built executables +``` + +Protocol packages MUST NOT import role implementations, network listeners, +databases, container tooling, or operator configuration. One role package MUST +NOT import another role's implementation. Cross-role behavior occurs only +through versioned protocol interfaces. + +The public SDK MUST expose message and packet construction contracts without +exposing node private APIs. Generic `utils`, shared mutable singletons, a +universal database package, and an in-process plugin bus are forbidden +substitutes for explicit boundaries. + +The following MUST NOT be split yet: + +- separate source repositories for every role; +- server daemons for `fog-drop`, `fog-mailbox`, and `fog-im`; +- a generic external bridge framework inside core nodes; +- operator-selectable cryptographic plugin systems; +- separate databases or queues where a role currently needs no durable state. + +## 20. Architectural Verification + +Before the local PoC, the repository MUST support or define tests for: + +- one executable activating only one role; +- dependency rules preventing role-to-role implementation imports; +- configuration schema rejection of multi-role or unknown-critical settings; +- default-deny reachability for every row in the network policy table; +- inability of Composer images to create or receive network traffic; +- absence of shared writable volumes and private key files across roles; +- fixed packet-class behavior across all native applications; +- complete traversal of entry and all mix layers; +- replay-state loss and fail-closed restart behavior; +- voucher consume-once behavior and atomic ratchet, outbox, deduplication, + inbox, and acknowledgment transactions; +- exact message-envelope and storage-request retry layering: immutable + courier-envelope bytes per request generation with fresh KEMSphinx, route, + entry, rendezvous, and reply material per transmission; +- deterministic storage shards, disjoint intermediates, receipt quorum, + empty-read non-advancement, tombstone precedence, and non-resurrection; +- exact Composer bundle headers, hostile import limits, commit-before-effect, + external-anchor reconciliation, non-resumable recovery, and update floors; +- exact FOG-SX object and frame headers, fixed profile tuple, size padding, + CRC domains, FEC and conflict budgets, no-ACK behavior, and physical + direction fixtures; +- consensus stale, rollback, split, and equivocation scenarios; +- queue, parser, retry, CPU, memory, connection, and storage bounds; +- log and metrics schemas that reject prohibited data classes; +- compromise fixtures showing the information available to each isolated role; +- update signature, version, rollback, and revocation behavior; +- deterministic conformance vectors for every trust-boundary interface. + +Tests MUST observe public contracts and externally visible behavior, not reach +through trust boundaries to share internal state. + +## 21. Threat Traceability + +| Threat | Primary architectural controls | +| --- | --- | +| `TM-NET-01` | uniform packet classes, relay scheduling, cover subsystem, stratified mixes | +| `TM-NET-02` | independent retrieval schedule, rotating capabilities, bounded rendezvous, storage separation | +| `TM-NET-03` | loop health, degraded-mode gate, bounded retries, no bypass | +| `TM-NET-04` | per-mix replay state, single-use reply material, idempotent terminal state | +| `TM-NET-05` | authenticated KEMSphinx processing, terminal validation, uniform failures | +| `TM-NET-06` | complete consensus, fixed profiles, application-independent core behavior | +| `TM-PKI-01` | permissioned admission, family declarations, route constraints | +| `TM-PKI-02` | threshold consensus, monotonic Composer state, consistency evidence | +| `TM-PKI-03` | distinct authorities, quorum rules, offline recovery, deterministic failure | +| `TM-ROLE-01` | blind relay, entry capsules, Composer route authority, no direct fallback | +| `TM-ROLE-02` | one fixed mix layer per process, family-separated routes, role-local keys | +| `TM-ROLE-03` | courier/replica separation, opaque commands, independent replicas | +| `TM-ENDPOINT-01` | networkless Composer, narrow devices, encrypted state, signed updates | +| `TM-ENDPOINT-02` | minimal transfer adapter, bounded formats, no reverse FOG-SX path | +| `TM-ENDPOINT-03` | role-owned state, separate backup keys, no database copying | +| `TM-APP-01` | Composer-side application state, bounded authenticated rendering, no active content | +| `TM-OPS-01` | separate observer, aggregate-only interface, prohibited data classes | +| `TM-SUPPLY-01` | offline release authority, provenance, signed updates, role-specific artifacts | +| `TM-CRYPTO-01` | sole key owners, purpose separation, lifecycle refinement requirement | +| `TM-CRYPTO-02` | consensus-authorized suites, no plugin negotiation or downgrade | +| `TM-AVAIL-01` | bounds at every interface, non-amplification, quotas, explicit degraded modes | + +Every later specification MUST reference the applicable threat IDs and +architecture invariants. If it changes a trust boundary or information flow, +this document and the threat model MUST be updated before implementation. + +## 22. Open Architectural Protocol Contracts + +This architecture intentionally leaves the following to narrower reviewed +specifications: + +- the exact entry submission capsule and packet binding; +- short-lived return rendezvous and offline Composer import mechanics; +- activation of an exact numeric message profile after the structural + `FOG-MESSAGING` contract and its non-active PQXDH, Triple Ratchet, and + ML-KEM Braid candidate pass byte-exact integration, vectors, implementation + review, and independent review; +- the exact reviewed entry and mutual Noise or PQNoise suites and numeric + wire-profile parameters within `FOG-WIRE-1`; X-Wing is the leading KEM to + evaluate, not an active profile; +- activation of a final `FOG-SPHINX-1` primitive suite after the fixed + structural profile and calculated candidate in `FOG-SPHINX-PROFILES.md` pass + benchmarks, vectors, simulation, and independent review; +- activation of an exact numeric storage profile after the structural + `FOG-STORAGE` contract and its non-active narrow BACAP/Pigeonhole candidate + pass primitive review, receipt analysis, geometry, vectors, simulation, and + independent review; +- activation of exact Composer MicroVM and Portable profiles after the + structural `FOG-COMPOSER` contract and its Linux vault, Qubes, Portable, and + TUF candidates pass platform, anchor, fault, recovery, and independent + review; +- cover, loop, delay, retry, retrieval, and shutdown distributions; +- activation of one exact numeric FOG-SX joint profile after the structural + `FOG-SX` contract and its non-active RaptorQ, Reed-Solomon, QR, Lightpipe, + and MIDI candidates pass implementation, license and IPR, resource, vector, + hardware-direction, and independent review; +- exact aggregate metrics and suppression thresholds; +- implementation languages, reviewed libraries, and activation evidence for + the non-active cryptographic candidates in `FOG-CRYPTO-SUITES.md`. + +These are explicit design dependencies. No daemon implementation may resolve +them through undocumented behavior. + +## 23. References + +- FOG threat model: `FOG-THREAT-MODEL.md` +- FOG public key infrastructure: `FOG-PKI.md` +- FOG wire protocol: `FOG-WIRE.md` +- FOG Sphinx profile framework: `FOG-SPHINX-PROFILES.md` +- FOG messaging protocol: `FOG-MESSAGING.md` +- FOG storage protocol: `FOG-STORAGE.md` +- FOG Composer protocol: `FOG-COMPOSER.md` +- FOG simplex transfer protocol: `FOG-SX.md` +- FOG observability protocol: `FOG-OBSERVABILITY.md` +- FOG cryptographic suite evaluation: `FOG-CRYPTO-SUITES.md` +- FOG traffic and topology simulation: `FOG-SIMULATION.md` +- FOG local Podman PoC: `FOG-LOCAL-POC.md` +- Katzenpost mix network specification: + <https://katzenpost.network/docs/specs/mixnet/> +- Katzenpost public key infrastructure specification: + <https://katzenpost.network/docs/specs/pki/> +- Katzenpost wire protocol specification: + <https://katzenpost.network/docs/specs/wireprotocol/> +- Katzenpost Pigeonhole protocol specification: + <https://katzenpost.network/docs/specs/pigeonhole/> +- Piotrowska et al., *The Loopix Anonymity System*: + <https://www.usenix.org/conference/usenixsecurity17/technical-sessions/presentation/piotrowska> +- Infeld et al., *Echomix: a Strong Anonymity System with Messaging*: + <https://arxiv.org/abs/2501.02933> +- Noise Protocol Framework: + <https://noiseprotocol.org/noise.html> + +These references inform role separation, stratified mixing, consensus, +transport, reply, and storage boundaries. FOG requires its own protocol +profiles, conformance evidence, simulation, deployment evidence, and review. |