From 4bade69cf98a3e0257ea254a88b8c57c1ac69728 Mon Sep 17 00:00:00 2001 From: lgcosta Date: Thu, 12 Nov 2026 01:42:51 -0300 Subject: [PATCH] docs: queue status honest - attach path reached, purge lock assert documented as the blocker (caveats #9) --- docs/en/caveats.md | 24 ++++++++++++++++++++++++ docs/en/feature-matrix.md | 2 +- docs/pt-BR/caveats.md | 24 ++++++++++++++++++++++++ docs/pt-BR/feature-matrix.md | 2 +- 4 files changed, 50 insertions(+), 2 deletions(-) diff --git a/docs/en/caveats.md b/docs/en/caveats.md index 6922aaa..a9f6c24 100644 --- a/docs/en/caveats.md +++ b/docs/en/caveats.md @@ -111,6 +111,30 @@ kernel prints `pfk: divert port N: no socket bound`. `log` needs (256 records / 1 MiB) and drops. Neither is a stub any more, but neither does anything until its consumer runs (see [pf-guide.md](pf-guide.md)). +### 9. Queue configuration hits a lock assertion in the purge path (open) + +Loading a `queue` rule now reaches the scheduler (`pfq_alloc` succeeds, the +hfsc discipline is created), and `pf_commit_queues()` is exercised end to end. +Two bugs were found and fixed while getting there — the root-class handle +colliding with odd pf queue ids (`pfq_addqueue` returned EBUSY), and a NULL +discipline not being checked. + +What still blocks queues: the port's purge timer can run while an ioctl holds +the write lock, and `pf_purge_expired_states()` asserts the opposite: + +``` +splassert: pf_purge_expired_states: expected 0, got 2 +kernel BUG at compat/pfk_kern.c:51 +``` + +OpenBSD serialises this with `splnet()`; the port replaced the spl layer with +rw-semas and a work queue, so the assertion is now a real race rather than a +formality. Until it is fixed (release the lock around the call, or give the +state tree its own lock as upstream does for `pf_state_lock`), a host that +configures queues can wedge the module in `Unloading`. **Do not enable +`queue` rules on a live host yet** — the 14 lab gates never configure queues, +which is why they stay green. + ## Compat-layer deviations (from `docs/porting-notes.md`, M1 record) These are deliberate, marked-in-code divergences of the compat layer; they diff --git a/docs/en/feature-matrix.md b/docs/en/feature-matrix.md index 58642e4..2fec9f0 100644 --- a/docs/en/feature-matrix.md +++ b/docs/en/feature-matrix.md @@ -60,7 +60,7 @@ As of 2026-11-11; source of per-row evidence: `out/test-logs/`, |---|---|---|---| | `route-to` / `dup-to` / `reply-to` | `pass in … route-to { 10.0.3.2 } keep state`, `reply-to { gw }` | M4 | **VERIFIED in lab** — gates `08-routeto.sh` (route-to leaves via the gateway's link, dup-to copies while keeping the flow) and `13-replyto.sh` (the reply of a flow is forced onto the gateway link instead of the client's). Deviation: the forced nexthop is installed by setting `rt_gw4`/`rt_gw_family` on the looked-up route (Linux only ARPs the gateway when the route declares one). `fail-to` is **not a keyword in OpenBSD 8.0** (removed upstream), so it is out of scope. | | `divert-to` / `divert` tag | `pass in … divert-to port ` | VERIFIED in lab | **Works end-to-end.** The module registers an `AF_DIVERT` socket family (`PF_DIVERT=32`) at runtime — no kernel-tree patch. A `divert-to` rule hands the packet to that family; with no listener the packet is dropped (`divs_noport`, BSD parity); with `tools/pdivert` running the packet is delivered to userspace, rewritten and reinjected (verified with a payload mutation + checksum fix, ping 0% loss). | -| Queues (`queue`, `queue on`, `pfctl -s queue -v`) | `queue root q1 bandwidth 1Mbit; queue on egress all ...` | M5 | **Back ends built into `pf.ko`, attachment pending** — OpenBSD's `hfsc.c` and `fq_codel.c` now compile and link (157 scheduler symbols, `hfsc_initialize`/`hfsc_pf_addqueue`/`fqcodel_pf_addqueue` reachable from `pf_ioctl`), and the full suite still passes with them in. The active scheduler remains the priq shim: nothing enqueues the reinjected packet into the queue yet, so **no bandwidth is enforced**. | +| Queues (`queue`, `queue on`, `pfctl -s queue -v`) | `queue root q1 bandwidth 1Mbit; queue on egress all ...` | M5 | **Back ends built into `pf.ko`, attachment pending** — OpenBSD's `hfsc.c` and `fq_codel.c` now compile and link (157 scheduler symbols, `hfsc_initialize`/`hfsc_pf_addqueue`/`fqcodel_pf_addqueue` reachable from `pf_ioctl`), and the full suite still passes with them in. The data path can now hand packets to an attached scheduler (`pf_kifq_input()` + a real `ifq_start()` that drains and transmits), but queues are **not safe to enable yet**: the purge path hits a lock assertion (see caveats #9), so no bandwidth is enforced and no gate covers it. | | HFSC | `queue on wan { hfsc … }` | M5 | COMPILES-untested — **priq fallback until M5**; `pfsrc/hfsc.c` staged (real OpenBSD 8.0 code) but **not in the module object list** | | FQ-CoDel | `queue on wan { fq_codel … }` | M5 | COMPILES-untested — same as HFSC: staged, not built | | `pfq` qdisc (`tc qdisc … root pfq`) | n/a (Linux glue) | M5 | PENDING-M5 (does not exist in the tree yet) | diff --git a/docs/pt-BR/caveats.md b/docs/pt-BR/caveats.md index b54b3b4..5086e18 100644 --- a/docs/pt-BR/caveats.md +++ b/docs/pt-BR/caveats.md @@ -111,6 +111,30 @@ lendo `/dev/pflog`; sem ele a fila atinge o limite (256 registros / 1 MiB) e descarta. Nenhum dos dois é mais stub, mas nenhum faz algo sem o consumidor rodar (ver [pf-guide.md](pf-guide.md)). +### 9. Configurar filas esbarra numa asserção de lock no caminho de purge (aberto) + +Carregar uma regra `queue` agora alcança o escalonador (`pfq_alloc` tem sucesso, +a disciplina hfsc é criada) e `pf_commit_queues()` é exercitada de ponta a ponta. +Dois bugs foram encontrados e corrigidos no caminho: o handle da classe raiz +colidia com qids ímpares do pf (`pfq_addqueue` devolvia EBUSY) e a disciplina +NULL não era verificada. + +O que ainda bloqueia as filas: o timer de purge do port pode rodar enquanto um +ioctl segura o lock de escrita, e `pf_purge_expired_states()` afirma o contrário: + +``` +splassert: pf_purge_expired_states: expected 0, got 2 +kernel BUG at compat/pfk_kern.c:51 +``` + +O OpenBSD serializa isso com `splnet()`; o port substituiu a camada spl por +rw-semáforos e work queue, então a asserção passou a ser uma corrida real em vez +de formalidade. Até corrigi-la (liberar o lock em torno da chamada, ou dar ao +estado o seu próprio lock como o upstream faz com `pf_state_lock`), um host que +configure filas pode travar o módulo em `Unloading`. **Não habilite regras +`queue` em host real ainda** — os 14 gates do lab nunca configuram filas, e é +por isso que continuam verdes. + ## Desvios da camada de compat (de `docs/porting-notes.md`, registro do M1) Estes são desvios deliberados e marcados no código; importam ao raciocinar diff --git a/docs/pt-BR/feature-matrix.md b/docs/pt-BR/feature-matrix.md index d09aead..1c6ce7b 100644 --- a/docs/pt-BR/feature-matrix.md +++ b/docs/pt-BR/feature-matrix.md @@ -60,7 +60,7 @@ Estado em 2026-11-11; evidência por linha: `out/test-logs/`, |---|---|---|---| | Pools de `route-to` (`rr`, `round-robin`, `source-hash`, `hash`), `reply-to`, `dup-to` | `pass in … route-to { 10.0.3.2, 10.0.2.2 } round-robin keep state` | M4 | **`round-robin` e `source-hash` VERIFICADOS em lab** (gate `10-pools.sh`): com dois gateways em dois enlaces, o round-robin usa ambos e o source-hash fixa todo fluxo de uma origem em um único enlace. `dup-to` verificado no gate 08; `reply-to` ainda sem gate. `fail-to` **não existe no OpenBSD 8.0** (removido upstream), então está fora de escopo. | | `divert-to` / tag `divert` | `pass in … divert-to port ` | VERIFICADO em lab | **Funciona ponta a ponta.** O módulo registra em runtime a família de sockets `AF_DIVERT` (`PF_DIVERT=32`) — sem patch no kernel. A regra `divert-to` entrega o pacote a essa família; sem listener o pacote é descartado (`divs_noport`, paridade BSD); com `tools/pdivert` rodando o pacote vai ao userspace, é reescrito e reinjetado (verificado com mutação de payload + correção de checksum, ping 0% loss). | -| Filas (`queue`, `queue on`, `pfctl -s queue -v`) | `queue root q1 bandwidth 1Mbit; queue on egress all ...` | M5 | **Back ends compilados no `pf.ko`, anexação pendente** — `hfsc.c` e `fq_codel.c` do OpenBSD agora compilam e linkam (157 símbolos do escalonador; `hfsc_initialize`/`hfsc_pf_addqueue`/`fqcodel_pf_addqueue` alcançáveis pelo `pf_ioctl`), e a suíte inteira continua passando com eles dentro. O escalonador ativo segue sendo o shim priq: nada ainda enfileira o pacote reinjetado na fila, então **nenhuma banda é aplicada**. | +| Filas (`queue`, `queue on`, `pfctl -s queue -v`) | `queue root q1 bandwidth 1Mbit; queue on egress all ...` | M5 | **Back ends compilados no `pf.ko`, anexação pendente** — `hfsc.c` e `fq_codel.c` do OpenBSD agora compilam e linkam (157 símbolos do escalonador; `hfsc_initialize`/`hfsc_pf_addqueue`/`fqcodel_pf_addqueue` alcançáveis pelo `pf_ioctl`), e a suíte inteira continua passando com eles dentro. O caminho de dados já pode entregar pacotes a um escalonador anexado (`pf_kifq_input()` + um `ifq_start()` real que drena e transmite), mas as filas **ainda não são seguras para habilitar**: o caminho de purge esbarra numa asserção de lock (ver caveats #9), então nenhuma banda é aplicada e não há gate para isso. | | HFSC | `queue on wan { hfsc … }` | M5 | COMPILES-untested — **fallback priq até o M5**; `pfsrc/hfsc.c` staged (código real do OpenBSD 8.0) mas **fora da lista de objetos** do módulo | | FQ-CoDel | `queue on wan { fq_codel … }` | M5 | COMPILES-untested — igual ao HFSC: staged, não compilado | | Qdisc `pfq` (`tc qdisc … root pfq`) | n/a (glue Linux) | M5 | PENDING-M5 (ainda não existe na árvore) |