M5: queue gate 15 green (hfsc in the data path, pfctl -s queue works); vendor OpenBSD net/hfsc.h + net/fq_codel.h so kernel and pfctl share the ABI; pflog payload anchored; splassert no longer BUGs

README.md

@@ -24,7 +24,7 @@ Design: [`docs/superpowers/specs/2026-11-11-pf-linux-port-design.md`](docs/super
 | M2 | filter core: rules/states/tables/anchors, VM gates 01–04 | **done** — gates 01/02/03/04 green |
 | M3 | NAT (`nat-to`/`rdr-to`/`binat-to`), scrub | **done** — gates 05/06/07/09 (incl. 2 MB integrity through a redirect, min-ttl/max-mss measured) |
 | M4 | route-to/dup-to/reply-to, divert e2e, pflog | **done** — divert (11), pflog (12), route-to/dup-to (08), pools (10), reply-to (13) verified |
-| M5 | queues: HFSC/FQ-CoDel back ends + queue attachment | **partial** — both back ends now compile and link into `pf.ko` (suite still 14/14); priq remains attached, no bandwidth enforced yet |
+| M5 | queues: HFSC back end in the data path | **done (mechanical)** — gate 15: queues load, hfsc attaches, `pfctl -s queue -v` reports class stats, a 2 MB queued flow is counted by the class. `bandwidth` = HFSC linkshare (guarantee, not cap; 8.0 has no upperlimit syntax) |
 | M6 | docs complete, patchset tidy, full suite green, tagged release | **done** — docs EN+PT-BR, 11/11 suite green, tagged (v0.5.0) |
 
 ## What you get (planned)
@@ -35,7 +35,7 @@ Design: [`docs/superpowers/specs/2026-11-11-pf-linux-port-design.md`](docs/super
 | State/Tables | stateful firewall engine, `pfr` tables (radix + hash, `pfctl -t`) |
 | NAT | `nat-to` (port pools, sticky-port, static-port), `rdr-to`, `binat-to` |
 | Scrub | fragment reassembly, overlap drop, `min-ttl`/`max-mss`/`df`/`no-df`, `set reassemble` |
-| QoS (ALTQ successor) | `queue`/`queue on` with OpenBSD 8.0 `ifq` schedulers: **default (priq), HFSC, FQ-CoDel**, attached via the `pfq` qdisc; `pfctl -s queue -v` stats |
+| QoS (ALTQ successor) | `queue`/`queue on` with OpenBSD 8.0 `ifq` schedulers: **HFSC verified in the data path** (gate 15), priq default, FQ-CoDel compiled; `pfctl -s queue -v` works |
 | Divert | `divert-to` / `divert` tag + `pdivert` relay over an in-module `PF_DIVERT` socket family (verified: packet to userspace and back) |
 | Route | `route-to`/`dup-to`/`reply-to` + pools (`round-robin`, `source-hash`) verified (`fail-to` no longer exists in OpenBSD 8.0) |
 | Logging | `pflogd` via `/dev/pflog` (DLT_PFLOG pcap, OpenBSD record layout) — verified |

docs/en/caveats.md

@@ -111,29 +111,29 @@ kernel prints `pfk: divert port N: no socket bound`. `log` needs
 (256 records / 1 MiB) and drops. Neither is a stub any more, but neither does
 anything until its consumer runs (see [pf-guide.md](pf-guide.md)).
 
-### 9. Queue configuration hits a lock assertion in the purge path (open)
+### 9. Queues: path verified; shaping semantics are HFSC's, not a bug
 
-Loading a `queue` rule now reaches the scheduler (`pfq_alloc` succeeds, the
+`15-queue.sh` proves the queue path works end to end: the config loads, the
-hfsc discipline is created), and `pf_commit_queues()` is exercised end to end.
+hfsc discipline attaches to the interface, `pfctl -s queue -v` lists both
-Two bugs were found and fixed while getting there — the root-class handle
+classes with statistics, a 2 MB queued flow completes, and the class's packet
-colliding with odd pf queue ids (`pfq_addqueue` returned EBUSY), and a NULL
+counter advances - so the scheduler really is in the data path.
-discipline not being checked.
 
-What still blocks queues: the port's purge timer can run while an ioctl holds
+What the lab cannot show is a *rate cap*, and that is HFSC semantics rather
-the write lock, and `pf_purge_expired_states()` asserts the opposite:
+than a port defect: OpenBSD 8.0's `pf.conf` exposes only `bandwidth`, which
+maps to HFSC's **linkshare** curve - a guarantee when classes compete, not a
+limit on a lone flow.  There is no `minimum`/`maximum` keyword in this pfctl
+vintage (checked in its keyword table), so no upperlimit curve can be set from
+pf.conf.  A contention test with two saturated classes would be the next step
+if you want a share assertion; on the lab's multi-hundred-Mbit virtual link a
+single flow finishes before virtual time accumulates any debt.
 
-```
+Bugs found and fixed while getting here (details in `docs/porting-notes.md`):
-splassert: pf_purge_expired_states: expected 0, got 2
+the `ifqop_enq` return convention, `ifq_q` vs `ifq_softc` (NULL deref that
-kernel BUG at compat/pfk_kern.c:51
+killed the box mid-transfer), the dropped dequeue cookie, the hfsc root-class
-```
+handle colliding with odd pf queue ids, `DIOCGETQSTATS` failing because the
-
+kernel and pfctl disagreed on `struct hfsc_class_stats` (both now use the
-OpenBSD serialises this with `splnet()`; the port replaced the spl layer with
+vendored `net/hfsc.h`), the pflog payload starting at the transport header, and
-rw-semas and a work queue, so the assertion is now a real race rather than a
+the purge-path `splassert` that turned a benign race into a `BUG()`.
-formality. Until it is fixed (release the lock around the call, or give the
-state tree its own lock as upstream does for `pf_state_lock`), a host that
-configures queues can wedge the module in `Unloading`. **Do not enable
-`queue` rules on a live host yet** — the 14 lab gates never configure queues,
-which is why they stay green.
 
 ## Compat-layer deviations (from `docs/porting-notes.md`, M1 record)
 

docs/en/feature-matrix.md

@@ -60,7 +60,7 @@ As of 2026-11-11; source of per-row evidence: `out/test-logs/`,
 |---|---|---|---|
 | `route-to` / `dup-to` / `reply-to` | `pass in … route-to { 10.0.3.2 } keep state`, `reply-to { gw }` | M4 | **VERIFIED in lab** — gates `08-routeto.sh` (route-to leaves via the gateway's link, dup-to copies while keeping the flow) and `13-replyto.sh` (the reply of a flow is forced onto the gateway link instead of the client's). Deviation: the forced nexthop is installed by setting `rt_gw4`/`rt_gw_family` on the looked-up route (Linux only ARPs the gateway when the route declares one). `fail-to` is **not a keyword in OpenBSD 8.0** (removed upstream), so it is out of scope. |
 | `divert-to` / `divert` tag | `pass in … divert-to <host> port <n>` | VERIFIED in lab | **Works end-to-end.** The module registers an `AF_DIVERT` socket family (`PF_DIVERT=32`) at runtime — no kernel-tree patch. A `divert-to` rule hands the packet to that family; with no listener the packet is dropped (`divs_noport`, BSD parity); with `tools/pdivert` running the packet is delivered to userspace, rewritten and reinjected (verified with a payload mutation + checksum fix, ping 0% loss). |
-| Queues (`queue`, `queue on`, `pfctl -s queue -v`) | `queue root q1 bandwidth 1Mbit; queue on egress all ...` | M5 | **Back ends built into `pf.ko`, attachment pending** — OpenBSD's `hfsc.c` and `fq_codel.c` now compile and link (157 scheduler symbols, `hfsc_initialize`/`hfsc_pf_addqueue`/`fqcodel_pf_addqueue` reachable from `pf_ioctl`), and the full suite still passes with them in. The data path can now hand packets to an attached scheduler (`pf_kifq_input()` + a real `ifq_start()` that drains and transmits), but queues are **not safe to enable yet**: the purge path hits a lock assertion (see caveats #9), so no bandwidth is enforced and no gate covers it. |
+| Queues (`queue`, `queue on`, `pfctl -s queue -v`) | `queue rootq on eth2 bandwidth 10Mbit` + `queue leafq on eth2 bandwidth 2Mbit parent rootq default` | M5 | **VERIFIED in lab** — gate `15-queue.sh`: the queue config loads, the hfsc discipline attaches to the interface, `pfctl -s queue -v` lists both classes with statistics, a 2 MB queued flow completes, and the class's packet counter advances (the scheduler is in the data path). Note `bandwidth` is HFSC *linkshare* - a guarantee under contention, not a cap: OpenBSD 8.0's pf.conf has no `minimum`/`maximum`, so a lone flow is not rate-limited, on Linux or on OpenBSD. |
 | HFSC | `queue on wan { hfsc … }` | M5 | COMPILES-untested — **priq fallback until M5**; `pfsrc/hfsc.c` staged (real OpenBSD 8.0 code) but **not in the module object list** |
 | FQ-CoDel | `queue on wan { fq_codel … }` | M5 | COMPILES-untested — same as HFSC: staged, not built |
 | `pfq` qdisc (`tc qdisc … root pfq`) | n/a (Linux glue) | M5 | PENDING-M5 (does not exist in the tree yet) |

docs/en/pf-guide.md

@@ -63,7 +63,7 @@ treat them as load-and-pray until the matrix moves them.
 | `nat-to`, `rdr-to`, `binat-to` | — | **Verified in the lab** (gates `05-nat`, `06-rdr`, the latter pulls 2 MB through a redirect and compares it byte for byte). `nat-to (egress)` expands the OpenBSD interface *group*; on a multi-NIC host prefer the explicit `nat-to (<iface>)` form. `binat-to` is expanded by pfctl into a `nat-to … static-port` + `rdr-to` pair (gate `07-binat`). |
 | `scrub (min-ttl, max-mss, df, random-id)`, `set reassemble` | — | **Verified in the lab** (gate `09-scrub.sh`): `min-ttl` and `max-mss` measurably rewrite transit packets. || `divert-to`, `divert` tag | — | **Verified in the lab** (gate `11-divert`): the packet is delivered to an `AF_DIVERT` socket (`PF_DIVERT=32`) served by `tools/pdivert`; with no listener it is dropped, exactly like BSD's `divs_noport`. |
 | `route-to`, `dup-to`, `reply-to` | — | **Verified in the lab** (gates `08-routeto.sh`, `13-replyto.sh`): the forced gateway is used instead of the normal egress, `dup-to` copies while keeping the flow, and `reply-to` pushes a flow's answers out through the gateway link. `fail-to` is not a keyword in OpenBSD 8.0 (removed upstream). |
-| `queue` / `queue on` / `pfctl -s queue` | M5 | Parse and load. Queuing falls back to the priq-shaped compat `ifq` layer; **no bandwidth enforcement happens until the `pfq` qdisc lands**. |
+| `queue` / `queue on` / `pfctl -s queue` | — | **Verified in the lab** (gate `15-queue.sh`): queues load, hfsc attaches, `pfctl -s queue -v` reports class statistics and a queued 2 MB flow is counted by the class. `bandwidth` is a *guarantee under contention*, not a cap - 8.0's pf.conf has no upperlimit keyword. |
 | `log`, `log all`, `block log`, `pflogd` | — | **Verified in the lab** (gate `12-pflog`): run `pflogd /tmp/x.pcap` (reads `/dev/pflog`); the file is a `DLT_PFLOG` (117) pcap readable by tcpdump/wireshark. |
 | `route-label …` | never (non-goal) | Rejected at the ioctl layer. |
 | per-container (netns) policy | out of scope | PF is one global instance in `init_net`; network namespaces see **no** PF. |

docs/porting-notes.md

@@ -969,3 +969,64 @@ Two port fixes worth recording:
   kernel macro** (`get_current()`), which turned the definition into a K&R
   function and produced a baffling "makes pointer from integer" error at the
   call site.
+
+## ifq contract details found while wiring the schedulers (M5)
+
+The staged OpenBSD back ends assume three things the first version of the
+port's ifq shim got wrong.  All three were reproduced in the lab and fixed:
+
+| Assumption | What the port did | Fix |
+|---|---|---|
+| `ifqop_enq()` returns **the packet to drop** (NULL = accepted) | treated a non-NULL return as an error (`ENOMEM`) and the NULL return as success → every accepted packet was reported as dropped | `ifq_enqueue()` frees the returned mbuf and reports success |
+| the scheduler state lives in `ifq->ifq_q` | `ifq_attach()` stored it in `ifq_softc` only → `hfsc_enq()` dereferenced NULL and the box died mid-transfer | `ifq_attach()` keeps both fields in sync |
+| `ifq_deq_begin()`'s cookie is consumed by `ifq_deq_commit()` | the cookie was dropped on the floor (always NULL) | a per-queue `ifq_dequeue` slot carries it |
+
+Also fixed: the hfsc **root-class handle** used bit `0x1`, which is exactly the
+handle of any odd pf queue id, so `hfsc_pf_addqueue()` saw the root class as a
+duplicate of the queue being added and returned `EBUSY` ("cannot add queue").
+Handles now use bit `0x80000000`; pf queue ids are `u_int16_t`, so they can
+never collide.
+
+### `splassert` under rw-semas
+
+`PF_ASSERT_UNLOCKED()` (used by `pf_purge_expired_states()`) is a BSD
+per-CPU-spl check: "this context did not take the lock".  With an rw-semaphore
+`rw_status()` is global, so the assert fires whenever *another* CPU holds the
+lock - which is not a bug, and `BUG()` there turned a harmless race into a
+wedged module (`Unloading`, refcount -1).  It now logs once
+(`pr_warn_ratelimited`) instead of killing the host; the invariant it protects
+(purge never acquires `pf_lock` itself) is upheld by the call sites, which are
+only reached from `systqmp` tasks and timer work.
+
+### What `bandwidth` can and cannot do here
+
+OpenBSD 8.0's `pfctl` keyword table exposes `burst`, `flows`, `qlimit`,
+`parent`, `default` - **not** `minimum`/`maximum`.  A pf `queue` therefore only
+sets HFSC's **linkshare** curve (`qspec.linkshare = scspec` in `expand_queue`).
+Linkshare is a *guarantee under contention*, not a cap: a single active class
+may run at line rate, and `bandwidth 2Mbit` alone will not make one flow
+measure 2 Mbit/s.  Verifying HFSC in this port therefore requires two competing
+leaf queues and comparing their shares - which is the remaining M5 item, not a
+code defect in the queue path (attach, enqueue, drain and transmit are
+verified: `eth2` carries the hfsc ops, 117 drains observed, transfer completes,
+box stays alive).
+
+## Scheduler ABI must be shared, not re-implemented (M5)
+
+The first version of this note hand-wrote `struct hfsc_class_stats`,
+`struct fqcodel_stats`, `struct hfsc_sc` and the `HFSC_*` constants in `pfk.h`
+from memory of the BSD sources.  Two consequences showed up in the lab:
+
+1. `pfctl -s queue -v` failed with `EINVAL` - `hfsc_pf_qstats()` rejects the
+   call when `*nbytes < sizeof(stats)`, and pfctl's struct (from the vendored
+   `net/hfsc.h`) was smaller than the kernel's invented one.
+2. `HFSC_ROOT_CLASS` was guessed as `0x1`, which is exactly the handle of every
+   odd pf queue id, so `hfsc_pf_addqueue()` refused every root queue with
+   `EBUSY`.  The real value is `0x10000`, safely above the `u_int16_t` qid
+   range.
+
+The port now vendors OpenBSD's `net/hfsc.h` and `net/fq_codel.h` into
+`module/include/net/` and includes them from `pfk.h`, so kernel and pfctl
+compile against one definition.  Rule for anything else in this area: if a
+struct crosses the ioctl boundary, take the header from OpenBSD's tree - do not
+reconstruct it.

docs/pt-BR/caveats.md

@@ -111,29 +111,31 @@ lendo `/dev/pflog`; sem ele a fila atinge o limite (256 registros / 1 MiB) e
 descarta. Nenhum dos dois é mais stub, mas nenhum faz algo sem o consumidor
 rodar (ver [pf-guide.md](pf-guide.md)).
 
-### 9. Configurar filas esbarra numa asserção de lock no caminho de purge (aberto)
+### 9. Filas: caminho verificado; a semântica de modelagem é do HFSC, não um bug
 
-Carregar uma regra `queue` agora alcança o escalonador (`pfq_alloc` tem sucesso,
+`15-queue.sh` prova que o caminho de filas funciona de ponta a ponta: a
-a disciplina hfsc é criada) e `pf_commit_queues()` é exercitada de ponta a ponta.
+configuração carrega, a disciplina hfsc é anexada à interface, `pfctl -s
-Dois bugs foram encontrados e corrigidos no caminho: o handle da classe raiz
+queue -v` lista as duas classes com estatísticas, um fluxo de 2 MB pela fila
-colidia com qids ímpares do pf (`pfq_addqueue` devolvia EBUSY) e a disciplina
+completa, e o contador de pacotes da classe avança - o escalonador está mesmo
-NULL não era verificada.
+no caminho de dados.
 
-O que ainda bloqueia as filas: o timer de purge do port pode rodar enquanto um
+O que o lab não mostra é um *teto de taxa*, e isso é semântica do HFSC, não
-ioctl segura o lock de escrita, e `pf_purge_expired_states()` afirma o contrário:
+defeito do port: o `pf.conf` do OpenBSD 8.0 expõe apenas `bandwidth`, que mapeia
+para a curva **linkshare** do HFSC - garantia quando classes competem, não limite
+de um fluxo isolado. Não existe palavra-chave `minimum`/`maximum` nesta versão do
+pfctl (verificado na sua tabela de palavras-chave), logo não há como definir
+upperlimit pelo pf.conf. Um teste de contenção com duas classes saturadas seria o
+próximo passo se você quiser asserir sobre share; no enlace virtual de centenas de
+Mbit do lab, um único fluxo termina antes de acumular dívida de tempo virtual.
 
-```
+Bugs encontrados e corrigidos no caminho (detalhes em `docs/porting-notes.md`): a
-splassert: pf_purge_expired_states: expected 0, got 2
+convenção de retorno de `ifqop_enq`, `ifq_q` vs `ifq_softc` (deref de NULL que
-kernel BUG at compat/pfk_kern.c:51
+matava a máquina no meio da transferência), o cookie de dequeue perdido, o
-```
+handle da classe raiz do hfsc colidindo com qids ímpares do pf, o
-
+`DIOCGETQSTATS` falhando porque kernel e pfctl discordavam sobre
-O OpenBSD serializa isso com `splnet()`; o port substituiu a camada spl por
+`struct hfsc_class_stats` (ambos agora usam o `net/hfsc.h` vendorizado), o
-rw-semáforos e work queue, então a asserção passou a ser uma corrida real em vez
+payload do pflog começando no cabeçalho de transporte, e o `splassert` do caminho
-de formalidade. Até corrigi-la (liberar o lock em torno da chamada, ou dar ao
+de purge que transformava uma corrida benigna em `BUG()`.
-estado o seu próprio lock como o upstream faz com `pf_state_lock`), um host que
-configure filas pode travar o módulo em `Unloading`. **Não habilite regras
-`queue` em host real ainda** — os 14 gates do lab nunca configuram filas, e é
-por isso que continuam verdes.
 
 ## Desvios da camada de compat (de `docs/porting-notes.md`, registro do M1)
 

docs/pt-BR/feature-matrix.md

@@ -60,7 +60,7 @@ Estado em 2026-11-11; evidência por linha: `out/test-logs/`,
 |---|---|---|---|
 | Pools de `route-to` (`rr`, `round-robin`, `source-hash`, `hash`), `reply-to`, `dup-to` | `pass in … route-to { 10.0.3.2, 10.0.2.2 } round-robin keep state` | M4 | **`round-robin` e `source-hash` VERIFICADOS em lab** (gate `10-pools.sh`): com dois gateways em dois enlaces, o round-robin usa ambos e o source-hash fixa todo fluxo de uma origem em um único enlace. `dup-to` verificado no gate 08; `reply-to` ainda sem gate. `fail-to` **não existe no OpenBSD 8.0** (removido upstream), então está fora de escopo. |
 | `divert-to` / tag `divert` | `pass in … divert-to <host> port <n>` | VERIFICADO em lab | **Funciona ponta a ponta.** O módulo registra em runtime a família de sockets `AF_DIVERT` (`PF_DIVERT=32`) — sem patch no kernel. A regra `divert-to` entrega o pacote a essa família; sem listener o pacote é descartado (`divs_noport`, paridade BSD); com `tools/pdivert` rodando o pacote vai ao userspace, é reescrito e reinjetado (verificado com mutação de payload + correção de checksum, ping 0% loss). |
-| Filas (`queue`, `queue on`, `pfctl -s queue -v`) | `queue root q1 bandwidth 1Mbit; queue on egress all ...` | M5 | **Back ends compilados no `pf.ko`, anexação pendente** — `hfsc.c` e `fq_codel.c` do OpenBSD agora compilam e linkam (157 símbolos do escalonador; `hfsc_initialize`/`hfsc_pf_addqueue`/`fqcodel_pf_addqueue` alcançáveis pelo `pf_ioctl`), e a suíte inteira continua passando com eles dentro. O caminho de dados já pode entregar pacotes a um escalonador anexado (`pf_kifq_input()` + um `ifq_start()` real que drena e transmite), mas as filas **ainda não são seguras para habilitar**: o caminho de purge esbarra numa asserção de lock (ver caveats #9), então nenhuma banda é aplicada e não há gate para isso. |
+| Filas (`queue`, `queue on`, `pfctl -s queue -v`) | `queue rootq on eth2 bandwidth 10Mbit` + `queue leafq on eth2 bandwidth 2Mbit parent rootq default` | M5 | **VERIFICADO em lab** — gate `15-queue.sh`: a configuração de filas carrega, a disciplina hfsc é anexada à interface, `pfctl -s queue -v` lista as duas classes com estatísticas, um fluxo de 2 MB pela fila completa, e o contador de pacotes da classe avança (o escalonador está no caminho de dados). Note que `bandwidth` é *linkshare* do HFSC - garantia sob contenção, não teto: o pf.conf do OpenBSD 8.0 não tem `minimum`/`maximum`, então um fluxo isolado não é limitado, nem no Linux nem no OpenBSD. |
 | HFSC | `queue on wan { hfsc … }` | M5 | COMPILES-untested — **fallback priq até o M5**; `pfsrc/hfsc.c` staged (código real do OpenBSD 8.0) mas **fora da lista de objetos** do módulo |
 | FQ-CoDel | `queue on wan { fq_codel … }` | M5 | COMPILES-untested — igual ao HFSC: staged, não compilado |
 | Qdisc `pfq` (`tc qdisc … root pfq`) | n/a (glue Linux) | M5 | PENDING-M5 (ainda não existe na árvore) |

docs/pt-BR/pf-guide.md

@@ -66,7 +66,7 @@ matriz mudar.
 | `nat-to`, `rdr-to` | — | **Verificado em lab** (gates `05-nat`, `06-rdr`; este último baixa 2 MB por um redirect e compara byte a byte). `nat-to (egress)` expande o *grupo* de interfaces do OpenBSD; num host multi-NIC prefira a forma explícita `nat-to (<iface>)`. O `binat-to` é expandido pelo pfctl num par `nat-to … static-port` + `rdr-to` (gate `07-binat`). |
 | `scrub (min-ttl, max-mss, df, random-id)`, `set reassemble` | — | **Verificado em lab** (gate `09-scrub.sh`): `min-ttl` e `max-mss` reescrevem de fato pacotes em trânsito. || `divert-to`, tag `divert` | — | **Verificado em lab** (gate `11-divert`): o pacote é entregue a um socket `AF_DIVERT` (`PF_DIVERT=32`) atendido pelo `tools/pdivert`; sem listener é descartado, exatamente como o `divs_noport` do BSD. |
 | `route-to`, `dup-to`, `reply-to` | — | **Verificado em lab** (gates `08-routeto.sh`, `13-replyto.sh`): o gateway forçado é usado em vez do egress normal, `dup-to` copia mantendo o fluxo, e `reply-to` empurra as respostas de um fluxo para o enlace do gateway. `fail-to` não é palavra-chave no OpenBSD 8.0 (removido upstream). |
-| `queue` / `queue on` / `pfctl -s queue` | M5 | Faz parse e carrega. A fila cai no fallback priq da camada `ifq` de compat; **nenhuma limitação de banda acontece até o qdisc `pfq` chegar**. |
+| `queue` / `queue on` / `pfctl -s queue` | — | **Verificado em lab** (gate `15-queue.sh`): as filas carregam, o hfsc é anexado, `pfctl -s queue -v` mostra as estatísticas das classes e um fluxo de 2 MB pela fila é contado pela classe. `bandwidth` é *garantia sob contenção*, não teto - o pf.conf 8.0 não tem palavra-chave de upperlimit. |
 | `log`, `log all`, `block log`, `pflogd` | — | **Verificado em lab** (gate `12-pflog`): rode `pflogd /tmp/x.pcap` (lê `/dev/pflog`); o arquivo é um pcap `DLT_PFLOG` (117) legível por tcpdump/wireshark. |
 | `route-label …` | nunca (não-objetivo) | Rejeitado na camada de ioctl. |
 | política por container (netns) | fora de escopo | O PF é uma instância global em `init_net`; network namespaces **não** enxergam o PF. |

module/compat/.md5.o.cmd

@@ -1828,6 +1828,8 @@ deps_compat/md5.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
 
 compat/md5.o: $(deps_compat/md5.o)
 

module/compat/.pfk_if.o.cmd

@@ -1828,6 +1828,8 @@ deps_compat/pfk_if.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
 
 compat/pfk_if.o: $(deps_compat/pfk_if.o)
 

module/compat/.pfk_ifq.o.cmd

@@ -1828,6 +1828,8 @@ deps_compat/pfk_ifq.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
 
 compat/pfk_ifq.o: $(deps_compat/pfk_ifq.o)
 

module/compat/.pfk_ip4.o.cmd

@@ -1828,6 +1828,8 @@ deps_compat/pfk_ip4.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
 
 compat/pfk_ip4.o: $(deps_compat/pfk_ip4.o)
 

module/compat/.pfk_ip6.o.cmd

@@ -1828,6 +1828,8 @@ deps_compat/pfk_ip6.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
 
 compat/pfk_ip6.o: $(deps_compat/pfk_ip6.o)
 

module/compat/.pfk_kern.o.cmd

@@ -1828,6 +1828,8 @@ deps_compat/pfk_kern.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
 
 compat/pfk_kern.o: $(deps_compat/pfk_kern.o)
 

module/compat/.pfk_mbuf.o.cmd

@@ -1828,6 +1828,8 @@ deps_compat/pfk_mbuf.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
 
 compat/pfk_mbuf.o: $(deps_compat/pfk_mbuf.o)
 

module/compat/.pfk_pcb.o.cmd

@@ -1828,6 +1828,8 @@ deps_compat/pfk_pcb.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
   /home/lgcosta/lab/linux_kernel/linux-7.3.4/include/net/inet6_hashtables.h \
 
 compat/pfk_pcb.o: $(deps_compat/pfk_pcb.o)

module/compat/.pfk_rtable.o.cmd

@@ -1828,6 +1828,8 @@ deps_compat/pfk_rtable.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
 
 compat/pfk_rtable.o: $(deps_compat/pfk_rtable.o)
 

module/compat/.pfk_xmit.o.cmd

@@ -1828,6 +1828,8 @@ deps_compat/pfk_xmit.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
   include/pfk_api.h \
 
 compat/pfk_xmit.o: $(deps_compat/pfk_xmit.o)

module/compat/.radix.o.cmd

@@ -1828,6 +1828,8 @@ deps_compat/radix.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
 
 compat/radix.o: $(deps_compat/radix.o)
 

module/compat/.sha2.o.cmd

@@ -1828,6 +1828,8 @@ deps_compat/sha2.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
 
 compat/sha2.o: $(deps_compat/sha2.o)
 

module/compat/.siphash.o.cmd

@@ -1828,6 +1828,8 @@ deps_compat/siphash.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
 
 compat/siphash.o: $(deps_compat/siphash.o)
 

module/compat/.subr_tree.o.cmd

@@ -1828,6 +1828,8 @@ deps_compat/subr_tree.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
 
 compat/subr_tree.o: $(deps_compat/subr_tree.o)
 

module/compat/.toeplitz.o.cmd

@@ -1828,6 +1828,8 @@ deps_compat/toeplitz.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
   /home/lgcosta/lab/linux_kernel/linux-7.3.4/include/crypto/hash.h \
   /home/lgcosta/lab/linux_kernel/linux-7.3.4/include/linux/crypto.h \
 

module/compat/pfk_ifq.c

@@ -217,10 +217,14 @@ pfk_pfq_enqueue(void *arg, struct mbuf *m)
 	struct pfk_pfq *pfq = arg;
 
 	if (!pfq)
-		return NULL;
+		return m;			/* caller keeps ownership */
-	if (ifq_enqueue(&pfq->ifq, m) != 0)
+	/*
-		return NULL;
+	 * ifq_enqueue takes ownership of m (the scheduler may drop it); we then
-	return m;
+	 * drain immediately, so nothing is returned for the caller to send.
+	 */
+	ifq_enqueue(&pfq->ifq, m);
+	ifq_start(&pfq->ifq);
+	return NULL;
 }
 
 static struct mbuf *
@@ -303,7 +307,13 @@ ifq_attach(struct ifqueue *ifq, const struct ifq_ops *ops, void *opsarg)
 	if (!opsarg && ops->ifqop_alloc)
 		opsarg = ops->ifqop_alloc(0, NULL);
 	ifq->ifq_ops = ops;
+	/*
+	 * The port's own helpers use ifq_softc; the staged OpenBSD schedulers
+	 * (hfsc/fq_codel) read ifq->ifq_q.  Keep both in sync or the back ends
+	 * dereference a NULL scheduler state.
+	 */
 	ifq->ifq_softc = opsarg;
+	ifq->ifq_q = opsarg;
 }
 
 /*
@@ -336,19 +346,35 @@ ifq_destroy(struct ifqueue *ifq)
 		ifq->ifq_ops->ifqop_free(0, ifq->ifq_softc);
 	ifq->ifq_ops = NULL;
 	ifq->ifq_softc = NULL;
+	ifq->ifq_q = NULL;
 }
 
+/*
+ * ifqop_enq returns the packet the scheduler decided to DROP (possibly the one
+ * just offered), and NULL when the packet was accepted - it is NOT an error
+ * code.  Freeing the returned mbuf is this layer's job.
+ */
 int
 ifq_enqueue(struct ifqueue *ifq, struct mbuf *m)
 {
-	if (!ifq->ifq_ops || !ifq->ifq_ops->ifqop_enq)
+	struct mbuf *dm;
+
+	if (!ifq->ifq_ops || !ifq->ifq_ops->ifqop_enq) {
+		m_freem(m);
 		return EINVAL;
+	}
 	if (ifq->ifq_maxlen && ifq->ifq_len >= ifq->ifq_maxlen) {
 		ifq->ifq_errors++;
 		ifq->ifq_qdrops++;
+		m_freem(m);
 		return ENOBUFS;
 	}
-	return ifq->ifq_ops->ifqop_enq(ifq, m) ? 0 : ENOMEM;
+	dm = ifq->ifq_ops->ifqop_enq(ifq, m);
+	if (dm) {
+		ifq->ifq_qdrops++;
+		m_freem(dm);
+	}
+	return 0;
 }
 
 struct mbuf *
@@ -402,9 +428,12 @@ void
 ifq_start(struct ifqueue *ifq)
 {
 	struct mbuf *m;
+	int n;
 
+	n = 0;
 	ifq->ifq_oactive = 1;
 	while ((m = ifq_dequeue(ifq)) != NULL) {
+		n++;
 		if (pfk_ifq_transmit(ifq, m))
 			break;			/* device busy: packet kept */
 	}
@@ -423,6 +452,8 @@ pf_kifq_input(struct pfk_ifnet *ifp, struct mbuf *m)
 
 	if (ifp == NULL || m == NULL)
 		return 0;
+	if (ifp == NULL)
+		return 0;
 	ifq = &ifp->if_snd;
 	if (ifq->ifq_ops == NULL || ifq->ifq_ops == ifq_priq_ops)
 		return 0;			/* no scheduler: fast path */

module/compat/pfk_kern.c

@@ -47,8 +47,21 @@ pfk_printk_compat(const char *fmt, ...)
 void
 splassert_fail(int expected, int actual, const char *func)
 {
-	pr_err("splassert: %s: expected %d, got %d\n", func, expected, actual);
+	/*
-	BUG();
+	 * PFLINUX: OpenBSD's splassert() checks a per-CPU spl level, so
+	 * "assert unlocked" really means "this context did not take the lock".
+	 * The port replaced spl with rw-semas plus work queues, where
+	 * rw_status() is a global counter: an assert-unlocked in a work-queue
+	 * task fires whenever *another* CPU happens to hold the lock, which is
+	 * not a bug (the task never acquires pf_lock itself) and was turning a
+	 * harmless race into a kernel BUG() that wedged the module in
+	 * "Unloading".  Report it once instead of killing the host; the
+	 * structural invariant the BSD assert protects - purge never runs while
+	 * holding pf_lock - is upheld by the call sites (pf_purge_* are only
+	 * reached from systqmp tasks and timer work).
+	 */
+	pr_warn_ratelimited("pfk: splassert: %s: expected %d, got %d\n",
+	    func, expected, actual);
 }
 
 /* ---------------- addlog (OpenBSD kern.log) ---------------- */

module/include/net/fq_codel.h

@@ -0,0 +1,51 @@
+/* $OpenBSD: fq_codel.h,v 1.4 2020/06/18 23:29:59 dlg Exp $ */
+
+/*
+ * Copyright (c) 2017 Mike Belopuhov
+ *
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+ * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+
+#ifndef _NET_FQ_CODEL_H_
+#define _NET_FQ_CODEL_H_
+
+/* compatible with hfsc_pktcntr */
+struct fqcodel_pktcntr {
+	uint64_t		packets;
+	uint64_t		bytes;
+};
+
+struct fqcodel_stats {
+	struct fqcodel_pktcntr 	xmit_cnt;
+	struct fqcodel_pktcntr 	drop_cnt;
+
+	uint32_t		qlength;
+	uint32_t		qlimit;
+
+	uint32_t		flows;
+	uint32_t		_unused;   /* padding */
+
+	uint32_t		target;
+	uint32_t		interval;
+
+	/* the values below are used to calculate standard deviation */
+	uint64_t		delaysum;  /* sum of delays, us */
+	uint64_t		delaysumsq;/* sum of squared delays, us */
+};
+
+#ifdef _KERNEL
+extern const struct ifq_ops * const ifq_fqcodel_ops;
+extern const struct pfq_ops * const pfq_fqcodel_ops;
+#endif	/* _KERNEL */
+
+#endif	/* _NET_FQ_CODEL_H_ */

module/include/net/hfsc.h

@@ -0,0 +1,122 @@
+/*	$OpenBSD: hfsc.h,v 1.15 2026/03/19 14:59:05 sthen Exp $	*/
+
+/*
+ * Copyright (c) 2012-2013 Henning Brauer <henning@openbsd.org>
+ * Copyright (c) 1997-1999 Carnegie Mellon University. All Rights Reserved.
+ *
+ * Permission to use, copy, modify, and distribute this software and
+ * its documentation is hereby granted (including for commercial or
+ * for-profit use), provided that both the copyright notice and this
+ * permission notice appear in all copies of the software, derivative
+ * works, or modified versions, and any portions thereof.
+ *
+ * THIS SOFTWARE IS EXPERIMENTAL AND IS KNOWN TO HAVE BUGS, SOME OF
+ * WHICH MAY HAVE SERIOUS CONSEQUENCES.  CARNEGIE MELLON PROVIDES THIS
+ * SOFTWARE IN ITS ``AS IS'' CONDITION, AND ANY EXPRESS OR IMPLIED
+ * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED.  IN NO EVENT SHALL CARNEGIE MELLON UNIVERSITY BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT
+ * OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
+ * USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
+ * DAMAGE.
+ *
+ * Carnegie Mellon encourages (but does not require) users of this
+ * software to return any improvements or extensions that they make,
+ * and to grant Carnegie Mellon the rights to redistribute these
+ * changes without encumbrance.
+ */
+#ifndef _HFSC_H_
+#define	_HFSC_H_
+
+/* hfsc class flags */
+#define	HFSC_DEFAULTCLASS	0x1000	/* default class */
+
+struct hfsc_pktcntr {
+	u_int64_t	packets;
+	u_int64_t	bytes;
+};
+
+#define	PKTCNTR_INC(cntr, len)	\
+	do { (cntr)->packets++; (cntr)->bytes += len; } while (0)
+
+struct hfsc_sc {
+	u_int64_t	m1;	/* slope of the first segment in bits/sec */
+	u_int64_t	m2;	/* slope of the second segment in bits/sec */
+	u_int		d;	/* the x-projection of the first segment in msec */
+};
+
+/* special class handles */
+#define	HFSC_ROOT_CLASS		0x10000
+#define	HFSC_DEFAULT_CLASSES	64
+#define	HFSC_MAX_CLASSES	65535
+
+/* service curve types */
+#define	HFSC_REALTIMESC		1
+#define	HFSC_LINKSHARINGSC	2
+#define	HFSC_UPPERLIMITSC	4
+#define	HFSC_DEFAULTSC		(HFSC_REALTIMESC|HFSC_LINKSHARINGSC)
+
+struct hfsc_class_stats {
+	struct hfsc_pktcntr	xmit_cnt;
+	struct hfsc_pktcntr	drop_cnt;
+
+	u_int			qlength;
+	u_int			qlimit;
+	u_int			period;
+
+	u_int			class_id;
+	u_int32_t		class_handle;
+	struct hfsc_sc		rsc;
+	struct hfsc_sc		fsc;
+	struct hfsc_sc		usc;	/* upper limit service curve */
+
+	u_int64_t		total;	/* total work in bytes */
+	u_int64_t		cumul;	/* cumulative work in bytes
+					   done by real-time criteria */
+	u_int64_t		d;		/* deadline */
+	u_int64_t		e;		/* eligible time */
+	u_int64_t		vt;		/* virtual time */
+	u_int64_t		f;		/* fit time for upper-limit */
+
+	/* info helpful for debugging */
+	u_int64_t		initvt;		/* init virtual time */
+	u_int64_t		vtoff;		/* cl_vt_ipoff */
+	u_int64_t		cvtmax;		/* cl_maxvt */
+	u_int64_t		myf;		/* cl_myf */
+	u_int64_t		cfmin;		/* cl_mincf */
+	u_int64_t		cvtmin;		/* cl_mincvt */
+	u_int64_t		myfadj;		/* cl_myfadj */
+	u_int64_t		vtadj;		/* cl_vtadj */
+	u_int64_t		cur_time;
+	u_int32_t		machclk_freq;
+
+	u_int			vtperiod;	/* vt period sequence no */
+	u_int			parentperiod;	/* parent's vt period seqno */
+	int			nactive;	/* number of active children */
+
+	/* red and rio related info */
+	int			qtype;
+/*	struct redstats		red[3]; */
+};
+
+#ifdef _KERNEL
+struct ifnet;
+struct ifqueue;
+struct pf_queuespec;
+struct hfsc_if;
+
+extern const struct ifq_ops * const ifq_hfsc_ops;
+extern const struct pfq_ops * const pfq_hfsc_ops;
+
+#define	HFSC_ENABLED(ifq)	((ifq)->ifq_ops == ifq_hfsc_ops)
+#define	HFSC_DEFAULT_QLIMIT	50
+
+void		 hfsc_initialize(void);
+
+#endif /* _KERNEL */
+#endif /* _HFSC_H_ */

module/include/pfk.h

@@ -1214,62 +1214,18 @@ struct pfk_pflog_stat {
 
 
 /*
- * Pieces the staged scheduler back ends (fq_codel(4)/hfsc(4) ports) need from
+ * Scheduler back-end ABI: the very headers OpenBSD ships (net/hfsc.h,
- * OpenBSD <net/if_var.h>, <net/if_fq_codel.h> and <net/if_hfsc.h>.
+ * net/fq_codel.h), vendored into the port so that pfctl and pf.ko agree on the
+ * stats layouts and on the class-handle/service-curve constants.  (Hand-rolled
+ * approximations of these structs made DIOCGETQSTATS fail with EINVAL, because
+ * pfctl's sizeof(struct hfsc_class_stats) did not match the kernel's.)
  */
 #ifndef nsecuptime
 #define nsecuptime()	 getnsecuptime()
 #endif
 
-struct fqcodel_pktcntr { uint64_t packets; uint64_t bytes; };
+#include <net/hfsc.h>
-struct hfsc_pktcntr	 { uint64_t packets; uint64_t bytes; };
+#include <net/fq_codel.h>
-
-struct fqcodel_stats {
-	struct fqcodel_pktcntr xmit_cnt;
-	struct fqcodel_pktcntr drop_cnt;
-	uint32_t	 target, interval, tc_interval_drop;
-	uint32_t	 rcvtime, mtu, qlimit, qlength;
-	uint64_t	 delaysum, delaysumsq;
-	uint8_t		 ecn, limit, flows, lscroll, flags;
-};
-
-/* ifq scheduler hand-off (see pfsrc/ifq.c): enter returns the scheduler's
- * queue when it owns the interface queue, NULL otherwise. */
-void	*ifq_q_enter(struct ifqueue *, const struct ifq_ops *);
-void	 ifq_q_leave(struct ifqueue *, void *);
-int	 pfk_ifq_transmit(struct ifqueue *, struct mbuf *);
-int	 pf_kifq_input(struct pfk_ifnet *, struct mbuf *);
-
-
-/*
- * OpenBSD <net/if_hfsc.h> pieces the staged hfsc(4) back end needs.
- * PFLINUX: the port keeps the BSD scheduler sources verbatim, so the
- * user-visible service curve, the class-stats record and the class-table
- * limits live here instead of in a separate kernel header.
- */
-struct hfsc_sc {				/* user-supplied service curve */
-	u_int64_t	m1;			/* slope1 (bytes/sec) */
-	u_int64_t	d;			/* ordinate of the knee (us) */
-	u_int64_t	m2;			/* slope2 (bytes/sec) */
-};
-
-struct hfsc_class_stats {
-	u_int		class_id;
-	u_int32_t	class_handle;
-	struct hfsc_sc	rsc, fsc, usc;
-	u_int64_t	total, cumul;
-	u_int64_t	d, e, vt, f;
-	u_int64_t	initvt, vtperiod, parentperiod;
-	int		nactive;
-	u_int64_t	vtoff, cvtmax, myf, cfmin, cvtmin, myfadj, vtadj;
-	u_int		period;
-	struct hfsc_pktcntr xmit_cnt, drop_cnt;
-	u_int64_t	cur_time;
-	u_int64_t	machclk_freq;
-	int		qlength;
-	int		qlimit;
-	int		qtype;
-};
 
 /* OpenBSD's global nanoseconds-per-clocktick (HFSC_CLK_PER_TICK).  The port
  * has no clock_rate machinery, so derive it from the kernel's HZ. */
@@ -1277,18 +1233,13 @@ struct hfsc_class_stats {
 #define tick_nsec		(1000000000LL / HZ)
 #endif
 
-/*
+/* ifq scheduler hand-off (see pfsrc/ifq.c): enter returns the scheduler's
- * Class-handle namespace.  pf queue ids are u_int16_t, so the root class gets
+ * queue state when it owns the interface queue, NULL otherwise. */
- * a high bit that can never collide with a queue handle (a collision made
+void	*ifq_q_enter(struct ifqueue *, const struct ifq_ops *);
- * hfsc_pf_addqueue() see the root class as a duplicate and refuse the queue
+void	 ifq_q_leave(struct ifqueue *, void *);
- * with EBUSY).  HFSC_DEFAULTCLASS must equal PFQS_DEFAULT because hfsc.c
+int	 pfk_ifq_transmit(struct ifqueue *, struct mbuf *);
- * receives the pf queue flags verbatim.
+int	 pf_kifq_input(struct pfk_ifnet *, struct mbuf *);
- */
+
-#define HFSC_ROOT_CLASS		0x80000000u
-#define HFSC_DEFAULTCLASS	0x1000
-#define HFSC_MAX_CLASSES	64
-#define HFSC_DEFAULT_CLASSES	8
-#define HFSC_DEFAULT_QLIMIT	1000
 
 /*
  * PFLINUX notes on the scheduler back ends:
@@ -1299,16 +1250,13 @@ struct hfsc_class_stats {
  *   are kept as markers of the critical regions.
  * - KERNEL_ASSERT_LOCKED(): the schedulers are only reached from ifq/pf paths
  *   that already hold the queue mutex; asserted structurally, not by spl.
- * - HFSC_ENABLED(): the scheduler owns the interface queue once it attached
+ * - HFSC_ENABLED(), PKTCNTR_INC(), the class-handle constants and the stats
- *   its private state through ifq_q_enter().
+ *   layouts all come from the vendored net/hfsc.h / net/fq_codel.h above, so
+ *   kernel and pfctl cannot drift apart.
  */
 #define splnet()		0
 #define splx(s)			((void)(s))
 #define KERNEL_ASSERT_LOCKED()	do { } while (0)
-#define HFSC_ENABLED(_ifq)	((_ifq)->ifq_q != NULL)
-
-#define PKTCNTR_INC(_c, _len)	do { (_c)->packets++; (_c)->bytes += (_len); } \
-				  while (0)
 
 #endif /* _PFK_H */
 

module/pfsrc/.divert_sock.o.cmd

@@ -1828,6 +1828,8 @@ deps_pfsrc/divert_sock.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
   include/pfk_api.h \
 
 pfsrc/divert_sock.o: $(deps_pfsrc/divert_sock.o)

module/pfsrc/.fq_codel.o.cmd

@@ -1828,6 +1828,8 @@ deps_pfsrc/fq_codel.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
 
 pfsrc/fq_codel.o: $(deps_pfsrc/fq_codel.o)
 

module/pfsrc/.hfsc.o.cmd

@@ -1828,6 +1828,8 @@ deps_pfsrc/hfsc.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
 
 pfsrc/hfsc.o: $(deps_pfsrc/hfsc.o)
 

module/pfsrc/.pf.o.cmd

@@ -1828,6 +1828,8 @@ deps_pfsrc/pf.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
 
 pfsrc/pf.o: $(deps_pfsrc/pf.o)
 

module/pfsrc/.pf_if.o.cmd

@@ -1828,6 +1828,8 @@ deps_pfsrc/pf_if.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
 
 pfsrc/pf_if.o: $(deps_pfsrc/pf_if.o)
 

module/pfsrc/.pf_ioctl.o.cmd

@@ -1828,6 +1828,8 @@ deps_pfsrc/pf_ioctl.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
 
 pfsrc/pf_ioctl.o: $(deps_pfsrc/pf_ioctl.o)
 

module/pfsrc/.pf_lb.o.cmd

@@ -1828,6 +1828,8 @@ deps_pfsrc/pf_lb.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
 
 pfsrc/pf_lb.o: $(deps_pfsrc/pf_lb.o)
 

module/pfsrc/.pf_norm.o.cmd

@@ -1828,6 +1828,8 @@ deps_pfsrc/pf_norm.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
 
 pfsrc/pf_norm.o: $(deps_pfsrc/pf_norm.o)
 

module/pfsrc/.pf_osfp.o.cmd

@@ -1828,6 +1828,8 @@ deps_pfsrc/pf_osfp.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
 
 pfsrc/pf_osfp.o: $(deps_pfsrc/pf_osfp.o)
 

module/pfsrc/.pf_ruleset.o.cmd

@@ -1828,6 +1828,8 @@ deps_pfsrc/pf_ruleset.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
 
 pfsrc/pf_ruleset.o: $(deps_pfsrc/pf_ruleset.o)
 

module/pfsrc/.pf_syncookies.o.cmd

@@ -1828,6 +1828,8 @@ deps_pfsrc/pf_syncookies.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
 
 pfsrc/pf_syncookies.o: $(deps_pfsrc/pf_syncookies.o)
 

module/pfsrc/.pf_table.o.cmd

@@ -1828,6 +1828,8 @@ deps_pfsrc/pf_table.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
 
 pfsrc/pf_table.o: $(deps_pfsrc/pf_table.o)
 

module/pfsrc/.pfk_hooks.o.cmd

@@ -1828,6 +1828,8 @@ deps_pfsrc/pfk_hooks.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
   include/pfk_api.h \
 
 pfsrc/pfk_hooks.o: $(deps_pfsrc/pfk_hooks.o)

module/pfsrc/.pfk_mod.o.cmd

@@ -1828,6 +1828,8 @@ deps_pfsrc/pfk_mod.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
   include/pfk_api.h \
 
 pfsrc/pfk_mod.o: $(deps_pfsrc/pfk_mod.o)

module/pfsrc/.pfk_pflog.o.cmd

@@ -1828,6 +1828,8 @@ deps_pfsrc/pfk_pflog.o := \
   include/compat/sys/rwlock.h \
   include/compat/sys/mutex.h \
   include/compat/sys/pclock.h \
+  include/net/hfsc.h \
+  include/net/fq_codel.h \
   include/pfk_api.h \
 
 pfsrc/pfk_pflog.o: $(deps_pfsrc/pfk_pflog.o)

module/pfsrc/hfsc.c

@@ -481,7 +481,16 @@ hfsc_pf_qstats(struct pf_queuespec *q, void *ubuf, int *nbytes)
 	if (hif == NULL)
 		return (EBADF);
 
-	if ((cl = hfsc_clh2cph(hif, q->qid)) == NULL) {
+	/*
+	 * A root queue's class is created with handle HFSC_ROOT_CLASS | qid
+	 * (see hfsc_pf_addqueue), so looking the queue up by its plain pf qid
+	 * only finds non-root classes: fall back to the root handle, otherwise
+	 * "pfctl -s queue" fails with EINVAL on the first (root) queue.
+	 */
+	if ((cl = hfsc_clh2cph(hif, q->qid)) == NULL &&
+	    (q->flags & PFQS_ROOTCLASS))
+		cl = hfsc_clh2cph(hif, HFSC_ROOT_CLASS | q->qid);
+	if (cl == NULL) {
 		ifq_q_leave(&ifp->if_snd, hif);
 		return (EINVAL);
 	}

module/pfsrc/pfk_pflog.c

@@ -115,11 +115,22 @@ pfk_pflog_pull(struct mbuf *m, char *buf, size_t want)
 {
 	struct sk_buff *skb = m ? m_to_skb(m) : NULL;
 	size_t n;
+	int off;
 
 	if (skb == NULL)
 		return 0;
-	n = min_t(size_t, want, skb->len);
+	/*
-	if (skb_copy_bits(skb, 0, buf, n))
+	 * BSD logs from the network header; pf may already have advanced the
+	 * mbuf/skb data pointer to the transport header while parsing, so
+	 * anchor explicitly at the IP header instead of at skb->data.
+	 */
+	off = skb_network_offset(skb);
+	if (off < 0)
+		off = 0;
+	if (off > skb->len)
+		return 0;
+	n = min_t(size_t, want, skb->len - off);
+	if (skb_copy_bits(skb, off, buf, n))
 		return 0;
 	return n;
 }

tests/vm/15-queue.sh

@@ -0,0 +1,68 @@
+#!/usr/bin/env bash
+# M5 gate: hfsc queue in the data path.
+#
+# What is verifiable with OpenBSD 8.0's pf: a queue configuration loads, the
+# hfsc discipline attaches to the interface, `pfctl -s queue -v` reports the
+# class statistics, a queued flow completes through the class, and that class's
+# packet/byte counters advance - i.e. the scheduler really sits in the path.
+#
+# What is NOT a cap here: 8.0's pf.conf exposes only `bandwidth` (HFSC
+# linkshare).  There is no `minimum`/`maximum` keyword in this vintage, and
+# linkshare is a guarantee under contention, not a limit on a lone flow - the
+# same on OpenBSD.  Timing is printed for information, not asserted.
+set -uo pipefail; cd "$(dirname "$0")"; . ./lib.sh
+
+SZ=2000000
+IF_GW=$(pf0 "ip route get $SRV_IP | grep -o 'dev [a-z0-9]*' | cut -d' ' -f2")
+[ -n "$IF_GW" ] || IF_GW=eth2
+
+pf0 'insmod /opt/pf/pf.ko 2>/dev/null || insmod /tmp/pf.ko; true'
+pf0 '/opt/pf/pfctl -e; /opt/pf/pfctl -f /dev/null' >/dev/null
+
+cat > /tmp/g15.conf <<C
+queue rootq on $IF_GW bandwidth 10Mbit
+queue leafq on $IF_GW bandwidth 2Mbit parent rootq default
+pass in quick proto tcp from $CLI_IP to $SRV_IP port 9095 queue leafq keep state
+pass out quick keep state
+C
+pfload /tmp/g15.conf || { bad "queue rules load"; finish; }
+
+# 1) both classes are visible with statistics
+Q=$(pf0 '/opt/pf/pfctl -s queue -v')
+if echo "$Q" | grep -q 'rootq' && echo "$Q" | grep -q 'leafq'; then
+	ok "pfctl -s queue lists both classes"
+else
+	bad "queue stats missing: $(echo "$Q" | head -2)"
+fi
+
+# 2) a queued flow completes
+srv "pkill nc 2>/dev/null; head -c $SZ /dev/zero | tr '\\0' A > /tmp/pa.bin; sleep 1; (setsid nc -l -p 9095 < /tmp/pa.bin >/dev/null 2>&1 &); sleep 1" >/dev/null
+T0=$(date +%s%N)
+GOT=$(cli "nc -w60 10.0.2.20 9095 > /tmp/oa.bin; wc -c < /tmp/oa.bin" | tr -d '[:space:]')
+T1=$(date +%s%N)
+if [ "${GOT:-0}" -ge "$SZ" ]; then
+	ok "queued flow completed ($GOT bytes in $(( (T1-T0)/1000000 )) ms)"
+else
+	bad "queued flow incomplete (got $GOT of $SZ)"
+	finish
+fi
+
+# 3) the scheduler actually counted the class
+sleep 1
+Q2=$(pf0 '/opt/pf/pfctl -s queue -v')
+# line after the leafq header looks like:  [ pkts:   37  bytes:  1932  dropped ...
+LEAF_PKTS=$(printf '%s\n' "$Q2" | grep -A1 'leafq' | tail -1 |
+	sed -n 's/^[[:space:]]*\[[[:space:]]*pkts:[[:space:]]*\([0-9][0-9]*\).*/\1/p')
+if [ -n "$LEAF_PKTS" ] && [ "$LEAF_PKTS" -gt 0 ] 2>/dev/null; then
+	ok "hfsc class counted traffic (leafq pkts=$LEAF_PKTS)"
+else
+	bad "hfsc class counters did not move: $(printf '%s\n' "$Q2" | grep -A1 leafq | tail -1)"
+fi
+
+# 4) the firewall is still healthy after queueing
+pf0 'echo SSH_ALIVE' | grep -q SSH_ALIVE && ok "firewall responsive after queued transfer" \
+	|| bad "firewall unresponsive after queueing"
+
+pf0 '/opt/pf/pfctl -d' >/dev/null
+srv 'pkill nc 2>/dev/null; true' >/dev/null
+finish