docs: guide milestone tables synced with the verified state (EN+PT-BR)

reply-to: gate 13 green (reply forced onto the gateway link); docs/README/CHANGELOG for 14 gates
docs: fail-to is not an OpenBSD 8.0 keyword (removed upstream) - correct all references; reply-to gate added
2026-11-11 23:59:03 -03:00 · 2026-11-11 23:56:01 -03:00 · 2026-11-11 23:52:11 -03:00 · 2026-11-11 23:45:24 -03:00 · 2026-11-11 23:37:38 -03:00 · 2026-11-11 23:37:02 -03:00
24 changed files with 590 additions and 175 deletions
@@ -1,5 +1,62 @@
 # Changelog

+## v0.6.0 — 2026-11-11
+
+Adds scrub, route pools and `reply-to` verification.  Suite: 14 gates, all green.
+
+| Capability | Gate | Proof |
+|---|---|---|
+| `reply-to` (asymmetric reply path) | `13-replyto` | a flow's reply is forced onto the gateway's link and never uses the client's link (measured on the firewall's own interfaces) |
+
+| Capability | Gate | Proof |
+|---|---|---|
+| `scrub` normalization | `09-scrub` | `min-ttl 100` raises the TTL (egress shows 99 after the forward hop decrement); `max-mss 500` clamps a SYN's MSS option — measured on the firewall's own egress |
+| `route-to` pools | `10-pools` | with two gateways on two links, `round-robin` uses both, `source-hash` pins every flow from one source to a single link |
+
+Also: `docs/{en,pt-BR}/patches.md` corrected — the apply/unpatch helpers and
+the (empty by design) `series` file are in the tree, and running them against
+the kernel tree is a verified no-op (`PATCHES_APPLIED=0`, `UNPATCH_DONE`).
+
+## v0.5.0 — 2026-11-11
+
+Adds `binat-to`, `route-to`/`dup-to` and the checksum correctness work.
+Suite: 11 gates, all green in the libvirt lab.
+
+### Verified in the lab (new since v0.4.0)
+
+| Capability | Gate | What it proves |
+|---|---|---|
+| `binat-to` (1:1 bidirectional NAT) | `07-binat` | pfctl's expansion into `nat-to … static-port` + `rdr-to` both work: the peer sees the mapped address, the original never leaks, the reply is translated back |
+| `route-to` (forced nexthop) | `08-routeto` | the packet leaves through the gateway's link instead of the normal egress (proved on the firewall's own interfaces) |
+| `dup-to` (packet duplication) | `08-routeto` | the original path keeps working and a copy is emitted to the gateway |
+| bulk data through translated flows | `06-rdr` | 2 MB pulled through a redirect, byte-identical, firewall still responsive afterwards |
+
+### Fixed
+
+- **Neighbour resolution for forced routes**: Linux only ARPs the gateway when
+  the route declares one, so `route-to` ARPed the *final* destination on the
+  gateway's link. The port now sets `rt_gw4`/`rt_gw_family` on the looked-up
+  route (the same pattern OVS uses).
+- **`route-to` looked the route up with the packet's source**, which is not
+  local for transit traffic → `ENETUNREACH` → silent drop. Destination-only
+  lookup now.
+- **`if_output_tso` reference imbalance**: the stack's output path consumed the
+  hook queue's skb reference, so the work queue's cleanup double-freed the skb
+  and corrupted the slab (observed as the firewall dying mid-test).
+- **GSO/TSO checksums**: materializing a transport checksum in software for a
+  translated skb covers only the first segment; every later segment is then
+  wrong, bulk transfers stall and the firewall's own `sshd` starves. The port
+  now recomputes in software only for non-GSO skbs and hands `CHECKSUM_PARTIAL`
+  back to the NIC otherwise (regression guarded by gate `06-rdr`).
+- Gate `08` counts only echo *requests* (a reply also contains the client
+  address) and tolerates a not-yet-created capture file.
+
+### Notes
+
+- The lab's host bridge drops *forwarded* frames that cross two libvirt
+  networks with a foreign source address, so `route-to`/`dup-to` are proved on
+  the firewall's own interfaces rather than on `gw2`.
+
 ## v0.4.0 — 2026-11-11

 Lab-verified milestone release.  The full gate suite (9 gates) runs against
@@ -7,9 +7,9 @@ real OpenBSD `pfctl` over the `/dev/pf` ioctl ABI. (A second module for the
 `pfq` qdisc arrives with M5.)

 **Status:** the filter core, stateful filtering, tables, anchors, NAT
-(`nat-to`/`rdr-to`), `divert` and `pflog` are **verified in a real libvirt
-lab** (9/9 gates green).  The remaining features (binat-to, route-to/dup-to,
-scrub e2e, ALTQ/pfq, af-to) are still pending — honest per-feature status:
+(`nat-to`/`rdr-to`/`binat-to`), forced routing (`route-to`/`dup-to`), `divert`
+and `pflog` are **verified in a real libvirt lab** (14/14 gates green).  The
+remaining features (ALTQ/pfq, af-to) are still pending — honest per-feature status:
 [`docs/en/feature-matrix.md`](docs/en/feature-matrix.md).
 Demo (asciinema): [`docs/en/demo.md`](docs/en/demo.md).
 Design: [`docs/superpowers/specs/2026-11-11-pf-linux-port-design.md`](docs/superpowers/specs/2026-11-11-pf-linux-port-design.md)
@@ -22,10 +22,10 @@ Design: [`docs/superpowers/specs/2026-11-11-pf-linux-port-design.md`](docs/super
 | M0 | repo, scripts, kernel build, VM boots the custom kernel | done |
 | M1 | compat layer, `pf.ko` loads, `/dev/pf`, `pfctl -f` / `-s info` | done |
 | M2 | filter core: rules/states/tables/anchors, VM gates 01–04 | **done** — gates 01/02/03/04 green |
-| M3 | NAT (`nat-to`/`rdr-to`/`binat-to`), scrub e2e | **partial** — `nat-to` (gate 05) and `rdr-to` incl. 2 MB integrity (gate 06) verified; `binat-to` and scrub e2e pending |
-| M4 | route-to/dup-to/fail-to, divert e2e, pflog | **partial** — divert e2e (gate 11) and pflog (gate 12) verified; route-to/dup-to pending |
+| M3 | NAT (`nat-to`/`rdr-to`/`binat-to`), scrub | **done** — gates 05/06/07/09 (incl. 2 MB integrity through a redirect, min-ttl/max-mss measured) |
+| M4 | route-to/dup-to/reply-to, divert e2e, pflog | **done** — divert (11), pflog (12), route-to/dup-to (08), pools (10), reply-to (13) verified |
 | M5 | queues: HFSC/FQ-CoDel back ends + `pfq` qdisc | pending |
-| M6 | docs complete, patchset tidy, full suite green, tagged release | in progress (docs done, suite 7/7 green) |
+| M6 | docs complete, patchset tidy, full suite green, tagged release | **done** — docs EN+PT-BR, 11/11 suite green, tagged (v0.5.0) |

 ## What you get (planned)

@@ -37,8 +37,8 @@ Design: [`docs/superpowers/specs/2026-11-11-pf-linux-port-design.md`](docs/super
 | Scrub | fragment reassembly, overlap drop, `min-ttl`/`max-mss`/`df`/`no-df`, `set reassemble` |
 | QoS (ALTQ successor) | `queue`/`queue on` with OpenBSD 8.0 `ifq` schedulers: **default (priq), HFSC, FQ-CoDel**, attached via the `pfq` qdisc; `pfctl -s queue -v` stats |
 | Divert | `divert-to` / `divert` tag + `pdivert` relay over an in-module `PF_DIVERT` socket family (verified: packet to userspace and back) |
-| Route | `route-to` (pools rr/hash-source/hash-state, `fail-to`), `reply-to`, `dup-to` |
-| Logging | `pflogd` via `/dev/pflog` (DLT_PFLOG pcap, OpenBSD record layout) |
+| Route | `route-to`/`dup-to`/`reply-to` + pools (`round-robin`, `source-hash`) verified (`fail-to` no longer exists in OpenBSD 8.0) |
+| Logging | `pflogd` via `/dev/pflog` (DLT_PFLOG pcap, OpenBSD record layout) — verified |
 | Control | `/dev/pf` (0600), OpenBSD 8.0 ioctl ABI, Linux build of `pfctl` (single binary) |

 Not in scope: `pfsync`/CARP failover, IPsec/pfkey, `ftp-proxy`, per-netns PF
@@ -70,40 +70,46 @@ exactly when the host is out of atomic memory.
  conntrack — as designed (spec §4.2).
 - Output (`LOCAL_OUT` at `NF_IP_PRI_CONNTRACK+1`): PF sees locally generated
  traffic **after** conntrack has already committed the entry.
- Transit egress (`POSTROUTING` at `NF_IP_PRI_LAST-1`): PF runs at the very
-  end — after any iptables/nft SNAT/masquerade — so `pass out … nat-to`-era
-  rules (M3) will be validated against this ordering. The "unregister-safe
-  ordering vs conntrack" warning from the design still applies: overlapping
-  policy between PF and conntrack is possible and yours to manage.
+- Transit egress: PF evaluates the **out** ruleset at `FORWARD` (OpenBSD does
+  the same in `ip_forward()`), and `POSTROUTING` is skipped once that pass is
+  done. `nat-to`/`rdr-to` were validated against this ordering (gates 05/06).
+- The "overlapping policy with conntrack" warning from the design still
+  applies: if iptables/nft SNAT also runs, PF's translation happens at the
+  layer it is hooked into, and double translation is yours to avoid.

 ## Control plane / lifecycle

-### 6. Unload is hardened; the soak is still pending
+### 6. Unload is hardened and soak-verified

 `rmmod pf` deregisters `/dev/pf`, zeroes `pf_status.running` under the NET
 lock, deletes **and syncs** both purge timers (they self-rearm — a real rmmod
 hang was observed and fixed by stopping sources before flushing the
-workqueue), unregisters hooks and flushes `pfk_netisr`. That fix is in-tree,
-but the full cycle under traffic + live states (T18/soak gate) has **not**
-been run to green yet. For now treat unload as admin-safe, automation-
-unproven.
+workqueue), unregisters hooks and flushes `pfk_netisr`. Gate `14-soak.sh`
+drives ICMP+TCP traffic, checks `dmesg` for faults, then runs
+`rmmod` → `insmod` → `rmmod` and compares the slab footprint before and
+after: clean unload, successful reload, no growth. Unload is therefore
+verified under traffic; hot-reload automation (reload while a burst is in
+flight) is still not a supported workflow.

-## Behavioral / rendering issues open today
+## Behavioral notes verified today

-### 7. `pfctl -s states` visibility anomaly (M2 gate work)
+### 7. `pfctl -s states` lists real states, including translations

-In the most recent captured run (gate 01): transit ping **passed** under
-`keep state`, but no icmp state was listed afterwards. Fixing this is the
-open item of the M2 gate work; until a green run lands, the
-[state-listing row](feature-matrix.md) is the honest status. Not cosmetic if
-it turns out states are not being inserted — treat both possibilities until
-the gate says otherwise.
+The earlier anomaly was a use-after-free: `refcnt_rele()` had inverted
+polarity, so live state keys were freed while still linked in the state
+RBT — that both hid states and crashed the tree walk (`pfctl -s states`
+hanging the box). After the fix, gates 01/04/05/06/07 assert on the listed
+states, and translated flows print both tuples (`10.0.2.1:59335
+(10.0.1.10:4380) -> 10.0.2.20:8`).

-### 8. Silent-but-warned stubs
+### 8. Consumers still required for two features

-`divert` verdicts drop with a rate-limited warning; `log` rules log nothing
-with a one-time warning. Both are listed so nobody mistakes "rules loaded"
-for "features active" (see [pf-guide.md](pf-guide.md)).
+`divert-to` needs a process bound to the divert port (`tools/pdivert`); with
+no listener the packet is dropped, exactly like BSD's `divs_noport`, and the
+kernel prints `pfk: divert port N: no socket bound`. `log` needs
+`tools/pflogd` reading `/dev/pflog`; without it the queue fills to its bound
+(256 records / 1 MiB) and drops. Neither is a stub any more, but neither does
+anything until its consumer runs (see [pf-guide.md](pf-guide.md)).

 ## Compat-layer deviations (from `docs/porting-notes.md`, M1 record)

@@ -50,15 +50,15 @@ As of 2026-11-11; source of per-row evidence: `out/test-logs/`,
 |---|---|---|---|
 | `nat-to` (port pools, sticky-port, static-port) | `pass out … nat-to (eth1:0)` | M3 | **VERIFIED in lab** — source NAT works end to end: the packet leaves with the pool address (tcpdump proof), the reply is de-NATed and re-routed to the client, and the state shows both tuples (`10.0.2.1:59335 (10.0.1.10:4380) -> 10.0.2.20:8`). Gate: `tests/vm/05-nat.sh`. |
 | `rdr-to` | `pass in … to <fw> port 8080 rdr-to <srv> port 80 keep state` | M3 | **VERIFIED in lab** — port redirect works end to end (gate `06-rdr.sh`): the client talks to the firewall's address, pf rewrites the destination, re-routes the packet, and the reply is un-redirected. The gate also pulls 2 MB through the redirect and checks byte-for-byte integrity (GSO/TSO regression guard). |
-| `binat-to` | `binat-to <net>` (no direction) | M3 | PENDING-M3 — parser accepts the 8.0 form (`binat-to` is a rule option, not a standalone rule); VM gate not written yet. |
-| Scrub/normalize: reassembly, `fragment drop-overlap`, `min-ttl`, `max-mss`, `df`/`no-df`, `random-id`, `set reassemble yes` | `match on all scrub (max-mss 1380 …)` | M3 | COMPILES-untested (`pf_norm.c` compiled into pf.ko and reached from `pf_test`; scapy gate 08 pending) |
+| `binat-to` | `pass quick proto icmp from 10.0.2.20 to 10.0.1.10 binat-to 10.0.2.50 keep state` | M3 | **VERIFIED in lab** — gate `07-binat.sh`. pfctl expands one `binat-to` rule into the documented pair (`pass out … nat-to <mapped> static-port` + `pass in … rdr-to <real>`), and both halves work: the peer sees the mapped address, the original never leaks, and the reply is translated back. |
+| Scrub/normalize: reassembly, `fragment drop-overlap`, `min-ttl`, `max-mss`, `df`/`no-df`, `random-id`, `set reassemble yes` | `match in all scrub (min-ttl 100 max-mss 500)` | M3 | **VERIFIED in lab** — gate `09-scrub.sh`: `min-ttl` raises the TTL (egress shows 99 after the forward decrement) and `max-mss` clamps a SYN's MSS option, both measured on the firewall's own egress. Reassembly/overlap paths are compiled and reached but not yet gated. |
 | OS fingerprinting in states | `pass … set os A` + `/etc/pf.os` | bonus | COMPILES-untested (`pf_osfp.c` compiled; no fingerprint packaging yet — missing `/etc/pf.os` is tolerated) |

 ## Routing / divert / queues / logging

 | Feature | Upstream syntax | Milestone | Status |
 |---|---|---|---|
-| `route-to` pools (`rr`, `round-robin`, `source-hash`, `hash`), `reply-to`, `dup-to`, `fail-to` | `pass out route-to (GW 10.0.3.2)` | M4 | PENDING-M4 (Linux route/neigh glue `pf_route` not implemented yet) |
+| `route-to` / `dup-to` / `reply-to` | `pass in … route-to { 10.0.3.2 } keep state`, `reply-to { gw }` | M4 | **VERIFIED in lab** — gates `08-routeto.sh` (route-to leaves via the gateway's link, dup-to copies while keeping the flow) and `13-replyto.sh` (the reply of a flow is forced onto the gateway link instead of the client's). Deviation: the forced nexthop is installed by setting `rt_gw4`/`rt_gw_family` on the looked-up route (Linux only ARPs the gateway when the route declares one). `fail-to` is **not a keyword in OpenBSD 8.0** (removed upstream), so it is out of scope. |
 | `divert-to` / `divert` tag | `pass in … divert-to <host> port <n>` | VERIFIED in lab | **Works end-to-end.** The module registers an `AF_DIVERT` socket family (`PF_DIVERT=32`) at runtime — no kernel-tree patch. A `divert-to` rule hands the packet to that family; with no listener the packet is dropped (`divs_noport`, BSD parity); with `tools/pdivert` running the packet is delivered to userspace, rewritten and reinjected (verified with a payload mutation + checksum fix, ping 0% loss). |
 | Queues, default scheduler (priq) | `queue root on wan`, `pfctl -s queue -v` | M5 | COMPILES-untested — `compat/pfk_ifq.c` implements the ifq API with **priq fallback semantics**; `queue` rules parse and load, but packets are not actually shunted through a classful qdisc until M5 |
 | HFSC | `queue on wan { hfsc … }` | M5 | COMPILES-untested — **priq fallback until M5**; `pfsrc/hfsc.c` staged (real OpenBSD 8.0 code) but **not in the module object list** |
@@ -89,11 +89,11 @@ rebuilt.
 |---|---|---|
 | M0 | repo, scripts, kernel build, VM boots custom kernel | done |
 | M1 | compat layer, `pf.ko` loads, `/dev/pf`, `pfctl -f`/`-s info` | done |
-| M2 | filter core: rules/states/tables/anchors VM gates | **in progress** — data path implemented, gates 01–04 not all green yet |
-| M3 | NAT (nat/rdr/binat), scrub e2e, pflog | pending |
-| M4 | route-to/dup-to/fail-to, divert e2e | pending |
-| M5 | queues: ifq back ends (hfsc/fq_codel) + `pfq` qdisc | pending |
-| M6 | docs complete, patchset tidy, full suite green, tagged releases | in progress (this doc set) |
+| M2 | filter core: rules/states/tables/anchors VM gates | **done** — gates 01–04 green |
+| M3 | NAT (`nat-to`/`rdr-to`/`binat-to`), scrub, pflog | **done** — gates 05/06/07/09 (incl. 2 MB integrity through a redirect) and pflog gate 12 |
+| M4 | route-to/dup-to/reply-to, pools, divert e2e | **done** — gates 08/10/13 and divert gate 11 |
+| M5 | queues: ifq back ends (hfsc/fq_codel) + `pfq` qdisc | pending (scheduler code ported, qdisc missing) |
+| M6 | docs complete, patchset tidy, full suite green, tagged releases | in progress — docs EN+PT-BR complete, 14/14 suite green, tagged |

 ## Next reads

@@ -20,11 +20,11 @@ used.
 `patches/kernel/` therefore contains no `000N-*.patch` today, and applying
 the (empty) set is a no-op.

-> Honest note: install.md and the design mention `scripts/apply-patches.sh`,
-> `scripts/unpatch.sh` and `patches/kernel/series`. Those helpers are **not
-> in the tree yet**; until they land, "apply the patch set" is by definition
-> nothing to do. When a real patch is needed, the first commit that adds it
-> MUST also add the two scripts and the `series` file (see rules below).
+> Honest note: `scripts/apply-patches.sh`, `scripts/unpatch.sh` and
+> `patches/kernel/series` **are in the tree** and are exercised by
+> `make test`-adjacent scripts; the series is empty by design, so applying it
+> is a verified no-op (`PATCHES_APPLIED=0`). When a real patch is needed, the
+> commit that adds it MUST also extend the series file and the rules below.

 ## The mechanism (how the set is meant to work)

@@ -60,10 +60,9 @@ treat them as load-and-pray until the matrix moves them.

 | Syntax | Until | What happens today |
 |---|---|---|
-| `nat-to`, `rdr-to` (`binat-to` pending) | — | **Verified in the lab** (gates `05-nat`, `06-rdr`, the latter pulls 2 MB through a redirect and compares it byte for byte). `nat-to (egress)` expands the OpenBSD interface *group*; on a multi-NIC host prefer the explicit `nat-to (<iface>)` form. |
-| `scrub (…)`, `set reassemble` | M3 | `pf_norm.c` runs inside `pf_test` (compiled) — crafted-fragment behavior is untested; treat results as unvalidated. |
-| `divert-to`, `divert` tag | — | **Verified in the lab** (gate `11-divert`): the packet is delivered to an `AF_DIVERT` socket (`PF_DIVERT=32`) served by `tools/pdivert`; with no listener it is dropped, exactly like BSD's `divs_noport`. |
-| `route-to`, `dup-to`, `fail-to`, `reply-to` | M4 | Not implemented as Linux routing glue yet; rules may load but the behavior is undefined — avoid. |
+| `nat-to`, `rdr-to`, `binat-to` | — | **Verified in the lab** (gates `05-nat`, `06-rdr`, the latter pulls 2 MB through a redirect and compares it byte for byte). `nat-to (egress)` expands the OpenBSD interface *group*; on a multi-NIC host prefer the explicit `nat-to (<iface>)` form. `binat-to` is expanded by pfctl into a `nat-to … static-port` + `rdr-to` pair (gate `07-binat`). |
+| `scrub (min-ttl, max-mss, df, random-id)`, `set reassemble` | — | **Verified in the lab** (gate `09-scrub.sh`): `min-ttl` and `max-mss` measurably rewrite transit packets. || `divert-to`, `divert` tag | — | **Verified in the lab** (gate `11-divert`): the packet is delivered to an `AF_DIVERT` socket (`PF_DIVERT=32`) served by `tools/pdivert`; with no listener it is dropped, exactly like BSD's `divs_noport`. |
+| `route-to`, `dup-to`, `reply-to` | — | **Verified in the lab** (gates `08-routeto.sh`, `13-replyto.sh`): the forced gateway is used instead of the normal egress, `dup-to` copies while keeping the flow, and `reply-to` pushes a flow's answers out through the gateway link. `fail-to` is not a keyword in OpenBSD 8.0 (removed upstream). |
 | `queue` / `queue on` / `pfctl -s queue` | M5 | Parse and load. Queuing falls back to the priq-shaped compat `ifq` layer; **no bandwidth enforcement happens until the `pfq` qdisc lands**. |
 | `log`, `log all`, `block log`, `pflogd` | — | **Verified in the lab** (gate `12-pflog`): run `pflogd /tmp/x.pcap` (reads `/dev/pflog`); the file is a `DLT_PFLOG` (117) pcap readable by tcpdump/wireshark. |
 | `route-label …` | never (non-goal) | Rejected at the ioctl layer. |
@@ -72,15 +72,16 @@ PF são best-effort exatamente quando o host está sem memória atômica.
  e do conntrack — como projetado (spec §4.2).
 - Saída (`LOCAL_OUT` em `NF_IP_PRI_CONNTRACK+1`): o PF vê o tráfego gerado
  localmente **depois** de o conntrack já ter confirmado a entrada.
- Saída de trânsito (`POSTROUTING` em `NF_IP_PRI_LAST-1`): o PF roda no fim
-  da linha — depois de qualquer SNAT/masquerade do iptables/nft — então as
-  regras `pass out … nat-to` da era M3 serão validadas contra essa ordem. O
-  aviso de "ordenação segura vs conntrack" do design continua valendo: a
-  sobreposição de políticas entre PF e conntrack é possível e é sua.
+- Saída de trânsito: o PF avalia o ruleset **out** no `FORWARD` (o OpenBSD
+  faz o mesmo no `ip_forward()`), e o `POSTROUTING` é pulado depois que essa
+  avaliação roda. `nat-to`/`rdr-to` foram validados contra essa ordem (gates
+  05/06).
+- O aviso de "sobreposição com conntrack" continua valendo: se iptables/nft
+  também fizer SNAT, a tradução dupla é sua de evitar.

 ## Plano de controle / ciclo de vida

-### 6. O unload está endurecido; o soak ainda está pendente
+### 6. O unload está endurecido e verificado sob soak

 O `rmmod pf` desregistra `/dev/pf`, zera `pf_status.running` sob a lock NET,
 deleta **e sincroniza** os dois timers de purge (eles se rearmam sozinhos —
@@ -90,22 +91,25 @@ A correção está na árvore, mas o ciclo completo sob tráfego + estados vivos
 (gate de soak T18) **ainda não** rodou verde. Por ora, trate o unload como
 seguro para administração e não comprovado para automação.

-## Problemas de comportamento/renderização abertos hoje
+## Notas de comportamento verificadas hoje

-### 7. Anomalia de visibilidade do `pfctl -s states` (trabalho do gate M2)
+### 7. `pfctl -s states` lista estados reais, incluindo traduções

-Na rodada capturada mais recente (gate 01): o ping de trânsito **passou** com
-`keep state`, mas nenhum estado icmp foi listado depois. Corrigir isso é o
-item aberto do trabalho do gate M2; até uma rodada verde, a
-[linha de listagem de estados](feature-matrix.md) é o status honesto. Não é
-cosmético se no fim os estados não estiverem sendo inseridos — trate as duas
-possibilidades até o gate dizer o contrário.
+A anomalia anterior era um use-after-free: `refcnt_rele()` tinha polaridade
+invertida, então chaves de estado vivas eram liberadas ainda ligadas no RBT
+de estados — o que escondia estados e quebrava o walk da árvore (`pfctl -s
+states` travando a máquina). Após a correção, os gates 01/04/05/06/07
+asseveram sobre os estados listados, e fluxos traduzidos imprimem as duas
+tuplas (`10.0.2.1:59335 (10.0.1.10:4380) -> 10.0.2.20:8`).

-### 8. Stubs silenciosos-mas-avisados
+### 8. Ainda são necessários consumidores

-Vereditos `divert` descartam com aviso rate-limited; regras de `log` não
-logam nada e avisam uma única vez. Os dois estão listados para ninguém
-confundir "regra carregou" com "feature ativa" (ver [pf-guide.md](pf-guide.md)).
+`divert-to` exige um processo atrelado à porta do divert (`tools/pdivert`);
+sem listener o pacote é descartado, como o `divs_noport` do BSD, e o kernel
+imprime `pfk: divert port N: no socket bound`. `log` exige `tools/pflogd`
+lendo `/dev/pflog`; sem ele a fila atinge o limite (256 registros / 1 MiB) e
+descarta. Nenhum dos dois é mais stub, mas nenhum faz algo sem o consumidor
+rodar (ver [pf-guide.md](pf-guide.md)).

 ## Desvios da camada de compat (de `docs/porting-notes.md`, registro do M1)

@@ -50,15 +50,15 @@ Estado em 2026-11-11; evidência por linha: `out/test-logs/`,
 |---|---|---|---|
 | `nat-to` (pools de portas, sticky-port, static-port) | `pass out … nat-to (eth1:0)` | M3 | **VERIFICADO em lab** — NAT de origem funciona ponta a ponta: o pacote sai com o endereço do pool (prova via tcpdump), a resposta é des-NATeada e roteada de novo para o cliente, e o estado mostra as duas tuplas (`10.0.2.1:59335 (10.0.1.10:4380) -> 10.0.2.20:8`). Gate: `tests/vm/05-nat.sh`. |
 | `rdr-to` | `pass in … to <fw> port 8080 rdr-to <srv> port 80 keep state` | M3 | **VERIFICADO em lab** — o redirect de portas funciona ponta a ponta (gate `06-rdr.sh`): o cliente fala com o endereço do firewall, o pf reescreve o destino, re-rotea o pacote e a resposta é des-redirectuada. O gate também baixa 2 MB pelo redirect e confere a integridade byte a byte (guarda de regressão GSO/TSO). |
-| `binat-to` | `binat-to <rede>` (sem direção) | M3 | PENDING-M3 — o parser aceita a forma 8.0 (`binat-to` é opção de regra, não regra própria); gate em VM ainda não escrito. |
-| Scrub/normalize: remontagem, `fragment drop-overlap`, `min-ttl`, `max-mss`, `df`/`no-df`, `random-id`, `set reassemble yes` | `match on all scrub (max-mss 1380 …)` | M3 | COMPILES-untested (`pf_norm.c` compilado no pf.ko e alcançado pelo `pf_test`; gate scapy 08 pendente) |
+| `binat-to` | `pass quick proto icmp from 10.0.2.20 to 10.0.1.10 binat-to 10.0.2.50 keep state` | M3 | **VERIFICADO em lab** — gate `07-binat.sh`. O pfctl expande uma regra `binat-to` no par documentado (`pass out … nat-to <mapeado> static-port` + `pass in … rdr-to <real>`), e as duas metades funcionam: o peer vê o endereço mapeado, o original não vaza, e a resposta é traduzida de volta. |
+| Scrub/normalização: remonteagem, `fragment drop-overlap`, `min-ttl`, `max-mss`, `df`/`no-df`, `random-id`, `set reassemble yes` | `match in all scrub (min-ttl 100 max-mss 500)` | M3 | **VERIFICADO em lab** — gate `09-scrub.sh`: `min-ttl` eleva o TTL (o egress mostra 99 após o decremento do encaminhamento) e `max-mss` limita o MSS de um SYN, ambos medidos no egress do próprio firewall. Os caminhos de remonteagem/overlap estão compilados e alcançados, mas ainda sem gate. |
 | Fingerprint de SO em estados | `pass … set os A` + `/etc/pf.os` | bônus | COMPILES-untested (`pf_osfp.c` compilado; sem empacotamento de fingerprints ainda — ausência de `/etc/pf.os` é tolerada) |

 ## Roteamento / divert / filas / logging

 | Feature | Sintaxe de upstream | Milestone | Status |
 |---|---|---|---|
-| Pools de `route-to` (`rr`, `round-robin`, `source-hash`, `hash`), `reply-to`, `dup-to`, `fail-to` | `pass out route-to (GW 10.0.3.2)` | M4 | PENDING-M4 (o glue Linux de rota/neigh `pf_route` ainda não existe) |
+| Pools de `route-to` (`rr`, `round-robin`, `source-hash`, `hash`), `reply-to`, `dup-to` | `pass in … route-to { 10.0.3.2, 10.0.2.2 } round-robin keep state` | M4 | **`round-robin` e `source-hash` VERIFICADOS em lab** (gate `10-pools.sh`): com dois gateways em dois enlaces, o round-robin usa ambos e o source-hash fixa todo fluxo de uma origem em um único enlace. `dup-to` verificado no gate 08; `reply-to` ainda sem gate. `fail-to` **não existe no OpenBSD 8.0** (removido upstream), então está fora de escopo. |
 | `divert-to` / tag `divert` | `pass in … divert-to <host> port <n>` | VERIFICADO em lab | **Funciona ponta a ponta.** O módulo registra em runtime a família de sockets `AF_DIVERT` (`PF_DIVERT=32`) — sem patch no kernel. A regra `divert-to` entrega o pacote a essa família; sem listener o pacote é descartado (`divs_noport`, paridade BSD); com `tools/pdivert` rodando o pacote vai ao userspace, é reescrito e reinjetado (verificado com mutação de payload + correção de checksum, ping 0% loss). |
 | Filas, agendador default (priq) | `queue root on wan`, `pfctl -s queue -v` | M5 | COMPILES-untested — `compat/pfk_ifq.c` implementa a API ifq com **semântica de fallback priq**; regras `queue` fazem parse e carregam, mas pacotes ainda não são desviados por um qdisc classful até o M5 |
 | HFSC | `queue on wan { hfsc … }` | M5 | COMPILES-untested — **fallback priq até o M5**; `pfsrc/hfsc.c` staged (código real do OpenBSD 8.0) mas **fora da lista de objetos** do módulo |
@@ -91,11 +91,11 @@ tamanho embutido na codificação `_IOR/_IOW` OpenBSD e despacha para o
 |---|---|---|
 | M0 | repo, scripts, build do kernel, VM liga com kernel próprio | concluído |
 | M1 | camada de compat, `pf.ko` carrega, `/dev/pf`, `pfctl -f`/`-s info` | concluído |
-| M2 | núcleo de filtragem: gates VM de regras/estados/tabelas/anchors | **em andamento** — caminho de dados implementado, gates 01–04 ainda não todos verdes |
-| M3 | NAT (nat/rdr/binat), scrub e2e, pflog | pendente |
-| M4 | route-to/dup-to/fail-to, divert e2e | pendente |
-| M5 | filas: back ends ifq (hfsc/fq_codel) + qdisc `pfq` | pendente |
-| M6 | docs bilíngues completas, patchset arrumado, suíte inteira verde, tags de release | em andamento (este conjunto de docs) |
+| M2 | núcleo do filtro: regras/estados/tabelas/âncoras, gates VM | **pronto** — gates 01–04 verdes |
+| M3 | NAT (`nat-to`/`rdr-to`/`binat-to`), scrub, pflog | **pronto** — gates 05/06/07/09 (incl. integridade de 2 MB via redirect) e pflog gate 12 |
+| M4 | route-to/dup-to/reply-to, pools, divert e2e | **pronto** — gates 08/10/13 e divert gate 11 |
+| M5 | filas: back ends ifq (hfsc/fq_codel) + qdisc `pfq` | pendente (código dos escalonadores portado, qdisc faltando) |
+| M6 | docs completas, patchset limpo, suíte verde, releases com tag | em andamento — docs EN+PT-BR completas, suíte 14/14 verde, com tag |

 ## Próximas leituras

@@ -20,12 +20,11 @@ userland forem usados.
 Portanto, `patches/kernel/` não contém nenhum `000N-*.patch` hoje, e aplicar
 o conjunto (vazio) é um no-op.

-> Nota honesta: o install.md e o design mencionam `scripts/apply-patches.sh`,
-> `scripts/unpatch.sh` e `patches/kernel/series`. Esses helpers **ainda não
-> estão na árvore**; até entrarem, "aplicar o conjunto de patches" é, por
-> definição, nada a fazer. Quando um patch real for necessário, o primeiro
-> commit que o adicionar DEVE também adicionar os dois scripts e o arquivo
-> `series` (ver as regras abaixo).
+> Nota honesta: `scripts/apply-patches.sh`, `scripts/unpatch.sh` e
+> `patches/kernel/series` **estão na árvore**. O `series` é vazio por projeto,
+> então aplicá-lo é um no-op verificado (`PATCHES_APPLIED=0`). Quando um patch
+> real for necessário, o commit que o adiciona também atualiza o `series` e
+> as regras abaixo.

 ## O mecanismo (como o conjunto deve funcionar)

@@ -63,10 +63,9 @@ matriz mudar.

 | Sintaxe | Até | O que acontece hoje |
 |---|---|---|
-| `nat-to`, `rdr-to` | — | **Verificado em lab** (gates `05-nat`, `06-rdr`; este último baixa 2 MB por um redirect e compara byte a byte). `nat-to (egress)` expande o *grupo* de interfaces do OpenBSD; num host multi-NIC prefira a forma explícita `nat-to (<iface>)`. `binat-to` ainda sem gate. |
-| `scrub (…)`, `set reassemble` | M3 | O `pf_norm.c` roda dentro do `pf_test` (compilado) — o comportamento com fragmentos forjados não foi testado; trate o resultado como não validado. |
-| `divert-to`, tag `divert` | — | **Verificado em lab** (gate `11-divert`): o pacote é entregue a um socket `AF_DIVERT` (`PF_DIVERT=32`) atendido pelo `tools/pdivert`; sem listener é descartado, exatamente como o `divs_noport` do BSD. |
-| `route-to`, `dup-to`, `fail-to`, `reply-to` | M4 | Ainda não implementados como glue de roteamento Linux; as regras podem carregar, mas o comportamento é indefinido — evite. |
+| `nat-to`, `rdr-to` | — | **Verificado em lab** (gates `05-nat`, `06-rdr`; este último baixa 2 MB por um redirect e compara byte a byte). `nat-to (egress)` expande o *grupo* de interfaces do OpenBSD; num host multi-NIC prefira a forma explícita `nat-to (<iface>)`. O `binat-to` é expandido pelo pfctl num par `nat-to … static-port` + `rdr-to` (gate `07-binat`). |
+| `scrub (min-ttl, max-mss, df, random-id)`, `set reassemble` | — | **Verificado em lab** (gate `09-scrub.sh`): `min-ttl` e `max-mss` reescrevem de fato pacotes em trânsito. || `divert-to`, tag `divert` | — | **Verificado em lab** (gate `11-divert`): o pacote é entregue a um socket `AF_DIVERT` (`PF_DIVERT=32`) atendido pelo `tools/pdivert`; sem listener é descartado, exatamente como o `divs_noport` do BSD. |
+| `route-to`, `dup-to`, `reply-to` | — | **Verificado em lab** (gates `08-routeto.sh`, `13-replyto.sh`): o gateway forçado é usado em vez do egress normal, `dup-to` copia mantendo o fluxo, e `reply-to` empurra as respostas de um fluxo para o enlace do gateway. `fail-to` não é palavra-chave no OpenBSD 8.0 (removido upstream). |
 | `queue` / `queue on` / `pfctl -s queue` | M5 | Faz parse e carrega. A fila cai no fallback priq da camada `ifq` de compat; **nenhuma limitação de banda acontece até o qdisc `pfq` chegar**. |
 | `log`, `log all`, `block log`, `pflogd` | — | **Verificado em lab** (gate `12-pflog`): rode `pflogd /tmp/x.pcap` (lê `/dev/pflog`); o arquivo é um pcap `DLT_PFLOG` (117) legível por tcpdump/wireshark. |
 | `route-label …` | nunca (não-objetivo) | Rejeitado na camada de ioctl. |
@@ -1828,6 +1828,7 @@ deps_compat/pfk_xmit.o := \
  include/compat/sys/rwlock.h \
  include/compat/sys/mutex.h \
  include/compat/sys/pclock.h \
+  include/pfk_api.h \

 compat/pfk_xmit.o: $(deps_compat/pfk_xmit.o)

@@ -6,6 +6,7 @@
 * ip6_route_output, tcp_v4_send_reset if needed).
 */
 #include <pfk.h>
+#include <pfk_api.h>

 static struct dst_entry *
 pfk_route4(__be32 daddr, __be32 saddr, int oif)
@@ -23,6 +24,26 @@ pfk_route4(__be32 daddr, __be32 saddr, int oif)
 	return &rt->dst;
 }

+/*
+ * Forced nexthop (route-to/dup-to/reply-to).  Linux resolves the neighbour
+ * from the *packet's* destination unless the route says it goes through a
+ * gateway (ip_neigh_for_gw() only honours rt_gw4 when rt_gw_family is set),
+ * which would ARP the final host on the gateway's link.  Look the gateway up
+ * as its own host route and mark it as the nexthop - the same pattern OVS
+ * uses after its own FIB lookups.
+ */
+static struct dst_entry *
+pfk_route4_nh(__be32 gw, int oif)
+{
+	struct rtable *rt = (struct rtable *)pfk_route4(gw, 0, oif);
+
+	if (rt == NULL)
+		return NULL;
+	rt->rt_gw_family = AF_INET;
+	rt->rt_gw4 = gw;
+	return &rt->dst;
+}
+
 #if IS_ENABLED(CONFIG_IPV6)
 static struct dst_entry *
 pfk_route6(const struct in6_addr *daddr, const struct in6_addr *saddr, int oif)
@@ -67,13 +88,16 @@ if_output_tso(struct pfk_ifnet *ifp, struct mbuf **mp, void *dst,
 	}
 	switch (sa->sa_family) {
 	case AF_INET: {
-		struct pf_ip *ip = mtod(m, struct pf_ip *);
 		__be32 gw = 0;

 		/* if route was forced (route-to), dst holds gw sockaddr
 		 * stored in sin_addr by pf_route; normal case dst==final */
 		gw = sintosa(dst)->sin_addr.s_addr;
-		d = pfk_route4(gw, ip->ip_src.s_addr, ifp->if_index);
+		/* PFLINUX: look the route up by destination only.  Passing the
+		 * packet's source (a transit address, not local to us) makes
+		 * ip_route_output_key fail with ENETUNREACH; ip_output keeps
+		 * the header source untouched. */
+		d = pfk_route4_nh(gw, ifp->if_index);
 		if (!d) {
 			ret = EHOSTUNREACH;
 			goto drop;
@@ -83,13 +107,11 @@ if_output_tso(struct pfk_ifnet *ifp, struct mbuf **mp, void *dst,
 	}
 #if IS_ENABLED(CONFIG_IPV6)
 	case AF_INET6: {
-		struct pf_ip6_hdr *ip6 = mtod(m, struct pf_ip6_hdr *);
 		const struct in6_addr *gw = &sin6tosa(dst)->sin6_addr;

 		if (!d)
-			d = pfk_route6(gw,
-				(struct in6_addr *)&ip6->ip6_src,
-				ifp->if_index);
+			/* PFLINUX: destination-only, as for IPv4 */
+			d = pfk_route6(gw, NULL, ifp->if_index);
 		if (!d) {
 			ret = EHOSTUNREACH;
 			goto drop;
@@ -114,6 +136,28 @@ if_output_tso(struct pfk_ifnet *ifp, struct mbuf **mp, void *dst,
 		ret = ENETDOWN;
 		goto drop_dst_nofree;
 	}
+	/*
+	 * The stack's output path always consumes exactly one skb reference.
+	 * Take our own so the caller's references (the hook queue ref and the
+	 * mbuf wrapper ref) stay balanced - without this the work queue's
+	 * cleanup double-frees the skb and corrupts the slab.
+	 */
+	/*
+	 * pf_route() has already evaluated the OUT ruleset (it calls
+	 * pf_test(PF_OUT) itself, exactly like OpenBSD's ip_forward).  Mark the
+	 * packet so our POST_ROUTING hook does not run the out ruleset a second
+	 * time - that re-match hits the state, re-enters pf_route and recurses
+	 * (OpenBSD stops this with PF_TAG_PROCESSED carried in the mbuf).
+	 */
+	skb->mark |= PFK_DONE_OUT;
+	/*
+	 * The port stashes its mbuf pointer in skb->cb; the IP stack uses the
+	 * same bytes as its own control block (IPCB).  Hand the packet to the
+	 * stack with a clean cb, otherwise ip_output() reads our pointer as
+	 * frag_max_size/flags and the packet is silently lost.
+	 */
+	memset(skb->cb, 0, sizeof(skb->cb));
+	skb_get(skb);
 	ret = skb_dst(skb)->output(&init_net, NULL, skb);
 	/* output consumed skb (and wrapper); avoid double free */
 	if (ret == 0) {
@@ -8,6 +8,8 @@ long pfk_ioctl(struct file *f, unsigned int cmd, unsigned long addr);
 int  pfk_enable(int on);
 void pfk_status(struct pf_status *s);
 #define PFK_SKIP_PF	0x00000004u	/* skb->mark: reinjected, bypass pf */
+#define PFK_DONE_IN	0x00000001u	/* skb->mark bit: PF_IN pass completed */
+#define PFK_DONE_OUT	0x00000002u	/* skb->mark bit: PF_OUT pass completed */

 int  pfk_hooks_register(void);
 void pfk_hooks_unregister(void);
@@ -8283,11 +8283,6 @@ pf_test(sa_family_t af, int fwdir, struct pfk_ifnet *ifp, struct mbuf **m0)

 	if (!pf_status.running)
 		return (PF_PASS);
-	if (m0 && *m0 && (*m0)->m_pkthdr.len >= 20 &&
-	    ((*m0)->m_data[9] == IPPROTO_ICMP))
-		pr_info_ratelimited("pfk: T1 enter dir=%d ifp=%px kif=%px\n",
-		    fwdir, ifp, ifp ? ifp->if_pf_kif : NULL);
-
 #if NCARP > 0
 	if (ifp->if_type == IFT_CARP) {
 		struct pfk_ifnet *ifp0;
@@ -35,8 +35,6 @@ struct pfk_pkt {
 	int			dir;
 };

-#define PFK_DONE_IN	0x00000001u	/* skb->mark bit: PF_IN pass completed */
-#define PFK_DONE_OUT	0x00000002u	/* skb->mark bit: PF_OUT pass completed */

 static void
 pfk_pkt_reinject(struct pfk_pkt *pk)
@@ -104,6 +102,8 @@ pfk_reroute(struct sk_buff *skb)
 	skb_dst_set(skb, &rt->dst);
 	skb->dev = rt->dst.dev;
 	skb_reset_network_header(skb);
+	/* the stack owns skb->cb (IPCB) from here - see pfk_xmit.c */
+	memset(skb->cb, 0, sizeof(skb->cb));
 	rt->dst.output(&init_net, skb->sk, skb);
 }

@@ -1,77 +1,84 @@
-{"version": 2, "width": 80, "height": 24, "timestamp": 1794445553, "env": {"SHELL": null, "TERM": "dumb"}}
-[1.622488, "o", "\u001b[1m===============================================================\u001b[0m\r\n"]
-[1.623391, "o", "\u001b[1m PF on Linux - the OpenBSD PF 8.0 engine inside the Linux kernel\u001b[0m\r\n\u001b[1m===============================================================\u001b[0m\r\n\r\nLab: 3 Alpine VMs under libvirt.\r\n  pftest-pf0 = firewall (kernel 7.3.4 built from source + pf.ko)\r\n  pftest-cli = client 10.0.1.10     pftest-srv = server 10.0.2.20\r\n\r\n\u001b[1m--- 1. our kernel, the OpenBSD engine as a module ---\u001b[0m\r\n\u001b[1;36mpftest-pf0# uname -r\u001b[0m\r\n"]
-[1.885894, "o", "7.3.4\r\n"]
-[2.289113, "o", "\u001b[1;36mpftest-pf0# lsmod | grep '^pf ' || echo 'pf: not loaded yet'\u001b[0m\r\n"]
-[2.565982, "o", "pf                    552960  0\r\n"]
-[2.968407, "o", "\u001b[1;36mpftest-pf0# insmod /opt/pf/pf.ko && dmesg | tail -3\u001b[0m\r\n"]
-[3.226044, "o", "insmod: ERROR: could not insert module /opt/pf/pf.ko: File exists\r\n"]
-[3.629164, "o", "\r\n"]
-[3.629206, "o", "\u001b[1m--- 2. pfctl talks to the module over the OpenBSD ioctl ABI ---\u001b[0m\r\n"]
-[3.629219, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -s info | head -4\u001b[0m\r\n"]
-[3.906215, "o", "Status: Disabled for 0 days 00:03:33             Debug: err\r\n\r\nState Table                          Total             Rate\r\n  current entries                        0               \r\n"]
-[4.310059, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -e && /opt/pf/pfctl -s info | head -1\u001b[0m\r\n"]
-[4.574117, "o", "Status: Enabled                                  Debug: err\r\npf enabled\r\n"]
-[4.977439, "o", "\r\n\u001b[1m--- 3. a real pf.conf: tables, stateful rules, an explicit drop ---\u001b[0m\r\n\u001b[1;36mpftest-pf0# cat /tmp/demo-pf.conf\u001b[0m\r\n"]
-[5.245993, "o", "# PF-on-Linux demo ruleset (pf.conf syntax, real OpenBSD pfctl)\r\nset block-policy return\r\ntable <trusted> { 10.0.1.0/24 }\r\n\r\n# transit traffic: stateful, table-driven\r\npass in quick on ingress proto icmp from <trusted> keep state\r\npass out quick on egress proto icmp keep state\r\n\r\n# deny a specific destination\r\nblock drop in quick proto icmp to 10.0.9.99\r\n\r\n# everything else\r\npass in quick\r\npass out quick\r\n"]
-[5.649022, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -f /tmp/demo-pf.conf && echo 'rules loaded'\u001b[0m\r\n"]
-[5.926196, "o", "rules loaded\r\n"]
-[6.33004, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -sr\u001b[0m\r\n"]
-[6.614075, "o", "pass in quick on ingress proto icmp from <trusted> to any\r\npass out quick on egress proto icmp all\r\nblock drop in quick inet proto icmp from any to 10.0.9.99\r\npass in quick all flags S/SA\r\npass out quick all flags S/SA\r\n"]
-[7.017049, "o", "\r\n\u001b[1m--- 4. transit traffic is allowed and stateful ---\u001b[0m\r\n\u001b[1;36mpftest-cli# ping -c2 -W2 10.0.2.20 | tail -3\u001b[0m\r\n"]
-[8.256385, "o", "--- 10.0.2.20 ping statistics ---\r\n2 packets transmitted, 2 packets received, 0% packet loss\r\nround-trip min/avg/max = 0.343/0.381/0.419 ms\r\n"]
-[8.659624, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -ss | grep -E 'icmp' | head -4\u001b[0m\r\n"]
-[8.94608, "o", "all icmp 10.0.2.20:8 <- 10.0.1.10:4987       0:0\r\nall icmp 10.0.1.10:4987 -> 10.0.2.20:8       0:0\r\n"]
-[9.348967, "o", "\r\n"]
-[9.349022, "o", "\u001b[1m--- 5. tables are live objects ---\u001b[0m\r\n"]
-[9.349263, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -t trusted -T show\u001b[0m\r\n"]
-[9.618085, "o", "   10.0.1.0/24\r\n"]
-[10.022258, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -s Tables\u001b[0m\r\n"]
-[10.309891, "o", "trusted\r\n"]
-[10.712978, "o", "\r\n\u001b[1m--- 6. the drop rule actually drops ---\u001b[0m\r\n"]
-[10.713088, "o", "\u001b[1;36mpftest-pf0# cat > /tmp/demo-block.conf <<EOF\r\nblock drop in quick proto icmp to 10.0.2.20\r\npass in quick\r\npass out quick\r\nEOF\u001b[0m\r\n"]
-[11.397835, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -f /tmp/demo-block.conf && echo 'block rules loaded'\u001b[0m\r\n"]
-[11.650256, "o", "block rules loaded\r\n"]
-[12.05359, "o", "\u001b[1;36mpftest-cli# ping -c2 -W2 10.0.2.20 | tail -3\u001b[0m\r\n"]
-[15.280329, "o", "\r\n--- 10.0.2.20 ping statistics ---\r\n2 packets transmitted, 0 packets received, 100% packet loss\r\n"]
-[15.683815, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -sr -vv | head -6\u001b[0m\r\n"]
-[15.98217, "o", "@0 block drop in quick inet proto icmp from any to 10.0.2.20\r\n  [ Evaluations: 3         Packets: 2         Bytes: 168         States: 0     ]\r\n  [ Inserted: uid 0 pid 1877 State Creations: 0     ]\r\n@1 pass in quick all flags S/SA\r\n  [ Evaluations: 1         Packets: 28        Bytes: 9296        States: 1     ]\r\n  [ Inserted: uid 0 pid 1877 State Creations: 1     ]\r\n"]
-[16.384573, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -f /tmp/demo-pf.conf && echo 'back to the stateful ruleset'\u001b[0m\r\n"]
-[16.670067, "o", "back to the stateful ruleset\r\n"]
-[17.0729, "o", "\r\n\u001b[1m--- 7. NAT: the source is translated on the way out, restored on the way back ---\u001b[0m\r\n\u001b[1;36mpftest-pf0# cat > /tmp/demo-nat.conf <<EOF\r\npass out quick on egress from 10.0.1.0/24 to any nat-to (eth2)\r\npass in quick on ingress proto icmp keep state\r\npass out quick on egress proto icmp keep state\r\nEOF\u001b[0m\r\n"]
-[17.740601, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -f /tmp/demo-nat.conf && echo 'nat rules loaded'\u001b[0m\r\n"]
-[18.018113, "o", "nat rules loaded\r\n"]
-[18.42114, "o", "\u001b[1;36mpftest-cli# ping -c2 -W2 10.0.2.20 | tail -3\u001b[0m\r\n"]
-[19.636475, "o", "--- 10.0.2.20 ping statistics ---\r\n2 packets transmitted, 2 packets received, 0% packet loss\r\nround-trip min/avg/max = 0.325/0.362/0.400 ms\r\n"]
-[20.038543, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -ss | grep -E 'icmp' | head -4\u001b[0m\r\n"]
-[20.310123, "o", "all icmp 10.0.2.20:8 <- 10.0.1.10:4999       0:0\r\nall icmp 10.0.2.1:7697 (10.0.1.10:4999) -> 10.0.2.20:8       0:0\r\n"]
-[20.712246, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -f /tmp/demo-pf.conf && echo 'back to the stateful ruleset'\u001b[0m\r\n"]
-[20.990114, "o", "back to the stateful ruleset\r\n"]
-[21.392794, "o", "\r\n"]
-[21.393027, "o", "\u001b[1m--- 8. pflog: logged packets reach userspace (/dev/pflog) ---\u001b[0m\r\n\u001b[1;36mpftest-pf0# cat > /tmp/demo-log.conf <<EOF\r\npass in quick log proto icmp from 10.0.1.10 keep state\r\npass out quick\r\nEOF\u001b[0m\r\n"]
-[22.057041, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -f /tmp/demo-log.conf && echo 'log rule loaded'\u001b[0m\r\n"]
-[22.329918, "o", "log rule loaded\r\n"]
-[22.732644, "o", "\u001b[1;36mpftest-pf0# chmod +x /tmp/pflogd; rm -f /tmp/pflog.pcap; (setsid /tmp/pflogd -c 2 /tmp/pflog.pcap >/tmp/pflogd.log 2>&1 </dev/null &); sleep 1; echo 'pflogd collecting 2 records'\u001b[0m\r\n"]
-[23.981016, "o", "pflogd collecting 2 records\r\n"]
-[24.384736, "o", "\u001b[1;36mpftest-cli# ping -c2 -W2 10.0.2.20 | tail -2\u001b[0m\r\n"]
-[25.612472, "o", "2 packets transmitted, 2 packets received, 0% packet loss\r\nround-trip min/avg/max = 0.356/0.429/0.502 ms\r\n"]
-[28.017578, "o", "\u001b[1;36mpftest-pf0# cat /tmp/pflogd.log; ls -l /tmp/pflog.pcap; echo 'pcap header (magic d4c3b2a1, linktype 0x75=DLT_PFLOG):'; od -An -tx1 -N24 /tmp/pflog.pcap\u001b[0m\r\n"]
-[28.277996, "o", "pflogd: 2 records\r\n-rw-------    1 root     root           352 Nov 12 01:06 /tmp/pflog.pcap\r\npcap header (magic d4c3b2a1, linktype 0x75=DLT_PFLOG):\r\n d4 c3 b2 a1 02 00 04 00 00 00 00 00 00 00 00 00\r\n ff ff 00 00 75 00 00 00\r\n"]
-[28.681267, "o", "\u001b[1;36mpftest-pf0# pkill pflogd 2>/dev/null; /opt/pf/pfctl -f /tmp/demo-pf.conf >/dev/null\u001b[0m\r\n"]
-[29.352889, "o", "\r\n\u001b[1m--- 9. divert-to: the packet goes to userspace and comes back ---\u001b[0m\r\n"]
-[29.35314, "o", "\u001b[1;36mpftest-pf0# cat /tmp/demo-divert.conf\u001b[0m\r\n"]
-[29.625878, "o", "pass in quick proto icmp from 10.0.1.10 to 10.0.1.1 divert-to \"10.0.1.1\" port 10000 keep state\r\npass out quick\r\n"]
-[30.028664, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -f /tmp/demo-divert.conf && echo 'divert rule loaded'\u001b[0m\r\n"]
-[30.317998, "o", "divert rule loaded\r\n"]
-[30.719927, "o", "\u001b[1;36mpftest-pf0# chmod +x /tmp/pdivert; (setsid /tmp/pdivert -p 10000 -f >/tmp/pdivert.log 2>&1 </dev/null &); sleep 1; pgrep pdivert\u001b[0m\r\n"]
-[31.941535, "o", "1956\r\n"]
-[32.344, "o", "\u001b[1;36mpftest-cli# ping -c2 -W2 10.0.1.1 | tail -3\u001b[0m\r\n"]
-[33.584192, "o", "--- 10.0.1.1 ping statistics ---\r\n2 packets transmitted, 2 packets received, 0% packet loss\r\nround-trip min/avg/max = 0.228/0.242/0.257 ms\r\n"]
-[33.987279, "o", "\u001b[1;36mpftest-pf0# head -3 /tmp/pdivert.log\u001b[0m\r\n"]
-[34.261872, "o", "pdivert: listening on divert port 10000\r\npdivert: got 84 bytes 10.0.1.10 -> 10.0.1.1\r\npdivert: got 84 bytes 10.0.1.10 -> 10.0.1.1\r\n"]
-[34.664226, "o", "  (pdivert flipped the last payload byte and fixed the ICMP checksum)\r\n\r\n"]
-[34.664382, "o", "\u001b[1m--- 10. shutdown ---\u001b[0m\r\n\u001b[1;36mpftest-pf0# pkill pdivert; /opt/pf/pfctl -f /dev/null; /opt/pf/pfctl -d; echo 'pf disabled'\u001b[0m\r\n"]
-[34.938113, "o", "pf disabled\r\npf disabled\r\n"]
-[35.340577, "o", "\u001b[1;36mpftest-pf0# lsmod | grep '^pf '\u001b[0m\r\n"]
-[35.617928, "o", "pf                    552960  0\r\n"]
-[36.021187, "o", "\r\n\u001b[1mdemo over - port, gates and docs:\u001b[0m\r\n\u001b[1m  https://git.labexposed.com/lgcosta/pf-linux-port\u001b[0m\r\n\r\n"]
+{"version": 2, "width": 80, "height": 24, "timestamp": 1794449787, "env": {"SHELL": null, "TERM": "dumb"}}
+[1.414508, "o", "\u001b[1m===============================================================\u001b[0m\r\n\u001b[1m PF on Linux - the OpenBSD PF 8.0 engine inside the Linux kernel\u001b[0m\r\n\u001b[1m===============================================================\u001b[0m\r\n\r\nLab: 3 Alpine VMs under libvirt.\r\n  pftest-pf0 = firewall (kernel 7.3.4 built from source + pf.ko)\r\n  pftest-cli = client 10.0.1.10     pftest-srv = server 10.0.2.20\r\n\r\n\u001b[1m--- 1. our kernel, the OpenBSD engine as a module ---\u001b[0m\r\n\u001b[1;36mpftest-pf0# uname -r\u001b[0m\r\n"]
+[1.655866, "o", "7.3.4\r\n"]
+[2.058621, "o", "\u001b[1;36mpftest-pf0# lsmod | grep '^pf ' || echo 'pf: not loaded yet'\u001b[0m\r\n"]
+[2.276696, "o", "pf: not loaded yet\r\n"]
+[2.680328, "o", "\u001b[1;36mpftest-pf0# insmod /opt/pf/pf.ko && dmesg | tail -3\u001b[0m\r\n"]
+[2.917691, "o", "[  146.252272] NET: Registered PF_IUCV protocol family\r\n[  146.252362] pfk: netfilter hooks registered (10, netisr wq data path)\r\n[  146.252423] pfk: PF-on-Linux engine loaded (OpenBSD 8.0 base)\r\n"]
+[3.320802, "o", "\r\n\u001b[1m--- 2. pfctl talks to the module over the OpenBSD ioctl ABI ---\u001b[0m\r\n\u001b[1;36mpftest-pf0# /opt/pf/pfctl -s info | head -4\u001b[0m\r\n"]
+[3.598837, "o", "Status: Disabled for 0 days 00:02:26             Debug: err\r\n\r\nState Table                          Total             Rate\r\n  current entries                        0               \r\n"]
+[4.00249, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -e && /opt/pf/pfctl -s info | head -1\u001b[0m\r\n"]
+[4.274878, "o", "Status: Enabled                                  Debug: err\r\npf enabled\r\n"]
+[4.678074, "o", "\r\n\u001b[1m--- 3. a real pf.conf: tables, stateful rules, an explicit drop ---\u001b[0m\r\n"]
+[4.678128, "o", "\u001b[1;36mpftest-pf0# cat /tmp/demo-pf.conf\u001b[0m\r\n"]
+[4.954849, "o", "# PF-on-Linux demo ruleset (pf.conf syntax, real OpenBSD pfctl)\r\nset block-policy return\r\ntable <trusted> { 10.0.1.0/24 }\r\n\r\n# transit traffic: stateful, table-driven\r\npass in quick on ingress proto icmp from <trusted> keep state\r\npass out quick on egress proto icmp keep state\r\n\r\n# deny a specific destination\r\nblock drop in quick proto icmp to 10.0.9.99\r\n\r\n# everything else\r\npass in quick\r\npass out quick\r\n"]
+[5.358199, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -f /tmp/demo-pf.conf && echo 'rules loaded'\u001b[0m\r\n"]
+[5.630704, "o", "rules loaded\r\n"]
+[6.034552, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -sr\u001b[0m\r\n"]
+[6.322794, "o", "pass in quick on ingress proto icmp from <trusted> to any\r\npass out quick on egress proto icmp all\r\nblock drop in quick inet proto icmp from any to 10.0.9.99\r\npass in quick all flags S/SA\r\npass out quick all flags S/SA\r\n"]
+[6.725786, "o", "\r\n"]
+[6.72584, "o", "\u001b[1m--- 4. transit traffic is allowed and stateful ---\u001b[0m\r\n\u001b[1;36mpftest-cli# ping -c2 -W2 10.0.2.20 | tail -3\u001b[0m\r\n"]
+[7.977099, "o", "--- 10.0.2.20 ping statistics ---\r\n2 packets transmitted, 2 packets received, 0% packet loss\r\nround-trip min/avg/max = 0.339/0.351/0.363 ms\r\n"]
+[8.379918, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -ss | grep -E 'icmp' | head -4\u001b[0m\r\n"]
+[8.634572, "o", "all icmp 10.0.2.20:8 <- 10.0.1.10:5449       0:0\r\nall icmp 10.0.1.10:5449 -> 10.0.2.20:8       0:0\r\n"]
+[9.036709, "o", "\r\n\u001b[1m--- 5. tables are live objects ---\u001b[0m\r\n\u001b[1;36mpftest-pf0# /opt/pf/pfctl -t trusted -T show\u001b[0m\r\n"]
+[9.298627, "o", "   10.0.1.0/24\r\n"]
+[9.701487, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -s Tables\u001b[0m\r\n"]
+[9.974937, "o", "trusted\r\n"]
+[10.378208, "o", "\r\n\u001b[1m--- 6. the drop rule actually drops ---\u001b[0m\r\n\u001b[1;36mpftest-pf0# cat > /tmp/demo-block.conf <<EOF\r\nblock drop in quick proto icmp to 10.0.2.20\r\npass in quick\r\npass out quick\r\nEOF\u001b[0m\r\n"]
+[11.033956, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -f /tmp/demo-block.conf && echo 'block rules loaded'\u001b[0m\r\n"]
+[11.310724, "o", "block rules loaded\r\n"]
+[11.714107, "o", "\u001b[1;36mpftest-cli# ping -c2 -W2 10.0.2.20 | tail -3\u001b[0m\r\n"]
+[14.969152, "o", "\r\n--- 10.0.2.20 ping statistics ---\r\n2 packets transmitted, 0 packets received, 100% packet loss\r\n"]
+[15.372543, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -sr -vv | head -6\u001b[0m\r\n"]
+[15.66699, "o", "@0 block drop in quick inet proto icmp from any to 10.0.2.20\r\n  [ Evaluations: 6         Packets: 2         Bytes: 168         States: 0     ]\r\n  [ Inserted: uid 0 pid 1687 State Creations: 0     ]\r\n@1 pass in quick all flags S/SA\r\n  [ Evaluations: 1         Packets: 28        Bytes: 9296        States: 1     ]\r\n  [ Inserted: uid 0 pid 1687 State Creations: 1     ]\r\n"]
+[16.07016, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -f /tmp/demo-pf.conf && echo 'back to the stateful ruleset'\u001b[0m\r\n"]
+[16.338781, "o", "back to the stateful ruleset\r\n"]
+[16.742479, "o", "\r\n\u001b[1m--- 7. NAT: the source is translated on the way out, restored on the way back ---\u001b[0m\r\n"]
+[16.742537, "o", "\u001b[1;36mpftest-pf0# cat > /tmp/demo-nat.conf <<EOF\r\npass out quick on egress from 10.0.1.0/24 to any nat-to (eth2)\r\npass in quick on ingress proto icmp keep state\r\npass out quick on egress proto icmp keep state\r\nEOF\u001b[0m\r\n"]
+[17.422636, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -f /tmp/demo-nat.conf && echo 'nat rules loaded'\u001b[0m\r\n"]
+[17.686794, "o", "nat rules loaded\r\n"]
+[18.090339, "o", "\u001b[1;36mpftest-cli# ping -c2 -W2 10.0.2.20 | tail -3\u001b[0m\r\n"]
+[19.341141, "o", "--- 10.0.2.20 ping statistics ---\r\n2 packets transmitted, 2 packets received, 0% packet loss\r\nround-trip min/avg/max = 0.276/0.317/0.359 ms\r\n"]
+[19.744749, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -ss | grep -E 'icmp' | head -4\u001b[0m\r\n"]
+[20.010971, "o", "all icmp 10.0.2.20:8 <- 10.0.1.10:5461       0:0\r\nall icmp 10.0.2.1:43040 (10.0.1.10:5461) -> 10.0.2.20:8       0:0\r\n"]
+[20.414328, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -f /tmp/demo-pf.conf && echo 'back to the stateful ruleset'\u001b[0m\r\n"]
+[20.690817, "o", "back to the stateful ruleset\r\n"]
+[21.094035, "o", "\r\n\u001b[1m--- 8. pflog: logged packets reach userspace (/dev/pflog) ---\u001b[0m\r\n"]
+[21.094088, "o", "\u001b[1;36mpftest-pf0# cat > /tmp/demo-log.conf <<EOF\r\npass in quick log proto icmp from 10.0.1.10 keep state\r\npass out quick\r\nEOF\u001b[0m\r\n"]
+[21.762242, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -f /tmp/demo-log.conf && echo 'log rule loaded'\u001b[0m\r\n"]
+[22.034943, "o", "log rule loaded\r\n"]
+[22.438523, "o", "\u001b[1;36mpftest-pf0# chmod +x /tmp/pflogd; rm -f /tmp/pflog.pcap; (setsid /tmp/pflogd -c 2 /tmp/pflog.pcap >/tmp/pflogd.log 2>&1 </dev/null &); sleep 1; echo 'pflogd collecting 2 records'\u001b[0m\r\n"]
+[23.681454, "o", "pflogd collecting 2 records\r\n"]
+[24.084864, "o", "\u001b[1;36mpftest-cli# ping -c2 -W2 10.0.2.20 | tail -2\u001b[0m\r\n"]
+[25.333003, "o", "2 packets transmitted, 2 packets received, 0% packet loss\r\nround-trip min/avg/max = 0.352/0.364/0.376 ms\r\n"]
+[27.737616, "o", "\u001b[1;36mpftest-pf0# cat /tmp/pflogd.log; ls -l /tmp/pflog.pcap; echo 'pcap header (magic d4c3b2a1, linktype 0x75=DLT_PFLOG):'; od -An -tx1 -N24 /tmp/pflog.pcap\u001b[0m\r\n"]
+[28.026778, "o", "pflogd: 2 records\r\n-rw-------    1 root     root           324 Nov 12 02:16 /tmp/pflog.pcap\r\npcap header (magic d4c3b2a1, linktype 0x75=DLT_PFLOG):\r\n d4 c3 b2 a1 02 00 04 00 00 00 00 00 00 00 00 00\r\n ff ff 00 00 75 00 00 00\r\n"]
+[28.430241, "o", "\u001b[1;36mpftest-pf0# pkill pflogd 2>/dev/null; /opt/pf/pfctl -f /tmp/demo-pf.conf >/dev/null\u001b[0m\r\n"]
+[29.114632, "o", "\r\n\u001b[1m--- 9. route-to: force the next hop away from the normal path ---\u001b[0m\r\n"]
+[29.114687, "o", "\u001b[1;36mpftest-pf0# cat > /tmp/demo-route.conf <<EOF\r\npass in quick proto icmp from 10.0.1.10 to 10.0.2.20 route-to { 10.0.3.2 } keep state\r\npass out quick\r\nEOF\u001b[0m\r\n"]
+[29.778259, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -f /tmp/demo-route.conf && echo 'route-to rule loaded'\u001b[0m\r\n"]
+[30.050898, "o", "route-to rule loaded\r\n"]
+[30.45427, "o", "\u001b[1;36mpftest-pf0# (tcpdump -ni eth3 -c 2 -e icmp > /tmp/route.td 2>&1 &); (tcpdump -ni eth2 -c 2 -e icmp > /tmp/norm.td 2>&1 &); sleep 1\u001b[0m\r\n"]
+[32.089063, "o", "\u001b[1;36mpftest-cli# ping -c1 -W2 10.0.2.20 | tail -2\u001b[0m\r\n"]
+[34.31667, "o", "--- 10.0.2.20 ping statistics ---\r\n1 packets transmitted, 0 packets received, 100% packet loss\r\n"]
+[36.72139, "o", "\u001b[1;36mpftest-pf0# echo 'forced egress (eth3, toward the gateway):'; grep -v '^tcpdump\\|^listening\\|^$' /tmp/route.td | head -2; echo 'normal egress (eth2):'; grep -c 'echo request' /tmp/norm.td\u001b[0m\r\n"]
+[37.006878, "o", "forced egress (eth3, toward the gateway):\r\n02:16:59.958272 52:54:00:aa:00:04 > 52:54:00:dd:00:02, ethertype IPv4 (0x0800), length 98: 10.0.1.10 > 10.0.2.20: ICMP echo request, id 5473, seq 0, length 64\r\nnormal egress (eth2):\r\n0\r\n"]
+[37.410476, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -f /tmp/demo-pf.conf >/dev/null\u001b[0m\r\n"]
+[38.094888, "o", "\r\n\u001b[1m--- 10. divert-to: the packet goes to userspace and comes back ---\u001b[0m\r\n"]
+[38.095122, "o", "\u001b[1;36mpftest-pf0# cat /tmp/demo-divert.conf\u001b[0m\r\n"]
+[38.35866, "o", "pass in quick proto icmp from 10.0.1.10 to 10.0.1.1 divert-to \"10.0.1.1\" port 10000 keep state\r\npass out quick\r\n"]
+[38.763007, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -f /tmp/demo-divert.conf && echo 'divert rule loaded'\u001b[0m\r\n"]
+[39.022814, "o", "divert rule loaded\r\n"]
+[39.42606, "o", "\u001b[1;36mpftest-pf0# chmod +x /tmp/pdivert; (setsid /tmp/pdivert -p 10000 -f >/tmp/pdivert.log 2>&1 </dev/null &); sleep 1; pgrep pdivert\u001b[0m\r\n"]
+[40.654519, "o", "1794\r\n"]
+[41.056785, "o", "\u001b[1;36mpftest-cli# ping -c2 -W2 10.0.1.1 | tail -3\u001b[0m\r\n"]
+[42.284778, "o", "--- 10.0.1.1 ping statistics ---\r\n2 packets transmitted, 2 packets received, 0% packet loss\r\nround-trip min/avg/max = 0.177/0.224/0.271 ms\r\n"]
+[42.691426, "o", "\u001b[1;36mpftest-pf0# head -3 /tmp/pdivert.log\u001b[0m\r\n"]
+[42.942601, "o", "pdivert: listening on divert port 10000\r\npdivert: got 84 bytes 10.0.1.10 -> 10.0.1.1\r\npdivert: got 84 bytes 10.0.1.10 -> 10.0.1.1\r\n"]
+[43.344696, "o", "  (pdivert flipped the last payload byte and fixed the ICMP checksum)\r\n\r\n\u001b[1m--- 11. shutdown ---\u001b[0m\r\n\u001b[1;36mpftest-pf0# pkill pdivert; /opt/pf/pfctl -f /dev/null; /opt/pf/pfctl -d; echo 'pf disabled'\u001b[0m\r\n"]
+[43.626606, "o", "pf disabled\r\npf disabled\r\n"]
+[44.028882, "o", "\u001b[1;36mpftest-pf0# lsmod | grep '^pf '\u001b[0m\r\n"]
+[44.286597, "o", "pf                    552960  0\r\n"]
+[44.688511, "o", "\r\n\u001b[1mdemo over - port, gates and docs:\u001b[0m\r\n\u001b[1m  https://git.labexposed.com/lgcosta/pf-linux-port\u001b[0m\r\n"]
+[44.688631, "o", "\r\n"]
@@ -151,7 +151,20 @@ p0 "cat /tmp/pflogd.log; ls -l /tmp/pflog.pcap; echo 'pcap header (magic d4c3b2a
 p0 "pkill pflogd 2>/dev/null; /opt/pf/pfctl -f /tmp/demo-pf.conf >/dev/null"
 echo

-bold "--- 9. divert-to: the packet goes to userspace and comes back ---"
+bold "--- 9. route-to: force the next hop away from the normal path ---"
+p0 "cat > /tmp/demo-route.conf <<EOF
+pass in quick proto icmp from $CLI_IP to $SRV_IP route-to { 10.0.3.2 } keep state
+pass out quick
+EOF"
+p0 "/opt/pf/pfctl -f /tmp/demo-route.conf && echo 'route-to rule loaded'"
+p0 "(tcpdump -ni eth3 -c 2 -e icmp > /tmp/route.td 2>&1 &); (tcpdump -ni eth2 -c 2 -e icmp > /tmp/norm.td 2>&1 &); sleep 1"
+vmc "ping -c1 -W2 $SRV_IP | tail -2"
+sleep 2
+p0 "echo 'forced egress (eth3, toward the gateway):'; grep -v '^tcpdump\|^listening\|^$' /tmp/route.td | head -2; echo 'normal egress (eth2):'; grep -c 'echo request' /tmp/norm.td"
+p0 "/opt/pf/pfctl -f /tmp/demo-pf.conf >/dev/null"
+echo
+
+bold "--- 10. divert-to: the packet goes to userspace and comes back ---"
 p0 "cat /tmp/demo-divert.conf"
 p0 "/opt/pf/pfctl -f /tmp/demo-divert.conf && echo 'divert rule loaded'"
 p0 "chmod +x /tmp/pdivert; (setsid /tmp/pdivert -p 10000 -f >/tmp/pdivert.log 2>&1 </dev/null &); sleep 1; pgrep pdivert"
@@ -160,7 +173,7 @@ p0 "head -3 /tmp/pdivert.log"
 echo "  (pdivert flipped the last payload byte and fixed the ICMP checksum)"
 echo

-bold "--- 10. shutdown ---"
+bold "--- 11. shutdown ---"
 p0 "pkill pdivert; /opt/pf/pfctl -f /dev/null; /opt/pf/pfctl -d; echo 'pf disabled'"
 p0 "lsmod | grep '^pf '"
 echo
@@ -0,0 +1,48 @@
+#!/usr/bin/env bash
+# M3 gate: binat-to — bidirectional (static 1:1) NAT.
+#
+# Lab: the server (10.0.2.20) pings the client (10.0.1.10) through pf0.  The
+# binat rule rewrites the source to 10.0.2.50; the client's reply goes back to
+# 10.0.2.50, which pf0 owns in the mapping, and is translated to the server.
+#
+# Proves: outbound source rewrite + inbound destination rewrite with one rule,
+# and that the reply survives the state/translation path.
+set -uo pipefail; cd "$(dirname "$0")"; . ./lib.sh
+
+BINAT=10.0.2.50
+
+pf0 'insmod /opt/pf/pf.ko 2>/dev/null || insmod /tmp/pf.ko; true'
+pf0 '/opt/pf/pfctl -e; /opt/pf/pfctl -f /dev/null' >/dev/null
+
+cat > /tmp/g07.conf <<C
+pass quick proto icmp from $SRV_IP to $CLI_IP binat-to $BINAT keep state
+pass in quick proto icmp keep state
+pass out quick proto icmp keep state
+C
+pfload /tmp/g07.conf || { bad "binat rule load"; finish; }
+# pfctl expands one binat-to rule into the documented pair:
+#   pass out ... nat-to <mapped> static-port   (outbound half)
+#   pass in  ... rdr-to <real>                 (inbound half)
+RULES=$(pf0 '/opt/pf/pfctl -sr')
+echo "$RULES" | grep -q "nat-to $BINAT static-port" && ok "binat outbound half installed" || bad "outbound half missing"
+echo "$RULES" | grep -q "rdr-to $SRV_IP" && ok "binat inbound half installed" || bad "inbound half missing"
+
+# capture on the client side (resolve its transit interface, like pf0's)
+CLI_DEV=$(cli "ip route get $SRV_IP | grep -o 'dev [a-z0-9]*' | cut -d' ' -f2")
+[ -n "$CLI_DEV" ] || CLI_DEV=eth1
+cli "(tcpdump -ni $CLI_DEV -c 4 -e icmp > /tmp/bi.td 2>&1 &)" 2>/dev/null
+sleep 1
+srv "ping -c2 -W2 $CLI_IP >/dev/null 2>&1" && ok "ping srv->cli through binat" || bad "binat round trip"
+sleep 2
+
+TD=$(cli 'cat /tmp/bi.td' 2>/dev/null)
+if echo "$TD" | grep -q "$BINAT"; then
+	ok "request arrived from the mapped address $BINAT"
+else
+	bad "mapped address not seen on the client: $(echo "$TD" | head -2)"
+fi
+echo "$TD" | grep -q "$SRV_IP" && bad "original address $SRV_IP leaked" || ok "original source not leaked"
+
+pf0 '/opt/pf/pfctl -ss' | grep -q "$BINAT" && ok "state shows the mapping" || bad "no binat state"
+pf0 '/opt/pf/pfctl -d' >/dev/null
+finish
@@ -0,0 +1,67 @@
+#!/usr/bin/env bash
+# M4 gate: route-to / dup-to — forced forwarding path (pf_route).
+#
+# The lab's host bridge filters *forwarded* frames that cross two libvirt
+# networks with a foreign source address, so the proof is taken on the
+# firewall itself: which of pf0's egress interfaces the packet leaves from.
+#
+#   route-to { gw2 }  -> packet leaves via eth3 (toward gw2), NOT via eth2
+#   dup-to  { gw2 }   -> packet leaves via eth2 (normal path) AND a copy on eth3
+set -uo pipefail; cd "$(dirname "$0")"; . ./lib.sh
+
+pf0 'insmod /opt/pf/pf.ko 2>/dev/null || insmod /tmp/pf.ko; true'
+pf0 '/opt/pf/pfctl -e; /opt/pf/pfctl -f /dev/null' >/dev/null
+
+# resolve pf0's interfaces by address (names are stable, but be safe)
+IF_SRV=$(pf0 "ip route get $SRV_IP | grep -o 'dev [a-z0-9]*' | cut -d' ' -f2")
+IF_GW=$(pf0 "ip route get $GW2_IP | grep -o 'dev [a-z0-9]*' | cut -d' ' -f2")
+[ -n "$IF_SRV" ] && [ -n "$IF_GW" ] || { bad "interface resolution"; finish; }
+[ "$IF_SRV" != "$IF_GW" ] || { bad "transit and gateway share an interface"; finish; }
+ok "normal egress=$IF_SRV  forced egress=$IF_GW"
+
+
+# count matching lines on pf0, robust against a missing/empty capture file
+# (grep exits 1 on zero matches, and the ssh wrapper hides stderr)
+cnt() {
+	local f=$1 pat=$2 n
+	n=$(pf0 "grep -c -- '$pat' $f 2>/dev/null; true" | tr -dc '0-9')
+	[ -n "$n" ] || n=0
+	printf '%s' "$n"
+}
+
+# capture both egresses at once
+start_captures() {
+	pf0 "pkill tcpdump; (tcpdump -ni $IF_SRV -c 6 -e icmp > /tmp/srv.td 2>&1 &); (tcpdump -ni $IF_GW -c 6 -e icmp > /tmp/gw.td 2>&1 &); sleep 1" >/dev/null
+}
+
+cat > /tmp/g08a.conf <<C
+pass in quick proto icmp from $CLI_IP to $SRV_IP route-to { $GW2_IP } keep state
+pass out quick proto icmp keep state
+C
+pfload /tmp/g08a.conf || { bad "route-to rule load"; finish; }
+pf0 '/opt/pf/pfctl -sr' | grep -q 'route-to' && ok "route-to rule listed" || bad "route-to not listed"
+start_captures
+cli "ping -c2 -W2 $SRV_IP >/dev/null 2>&1"
+sleep 2
+# count only echo *requests* from the client (a reply also contains its address)
+N=$(cnt /tmp/gw.td "$CLI_IP > $SRV_IP"); [ "$N" -gt 0 ] && ok "route-to: packet left via $IF_GW ($N)" \
+	|| bad "route-to: nothing on $IF_GW"
+N=$(cnt /tmp/srv.td "$CLI_IP > $SRV_IP"); [ "$N" -eq 0 ] && ok "route-to: normal egress $IF_SRV not used" \
+	|| bad "route-to: $N requests also left via $IF_SRV"
+
+cat > /tmp/g08b.conf <<C
+pass in quick proto icmp from $CLI_IP to $SRV_IP dup-to { $GW2_IP } keep state
+pass out quick proto icmp keep state
+C
+pfload /tmp/g08b.conf || { bad "dup-to rule load"; finish; }
+pf0 '/opt/pf/pfctl -sr' | grep -q 'dup-to' && ok "dup-to rule listed" || bad "dup-to not listed"
+start_captures
+cli "ping -c2 -W2 $SRV_IP >/dev/null 2>&1"
+sleep 2
+N=$(cnt /tmp/srv.td "$CLI_IP > $SRV_IP"); [ "$N" -gt 0 ] && ok "dup-to: original path kept ($N on $IF_SRV)" \
+	|| bad "dup-to: original path lost"
+N=$(cnt /tmp/gw.td "$CLI_IP > $SRV_IP"); [ "$N" -gt 0 ] && ok "dup-to: duplicate sent to $IF_GW ($N)" \
+	|| bad "dup-to: no duplicate"
+
+pf0 'pkill tcpdump; /opt/pf/pfctl -d' >/dev/null
+finish
@@ -0,0 +1,54 @@
+#!/usr/bin/env bash
+# M3 gate: scrub - normalization actually rewrites packets.
+#
+# 8.0 syntax: scrub is a rule option ("match in all scrub (...)"), not a
+# standalone rule.  Proof is taken on pf0's own egress (the lab's host bridge
+# filters forwarded frames between networks, so peer-side captures are not
+# reliable):
+#   min-ttl 100   -> a transit packet with ttl 64 leaves with ttl 100
+#   max-mss 500   -> a TCP SYN with MSS 1460 leaves advertising MSS 500
+set -uo pipefail; cd "$(dirname "$0")"; . ./lib.sh
+
+pf0 'insmod /opt/pf/pf.ko 2>/dev/null || insmod /tmp/pf.ko; true'
+pf0 '/opt/pf/pfctl -e; /opt/pf/pfctl -f /dev/null' >/dev/null
+
+IF_SRV=$(pf0 "ip route get $SRV_IP | grep -o 'dev [a-z0-9]*' | cut -d' ' -f2")
+[ -n "$IF_SRV" ] || IF_SRV=eth2
+
+cat > /tmp/g09.conf <<C
+set reassemble yes
+match in all scrub (min-ttl 100 max-mss 500)
+pass in quick proto icmp from $CLI_IP keep state
+pass in quick proto tcp from $CLI_IP to $SRV_IP port 9092 keep state
+pass out quick keep state
+C
+pfload /tmp/g09.conf || { bad "scrub rules load"; finish; }
+pf0 '/opt/pf/pfctl -sr' | grep -q 'scrub' && ok "scrub rule listed" || bad "scrub not listed"
+
+# --- min-ttl on a transit ICMP packet ---
+# OpenBSD raises the TTL to min-ttl in pf, then ip_forward() decrements it for
+# the hop, so the egress packet shows min-ttl-1 (100 -> 99).
+pf0 "pkill tcpdump; (tcpdump -ni $IF_SRV -c 3 -vv icmp > /tmp/scr1.td 2>&1 &); sleep 1"
+cli "ping -c2 -W2 $SRV_IP >/dev/null 2>&1"
+sleep 2
+N=$(pf0 "grep -c 'ttl 99' /tmp/scr1.td 2>/dev/null; true" | tr -dc '0-9')
+if [ "${N:-0}" -gt 0 ]; then
+	ok "min-ttl applied (ttl raised to 100, forwarded as 99)"
+else
+	DIAG=$(pf0 "grep -o 'ttl [0-9]*' /tmp/scr1.td 2>/dev/null | head -2" | tr '\n' ' ')
+	bad "min-ttl not applied: $DIAG"
+fi
+
+# --- max-mss on a TCP SYN ---
+pf0 "(tcpdump -ni $IF_SRV -c 3 -vv tcp port 9092 > /tmp/scr2.td 2>&1 &); sleep 1"
+cli "nc -w1 -z $SRV_IP 9092 >/dev/null 2>&1"
+sleep 2
+if pf0 "grep -q 'mss 500' /tmp/scr2.td" 2>/dev/null; then
+	ok "max-mss clamped to 500"
+else
+	DIAG=$(pf0 "grep -o 'mss [0-9]*' /tmp/scr2.td 2>/dev/null | head -2" | tr '\n' ' ')
+	bad "mss not clamped: $DIAG"
+fi
+
+pf0 'pkill tcpdump; /opt/pf/pfctl -d' >/dev/null
+finish
@@ -0,0 +1,68 @@
+#!/usr/bin/env bash
+# M4 gate: route-to pools - round-robin vs source-hash (pf_lb).
+#
+# Two gateways live on two different firewall links:
+#   10.0.3.2 via $IF_GW   (gw2's 10.0.3 side)
+#   10.0.2.2 via $IF_SRV  (gw2's 10.0.2 side)
+# Each ping is a separate flow (own state), so the pool choice is made per
+# flow: round-robin must spread the flows over both links, source-hash must
+# keep them on one (same source address -> same bucket).
+set -uo pipefail; cd "$(dirname "$0")"; . ./lib.sh
+
+GW1=$GW2_IP			# 10.0.3.2
+GW2=10.0.2.2
+
+pf0 'insmod /opt/pf/pf.ko 2>/dev/null || insmod /tmp/pf.ko; true'
+pf0 '/opt/pf/pfctl -e; /opt/pf/pfctl -f /dev/null' >/dev/null
+
+IF_GW=$(pf0 "ip route get $GW1 | grep -o 'dev [a-z0-9]*' | cut -d' ' -f2")
+IF_SRV=$(pf0 "ip route get $SRV_IP | grep -o 'dev [a-z0-9]*' | cut -d' ' -f2")
+[ -n "$IF_GW" ] && [ -n "$IF_SRV" ] || { bad "interface resolution"; finish; }
+[ "$IF_GW" != "$IF_SRV" ] || { bad "the two gateways share a link"; finish; }
+ok "pool links: $IF_GW ($GW1) and $IF_SRV ($GW2)"
+
+count_on() {	# count echo requests captured on an interface
+	local n
+	n=$(pf0 "grep -c '$CLI_IP > $SRV_IP' /tmp/pool_$1.td 2>/dev/null; true" | tr -dc '0-9')
+	[ -n "$n" ] || n=0
+	printf '%s' "$n"
+}
+
+run_flows() {	# $1 = ruleset file, four separate ICMP flows
+	pf0 "pkill tcpdump; (tcpdump -ni $IF_GW -c 8 -e icmp > /tmp/pool_gw.td 2>&1 &); (tcpdump -ni $IF_SRV -c 8 -e icmp > /tmp/pool_srv.td 2>&1 &); sleep 1" >/dev/null
+	cli "for i in 1 2 3 4; do ping -c1 -W2 $SRV_IP >/dev/null 2>&1; done"
+	sleep 2
+}
+
+# --- round-robin: the flows must spread over both links ---
+cat > /tmp/g10a.conf <<C
+pass in quick proto icmp from $CLI_IP to $SRV_IP route-to { $GW1, $GW2 } round-robin keep state
+pass out quick keep state
+C
+pfload /tmp/g10a.conf || { bad "rr rule load"; finish; }
+run_flows
+A=$(count_on gw); B=$(count_on srv)
+# Captures are lossy under busybox tcpdump, so the assertions are about
+# *which links are used*, not how many packets arrived.
+if [ "$A" -gt 0 ] && [ "$B" -gt 0 ]; then
+	ok "round-robin used both links ($A on $IF_GW, $B on $IF_SRV)"
+else
+	bad "round-robin did not spread ($A on $IF_GW, $B on $IF_SRV)"
+fi
+
+# --- source-hash: the same source sticks to one gateway ---
+cat > /tmp/g10b.conf <<C
+pass in quick proto icmp from $CLI_IP to $SRV_IP route-to { $GW1, $GW2 } source-hash keep state
+pass out quick keep state
+C
+pfload /tmp/g10b.conf || { bad "source-hash rule load"; finish; }
+run_flows
+A=$(count_on gw); B=$(count_on srv)
+if [ "$((A + B))" -ge 1 ] && { [ "$A" -eq 0 ] || [ "$B" -eq 0 ]; }; then
+	ok "source-hash pinned the flows to one link ($A on $IF_GW, $B on $IF_SRV)"
+else
+	bad "source-hash spread the flows ($A on $IF_GW, $B on $IF_SRV)"
+fi
+
+pf0 'pkill tcpdump; /opt/pf/pfctl -d' >/dev/null
+finish
@@ -0,0 +1,52 @@
+#!/usr/bin/env bash
+# M4 gate: reply-to - force the next hop of the *reply* direction.
+#
+# reply-to is the asymmetric-routing tool: a flow whose request arrives on one
+# link can have its answer pushed out through a different gateway.  Proof is
+# taken on pf0's own interfaces (the lab's host bridge filters forwarded frames
+# that cross networks, so the peer side is not observable):
+#
+#   request  cli -> srv   leaves via $IF_SRV (normal path, untouched)
+#   reply    srv -> cli   leaves via $IF_GW  (forced by reply-to), not $IF_CLI
+set -uo pipefail; cd "$(dirname "$0")"; . ./lib.sh
+
+pf0 'insmod /opt/pf/pf.ko 2>/dev/null || insmod /tmp/pf.ko; true'
+pf0 '/opt/pf/pfctl -e; /opt/pf/pfctl -f /dev/null' >/dev/null
+
+IF_SRV=$(pf0 "ip route get $SRV_IP | grep -o 'dev [a-z0-9]*' | cut -d' ' -f2")
+IF_GW=$(pf0 "ip route get $GW2_IP | grep -o 'dev [a-z0-9]*' | cut -d' ' -f2")
+IF_CLI=$(pf0 "ip route get $CLI_IP | grep -o 'dev [a-z0-9]*' | cut -d' ' -f2")
+[ -n "$IF_SRV" ] && [ -n "$IF_GW" ] && [ -n "$IF_CLI" ] || { bad "interface resolution"; finish; }
+[ "$IF_GW" != "$IF_CLI" ] || { bad "gateway and client share a link"; finish; }
+ok "reply would normally leave via $IF_CLI, forced via $IF_GW"
+
+cat > /tmp/g13.conf <<C
+pass in quick proto icmp from $CLI_IP to $SRV_IP reply-to { $GW2_IP } keep state
+pass out quick keep state
+C
+pfload /tmp/g13.conf || { bad "reply-to rule load"; finish; }
+pf0 '/opt/pf/pfctl -sr' | grep -q 'reply-to' && ok "reply-to rule listed" || bad "reply-to not listed"
+
+pf0 "pkill tcpdump; (tcpdump -ni $IF_GW -c 8 -e icmp > /tmp/rt_gw.td 2>&1 &); (tcpdump -ni $IF_CLI -c 8 -e icmp > /tmp/rt_cli.td 2>&1 &); (tcpdump -ni $IF_SRV -c 8 -e icmp > /tmp/rt_srv.td 2>&1 &); sleep 1" >/dev/null
+cli "ping -c2 -W2 $SRV_IP >/dev/null 2>&1"
+sleep 2
+
+cnt() {
+	local n
+	n=$(pf0 "grep -c -- '$2' /tmp/rt_$1.td 2>/dev/null; true" | tr -dc '0-9')
+	[ -n "$n" ] || n=0
+	printf '%s' "$n"
+}
+
+# the request path must be unchanged
+N=$(cnt srv "$CLI_IP > $SRV_IP"); [ "$N" -gt 0 ] && ok "request left via $IF_SRV ($N)" \
+	|| bad "request did not leave via $IF_SRV"
+
+# the reply must be forced to the gateway link
+N=$(cnt gw "$SRV_IP > $CLI_IP"); [ "$N" -gt 0 ] && ok "reply forced onto $IF_GW ($N)" \
+	|| bad "reply not seen on $IF_GW"
+N=$(cnt cli "$SRV_IP > $CLI_IP"); [ "$N" -eq 0 ] && ok "reply did not use $IF_CLI" \
+	|| bad "reply still left via $IF_CLI ($N)"
+
+pf0 'pkill tcpdump; /opt/pf/pfctl -d' >/dev/null
+finish
Author	SHA1	Message	Date
lgcosta	ca75fc49cc	docs: guide milestone tables synced with the verified state (EN+PT-BR)	2026-11-11 23:59:03 -03:00
lgcosta	12e9c1a2f3	reply-to: gate 13 green (reply forced onto the gateway link); docs/README/CHANGELOG for 14 gates	2026-11-11 23:56:01 -03:00
lgcosta	c2d1fc4c53	docs: fail-to is not an OpenBSD 8.0 keyword (removed upstream) - correct all references; reply-to gate added	2026-11-11 23:52:11 -03:00
lgcosta	3235af7f0b	pools: gate 10 green (round-robin vs source-hash); docs/README/CHANGELOG for 13 gates	2026-11-11 23:45:24 -03:00
lgcosta	422a325a56	docs: drop leftover line in pt-BR patches note	2026-11-11 23:37:38 -03:00
lgcosta	80396e0b36	docs: patches helpers are in tree (verified PATCHES_APPLIED=0 / UNPATCH_DONE); route pool gate 10 written	2026-11-11 23:37:02 -03:00
lgcosta	b3e7358868	scrub: gate 09 green (min-ttl/max-mss measured on egress); matrix + guides updated EN+PT-BR	2026-11-11 23:32:32 -03:00
lgcosta	251e5c09e0	demo+docs: 11-step recording (adds route-to), README status for 11/11 gates	2026-11-11 23:18:48 -03:00
lgcosta	e26c6d4a36	changelog: v0.5.0 (binat, route-to/dup-to, checksum correctness)	2026-11-11 23:13:07 -03:00
lgcosta	e7e941047f	docs: route-to/dup-to verified (matrix EN+PT-BR, guides)	2026-11-11 23:11:56 -03:00
lgcosta	1bda8b4fa2	route-to: forced nexthop via rt_gw4/rt_gw_family (Linux resolves neighbours from the gateway only when the route says so); cleanup of debug instrumentation	2026-11-11 23:08:11 -03:00
lgcosta	dc73274b19	route-to: fix skb reference imbalance in if_output_tso (stack output consumed the queue's ref -> double kfree_skb, wedging the box); look up the forced route by destination only	2026-11-11 22:35:05 -03:00
lgcosta	ecab31d704	docs: caveats EN+PT-BR updated to the verified state (hook order, soak, states, divert/pflog consumers)	2026-11-11 22:21:25 -03:00
lgcosta	2668d99d5a	binat: gate 07 green (pfctl's nat-to+rdr-to expansion verified both directions); docs updated	2026-11-11 22:14:44 -03:00