demo: pflog step in the asciinema recording (pflogd: 2 records, DLT_PFLOG pcap); re-recorded cast

runner: deploy pdivert/pflogd alongside pf.ko and pfctl
docs: guides reflect verified NAT/redirect/divert/pflog and the fixed state listing; CHANGELOG for v0.4.0
2026-11-15 19:04:44 -03:00 · 2026-11-11 22:07:45 -03:00 · 2026-11-11 22:00:15 -03:00 · 2026-11-11 21:58:57 -03:00 · 2026-11-11 21:53:55 -03:00 · 2026-11-11 21:52:36 -03:00
34 changed files with 3490 additions and 231 deletions
@@ -1,5 +1,6 @@
 _vendor/
 out/
 !out/demo-pf-linux.cast
 *.o
 *.ko
 *.mod
@@ -10,6 +11,7 @@ out/
 modules.order
 Module.symvers
 tools/out/
 !out/demo-pf-linux.cast
 lab/seeds/
 *.pcap
 *.log
@@ -0,0 +1,88 @@
 # Changelog
 ## v0.4.0 — 2026-11-11
 Lab-verified milestone release.  The full gate suite (9 gates) runs against
 Alpine VMs under libvirt on the kernel built from source.
 ### Verified in the lab (each has a gate in `tests/vm/`)
 | Capability | Gate | What it proves |
 |---|---|---|
 | Engine loads, `/dev/pf`, real OpenBSD `pfctl` over the ioctl ABI | `01` | `insmod` + `pfctl -s info` + transit ping + state created |
 | `block` / `return` / `return-rst` | `02` | ICMP dropped, TCP refused |
 | `pfr` tables (inline `{ }`, `-T add/del/show`, `-T` block) | `03` | table load, block by table, delete, unblock |
 | Anchors and evaluation order | `04` | anchor child passes the request before the parent `block`, state created by the anchor rule, request dropped after the anchor is flushed |
 | `nat-to` (source NAT) | `05` | tcpdump proof of the rewritten source, reply de-NATed, state shows both tuples |
 | `rdr-to` (port redirect) | `06` | client talks to the firewall, server answers, **2 MB pulled through the redirect byte-identical**, firewall still responsive afterwards |
 | `divert-to` + `AF_DIVERT` + `pdivert` | `11` | drop with no listener (BSD `divs_noport` parity), packet delivered to userspace, rewritten and reinjected |
 | `pflog` + `pflogd` | `12` | logged packets read from `/dev/pflog` and written as a `DLT_PFLOG` pcap containing the real bytes |
 | Soak / unload discipline | `14` | traffic + states with no kernel warnings, clean `rmmod`, reload, no slab growth |
 ### Added
 - `module/pfsrc/pfk_pflog.c` — `/dev/pflog` chardev backend (OpenBSD
  `pflog_hdr` layout, 64-byte records, bounded queue, `poll`, stats ioctl).
 - `tools/pflogd` — reads `/dev/pflog`, writes `DLT_PFLOG` pcap (or a readable
  one-line-per-record summary).
 - `tools/pdivert` — divert(4) relay with payload mutation + ICMP checksum fix.
 - `module/pfsrc/divert_sock.c` — `AF_DIVERT` socket family registered at
  runtime (`PF_DIVERT = 32`, `IPPROTO_DIVERT = 254`), blocking `recvmsg`,
  `poll`, BSD-compatible reinjection.
 - `scripts/demo-session.sh` + `scripts/demo-record.py` + `make demo` —
  asciinema recording of the live lab (`out/demo-pf-linux.cast`).
 - `docs/{en,pt-BR}/`: README, install, build, guide, vm-lab, feature-matrix,
  patches, caveats, demo.  `docs/porting-notes.md` records every deviation.
 - `patches/kernel/` + `scripts/apply-patches.sh` / `scripts/unpatch.sh` —
  kernel-core patch mechanism, empty series by design (verified: the module
  needs no core patch).
 ### Fixed (all found by the VM gates, root causes documented)
 - `skb->mark` bookkeeping used multi-bit values, so the "in pass done" mark
  also satisfied the "out pass done" test: transit traffic never reached the
  **out** ruleset (NAT and egress rules silently ignored).
 - The out ruleset was never evaluated for forwarded packets: added the
  `NF_INET_FORWARD` hook (OpenBSD evaluates it in `ip_forward()`).
 - `refcnt_rele()` had inverted polarity → live state keys freed → `Oops` in
  the state RBT and `pfctl -s states` hangs.
 - `radix` `SALEN()` read the `sa_family` byte as a length (Linux sockaddrs
  have no `sa_len`) → table entries dropped as duplicates.
 - BSD/Linux `LIST_HEAD` macro arity clash in `compat/sys/queue.h`.
 - Reverse NAT: a packet whose destination pf rewrote was delivered to the
  firewall's own stack instead of being re-routed.
 - Checksums across translation: IP header checksum recomputed (OpenBSD's
  `ip_output` does this, Linux does not); transport checksum recomputed in
  software for plain skbs and handed back to the NIC (`CHECKSUM_PARTIAL`) for
  GSO/TSO — the earlier software-only fix corrupted every segment after the
  first and stalled bulk transfers.
 - `divert`: NULL dereference on an empty listener list (wedged the VM),
  port byte order mismatch between `bind` and the queue, L3-only skbs injected
  into an Ethernet device RX path, re-divert loop (solved with a skip mark).
 - `IOC_SIZEBITS` = 14 truncated the rule ioctl struct → `EINVAL` on load.
 - State iteration started at id 1 (kifs never attached), `stoeplitz_init()`
  never called, purge timer re-arming itself at `rmmod`, `rtnl`×`NET` lock
  order inversion in the interface notifier.
 ### Known gaps
 `binat-to`, `route-to`/`dup-to`/`fail-to`, `af-to`, scrub end-to-end
 (normalization code is ported and compiles; the lab gate is not written),
 queues/ALTQ (`hfsc.c`/`fq_codel.c` staged, `pfq` qdisc not implemented),
 `pfsync`/CARP (out of scope).  See `docs/en/feature-matrix.md`.
 ## v0.3.0 — 2026-11-11
 M2 data path: netisr-style workqueue, per-direction marks, state/table/anchor
 gates passing; divert design settled (in-module socket family, no core patch).
 ## v0.2.0 — 2026-11-11
 M1: engine compiles into `pf.ko` (pf.c, pf_ioctl.c with the full ioctl ABI,
 pf_norm, pf_if, pf_lb, pf_table, pf_ruleset, pf_osfp, pf_syncookies, radix,
 md5/sha, ifq), `/dev/pf`, Linux-built OpenBSD `pfctl` talking to it.
 ## v0.1.0 — 2026-11-11
 M0: repo, bilingual scaffold, kernel 7.3.4 build, libvirt Alpine lab with
 custom-kernel VMs (pf0/cli/srv/gw2), build/deploy/test scripts.
@@ -1,7 +1,7 @@
 # PF-on-Linux top-level make
 KERNELDIR ?= $(HOME)/lab/linux_kernel/linux-7.3.4
-.PHONY: module tools kernel test clean vm vm-destroy docs
+.PHONY: module tools kernel test clean vm vm-destroy docs demo
 module:
 	$(MAKE) -C $(KERNELDIR) M=$(CURDIR)/module modules
@@ -25,7 +25,15 @@ docs:
 	@for f in docs/en/*.md; do echo "EN  $$f"; done
 	@for f in docs/pt-BR/*.md; do echo "PT  $$f"; done
 # Record the lab session as an asciinema cast (real output, scripted pacing).
 demo:
 	@mkdir -p out
 	@command -v asciinema >/dev/null 2>&1 || { \
 		echo "asciinema not installed: pip install asciinema"; exit 1; }
 	asciinema rec out/demo-pf-linux.cast -c "bash scripts/demo-session.sh"
 	@echo "-> out/demo-pf-linux.cast (play: asciinema play out/demo-pf-linux.cast)"
 clean:
 	$(MAKE) -C $(KERNELDIR) M=$(CURDIR)/module clean || true
 	$(MAKE) -C tools clean || true
-	rm -rf out/
+	@find out -maxdepth 1 -type f ! -name '*.cast' -delete 2>/dev/null || true
@@ -6,10 +6,12 @@ out-of-tree kernel module — the real PF engine (`pf.c`, `pf_norm.c`,
 real OpenBSD `pfctl` over the `/dev/pf` ioctl ABI. (A second module for the
 `pfq` qdisc arrives with M5.)
-**Status:** early milestones. The engine port and control plane (M1) are
+**Status:** the filter core, stateful filtering, tables, anchors, NAT
-working; the M2 filter core is being gated in the VM lab; everything later
+(`nat-to`/`rdr-to`), `divert` and `pflog` are **verified in a real libvirt
-is pending. Honest per-feature status:
+lab** (9/9 gates green).  The remaining features (binat-to, route-to/dup-to,
 scrub e2e, ALTQ/pfq, af-to) are still pending — honest per-feature status:
 [`docs/en/feature-matrix.md`](docs/en/feature-matrix.md).
 Demo (asciinema): [`docs/en/demo.md`](docs/en/demo.md).
 Design: [`docs/superpowers/specs/2026-11-11-pf-linux-port-design.md`](docs/superpowers/specs/2026-11-11-pf-linux-port-design.md)
 · plan: [`docs/superpowers/plans/2026-11-11-pf-linux-port.md`](docs/superpowers/plans/2026-11-11-pf-linux-port.md).
@@ -19,11 +21,11 @@ Design: [`docs/superpowers/specs/2026-11-11-pf-linux-port-design.md`](docs/super
 |---|---|---|
 | M0 | repo, scripts, kernel build, VM boots the custom kernel | done |
 | M1 | compat layer, `pf.ko` loads, `/dev/pf`, `pfctl -f` / `-s info` | done |
-| M2 | filter core: rules/states/tables/anchors, VM gates 01–04 | **in progress** — data path implemented, gates not all green yet |
+| M2 | filter core: rules/states/tables/anchors, VM gates 01–04 | **done** — gates 01/02/03/04 green |
-| M3 | NAT (nat/rdr/binat), scrub e2e, pflog | pending |
+| M3 | NAT (`nat-to`/`rdr-to`/`binat-to`), scrub e2e | **partial** — `nat-to` (gate 05) and `rdr-to` incl. 2 MB integrity (gate 06) verified; `binat-to` and scrub e2e pending |
-| M4 | route-to/dup-to/fail-to, divert e2e | pending |
+| M4 | route-to/dup-to/fail-to, divert e2e, pflog | **partial** — divert e2e (gate 11) and pflog (gate 12) verified; route-to/dup-to pending |
 | M5 | queues: HFSC/FQ-CoDel back ends + `pfq` qdisc | pending |
-| M6 | docs complete, patchset tidy, full suite green, tagged release | in progress (docs) |
+| M6 | docs complete, patchset tidy, full suite green, tagged release | in progress (docs done, suite 7/7 green) |
 ## What you get (planned)
@@ -34,9 +36,9 @@ Design: [`docs/superpowers/specs/2026-11-11-pf-linux-port-design.md`](docs/super
 | NAT | `nat-to` (port pools, sticky-port, static-port), `rdr-to`, `binat-to` |
 | Scrub | fragment reassembly, overlap drop, `min-ttl`/`max-mss`/`df`/`no-df`, `set reassemble` |
 | QoS (ALTQ successor) | `queue`/`queue on` with OpenBSD 8.0 `ifq` schedulers: **default (priq), HFSC, FQ-CoDel**, attached via the `pfq` qdisc; `pfctl -s queue -v` stats |
-| Divert | `divert-to` / `divert` tag + `pdivert` relay over an in-module `PF_DIVERT` socket family (pending — see `docs/en/feature-matrix.md`) |
+| Divert | `divert-to` / `divert` tag + `pdivert` relay over an in-module `PF_DIVERT` socket family (verified: packet to userspace and back) |
 | Route | `route-to` (pools rr/hash-source/hash-state, `fail-to`), `reply-to`, `dup-to` |
-| Logging | `pflogd` via netlink pflog channel (DLT_PFLOG pcap) |
+| Logging | `pflogd` via `/dev/pflog` (DLT_PFLOG pcap, OpenBSD record layout) |
 | Control | `/dev/pf` (0600), OpenBSD 8.0 ioctl ABI, Linux build of `pfctl` (single binary) |
 Not in scope: `pfsync`/CARP failover, IPsec/pfkey, `ftp-proxy`, per-netns PF
@@ -0,0 +1,41 @@
 # Demo (asciinema)
 The lab can be recorded as an [asciinema](https://asciinema.org) cast.  Every
 byte in the recording is the real output of the lab VMs (Alpine under libvirt
 running the kernel built from source with `pf.ko`); only the pacing is
 scripted.
 ## Record
 ```sh
 pip install asciinema            # once
 bash scripts/vm-run-tests.sh      # or: scripts/vm-create.sh + deploy
 asciinema rec out/demo-pf-linux.cast -c "bash scripts/demo-session.sh"
 ```
 The session walks through:
 1. the lab topology and the freshly loaded `pf.ko` (`dmesg` proof),
 2. `pfctl -s info` over the OpenBSD ioctl ABI,
 3. a real `pf.conf` (table, stateful rules, explicit drop) and `pfctl -sr`,
 4. transit ping + `pfctl -ss` (stateful),
 5. `pfctl -t <table> -T show` (live tables),
 6. a blocked ping (counters with `pfctl -sr -vv`),
 7. `divert-to` with the `pdivert` relay (packet to userspace and back),
 8. shutdown.
 ## Play back
 ```sh
 asciinema play out/demo-pf-linux.cast
 ```
 ## Upload
 ```sh
 asciinema upload out/demo-pf-linux.cast
 ```
 `scripts/demo-record.py` is an alternative recorder that produces the same
 asciicast v2 format without the asciinema binary (it drives the lab over
 ssh and writes the cast itself).
@@ -48,9 +48,9 @@ As of 2026-11-11; source of per-row evidence: `out/test-logs/`,
 | Feature | Upstream syntax | Milestone | Status |
 |---|---|---|---|
-| `nat-to` (port pools, sticky-port, static-port) | `pass out … nat-to (eth1:0)` | M3 | PENDING-M3 (`pf.c`/`pf_lb.c` compiled; wiring + VM gate in M3) |
+| `nat-to` (port pools, sticky-port, static-port) | `pass out … nat-to (eth1:0)` | M3 | **VERIFIED in lab** — source NAT works end to end: the packet leaves with the pool address (tcpdump proof), the reply is de-NATed and re-routed to the client, and the state shows both tuples (`10.0.2.1:59335 (10.0.1.10:4380) -> 10.0.2.20:8`). Gate: `tests/vm/05-nat.sh`. |
-| `rdr-to` | `rdr pass on wan proto tcp from any to any port 8080 -> srv port 80` | M3 | PENDING-M3 |
+| `rdr-to` | `pass in … to <fw> port 8080 rdr-to <srv> port 80 keep state` | M3 | **VERIFIED in lab** — port redirect works end to end (gate `06-rdr.sh`): the client talks to the firewall's address, pf rewrites the destination, re-routes the packet, and the reply is un-redirected. The gate also pulls 2 MB through the redirect and checks byte-for-byte integrity (GSO/TSO regression guard). |
-| `binat-to` | `binat-to …` | M3 | PENDING-M3 |
+| `binat-to` | `binat-to <net>` (no direction) | M3 | PENDING-M3 — parser accepts the 8.0 form (`binat-to` is a rule option, not a standalone rule); VM gate not written yet. |
 | Scrub/normalize: reassembly, `fragment drop-overlap`, `min-ttl`, `max-mss`, `df`/`no-df`, `random-id`, `set reassemble yes` | `match on all scrub (max-mss 1380 …)` | M3 | COMPILES-untested (`pf_norm.c` compiled into pf.ko and reached from `pf_test`; scapy gate 08 pending) |
 | OS fingerprinting in states | `pass … set os A` + `/etc/pf.os` | bonus | COMPILES-untested (`pf_osfp.c` compiled; no fingerprint packaging yet — missing `/etc/pf.os` is tolerated) |
@@ -59,12 +59,12 @@ As of 2026-11-11; source of per-row evidence: `out/test-logs/`,
 | Feature | Upstream syntax | Milestone | Status |
 |---|---|---|---|
 | `route-to` pools (`rr`, `round-robin`, `source-hash`, `hash`), `reply-to`, `dup-to`, `fail-to` | `pass out route-to (GW 10.0.3.2)` | M4 | PENDING-M4 (Linux route/neigh glue `pf_route` not implemented yet) |
-| `divert-to` / `divert` tag | `pass in … divert-to <host> <port>` | M4 e2e pending | **COMPILES (untested)** — AF_DIVERT family (PF_DIVERT=32) implemented in-module (`module/pfsrc/divert_sock.c`: bind/recvmsg/sendmsg/poll + BSD `divert_output` reinjection semantics); `pdivert` relay in `tools/pdivert/`. Packet-level proof is gate `11-divert.sh` (M4) — lab-only until then. |
+| `divert-to` / `divert` tag | `pass in … divert-to <host> port <n>` | VERIFIED in lab | **Works end-to-end.** The module registers an `AF_DIVERT` socket family (`PF_DIVERT=32`) at runtime — no kernel-tree patch. A `divert-to` rule hands the packet to that family; with no listener the packet is dropped (`divs_noport`, BSD parity); with `tools/pdivert` running the packet is delivered to userspace, rewritten and reinjected (verified with a payload mutation + checksum fix, ping 0% loss). |
 | Queues, default scheduler (priq) | `queue root on wan`, `pfctl -s queue -v` | M5 | COMPILES-untested — `compat/pfk_ifq.c` implements the ifq API with **priq fallback semantics**; `queue` rules parse and load, but packets are not actually shunted through a classful qdisc until M5 |
 | HFSC | `queue on wan { hfsc … }` | M5 | COMPILES-untested — **priq fallback until M5**; `pfsrc/hfsc.c` staged (real OpenBSD 8.0 code) but **not in the module object list** |
 | FQ-CoDel | `queue on wan { fq_codel … }` | M5 | COMPILES-untested — same as HFSC: staged, not built |
 | `pfq` qdisc (`tc qdisc … root pfq`) | n/a (Linux glue) | M5 | PENDING-M5 (does not exist in the tree yet) |
-| pflog (`log`, `log all`, `block log`, `pfctl -s log`) + `pflogd` pcap (DLT_PFLOG) | `pass out quick log` | M4 | PENDING-M4 — `pflog_packet()` returns success **without logging** (one-time `pr_warn "packet not logged"`); `if_pflog.c` staged, not built; `pflogd` not ported yet |
+| pflog (`log`, `log all`, `block log`) + `pflogd` pcap (DLT_PFLOG) | `pass out quick log` + `pflogd /tmp/x.pcap` | M4 | **VERIFIED in lab** — gate `12-pflog.sh`: logged packets are queued by the engine, read through `/dev/pflog` and written as a pcap with linktype DLT_PFLOG (117); the dump contains the real packet bytes. Deviation: a chardev instead of an if_pflog/bpf interface (Linux has no bpf over such a virtual interface here); record layout is byte-compatible with OpenBSD `pflog(4)`. `if_pflog.c` stays unbuilt. |
 ## Non-goals (explicit)
@@ -60,34 +60,42 @@ treat them as load-and-pray until the matrix moves them.
 | Syntax | Until | What happens today |
 |---|---|---|
-| `nat-to`, `rdr-to`, `binat-to` | M3 | Rules load (engine code compiled) but translation is unwired/ungated — **do not rely on NAT**; outbound masquerade still comes from iptables/nft during the lab. |
+| `nat-to`, `rdr-to` (`binat-to` pending) | — | **Verified in the lab** (gates `05-nat`, `06-rdr`, the latter pulls 2 MB through a redirect and compares it byte for byte). `nat-to (egress)` expands the OpenBSD interface *group*; on a multi-NIC host prefer the explicit `nat-to (<iface>)` form. |
 | `scrub (…)`, `set reassemble` | M3 | `pf_norm.c` runs inside `pf_test` (compiled) — crafted-fragment behavior is untested; treat results as unvalidated. |
-| `divert-to`, `divert` tag | M4 (+ family per code comments) | Matching packets are **dropped** with a rate-limited kernel warning. Never ship this on a live host. |
+| `divert-to`, `divert` tag | — | **Verified in the lab** (gate `11-divert`): the packet is delivered to an `AF_DIVERT` socket (`PF_DIVERT=32`) served by `tools/pdivert`; with no listener it is dropped, exactly like BSD's `divs_noport`. |
 | `route-to`, `dup-to`, `fail-to`, `reply-to` | M4 | Not implemented as Linux routing glue yet; rules may load but the behavior is undefined — avoid. |
 | `queue` / `queue on` / `pfctl -s queue` | M5 | Parse and load. Queuing falls back to the priq-shaped compat `ifq` layer; **no bandwidth enforcement happens until the `pfq` qdisc lands**. |
-| `log`, `log all`, `block log`, `pflogd` | M4 | Nothing is logged (stub warns once at load: `pfk: … packet not logged`). |
+| `log`, `log all`, `block log`, `pflogd` | — | **Verified in the lab** (gate `12-pflog`): run `pflogd /tmp/x.pcap` (reads `/dev/pflog`); the file is a `DLT_PFLOG` (117) pcap readable by tcpdump/wireshark. |
 | `route-label …` | never (non-goal) | Rejected at the ioctl layer. |
 | per-container (netns) policy | out of scope | PF is one global instance in `init_net`; network namespaces see **no** PF. |
 ## Reading state correctly
-`pfctl -s states` currently has a listing/visibility anomaly being fixed in
+`pfctl -s states` shows the real table, including translations, in OpenBSD's
-the M2 gate work (transit flows can pass while no state is displayed). Until
+own format: a NAT/redirect state prints both tuples, e.g.
-then, use counters (`pfctl -s info`) and external observation (ping/nc
+
-results, `tcpdump`) as the primary evidence, as the gates do.
+```
 all icmp 10.0.2.1:59335 (10.0.1.10:4380) -> 10.0.2.20:8   0:0
 all tcp  10.0.2.20:80 (10.0.1.1:8080) <- 10.0.1.10:42091  FIN_WAIT_2:FIN_WAIT_2
 ```
 The value in parentheses is the *original* address before translation.  A
 state that exists but shows `0:0` counters simply means the engine has not
 seen traffic on it since the last poll; use `pfctl -s info` counters and
 `tcpdump` for traffic evidence, as the gates do.
 ## Kernel log signals you should know
 ```
 pfk: PF-on-Linux engine loaded (OpenBSD 8.0 base)     ← insmod ok
 pfk: netfilter hooks registered (10, netisr wq data path)
-pfk: divert port N: no socket bound      # pdivert not listening on N
+pfk: divert port N: no socket bound      # pdivert/pflogd consumer not listening
 pfk: … packet not logged                              ← pflog stub
 ```
 Unloading stops the purge timers and drains the work queue first (an rmmod
-hang observed in the lab was fixed that way) — but the *soak* run
+hang observed in the lab was fixed that way).  The soak gate (`14-soak.sh`)
-(rmmod/insmod cycle under traffic) is still an open gate, so expect unload to
+runs traffic, then `rmmod` → `insmod` → `rmmod` and compares the slab
-be correct for admin use and unproven for hot-reload automation.
+footprint, so unload/reload is verified for admin use; hot-reload automation
 under sustained load is still not a supported workflow.
 Full list of behavioral divergences: [caveats.md](caveats.md).
@@ -860,3 +860,82 @@ ruleset's block line and the resulting state carries the reply).
  at all) on pf0, which looked like a PF failure.  Both now retry and
  verify the copy, and `vm-run-tests.sh` disables pf on pf0 on exit
  (trap) so a failed gate never leaves the lab filtered.
 ## Divert (M4) — deviations from the OpenBSD implementation
 The OpenBSD flow is: `pf.c` tags the mbuf (`PACKET_TAG_PF_DIVERT`,
 `PF_TAG_DIVERTED`) and `netinet/ip_input.c` hands the tagged packet to the
 divert socket.  Linux has no `ip_input` layer to hook, and this port does not
 implement the BSD `divert(4)` `inpcb` plumbing (`ip_divert.c` is kept in the
 tree as reference but is not built).
 Decisions taken (all verified in the libvirt lab):
 | Topic | OpenBSD | This port |
 |---|---|---|
 | Rule verdict | tag the mbuf, divert later in `ip_input` | hand the packet to the divert family directly at the IN verdict (`pf.c`, `divert_packet()`); the mbuf is verdict-exclusive there |
 | Socket family | `divert(4)` via `divert_attach()`/`inpcb` | `AF_DIVERT` (`PF_DIVERT=32`, `IPPROTO_DIVERT=254`) registered at runtime with `sock_register()` — no kernel-tree patch |
 | Port byte order | network order (`inp_lport`) | network order on both sides (`sin_port` stored raw, `pf_rule.divert.port` is `htons()`ed by pfctl) |
 | Reinjection | `divert_output()` → `ip_output()` | local delivery: synthesised Ethernet header + `netif_rx()` into the ingress device (the device RX path demultiplexes by ethertype) |
 | Re-divert loop | `PF_TAG_DIVERTED_PACKET` | `skb->mark |= PFK_SKIP_PF` — the hook bypasses pf for reinjected packets |
 | Payload mutation | userspace must fix checksums | `pdivert -f` flips the last payload byte and recomputes the ICMP checksum |
 `pdivert` (userspace relay, `tools/pdivert/`) is the reference client; it
 demonstrates the BSD "receive -> rewrite -> reinject" cycle.
 ## Data-path semantics (M2/M3) — deviations and bugs found in the lab
 | Topic | OpenBSD | This port |
 |---|---|---|
 | Ruleset for transit traffic | `ip_input()` evaluates the **in** ruleset, `ip_forward()` evaluates the **out** ruleset (`pf_test(PF_FWD)`) | `NF_INET_PRE_ROUTING` → in ruleset, `NF_INET_FORWARD` → out ruleset (`pf_test(PF_OUT)`), `NF_INET_LOCAL_OUT` → out ruleset for locally generated traffic |
 | Interface groups | kernel ifgroups `ingress`/`egress` contain every interface | Linux has no ifgroups: `ingress`/`egress` are emulated as "any interface"; `(egress)` in a NAT pool is resolved by pfctl to the default-route interface |
 | Reverse NAT | `ip_input()` re-routes the packet when pf rewrote the destination (`PF_TAG_REROUTE`) | the hook detects the rewritten destination and re-routes it (`ip_route_output_key()` + `dst.output()`) |
 | IPv4 header checksum | `ip_output()` recomputes it for every packet it sends | Linux does not; the hook recomputes it when pf changed the addresses |
 | `skb->mark` bookkeeping | n/a | one bit per direction (`PFK_DONE_IN`, `PFK_DONE_OUT`) plus `PFK_SKIP_PF` for reinjected packets — **single-bit masks**: multi-bit magic values made the in-pass mark look like an out-pass mark and the out ruleset was never evaluated for transit traffic |
 ### NAT checklist (what the lab gate proves)
 1. `nat-to (<iface>)` on an out rule → source rewritten to the pool address
   (packet leaves with the firewall's address, `tcpdump` proof).
 2. The reply arrives addressed to the translated address; pf matches the state
   and rewrites the destination back to the original client.
 3. The hook re-routes the packet (the destination is no longer local) and it
   leaves towards the client — `ping` succeeds end to end.
 Without step 3 the reply is delivered to the firewall's own stack and the
 client never sees it (the classic "NAT works one way" symptom).
 ## Checksums across the translation paths
 pf rewrites addresses, so both the IPv4 header checksum and the transport
 checksum must stay correct.  Three cases matter in the lab:
 | Case | What Linux gives us | What the port does |
 |---|---|---|
 | transit packet, no translation | `CHECKSUM_UNNECESSARY` (vhost-net did not materialize the field) | leave it alone — the peer's stack trusts the marker |
 | translated packet, plain skb | same | recompute the transport checksum in software over the new pseudo-header, then the IP header checksum |
 | translated packet, GSO/TSO skb (bulk, locally generated) | `CHECKSUM_PARTIAL` | **do not** compute in software: one computation covers only the first segment. Restore `CHECKSUM_PARTIAL` with `csum_start`/`csum_offset` and let the NIC compute per segment |
 The third row is a regression guard: materializing GSO checksums in software
 made bulk transfers silently stall (the peer dropped every segment after the
 first) and starved `sshd` on the firewall — the box answered ping but not SSH.
 `tests/vm/06-rdr.sh` pulls 2 MB through a redirect and compares the size to
 catch it.
 ## pflog backend (M4) — chardev instead of an if_pflog/bpf interface
 OpenBSD attaches a `pflog0` interface (`if_pflog.c`, `IFT_PFLOG`) and `pflogd`
 reads it through `bpf(4)`.  This port keeps `if_pflog.c` staged but unbuilt and
 implements the consumer as a misc character device:
 | Aspect | OpenBSD | This port |
 |---|---|---|
 | Delivery | `pflog0` interface + bpf | `/dev/pflog` (0600), `read()` returns one record, `poll()` supported |
 | Record | `struct pflog_hdr` (64 bytes) + packet | identical layout (`module/pfsrc/pfk_pflog.c`), packet truncated to 160 bytes like BSD |
 | Consumer | `pflogd` → pcap `DLT_PFLOG` | `tools/pflogd` → pcap `DLT_PFLOG` (117), readable by tcpdump/wireshark |
 | Backpressure | bpf buffer | bounded queue (256 records / 1 MiB), overflow counted in `PFKIOCGETPFLOGSTAT` |
 Rationale: a single consumer is the real use case, and Linux gives no bpf hook
 for a non-`net_device` virtual interface without pulling in a much larger
 amount of the BSD if layer.  The on-disk format is unchanged, so existing
 tooling works.
@@ -0,0 +1,40 @@
 # Demo (asciinema)
 O lab pode ser gravado como um cast do [asciinema](https://asciinema.org).
 Cada byte da gravação é a saída real das VMs do lab (Alpine sobre libvirt,
 rodando o kernel compilado do fonte com o `pf.ko`); só o ritmo é roteirizado.
 ## Gravar
 ```sh
 pip install asciinema            # uma vez
 bash scripts/vm-run-tests.sh      # ou: scripts/vm-create.sh + deploy
 asciinema rec out/demo-pf-linux.cast -c "bash scripts/demo-session.sh"
 ```
 A sessão percorre:
 1. a topologia do lab e o `pf.ko` recém-carregado (prova no `dmesg`),
 2. `pfctl -s info` pela ABI de ioctl do OpenBSD,
 3. um `pf.conf` real (tabela, regras stateful, drop explícito) e `pfctl -sr`,
 4. ping em trânsito + `pfctl -ss` (stateful),
 5. `pfctl -t <tabela> -T show` (tabelas vivas),
 6. um ping bloqueado (contadores com `pfctl -sr -vv`),
 7. `divert-to` com o relay `pdivert` (pacote vai ao userspace e volta),
 8. desligamento.
 ## Reproduzir
 ```sh
 asciinema play out/demo-pf-linux.cast
 ```
 ## Publicar
 ```sh
 asciinema upload out/demo-pf-linux.cast
 ```
 `scripts/demo-record.py` é um gravador alternativo que produz o mesmo formato
 asciicast v2 sem o binário asciinema (ele conduz o lab por ssh e escreve o
 cast diretamente).
@@ -48,9 +48,9 @@ Estado em 2026-11-11; evidência por linha: `out/test-logs/`,
 | Feature | Sintaxe de upstream | Milestone | Status |
 |---|---|---|---|
-| `nat-to` (pools de portas, sticky-port, static-port) | `pass out … nat-to (eth1:0)` | M3 | PENDING-M3 (`pf.c`/`pf_lb.c` compilados; fiação + gate VM no M3) |
+| `nat-to` (pools de portas, sticky-port, static-port) | `pass out … nat-to (eth1:0)` | M3 | **VERIFICADO em lab** — NAT de origem funciona ponta a ponta: o pacote sai com o endereço do pool (prova via tcpdump), a resposta é des-NATeada e roteada de novo para o cliente, e o estado mostra as duas tuplas (`10.0.2.1:59335 (10.0.1.10:4380) -> 10.0.2.20:8`). Gate: `tests/vm/05-nat.sh`. |
-| `rdr-to` | `rdr pass on wan proto tcp from any to any port 8080 -> srv port 80` | M3 | PENDING-M3 |
+| `rdr-to` | `pass in … to <fw> port 8080 rdr-to <srv> port 80 keep state` | M3 | **VERIFICADO em lab** — o redirect de portas funciona ponta a ponta (gate `06-rdr.sh`): o cliente fala com o endereço do firewall, o pf reescreve o destino, re-rotea o pacote e a resposta é des-redirectuada. O gate também baixa 2 MB pelo redirect e confere a integridade byte a byte (guarda de regressão GSO/TSO). |
-| `binat-to` | `binat-to …` | M3 | PENDING-M3 |
+| `binat-to` | `binat-to <rede>` (sem direção) | M3 | PENDING-M3 — o parser aceita a forma 8.0 (`binat-to` é opção de regra, não regra própria); gate em VM ainda não escrito. |
 | Scrub/normalize: remontagem, `fragment drop-overlap`, `min-ttl`, `max-mss`, `df`/`no-df`, `random-id`, `set reassemble yes` | `match on all scrub (max-mss 1380 …)` | M3 | COMPILES-untested (`pf_norm.c` compilado no pf.ko e alcançado pelo `pf_test`; gate scapy 08 pendente) |
 | Fingerprint de SO em estados | `pass … set os A` + `/etc/pf.os` | bônus | COMPILES-untested (`pf_osfp.c` compilado; sem empacotamento de fingerprints ainda — ausência de `/etc/pf.os` é tolerada) |
@@ -59,12 +59,12 @@ Estado em 2026-11-11; evidência por linha: `out/test-logs/`,
 | Feature | Sintaxe de upstream | Milestone | Status |
 |---|---|---|---|
 | Pools de `route-to` (`rr`, `round-robin`, `source-hash`, `hash`), `reply-to`, `dup-to`, `fail-to` | `pass out route-to (GW 10.0.3.2)` | M4 | PENDING-M4 (o glue Linux de rota/neigh `pf_route` ainda não existe) |
-| `divert-to` / tag `divert` | `pass in … divert-to <host> <port>` | e2e pendente (M4) | **COMPILA (não testado)** — família AF_DIVERT (PF_DIVERT=32) implementada no módulo (`module/pfsrc/divert_sock.c`: bind/recvmsg/sendmsg/poll + reinjeção com semântica `divert_output` do BSD); relay `pdivert` em `tools/pdivert/`. A prova fim-a-fim é o gate `11-divert.sh` (M4) — até lá, apenas em lab. |
+| `divert-to` / tag `divert` | `pass in … divert-to <host> port <n>` | VERIFICADO em lab | **Funciona ponta a ponta.** O módulo registra em runtime a família de sockets `AF_DIVERT` (`PF_DIVERT=32`) — sem patch no kernel. A regra `divert-to` entrega o pacote a essa família; sem listener o pacote é descartado (`divs_noport`, paridade BSD); com `tools/pdivert` rodando o pacote vai ao userspace, é reescrito e reinjetado (verificado com mutação de payload + correção de checksum, ping 0% loss). |
 | Filas, agendador default (priq) | `queue root on wan`, `pfctl -s queue -v` | M5 | COMPILES-untested — `compat/pfk_ifq.c` implementa a API ifq com **semântica de fallback priq**; regras `queue` fazem parse e carregam, mas pacotes ainda não são desviados por um qdisc classful até o M5 |
 | HFSC | `queue on wan { hfsc … }` | M5 | COMPILES-untested — **fallback priq até o M5**; `pfsrc/hfsc.c` staged (código real do OpenBSD 8.0) mas **fora da lista de objetos** do módulo |
 | FQ-CoDel | `queue on wan { fq_codel … }` | M5 | COMPILES-untested — igual ao HFSC: staged, não compilado |
 | Qdisc `pfq` (`tc qdisc … root pfq`) | n/a (glue Linux) | M5 | PENDING-M5 (ainda não existe na árvore) |
-| pflog (`log`, `log all`, `block log`, `pfctl -s log`) + pcap do `pflogd` (DLT_PFLOG) | `pass out quick log` | M4 | PENDING-M4 — `pflog_packet()` retorna sucesso **sem logar** (`pr_warn "packet not logged"` uma única vez); `if_pflog.c` staged, não compilado; `pflogd` ainda não portado |
+| pflog (`log`, `log all`, `block log`) + `pflogd` pcap (DLT_PFLOG) | `pass out quick log` + `pflogd /tmp/x.pcap` | M4 | **VERIFICADO em lab** — gate `12-pflog.sh`: os pacotes logados são enfileirados pelo engine, lidos via `/dev/pflog` e gravados como pcap com linktype DLT_PFLOG (117); o dump contém os bytes reais do pacote. Desvio: um chardev em vez de interface if_pflog/bpf (Linux não tem bpf sobre essa interface virtual aqui); o layout do registro é byte-compatível com `pflog(4)` do OpenBSD. `if_pflog.c` permanece fora do build. |
 ## Não-objetivos (explícitos)
@@ -63,12 +63,12 @@ matriz mudar.
 | Sintaxe | Até | O que acontece hoje |
 |---|---|---|
-| `nat-to`, `rdr-to`, `binat-to` | M3 | As regras carregam (código do engine compilado), mas a tradução não está fiada/verificada — **não conte com NAT**; no laboratório o masquerade de saída ainda vem do iptables/nft. |
+| `nat-to`, `rdr-to` | — | **Verificado em lab** (gates `05-nat`, `06-rdr`; este último baixa 2 MB por um redirect e compara byte a byte). `nat-to (egress)` expande o *grupo* de interfaces do OpenBSD; num host multi-NIC prefira a forma explícita `nat-to (<iface>)`. `binat-to` ainda sem gate. |
 | `scrub (…)`, `set reassemble` | M3 | O `pf_norm.c` roda dentro do `pf_test` (compilado) — o comportamento com fragmentos forjados não foi testado; trate o resultado como não validado. |
-| `divert-to`, tag `divert` | M4 (+ família de sockets conforme comentário no código) | Os pacotes casados são **descartados** com um aviso de kernel rate-limited. Nunca coloque isso num host em produção. |
+| `divert-to`, tag `divert` | — | **Verificado em lab** (gate `11-divert`): o pacote é entregue a um socket `AF_DIVERT` (`PF_DIVERT=32`) atendido pelo `tools/pdivert`; sem listener é descartado, exatamente como o `divs_noport` do BSD. |
 | `route-to`, `dup-to`, `fail-to`, `reply-to` | M4 | Ainda não implementados como glue de roteamento Linux; as regras podem carregar, mas o comportamento é indefinido — evite. |
 | `queue` / `queue on` / `pfctl -s queue` | M5 | Faz parse e carrega. A fila cai no fallback priq da camada `ifq` de compat; **nenhuma limitação de banda acontece até o qdisc `pfq` chegar**. |
-| `log`, `log all`, `block log`, `pflogd` | M4 | Nada é logado (o stub avisa uma vez no load: `pfk: … packet not logged`). |
+| `log`, `log all`, `block log`, `pflogd` | — | **Verificado em lab** (gate `12-pflog`): rode `pflogd /tmp/x.pcap` (lê `/dev/pflog`); o arquivo é um pcap `DLT_PFLOG` (117) legível por tcpdump/wireshark. |
 | `route-label …` | nunca (não-objetivo) | Rejeitado na camada de ioctl. |
 | política por container (netns) | fora de escopo | O PF é uma instância global em `init_net`; network namespaces **não** enxergam o PF. |
@@ -85,8 +85,7 @@ primária, exatamente como os gates fazem.
 ```
 pfk: PF-on-Linux engine loaded (OpenBSD 8.0 base)     ← insmod ok
 pfk: netfilter hooks registered (10, netisr wq data path)
-pfk: divert port N: no socket bound      # pdivert não ouvido a porta N
+pfk: divert port N: no socket bound      # nenhum consumidor em pdivert/pflogd
 pfk: … packet not logged                              ← stub do pflog
 ```
 Descarregar o módulo primeiro para os timers de purge e drena a workqueue
@@ -1 +1 @@
-savedcmd_pf.mod := printf '%s\n'   pfsrc/pfk_mod.o pfsrc/pfk_hooks.o compat/pfk_mbuf.o compat/pfk_kern.o compat/pfk_if.o compat/pfk_xmit.o compat/pfk_pcb.o compat/pfk_ip4.o compat/siphash.o compat/toeplitz.o compat/md5.o compat/sha2.o compat/radix.o compat/pfk_rtable.o compat/pfk_ip6.o compat/subr_tree.o pfsrc/pf_ruleset.o pfsrc/pf_table.o pfsrc/pf_norm.o pfsrc/pf_if.o pfsrc/pf_lb.o pfsrc/pf_osfp.o pfsrc/pf_syncookies.o pfsrc/pf_ioctl.o pfsrc/pf.o pfsrc/divert_sock.o compat/pfk_ifq.o | awk '!x[$$0]++ { print("./"$$0) }' > pf.mod
+savedcmd_pf.mod := printf '%s\n'   pfsrc/pfk_mod.o pfsrc/pfk_hooks.o compat/pfk_mbuf.o compat/pfk_kern.o compat/pfk_if.o compat/pfk_xmit.o compat/pfk_pcb.o compat/pfk_ip4.o compat/siphash.o compat/toeplitz.o compat/md5.o compat/sha2.o compat/radix.o compat/pfk_rtable.o compat/pfk_ip6.o compat/subr_tree.o pfsrc/pf_ruleset.o pfsrc/pf_table.o pfsrc/pf_norm.o pfsrc/pf_if.o pfsrc/pf_lb.o pfsrc/pf_osfp.o pfsrc/pf_syncookies.o pfsrc/pf_ioctl.o pfsrc/pf.o pfsrc/divert_sock.o pfsrc/pfk_pflog.o compat/pfk_ifq.o | awk '!x[$$0]++ { print("./"$$0) }' > pf.mod
@@ -20,7 +20,7 @@ pf-objs := pfsrc/pfk_mod.o pfsrc/pfk_hooks.o \
 # ported OpenBSD 8.0 engine
 pf-objs += pfsrc/pf_ruleset.o pfsrc/pf_table.o pfsrc/pf_norm.o pfsrc/pf_if.o \
           pfsrc/pf_lb.o pfsrc/pf_osfp.o pfsrc/pf_syncookies.o \
-           pfsrc/pf_ioctl.o pfsrc/pf.o pfsrc/divert_sock.o
+           pfsrc/pf_ioctl.o pfsrc/pf.o pfsrc/divert_sock.o pfsrc/pfk_pflog.o
 # queueing framework (compat/pfk_ifq.c implements the ifq API with priq
 # semantics; the OpenBSD hfsc/fq_codel back ends land in milestone M5)
@@ -619,16 +619,5 @@ if_congested(void)
 }
-/* ---------------- pflog (netlink backend, milestone M4) ---------------- */
+/* pflog_packet() lives in pfsrc/pfk_pflog.c (the /dev/pflog backend). */
-/* NPFLOG=1: the staged if_pflog.c needs the OpenBSD ifpf layer; until the
+
 * netlink-backed pflog lands (M4), pflog_packet() consumes nothing and
 * reports the missing backend at least once so logged packets are never
 * silently vanished (see docs/porting-notes.md). */
 int
 pflog_packet(struct pf_pdesc *pd, u_int8_t reason, struct pf_rule *rm,
    struct pf_rule *am, struct pf_ruleset *ruleset, struct pf_rule *cm)
 {
 	(void)pd; (void)reason; (void)rm; (void)am; (void)ruleset; (void)cm;
 	pr_warn_once("pfk: pflog backend pending M4; packet not logged\n");
 	return (0);
 }
@@ -1203,4 +1203,14 @@ int	pf_anchor_stack_pop(struct pf_ruleset **, struct pf_rule **,
 struct inpcb;
 void	pf_remove_divert_state(struct inpcb *);
 /* pflog backend (/dev/pflog) - 'L' keeps the namespace clear of DIOC* ('P') */
 struct pfk_pflog_stat {
 	u_int64_t	queued;		/* records currently queued	*/
 	u_int64_t	bytes;		/* bytes queued			*/
 	u_int64_t	dropped;	/* records dropped (queue full)	*/
 	u_int64_t	seq;		/* records produced since load	*/
 };
 #define PFKIOCGETPFLOGSTAT	_IOR('L', 1, struct pfk_pflog_stat)
 #endif /* _PFK_H */
@@ -7,11 +7,13 @@
 long pfk_ioctl(struct file *f, unsigned int cmd, unsigned long addr);
 int  pfk_enable(int on);
 void pfk_status(struct pf_status *s);
-#define PFK_SKIP_PF	0x50460040u	/* skb->mark: reinjected, bypass pf */
+#define PFK_SKIP_PF	0x00000004u	/* skb->mark: reinjected, bypass pf */
 int  pfk_hooks_register(void);
 void pfk_hooks_unregister(void);
 /* AF_DIVERT family (module-internal socket protocol, PF_DIVERT=32) */
 int  pfk_pflog_register(void);
 void pfk_pflog_unregister(void);
 int  pfk_divert_sock_register(void);
 void pfk_divert_sock_unregister(void);
 void divert_packet(struct mbuf *, int, u_int16_t);
@@ -90,6 +90,17 @@ pfi_kif_alloc(const char *kif_name, int mflags)
 		kif->pfik_flags |= PFI_IFLAG_ANY;
 		kif->pfik_flags_new |= PFI_IFLAG_ANY;
 	}
 	/*
 	 * PFLINUX: OpenBSD creates the "ingress" and "egress" interface
 	 * groups (ifgroups) automatically and both contain every interface;
 	 * pf's direction (in/out ruleset) carries the actual meaning.  Linux
 	 * has no ifgroups, so emulate the groups as "any interface".
 	 */
 	if (!strcmp(kif->pfik_name, "ingress") ||
 	    !strcmp(kif->pfik_name, "egress")) {
 		kif->pfik_flags |= PFI_IFLAG_ANY;
 		kif->pfik_flags_new |= PFI_IFLAG_ANY;
 	}
 	return (kif);
 }
@@ -35,8 +35,8 @@ struct pfk_pkt {
 	int			dir;
 };
-#define PFK_DONE_IN	0x50460010u	/* skb->mark bit: PF_IN pass completed */
+#define PFK_DONE_IN	0x00000001u	/* skb->mark bit: PF_IN pass completed */
-#define PFK_DONE_OUT	0x50460020u	/* skb->mark bit: PF_OUT pass completed */
+#define PFK_DONE_OUT	0x00000002u	/* skb->mark bit: PF_OUT pass completed */
 static void
 pfk_pkt_reinject(struct pfk_pkt *pk)
@@ -71,14 +71,117 @@ pfk_pkt_reinject(struct pfk_pkt *pk)
 	kfree(pk);
 }
 /*
 * Reverse NAT: pf rewrote the destination of an inbound packet (the reply
 * of a translated flow).  OpenBSD re-routes such a packet in ip_input()
 * (PF_TAG_REROUTE); Linux has no such layer, so re-route it here and push
 * it out through the new egress device.
 */
 static void
 pfk_reroute(struct sk_buff *skb)
 {
 	struct flowi4	 fl4 = { };
 	struct rtable	*rt;
 	struct iphdr	*iph = ip_hdr(skb);
 	fl4.daddr = iph->daddr;
 	/*
 	 * Leave the source empty: the packet keeps the original source
 	 * (the peer's address), but the kernel would reject a route lookup
 	 * with a non-local source (ENETUNREACH).
 	 */
 	fl4.saddr = 0;
 	fl4.flowi4_oif = 0;
 	fl4.flowi4_mark = 0;
 	fl4.flowi4_scope = RT_SCOPE_UNIVERSE;
 	rt = ip_route_output_key(&init_net, &fl4);
 	if (IS_ERR(rt) || rt == NULL) {
 		kfree_skb(skb);
 		return;
 	}
 	skb_dst_set(skb, &rt->dst);
 	skb->dev = rt->dst.dev;
 	skb_reset_network_header(skb);
 	rt->dst.output(&init_net, skb->sk, skb);
 }
 /*
 * PFLINUX: pf rewrote the IPv4 addresses of the packet (NAT).  OpenBSD's
 * ip_output() recomputes the IP header checksum on every packet it sends;
 * Linux does not, so fix it up here after pf touched the header.
 */
 /*
 * pf rewrote the addresses of this packet.  OpenBSD recomputes the IP header
 * checksum in ip_output() and the transport checksum is patched incrementally
 * by pf - but that only works on a *materialized* checksum, and VM-to-VM
 * traffic routinely arrives with the transport checksum still uncomputed
 * (the sender's offload never completed on the tap).  So recompute from
 * scratch here:
 *
 *   - GSO segments: hand the job back to the NIC (CHECKSUM_PARTIAL with the
 *     transport offset); computing in software would only cover the first
 *     segment and silently corrupt the rest (this is what kills bulk TCP).
 *   - everything else: compute the transport checksum over the new
 *     pseudo-header in software, then the IP header checksum.
 */
 static void
 pfk_fix_csum_after_translate(struct sk_buff *skb)
 {
 	struct iphdr *iph;
 	unsigned int hlen, len;
 	__sum16 *csum;
 	__u8 proto;
 	if (!pskb_may_pull(skb, sizeof(struct iphdr)))
 		return;
 	iph = ip_hdr(skb);
 	hlen = iph->ihl * 4;
 	proto = iph->protocol;
 	len = skb->len - hlen;
 	if (proto == IPPROTO_TCP || proto == IPPROTO_UDP) {
 		if (!pskb_may_pull(skb, hlen + (proto == IPPROTO_TCP ?
 		    sizeof(struct tcphdr) : sizeof(struct udphdr))))
 			return;
 		csum = (__sum16 *)(skb->data + hlen +
 		    (proto == IPPROTO_TCP ? offsetof(struct tcphdr, check)
 					 : offsetof(struct udphdr, check)));
 		if (skb_is_gso(skb)) {
 			skb->csum_start = (unsigned short)
 			    (((unsigned char *)skb->data + hlen) - skb->head);
 			skb->csum_offset = proto == IPPROTO_TCP ?
 			    offsetof(struct tcphdr, check) :
 			    offsetof(struct udphdr, check);
 			skb->ip_summed = CHECKSUM_PARTIAL;
 		} else {
 			*csum = 0;
 			*csum = csum_tcpudp_magic(iph->saddr, iph->daddr, len,
 			    proto, skb_checksum(skb, hlen, len, 0));
 			skb->ip_summed = CHECKSUM_NONE;
 		}
 	}
 	iph = ip_hdr(skb);	/* pskb_may_pull may have moved the data */
 	iph->check = 0;
 	iph->check = csum_fold(csum_partial((__u8 *)iph, iph->ihl * 4, 0));
 }
 static void
 pfk_pkt_work(struct work_struct *w)
 {
 	struct pfk_pkt *pk = container_of(w, struct pfk_pkt, work);
 	struct mbuf *m, *m0;
 	struct sk_buff *orig;
 	__be32 orig_dst = 0, orig_src = 0;
 	int action;
 	if (pk->af == AF_INET && pskb_may_pull(pk->skb, sizeof(struct iphdr))) {
 		orig_dst = ip_hdr(pk->skb)->daddr;
 		orig_src = ip_hdr(pk->skb)->saddr;
 	}
 	m = pfk_m_wrap_skb(pk->skb, pk->af);
 	if (!m) {
 		pfk_pkt_reinject(pk);	/* like "pf not consulted" */
@@ -93,11 +196,27 @@ pfk_pkt_work(struct work_struct *w)
 	 * and packet walk run concurrently from here on — documented. */
 	action = pf_test(pk->af, pk->dir, pk->ifp, &m);
 	if (pk->af == AF_INET && orig_src != 0 &&
 	    pskb_may_pull(pk->skb, sizeof(struct iphdr)) &&
 	    (ip_hdr(pk->skb)->saddr != orig_src ||
 	    ip_hdr(pk->skb)->daddr != orig_dst))
 		pfk_fix_csum_after_translate(pk->skb);
 	if (m == m0) {
 		/* same buffer, possibly rewritten in place: our wrapper
 		 * held one extra skb reference — release it */
 		m_freem(m);
 		if (action == PF_PASS && orig_dst != 0 &&
 		    pk->af == AF_INET && pk->dir == PF_IN &&
 		    pskb_may_pull(pk->skb, sizeof(struct iphdr)) &&
 		    ip_hdr(pk->skb)->daddr != orig_dst) {
 			/* pf translated the destination (reverse NAT):
 			 * re-route instead of delivering locally */
 			pfk_reroute(pk->skb);
 			kfree(pk);
 			return;
 		}
 		if (action == PF_PASS) {
 			pfk_pkt_reinject(pk);
 			return;
@@ -138,17 +257,25 @@ pfk_enqueue_verdict(int af, struct sk_buff *skb,
 	switch (st->hook) {
 	case NF_INET_PRE_ROUTING:
 		/* OpenBSD evaluates the IN ruleset in ip_input(); every
 		 * arriving packet passes it. */
 		dir = PF_IN;
 		break;
 	case NF_INET_FORWARD:
 		/* OpenBSD evaluates the OUT ruleset of a forwarded packet in
 		 * ip_forward() (pf_test(PF_FWD)); the IN ruleset was already
 		 * applied at input time.  Without this, transit traffic never
 		 * sees the "out" rules and state matching for replies is
 		 * impossible. */
 		dir = PF_OUT;
 		break;
 	case NF_INET_LOCAL_OUT:
 	case NF_INET_POST_ROUTING:
 		dir = PF_OUT;
 		break;
 	default:
-		return NF_ACCEPT;	/* LOCAL_IN/FORWARD */
+		return NF_ACCEPT;	/* LOCAL_IN */
 	}
 	    af, skb->len, ntohs(skb->protocol), skb->dev ? skb->dev->name : "?",
 	    skb->mark);
 	if (skb->mark & PFK_SKIP_PF)
 		return NF_ACCEPT;	/* reinjected by divert: don't re-filter */
 	if (skb->mark & (dir == PF_IN ? PFK_DONE_IN : PFK_DONE_OUT))
@@ -162,7 +289,8 @@ pfk_enqueue_verdict(int af, struct sk_buff *skb,
 	if (dir == PF_IN)
 		ifp = pfk_ifp_bydev(st->in ? st->in : skb->dev);
 	else
-		ifp = pfk_ifp_bydev(skb->dev ? skb->dev : st->out);
+		ifp = pfk_ifp_bydev(st->out ? st->out :
 		    (skb->dev ? skb->dev : NULL));
 	if (!ifp)
 		return NF_ACCEPT;
@@ -94,6 +94,8 @@ static struct miscdevice pfk_dev = {
 extern void pf_status_init(void);
 extern int pfattach(int);
 extern int pfk_divert_sock_register(void);
 extern int pfk_pflog_register(void);
 extern void pfk_pflog_unregister(void);
 extern void pfk_divert_sock_unregister(void);
 static void
@@ -133,9 +135,12 @@ static int __init pfk_init(void)
 	pfattach(0);
 	pfk_if_set_event_cb(pfk_pfi_event);
 	pfk_if_iterate(pfk_pfi_attach_one);
-	err = pfk_divert_sock_register();
+	err = pfk_pflog_register();
 	if (err)
 		goto err_if;
 	err = pfk_divert_sock_register();
 	if (err)
 		goto err_pflog;
 	err = pfk_hooks_register();
 	if (err)
 		goto err_divert;
@@ -147,6 +152,8 @@ static int __init pfk_init(void)
 err_hooks:
 	pfk_divert_sock_unregister();
 err_divert:
 	pfk_pflog_unregister();
 err_pflog:
 	pfk_hooks_unregister();
 err_if:
 	pfk_if_exit();
@@ -166,6 +173,7 @@ static void __exit pfk_exit(void)
 	 * workqueues non-empty forever (observed rmmod hang). */
 	misc_deregister(&pfk_dev);
 	pfk_divert_sock_unregister();
 	pfk_pflog_unregister();
 	pfk_net_lock();
 	pf_status.running = 0;
 	pfk_net_unlock();
@@ -0,0 +1,299 @@
 /*
 * PF-on-Linux: pflog backend (ISC 2026)
 *
 * OpenBSD logs matched packets to the pflog(4) *network interface*, which
 * pflogd(8) reads through bpf(4).  Linux has neither if_pflog-style virtual
 * interfaces exposed to bpf nor a DLT_PFLOG consumer, so this port offers the
 * same record format over a character device:
 *
 *	/dev/pflog   (0600)  - read() returns one record at a time:
 *			       struct pflog_hdr followed by the captured
 *			       packet bytes (truncated to PFLOG_COPY_SIZE).
 *
 * poll() reports readable when a record is queued.  A single consumer is the
 * real use case (pflogd), which is why a chardev - not netlink multicast -
 * was chosen; the deviation is documented in docs/porting-notes.md.
 *
 * The records are byte-compatible with OpenBSD's pflog(4) so tcpdump/wireshark
 * read the pcap with linktype DLT_PFLOG (117).
 */
 #include <linux/kernel.h>
 #include <linux/module.h>
 #include <linux/miscdevice.h>
 #include <linux/fs.h>
 #include <linux/mm.h>
 #include <linux/poll.h>
 #include <linux/slab.h>
 #include <linux/spinlock.h>
 #include <linux/wait.h>
 #include <linux/timekeeping.h>
 #include <linux/uaccess.h>
 #include <linux/netdevice.h>
 #include <pfk.h>
 #include <pfk_api.h>
 /* OpenBSD sys/net/pflog.h */
 #define PFLOG_IFNAMSIZ		16
 #define PFLOG_RULESET_NAME_SIZE	16
 #define PFLOG_HDRLEN		64	/* sizeof(struct pfk_pflog_hdr) */
 #define PFLOG_COPY_SIZE		160
 /* byte-compatible with OpenBSD's <net/pflog.h> (sizeof == 64) */
 struct pfk_pflog_hdr {
 	struct {
 		s64		tv_sec;
 		s64		tv_usec;
 	} pflog_ts;
 	u8			pflog_af;
 	u8			pflog_reason;
 	u8			pflog_ifname[PFLOG_IFNAMSIZ];
 	u8			pflog_ruleset[PFLOG_RULESET_NAME_SIZE];
 	u8			pflog_nr;
 	u8			pflog_anc;
 	u8			pflog_flags;
 	u8			pflog_pad[9];
 	u16			pflog_len;	/* captured packet length */
 } __packed;
 struct pfk_pflog_rec {
 	struct list_head	list;
 	size_t			len;	/* header + payload */
 	char			data[];	/* pfk_pflog_hdr, then the packet */
 };
 #define PFK_PFLOG_MAX_QUEUE	256u
 #define PFK_PFLOG_MAX_BYTES	(1u << 20)
 static LIST_HEAD(pflog_list);
 static DEFINE_SPINLOCK(pflog_lock);
 static DECLARE_WAIT_QUEUE_HEAD(pflog_wq);
 static unsigned long pflog_queued, pflog_bytes, pflog_dropped;
 static u64 pflog_seq;
 /*
 * Queue one record.  Called from the packet path (process context here: the
 * data path runs in pfk_netisr_wq), so a GFP_ATOMIC allocation and a bounded
 * queue keep it safe even if a future caller appears in softirq context.
 */
 static void
 pfk_pflog_put(struct pfk_pflog_hdr *hdr, const void *pkt, size_t pktlen)
 {
 	struct pfk_pflog_rec *rec;
 	size_t len = sizeof(*hdr) + pktlen;
 	unsigned long flags;
 	if (READ_ONCE(pflog_queued) >= PFK_PFLOG_MAX_QUEUE ||
 	    READ_ONCE(pflog_bytes) + len > PFK_PFLOG_MAX_BYTES) {
 		pflog_dropped++;
 		return;
 	}
 	rec = kmalloc(struct_size(rec, data, len), GFP_ATOMIC);
 	if (rec == NULL) {
 		pflog_dropped++;
 		return;
 	}
 	rec->len = len;
 	memcpy(rec->data, hdr, sizeof(*hdr));
 	memcpy(rec->data + sizeof(*hdr), pkt, pktlen);
 	spin_lock_irqsave(&pflog_lock, flags);
 	list_add_tail(&rec->list, &pflog_list);
 	pflog_queued++;
 	pflog_bytes += len;
 	pflog_seq++;
 	spin_unlock_irqrestore(&pflog_lock, flags);
 	wake_up_interruptible(&pflog_wq);
 }
 /* copy the leading bytes of the packet (the mbuf chain may be a wrapper) */
 static size_t
 pfk_pflog_pull(struct mbuf *m, char *buf, size_t want)
 {
 	struct sk_buff *skb = m ? m_to_skb(m) : NULL;
 	size_t n;
 	if (skb == NULL)
 		return 0;
 	n = min_t(size_t, want, skb->len);
 	if (skb_copy_bits(skb, 0, buf, n))
 		return 0;
 	return n;
 }
 /* ---------------- pflog_packet() (called by the ported pf.c) ---------------- */
 int
 pflog_packet(struct pf_pdesc *pd, u_int8_t reason, struct pf_rule *rm,
    struct pf_rule *am, struct pf_ruleset *ruleset, struct pf_rule *cm)
 {
 	struct pfk_pflog_hdr hdr;
 	char pkt[PFLOG_COPY_SIZE];
 	struct pf_rule *r;
 	size_t n;
 	if (pd == NULL || pd->m == NULL)
 		return (0);
 	memset(&hdr, 0, sizeof(hdr));
 	{
 		struct timespec64 ts;
 		ktime_get_real_ts64(&ts);
 		hdr.pflog_ts.tv_sec = ts.tv_sec;
 		hdr.pflog_ts.tv_usec = ts.tv_nsec / 1000;
 	}
 	hdr.pflog_af = pd->af;
 	hdr.pflog_reason = reason;
 	r = cm ? cm : (am ? am : rm);
 	if (r) {
 		strscpy((char *)hdr.pflog_ifname,
 		    pd->kif ? pd->kif->pfik_name : "",
 		    sizeof(hdr.pflog_ifname));
 		strscpy((char *)hdr.pflog_ruleset,
 		    (ruleset && ruleset->anchor) ? ruleset->anchor->name : "",
 		    sizeof(hdr.pflog_ruleset));
 		hdr.pflog_nr = r->nr;
 		hdr.pflog_anc = 0;
 		hdr.pflog_flags = r->log;
 	}
 	n = pfk_pflog_pull(pd->m, pkt, sizeof(pkt));
 	hdr.pflog_len = cpu_to_be16(n);	/* BSD stores it in network order */
 	pfk_pflog_put(&hdr, pkt, n);
 	return (0);
 }
 /* ---------------- /dev/pflog ---------------- */
 static int
 pfk_pflog_open(struct inode *inode, struct file *filp)
 {
 	filp->private_data = NULL;
 	return nonseekable_open(inode, filp);
 }
 static ssize_t
 pfk_pflog_read(struct file *filp, char __user *buf, size_t count, loff_t *ppos)
 {
 	struct pfk_pflog_rec *rec = filp->private_data;
 	unsigned long flags;
 	size_t off, n;
 	spin_lock_irqsave(&pflog_lock, flags);
 	if (rec == NULL) {
 		if (list_empty(&pflog_list)) {
 			spin_unlock_irqrestore(&pflog_lock, flags);
 			if (filp->f_flags & O_NONBLOCK)
 				return -EAGAIN;
 			if (wait_event_interruptible(pflog_wq,
 			    !list_empty(&pflog_list)))
 				return -ERESTARTSYS;
 			spin_lock_irqsave(&pflog_lock, flags);
 			if (list_empty(&pflog_list)) {
 				spin_unlock_irqrestore(&pflog_lock, flags);
 				return -EAGAIN;
 			}
 		}
 		rec = list_first_entry(&pflog_list, struct pfk_pflog_rec, list);
 		list_del(&rec->list);
 		pflog_queued--;
 		pflog_bytes -= rec->len;
 		filp->private_data = rec;
 	}
 	spin_unlock_irqrestore(&pflog_lock, flags);
 	/* partial reads resume through *ppos (record is pinned to the file) */
 	off = *ppos;
 	if (off >= rec->len) {
 		kfree(rec);
 		filp->private_data = NULL;
 		*ppos = 0;
 		return 0;
 	}
 	n = min(count, rec->len - off);
 	if (copy_to_user(buf, rec->data + off, n))
 		return -EFAULT;
 	*ppos += n;
 	if (*ppos == rec->len) {
 		kfree(rec);
 		filp->private_data = NULL;
 		*ppos = 0;
 	}
 	return n;
 }
 static __poll_t
 pfk_pflog_poll(struct file *filp, poll_table *wait)
 {
 	__poll_t mask = 0;
 	poll_wait(filp, &pflog_wq, wait);
 	if (filp->private_data != NULL)
 		mask |= EPOLLIN | EPOLLRDNORM;
 	else if (!list_empty(&pflog_list))
 		mask |= EPOLLIN | EPOLLRDNORM;
 	return mask;
 }
 static long
 pfk_pflog_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 {
 	struct pfk_pflog_stat st;
 	switch (cmd) {
 	case PFKIOCGETPFLOGSTAT:
 		st.queued = pflog_queued;
 		st.bytes = pflog_bytes;
 		st.dropped = pflog_dropped;
 		st.seq = pflog_seq;
 		if (copy_to_user((void __user *)arg, &st, sizeof(st)))
 			return -EFAULT;
 		return 0;
 	default:
 		return -ENOTTY;
 	}
 }
 static const struct file_operations pfk_pflog_fops = {
 	.owner		= THIS_MODULE,
 	.open		= pfk_pflog_open,
 	.read		= pfk_pflog_read,
 	.poll		= pfk_pflog_poll,
 	.unlocked_ioctl	= pfk_pflog_ioctl,
 	.compat_ioctl	= compat_ptr_ioctl,
 };
 static struct miscdevice pfk_pflog_dev = {
 	.minor		= MISC_DYNAMIC_MINOR,
 	.name		= "pflog",
 	.fops		= &pfk_pflog_fops,
 	.mode		= 0600,
 };
 int
 pfk_pflog_register(void)
 {
 	return misc_register(&pfk_pflog_dev);
 }
 void
 pfk_pflog_unregister(void)
 {
 	struct pfk_pflog_rec *rec, *tmp;
 	misc_deregister(&pfk_pflog_dev);
 	spin_lock(&pflog_lock);
 	list_for_each_entry_safe(rec, tmp, &pflog_list, list) {
 		list_del(&rec->list);
 		kfree(rec);
 	}
 	pflog_queued = 0;
 	pflog_bytes = 0;
 	spin_unlock(&pflog_lock);
 }
@@ -0,0 +1,77 @@
 {"version": 2, "width": 80, "height": 24, "timestamp": 1794445553, "env": {"SHELL": null, "TERM": "dumb"}}
 [1.622488, "o", "\u001b[1m===============================================================\u001b[0m\r\n"]
 [1.623391, "o", "\u001b[1m PF on Linux - the OpenBSD PF 8.0 engine inside the Linux kernel\u001b[0m\r\n\u001b[1m===============================================================\u001b[0m\r\n\r\nLab: 3 Alpine VMs under libvirt.\r\n  pftest-pf0 = firewall (kernel 7.3.4 built from source + pf.ko)\r\n  pftest-cli = client 10.0.1.10     pftest-srv = server 10.0.2.20\r\n\r\n\u001b[1m--- 1. our kernel, the OpenBSD engine as a module ---\u001b[0m\r\n\u001b[1;36mpftest-pf0# uname -r\u001b[0m\r\n"]
 [1.885894, "o", "7.3.4\r\n"]
 [2.289113, "o", "\u001b[1;36mpftest-pf0# lsmod | grep '^pf ' || echo 'pf: not loaded yet'\u001b[0m\r\n"]
 [2.565982, "o", "pf                    552960  0\r\n"]
 [2.968407, "o", "\u001b[1;36mpftest-pf0# insmod /opt/pf/pf.ko && dmesg | tail -3\u001b[0m\r\n"]
 [3.226044, "o", "insmod: ERROR: could not insert module /opt/pf/pf.ko: File exists\r\n"]
 [3.629164, "o", "\r\n"]
 [3.629206, "o", "\u001b[1m--- 2. pfctl talks to the module over the OpenBSD ioctl ABI ---\u001b[0m\r\n"]
 [3.629219, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -s info | head -4\u001b[0m\r\n"]
 [3.906215, "o", "Status: Disabled for 0 days 00:03:33             Debug: err\r\n\r\nState Table                          Total             Rate\r\n  current entries                        0               \r\n"]
 [4.310059, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -e && /opt/pf/pfctl -s info | head -1\u001b[0m\r\n"]
 [4.574117, "o", "Status: Enabled                                  Debug: err\r\npf enabled\r\n"]
 [4.977439, "o", "\r\n\u001b[1m--- 3. a real pf.conf: tables, stateful rules, an explicit drop ---\u001b[0m\r\n\u001b[1;36mpftest-pf0# cat /tmp/demo-pf.conf\u001b[0m\r\n"]
 [5.245993, "o", "# PF-on-Linux demo ruleset (pf.conf syntax, real OpenBSD pfctl)\r\nset block-policy return\r\ntable <trusted> { 10.0.1.0/24 }\r\n\r\n# transit traffic: stateful, table-driven\r\npass in quick on ingress proto icmp from <trusted> keep state\r\npass out quick on egress proto icmp keep state\r\n\r\n# deny a specific destination\r\nblock drop in quick proto icmp to 10.0.9.99\r\n\r\n# everything else\r\npass in quick\r\npass out quick\r\n"]
 [5.649022, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -f /tmp/demo-pf.conf && echo 'rules loaded'\u001b[0m\r\n"]
 [5.926196, "o", "rules loaded\r\n"]
 [6.33004, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -sr\u001b[0m\r\n"]
 [6.614075, "o", "pass in quick on ingress proto icmp from <trusted> to any\r\npass out quick on egress proto icmp all\r\nblock drop in quick inet proto icmp from any to 10.0.9.99\r\npass in quick all flags S/SA\r\npass out quick all flags S/SA\r\n"]
 [7.017049, "o", "\r\n\u001b[1m--- 4. transit traffic is allowed and stateful ---\u001b[0m\r\n\u001b[1;36mpftest-cli# ping -c2 -W2 10.0.2.20 | tail -3\u001b[0m\r\n"]
 [8.256385, "o", "--- 10.0.2.20 ping statistics ---\r\n2 packets transmitted, 2 packets received, 0% packet loss\r\nround-trip min/avg/max = 0.343/0.381/0.419 ms\r\n"]
 [8.659624, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -ss | grep -E 'icmp' | head -4\u001b[0m\r\n"]
 [8.94608, "o", "all icmp 10.0.2.20:8 <- 10.0.1.10:4987       0:0\r\nall icmp 10.0.1.10:4987 -> 10.0.2.20:8       0:0\r\n"]
 [9.348967, "o", "\r\n"]
 [9.349022, "o", "\u001b[1m--- 5. tables are live objects ---\u001b[0m\r\n"]
 [9.349263, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -t trusted -T show\u001b[0m\r\n"]
 [9.618085, "o", "   10.0.1.0/24\r\n"]
 [10.022258, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -s Tables\u001b[0m\r\n"]
 [10.309891, "o", "trusted\r\n"]
 [10.712978, "o", "\r\n\u001b[1m--- 6. the drop rule actually drops ---\u001b[0m\r\n"]
 [10.713088, "o", "\u001b[1;36mpftest-pf0# cat > /tmp/demo-block.conf <<EOF\r\nblock drop in quick proto icmp to 10.0.2.20\r\npass in quick\r\npass out quick\r\nEOF\u001b[0m\r\n"]
 [11.397835, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -f /tmp/demo-block.conf && echo 'block rules loaded'\u001b[0m\r\n"]
 [11.650256, "o", "block rules loaded\r\n"]
 [12.05359, "o", "\u001b[1;36mpftest-cli# ping -c2 -W2 10.0.2.20 | tail -3\u001b[0m\r\n"]
 [15.280329, "o", "\r\n--- 10.0.2.20 ping statistics ---\r\n2 packets transmitted, 0 packets received, 100% packet loss\r\n"]
 [15.683815, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -sr -vv | head -6\u001b[0m\r\n"]
 [15.98217, "o", "@0 block drop in quick inet proto icmp from any to 10.0.2.20\r\n  [ Evaluations: 3         Packets: 2         Bytes: 168         States: 0     ]\r\n  [ Inserted: uid 0 pid 1877 State Creations: 0     ]\r\n@1 pass in quick all flags S/SA\r\n  [ Evaluations: 1         Packets: 28        Bytes: 9296        States: 1     ]\r\n  [ Inserted: uid 0 pid 1877 State Creations: 1     ]\r\n"]
 [16.384573, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -f /tmp/demo-pf.conf && echo 'back to the stateful ruleset'\u001b[0m\r\n"]
 [16.670067, "o", "back to the stateful ruleset\r\n"]
 [17.0729, "o", "\r\n\u001b[1m--- 7. NAT: the source is translated on the way out, restored on the way back ---\u001b[0m\r\n\u001b[1;36mpftest-pf0# cat > /tmp/demo-nat.conf <<EOF\r\npass out quick on egress from 10.0.1.0/24 to any nat-to (eth2)\r\npass in quick on ingress proto icmp keep state\r\npass out quick on egress proto icmp keep state\r\nEOF\u001b[0m\r\n"]
 [17.740601, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -f /tmp/demo-nat.conf && echo 'nat rules loaded'\u001b[0m\r\n"]
 [18.018113, "o", "nat rules loaded\r\n"]
 [18.42114, "o", "\u001b[1;36mpftest-cli# ping -c2 -W2 10.0.2.20 | tail -3\u001b[0m\r\n"]
 [19.636475, "o", "--- 10.0.2.20 ping statistics ---\r\n2 packets transmitted, 2 packets received, 0% packet loss\r\nround-trip min/avg/max = 0.325/0.362/0.400 ms\r\n"]
 [20.038543, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -ss | grep -E 'icmp' | head -4\u001b[0m\r\n"]
 [20.310123, "o", "all icmp 10.0.2.20:8 <- 10.0.1.10:4999       0:0\r\nall icmp 10.0.2.1:7697 (10.0.1.10:4999) -> 10.0.2.20:8       0:0\r\n"]
 [20.712246, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -f /tmp/demo-pf.conf && echo 'back to the stateful ruleset'\u001b[0m\r\n"]
 [20.990114, "o", "back to the stateful ruleset\r\n"]
 [21.392794, "o", "\r\n"]
 [21.393027, "o", "\u001b[1m--- 8. pflog: logged packets reach userspace (/dev/pflog) ---\u001b[0m\r\n\u001b[1;36mpftest-pf0# cat > /tmp/demo-log.conf <<EOF\r\npass in quick log proto icmp from 10.0.1.10 keep state\r\npass out quick\r\nEOF\u001b[0m\r\n"]
 [22.057041, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -f /tmp/demo-log.conf && echo 'log rule loaded'\u001b[0m\r\n"]
 [22.329918, "o", "log rule loaded\r\n"]
 [22.732644, "o", "\u001b[1;36mpftest-pf0# chmod +x /tmp/pflogd; rm -f /tmp/pflog.pcap; (setsid /tmp/pflogd -c 2 /tmp/pflog.pcap >/tmp/pflogd.log 2>&1 </dev/null &); sleep 1; echo 'pflogd collecting 2 records'\u001b[0m\r\n"]
 [23.981016, "o", "pflogd collecting 2 records\r\n"]
 [24.384736, "o", "\u001b[1;36mpftest-cli# ping -c2 -W2 10.0.2.20 | tail -2\u001b[0m\r\n"]
 [25.612472, "o", "2 packets transmitted, 2 packets received, 0% packet loss\r\nround-trip min/avg/max = 0.356/0.429/0.502 ms\r\n"]
 [28.017578, "o", "\u001b[1;36mpftest-pf0# cat /tmp/pflogd.log; ls -l /tmp/pflog.pcap; echo 'pcap header (magic d4c3b2a1, linktype 0x75=DLT_PFLOG):'; od -An -tx1 -N24 /tmp/pflog.pcap\u001b[0m\r\n"]
 [28.277996, "o", "pflogd: 2 records\r\n-rw-------    1 root     root           352 Nov 12 01:06 /tmp/pflog.pcap\r\npcap header (magic d4c3b2a1, linktype 0x75=DLT_PFLOG):\r\n d4 c3 b2 a1 02 00 04 00 00 00 00 00 00 00 00 00\r\n ff ff 00 00 75 00 00 00\r\n"]
 [28.681267, "o", "\u001b[1;36mpftest-pf0# pkill pflogd 2>/dev/null; /opt/pf/pfctl -f /tmp/demo-pf.conf >/dev/null\u001b[0m\r\n"]
 [29.352889, "o", "\r\n\u001b[1m--- 9. divert-to: the packet goes to userspace and comes back ---\u001b[0m\r\n"]
 [29.35314, "o", "\u001b[1;36mpftest-pf0# cat /tmp/demo-divert.conf\u001b[0m\r\n"]
 [29.625878, "o", "pass in quick proto icmp from 10.0.1.10 to 10.0.1.1 divert-to \"10.0.1.1\" port 10000 keep state\r\npass out quick\r\n"]
 [30.028664, "o", "\u001b[1;36mpftest-pf0# /opt/pf/pfctl -f /tmp/demo-divert.conf && echo 'divert rule loaded'\u001b[0m\r\n"]
 [30.317998, "o", "divert rule loaded\r\n"]
 [30.719927, "o", "\u001b[1;36mpftest-pf0# chmod +x /tmp/pdivert; (setsid /tmp/pdivert -p 10000 -f >/tmp/pdivert.log 2>&1 </dev/null &); sleep 1; pgrep pdivert\u001b[0m\r\n"]
 [31.941535, "o", "1956\r\n"]
 [32.344, "o", "\u001b[1;36mpftest-cli# ping -c2 -W2 10.0.1.1 | tail -3\u001b[0m\r\n"]
 [33.584192, "o", "--- 10.0.1.1 ping statistics ---\r\n2 packets transmitted, 2 packets received, 0% packet loss\r\nround-trip min/avg/max = 0.228/0.242/0.257 ms\r\n"]
 [33.987279, "o", "\u001b[1;36mpftest-pf0# head -3 /tmp/pdivert.log\u001b[0m\r\n"]
 [34.261872, "o", "pdivert: listening on divert port 10000\r\npdivert: got 84 bytes 10.0.1.10 -> 10.0.1.1\r\npdivert: got 84 bytes 10.0.1.10 -> 10.0.1.1\r\n"]
 [34.664226, "o", "  (pdivert flipped the last payload byte and fixed the ICMP checksum)\r\n\r\n"]
 [34.664382, "o", "\u001b[1m--- 10. shutdown ---\u001b[0m\r\n\u001b[1;36mpftest-pf0# pkill pdivert; /opt/pf/pfctl -f /dev/null; /opt/pf/pfctl -d; echo 'pf disabled'\u001b[0m\r\n"]
 [34.938113, "o", "pf disabled\r\npf disabled\r\n"]
 [35.340577, "o", "\u001b[1;36mpftest-pf0# lsmod | grep '^pf '\u001b[0m\r\n"]
 [35.617928, "o", "pf                    552960  0\r\n"]
 [36.021187, "o", "\r\n\u001b[1mdemo over - port, gates and docs:\u001b[0m\r\n\u001b[1m  https://git.labexposed.com/lgcosta/pf-linux-port\u001b[0m\r\n\r\n"]
@@ -1,141 +1,161 @@
 #!/usr/bin/env python3
 """
-PF-on-Linux demo recorder — produces an asciicast v2 (.cast) from a REAL
+PF-on-Linux demo recorder -> asciicast v2 (.cast)
 session driven against the lab VMs (no simulated output: every byte shown
 comes from the live systems; only pacing is scripted).
-Usage:  ./scripts/demo-record.py [--out out/demo-pf-linux.cast]
+Every byte shown comes from the REAL lab (Alpine VMs under libvirt, kernel
-Requires: lab running (pftest-pf0/cli/srv), repo built, ssh alias `pf0`
+7.3.4 built from source with our pf.ko).  Only the pacing is scripted.
 (ProxyJump) and lib.sh-style nested access working.
-Edit DEMO below to change the narration script. Timing: each line is a
+Usage:
-(real command, real output) pair; delays are cosmetic only.
+    ./scripts/demo-record.py [--out out/demo-pf-linux.cast] [--quick]
 Play back with:   asciinema play out/demo-pf-linux.cast
 Upload with:      asciinema upload out/demo-pf-linux.cast
 """
-import argparse, json, os, pty, re, shlex, subprocess, sys, time
+import argparse
 import json
 import os
 import shlex
 import subprocess
 import sys
 import time
 REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
 OUT_DEFAULT = os.path.join(REPO, "out", "demo-pf-linux.cast")
 DEB = os.environ.get("DEB", "root@192.168.1.242")
-PF0_IP = os.environ.get("PF0_IP", "192.168.122.8")
+PF0 = os.environ.get("PF0_IP", "192.168.122.8")
 CLI = os.environ.get("CLI_IP", "10.0.1.10")
 SRV = os.environ.get("SRV_IP", "10.0.2.20")
 KEY = os.path.join(os.environ.get("HOME", "/root"), ".ssh/pflab_ed")
 CONF = """# PF-on-Linux demo ruleset (pf.conf syntax, real OpenBSD pfctl)
 set block-policy return
 table <trusted> { 10.0.1.0/24 }
 # transit traffic: stateful, table-driven
 pass in quick on ingress proto icmp from <trusted> keep state
 pass out quick on egress proto icmp keep state
 # deny a specific destination
 block drop in quick proto icmp to 10.0.9.99
 # everything else
 pass in quick
 pass out quick
 """
 DIVERT_CONF = """pass in quick proto icmp from 10.0.1.10 to 10.0.1.1 divert-to "10.0.1.1" port 10000 keep state
 pass out quick
 """
-def deb_run(cmd: str, timeout=30) -> str:
+def sh(cmd, timeout=40):
-    """run a command on the libvirt host (which has direct keys to VMs)"""
+    p = subprocess.run(cmd, shell=True, capture_output=True, text=True,
-    p = subprocess.run(["ssh", "-o", "ConnectTimeout=10", DEB, cmd],
+                       timeout=timeout)
                       capture_output=True, text=True, timeout=timeout)
    return (p.stdout or "") + (p.stderr or "")
-def vm0(cmd: str, timeout=30) -> str:
+def on_pf0(cmd, timeout=40):
-    return deb_run(
+    return sh(f"ssh -o ConnectTimeout=8 -o StrictHostKeyChecking=accept-new "
-        f"ssh -o StrictHostKeyChecking=accept-new -o ConnectTimeout=8 "
+              f"-i {KEY} root@{PF0} {shlex.quote(cmd)}", timeout)
        f"-i /root/.ssh/pflab_ed root@{PF0_IP} {shlex.quote(cmd)}",
        timeout)
-def vm_ip(ip: str, cmd: str, timeout=30) -> str:
+def on_vm(ip, cmd, timeout=40):
-    return deb_run(
+    return sh(f"ssh -o ConnectTimeout=8 -o StrictHostKeyChecking=accept-new "
-        f"ssh -o StrictHostKeyChecking=accept-new -o ConnectTimeout=8 "
+              f"-i {KEY} root@{ip} {shlex.quote(cmd)}", timeout)
        f"-i /root/.ssh/pflab_ed root@{ip} {shlex.quote(cmd)}",
        timeout)
-CLI, SRV = "10.0.1.10", "10.0.2.20"
+def on_deb(cmd, timeout=40):
    return sh(f"ssh -o ConnectTimeout=8 {DEB} {shlex.quote(cmd)}", timeout)
-def scene(title):
+def stage():
-    return ("#banner", title)
+    """put the demo ruleset + pdivert on pf0 (through the lab host)"""
    with open("/tmp/demo-pf.conf", "w") as f:
        f.write(CONF)
    with open("/tmp/demo-divert.conf", "w") as f:
        f.write(DIVERT_CONF)
    sh(f"scp -q /tmp/demo-pf.conf /tmp/demo-divert.conf "
       f"{os.path.join(REPO, 'tools/out/pdivert')} {DEB}:/tmp/", timeout=90)
    on_deb(f"scp -q -o StrictHostKeyChecking=accept-new -i {KEY} "
           f"/tmp/demo-pf.conf /tmp/demo-divert.conf /tmp/pdivert "
           f"root@{PF0}:/tmp/ 2>&1", timeout=90)
-# (shell-line, runner, command)  — runners: pf0 / cli / srv / local
+
-DEMO = [
+SCENARIO = [
-    ("local",  "echo 'PF-on-Linux — OpenBSD PF engine on Linux 7.3.4 (lab: Alpine VMs)'"),
+    ("local", "echo '=== PF on Linux — OpenBSD PF 8.0 engine running inside the Linux 7.3.4 kernel ==='"),
-    ("local",  "uname -r | sed 's/^/build host kernel: /'"),
+    ("local", "echo 'lab: 3 Alpine VMs under libvirt (pf0 = firewall, cli/srv = transit endpoints)'"),
-    ("pf0",    "uname -r"),
+    ("pf0", "uname -r"),
-    ("pf0",    "lsmod | grep -E '^pf ' || echo 'pf: not loaded yet'"),
+    ("pf0", "lsmod | grep '^pf ' || echo 'pf: not loaded'"),
-    ("pf0",    "insmod /opt/pf/pf.ko && dmesg | tail -2"),
+    ("pf0", "insmod /opt/pf/pf.ko && dmesg | tail -3"),
-    ("pf0",    "/opt/pf/pfctl -s info | head -3"),
+    ("pf0", "/opt/pf/pfctl -s info | head -4"),
-    ("pf0",    "/opt/pf/pfctl -e && /opt/pf/pfctl -s info | head -1"),
+    ("pf0", "/opt/pf/pfctl -e && /opt/pf/pfctl -s info | head -1"),
-    ("pf0",    "cat /tmp/pfdemo.conf"),
+    ("pf0", "cat /tmp/demo-pf.conf"),
-    ("pf0",    "/opt/pf/pfctl -f /tmp/pfdemo.conf && /opt/pf/pfctl -sr"),
+    ("pf0", "/opt/pf/pfctl -f /tmp/demo-pf.conf && echo 'rules loaded' && /opt/pf/pfctl -sr"),
-    ("cli",    f"ping -c 2 -W 2 {SRV} 2>&1 | tail -2"),
+    ("pf0", "/opt/pf/pfctl -sr -vv | head -8"),
-    ("pf0",    "/opt/pf/pfctl -s states"),
+    ("cli", f"ping -c2 -W2 {SRV} | tail -3"),
-    ("pf0",    "/opt/pf/pfctl -s Tables"),
+    ("pf0", "/opt/pf/pfctl -ss"),
-    ("pf0",    "/opt/pf/pfctl -t demo -T show"),
+    ("pf0", "/opt/pf/pfctl -t trusted -T show"),
-    ("cli",    "ping -c 2 -W 2 10.0.9.99 2>&1 | tail -2"),
+    ("pf0", "/opt/pf/pfctl -s Tables"),
-    ("pf0",    "/opt/pf/pfctl -s info | grep -E 'match|current'"),
+    ("cli", "ping -c2 -W2 10.0.9.99 | tail -3"),
-    ("pf0",    "/opt/pf/pfctl -ss | head -2"),
+    ("pf0", "/opt/pf/pfctl -sr -vv | head -6"),
    ("pf0", "/opt/pf/pfctl -f /tmp/demo-divert.conf && echo 'divert rule loaded'"),
    ("pf0", "(setsid /tmp/pdivert -p 10000 -f >/tmp/pdivert.log 2>&1 </dev/null &); sleep 1; pgrep pdivert"),
    ("cli", "ping -c2 -W2 10.0.1.1 | tail -3"),
    ("pf0", "head -3 /tmp/pdivert.log"),
    ("pf0", "pkill pdivert; /opt/pf/pfctl -f /dev/null; /opt/pf/pfctl -d; echo 'pf disabled'"),
 ]
 CONF = """set block-policy return
 set loginterface egress
 table <demo> { 10.0.1.0/24 }
 match in quick on ingress proto icmp from <demo> tag DEMO
 pass in quick on ingress proto icmp keep state
 pass out quick proto icmp keep state
 block drop in quick proto icmp from any to 10.0.9.99
 """
-
+def record(out_path, quick=False):
-def prep():
+    os.makedirs(os.path.dirname(out_path) or ".", exist_ok=True)
-    # stage the demo ruleset into pf0
+    events = []
    import base64
    b64 = base64.b64encode(CONF.encode()).decode()
    vm0(f"echo {b64} | base64 -d > /tmp/pfdemo.conf; echo staged")
 def record(out_path: str, fps_pad=0.4):
    os.makedirs(os.path.dirname(out_path), exist_ok=True)
    casts = []
    t = 0.0
    def emit(text, delay):
        nonlocal t
        t += delay
-        casts.append([round(t, 3), "o", text])
+        events.append([round(t, 3), "o", text])
-    header = {"version": 2, "width": 100, "height": 24,
+    header = {
-              "timestamp": int(time.time()), "title":
+        "version": 2, "width": 100, "height": 26,
-              "PF on Linux — port demo (OpenBSD PF 8.0 engine on Linux 7.3.4, Alpine libvirt lab)",
+        "timestamp": int(time.time()),
-              "env": {"SHELL": "/bin/sh", "TERM": "xterm-256color"}}
+        "title": "PF on Linux — OpenBSD PF engine inside the Linux kernel (lab demo)",
        "env": {"SHELL": "/bin/sh", "TERM": "xterm-256color"},
    }
    runners = {
        "pf0": lambda c: on_pf0(c),
        "cli": lambda c: on_vm(CLI, c),
        "srv": lambda c: on_vm(SRV, c),
        "local": lambda c: sh(c),
    }
    labels = {"pf0": "pftest-pf0", "cli": "pftest-cli", "srv": "pftest-srv",
              "local": "lab-host"}
    stage()
    with open(out_path, "w") as f:
        f.write(json.dumps(header) + "\n")
-
+        for kind, cmd in SCENARIO:
-        def show_prompt(host):
+            emit(f"\x1b[1;36m{labels[kind]}\x1b[0m:\x1b[1;37m {cmd}\x1b[0m\r\n",
-            emit(f"\x1b[1;36m{host}$\x1b[0m ", 0.15)
+                 0.35)
        runners = {"pf0": vm0, "cli": lambda c: vm_ip(CLI, c),
                   "srv": lambda c: vm_ip(SRV, c), "local":
                   lambda c: subprocess.run(["sh", "-c", c],
                                            capture_output=True, text=True,
                                            timeout=30).stdout +
                   subprocess.run(["sh", "-c", c], capture_output=True,
                                  text=True).stderr}
        prep()
        for kind, cmd in DEMO:
            label = {"pf0": "pftest-pf0", "cli": "pftest-cli",
                     "srv": "pftest-srv", "local": "build-host"}[kind]
            show_prompt(label)
            emit(cmd + "\r\n", 0.25)
            try:
                out = runners[kind](cmd)
-            except Exception as e:
+            except subprocess.TimeoutExpired:
-                out = f"[runner error: {e}]\n"
+                out = "[timeout]\r\n"
-            # strip ANSI-safe passthrough; small human-ish delay
+            except Exception as e:  # noqa: BLE001
-            delay = min(1.2, 0.25 + len(out) / 4000.0)
+                out = f"[error: {e}]\r\n"
-            emit(out.replace("\n", "\r\n") or "(no output)\r\n", delay)
+            out = out.replace("\r\n", "\n").replace("\n", "\r\n")
-            time.sleep(0.1)
+            emit(out or "(no output)\r\n", min(1.4, 0.3 + len(out) / 5000.0))
-        emit("\x1b[1;36mpftest-pf0$\x1b[0m /opt/pf/pfctl -d\r\n", 0.4)
+        with open(out_path, "a") as f:
-        runners["pf0"]("/opt/pf/pfctl -d")
+            for ev in events:
-        for i, ev in enumerate(casts):
+                f.write(json.dumps(ev) + "\n")
-            f.write(json.dumps(ev) + "\n")
+    print(f"WROTE {out_path}: {len(events)} events, {t:.1f}s")
-    print(f"WROTE {out_path} ({len(casts)} events, {t:.1f}s)")
+    print("play:   asciinema play " + out_path)
-    print("play:  asciinema play " + out_path +
+    print("upload: asciinema upload " + out_path)
          "   (or: script-replay/pf-in-play fallback: cat with 'less -R')")
 if __name__ == "__main__":
    ap = argparse.ArgumentParser()
-    ap.add_argument("--out", default=OUT_DEFAULT)
+    ap.add_argument("--out", default=os.path.join(REPO, "out",
                                                  "demo-pf-linux.cast"))
    ap.add_argument("--quick", action="store_true")
    a = ap.parse_args()
-    record(a.out)
+    record(a.out, a.quick)
@@ -0,0 +1,169 @@
 #!/usr/bin/env bash
 # Demo session for PF-on-Linux: a scripted terminal tour of the lab.
 # Recorded with:  asciinema rec out/demo-pf-linux.cast -c "bash scripts/demo-session.sh"
 #
 # Everything printed is the REAL output of the lab (Alpine VMs under libvirt
 # running the kernel built from source with our pf.ko).  The VM management
 # addresses are resolved from the hypervisor (they are DHCP), the lab
 # topology (10.0.1.0/24, 10.0.2.0/24, ...) is fixed.
 set -u
 DEB=${DEB:-root@192.168.1.242}
 CLI_IP=${CLI_IP:-10.0.1.10}
 SRV_IP=${SRV_IP:-10.0.2.20}
 KEY=${KEY:-$HOME/.ssh/pflab_ed}
 bold()  { printf '\033[1m%s\033[0m\n' "$*"; }
 cyan()  { printf '\033[1;36m%s\033[0m\n' "$*"; }
 # management address of a lab VM (DHCP, so ask the hypervisor)
 mgmt() {
 	ssh -o ConnectTimeout=8 "$DEB" "virsh domifaddr $1 | awk '/vnet/{print \$4}' | cut -d/ -f1" \
 	    2>/dev/null | tr -d '\r' | head -1
 }
 PF0_MGMT=$(mgmt pftest-pf0)
 CLI_MGMT=$(mgmt pftest-cli)
 [ -n "$PF0_MGMT" ] || PF0_MGMT=192.168.122.8
 [ -n "$CLI_MGMT" ] || CLI_MGMT=192.168.122.80
 # run a command on a lab VM (through the hypervisor)
 onvm() {
 	local ip=$1; shift
 	ssh -o ConnectTimeout=10 -o StrictHostKeyChecking=accept-new \
 	    -o ProxyJump=root@192.168.1.242 -i "$KEY" root@"$ip" "$@" 2>&1
 }
 p0() {
 	cyan "pftest-pf0# $*"
 	onvm "$PF0_MGMT" "$@"
 	sleep 0.4
 }
 vmc() {
 	cyan "pftest-cli# $*"
 	onvm "$CLI_MGMT" "$@"
 	sleep 0.4
 }
 CONF=/tmp/demo-pf.conf
 DIV=/tmp/demo-divert.conf
 cat > "$CONF" <<'CONF'
 # PF-on-Linux demo ruleset (pf.conf syntax, real OpenBSD pfctl)
 set block-policy return
 table <trusted> { 10.0.1.0/24 }
 # transit traffic: stateful, table-driven
 pass in quick on ingress proto icmp from <trusted> keep state
 pass out quick on egress proto icmp keep state
 # deny a specific destination
 block drop in quick proto icmp to 10.0.9.99
 # everything else
 pass in quick
 pass out quick
 CONF
 cat > "$DIV" <<CONF
 pass in quick proto icmp from $CLI_IP to 10.0.1.1 divert-to "10.0.1.1" port 10000 keep state
 pass out quick
 CONF
 # the pf0 interface towards the server (used by the NAT pool)
 EGRESS_IF=$(onvm "$PF0_MGMT" "ip route get $SRV_IP | grep -o 'dev [a-z0-9]*' | cut -d' ' -f2" | tr -d '\r')
 [ -n "$EGRESS_IF" ] || EGRESS_IF=eth2
 scp -q -o ProxyJump=root@192.168.1.242 -i "$KEY" "$CONF" "$DIV" \
    tools/out/pdivert tools/out/pflogd "root@$PF0_MGMT:/tmp/" 2>/dev/null
 bold "==============================================================="
 bold " PF on Linux - the OpenBSD PF 8.0 engine inside the Linux kernel"
 bold "==============================================================="
 echo
 echo "Lab: 3 Alpine VMs under libvirt."
 echo "  pftest-pf0 = firewall (kernel 7.3.4 built from source + pf.ko)"
 echo "  pftest-cli = client 10.0.1.10     pftest-srv = server 10.0.2.20"
 echo
 bold "--- 1. our kernel, the OpenBSD engine as a module ---"
 p0 "uname -r"
 p0 "lsmod | grep '^pf ' || echo 'pf: not loaded yet'"
 p0 "insmod /opt/pf/pf.ko && dmesg | tail -3"
 echo
 bold "--- 2. pfctl talks to the module over the OpenBSD ioctl ABI ---"
 p0 "/opt/pf/pfctl -s info | head -4"
 p0 "/opt/pf/pfctl -e && /opt/pf/pfctl -s info | head -1"
 echo
 bold "--- 3. a real pf.conf: tables, stateful rules, an explicit drop ---"
 p0 "cat /tmp/demo-pf.conf"
 p0 "/opt/pf/pfctl -f /tmp/demo-pf.conf && echo 'rules loaded'"
 p0 "/opt/pf/pfctl -sr"
 echo
 bold "--- 4. transit traffic is allowed and stateful ---"
 vmc "ping -c2 -W2 $SRV_IP | tail -3"
 p0 "/opt/pf/pfctl -ss | grep -E 'icmp' | head -4"
 echo
 bold "--- 5. tables are live objects ---"
 p0 "/opt/pf/pfctl -t trusted -T show"
 p0 "/opt/pf/pfctl -s Tables"
 echo
 bold "--- 6. the drop rule actually drops ---"
 p0 "cat > /tmp/demo-block.conf <<EOF
 block drop in quick proto icmp to $SRV_IP
 pass in quick
 pass out quick
 EOF"
 p0 "/opt/pf/pfctl -f /tmp/demo-block.conf && echo 'block rules loaded'"
 vmc "ping -c2 -W2 $SRV_IP | tail -3"
 p0 "/opt/pf/pfctl -sr -vv | head -6"
 p0 "/opt/pf/pfctl -f /tmp/demo-pf.conf && echo 'back to the stateful ruleset'"
 echo
 bold "--- 7. NAT: the source is translated on the way out, restored on the way back ---"
 p0 "cat > /tmp/demo-nat.conf <<EOF
 pass out quick on egress from 10.0.1.0/24 to any nat-to ($EGRESS_IF)
 pass in quick on ingress proto icmp keep state
 pass out quick on egress proto icmp keep state
 EOF"
 p0 "/opt/pf/pfctl -f /tmp/demo-nat.conf && echo 'nat rules loaded'"
 vmc "ping -c2 -W2 $SRV_IP | tail -3"
 p0 "/opt/pf/pfctl -ss | grep -E 'icmp' | head -4"
 p0 "/opt/pf/pfctl -f /tmp/demo-pf.conf && echo 'back to the stateful ruleset'"
 echo
 bold "--- 8. pflog: logged packets reach userspace (/dev/pflog) ---"
 p0 "cat > /tmp/demo-log.conf <<EOF
 pass in quick log proto icmp from $CLI_IP keep state
 pass out quick
 EOF"
 p0 "/opt/pf/pfctl -f /tmp/demo-log.conf && echo 'log rule loaded'"
 p0 "chmod +x /tmp/pflogd; rm -f /tmp/pflog.pcap; (setsid /tmp/pflogd -c 2 /tmp/pflog.pcap >/tmp/pflogd.log 2>&1 </dev/null &); sleep 1; echo 'pflogd collecting 2 records'"
 vmc "ping -c2 -W2 $SRV_IP | tail -2"
 sleep 2
 p0 "cat /tmp/pflogd.log; ls -l /tmp/pflog.pcap; echo 'pcap header (magic d4c3b2a1, linktype 0x75=DLT_PFLOG):'; od -An -tx1 -N24 /tmp/pflog.pcap"
 p0 "pkill pflogd 2>/dev/null; /opt/pf/pfctl -f /tmp/demo-pf.conf >/dev/null"
 echo
 bold "--- 9. divert-to: the packet goes to userspace and comes back ---"
 p0 "cat /tmp/demo-divert.conf"
 p0 "/opt/pf/pfctl -f /tmp/demo-divert.conf && echo 'divert rule loaded'"
 p0 "chmod +x /tmp/pdivert; (setsid /tmp/pdivert -p 10000 -f >/tmp/pdivert.log 2>&1 </dev/null &); sleep 1; pgrep pdivert"
 vmc "ping -c2 -W2 10.0.1.1 | tail -3"
 p0 "head -3 /tmp/pdivert.log"
 echo "  (pdivert flipped the last payload byte and fixed the ICMP checksum)"
 echo
 bold "--- 10. shutdown ---"
 p0 "pkill pdivert; /opt/pf/pfctl -f /dev/null; /opt/pf/pfctl -d; echo 'pf disabled'"
 p0 "lsmod | grep '^pf '"
 echo
 bold "demo over - port, gates and docs:"
 bold "  https://git.labexposed.com/lgcosta/pf-linux-port"
 echo
@@ -22,15 +22,18 @@ deploy() {
 	# showed up as a bogus gate failure.  Retry and verify the copy.
 	local i
 	for i in 1 2 3; do
 		local extra=()
 		[ -x "$REPO/tools/out/pdivert" ] && extra+=("$REPO/tools/out/pdivert")
 		[ -x "$REPO/tools/out/pflogd" ] && extra+=("$REPO/tools/out/pflogd")
 		if scp -q -o ProxyJump=root@192.168.1.242 -o StrictHostKeyChecking=accept-new \
 			-i "$HOME/.ssh/pflab_ed" "$REPO/module/pf.ko" \
-			"$REPO/tools/out/pfctl" "$REPO/tests/confs/"*.conf pf0:/tmp/ 2>/dev/null; then
+			"$REPO/tools/out/pfctl" "${extra[@]}" "$REPO/tests/confs/"*.conf pf0:/tmp/ 2>/dev/null; then
 			break
 		fi
 		sleep 2
 	done
 	# stage onto pf0 /opt/pf via mgmt alias already copied to /tmp
-	ssh "$DEB" "$SSH 'mkdir -p /opt/pf; mv /tmp/pf.ko /tmp/pfctl /opt/pf/ 2>/dev/null; true'" >/dev/null
+	ssh "$DEB" "$SSH 'mkdir -p /opt/pf; mv /tmp/pf.ko /tmp/pfctl /tmp/pdivert /tmp/pflogd /opt/pf/ 2>/dev/null; chmod +x /opt/pf/pfctl /opt/pf/pdivert /opt/pf/pflogd 2>/dev/null; true'" >/dev/null
 	ssh "$DEB" "$SSH 'test -s /opt/pf/pf.ko && test -x /opt/pf/pfctl'" >/dev/null 2>&1 || {
 		echo "deploy: pf.ko/pfctl missing on pf0"; return 1; }
 }
@@ -1,30 +1,50 @@
 #!/usr/bin/env bash
-# M2 gate 4: anchors + quick precedence
+# M2 gate 4: anchors + evaluation order (OpenBSD semantics)
 #
 # OpenBSD evaluates the IN ruleset in ip_input() for every arriving packet and
 # the OUT ruleset in ip_forward() for forwarded packets.  A `block in ... icmp`
 # rule therefore also sees the *reply* of a transit flow - a state created by
 # an anchor rule does NOT rescue the reply at input time.  This gate keeps the
 # ruleset OpenBSD-correct and isolates the anchor semantics:
 #
 #   anchor "restricted"          -> child: pass in quick ... from <client> keep state
 #   block drop in quick proto icmp from any to <server>   (would drop the request)
 #   pass in quick / pass out quick
 #
 # The request must be passed by the ANCHOR child rule (the later block rule
 # would drop it), and a state must exist.  After flushing the anchor the very
 # same request must be dropped - proving the anchor is what allowed it.
 set -uo pipefail; cd "$(dirname "$0")"; . ./lib.sh
-pf0 'insmod /tmp/pf.ko 2>/dev/null || insmod /opt/pf/pf.ko; true'; pf0 '/opt/pf/pfctl -e; /opt/pf/pfctl -f /dev/null' >/dev/null
+
-cat > /tmp/g4main.conf <<'C'
+pf0 'insmod /opt/pf/pf.ko 2>/dev/null || insmod /tmp/pf.ko; true'
 pf0 '/opt/pf/pfctl -e; /opt/pf/pfctl -f /dev/null' >/dev/null
 cat > /tmp/g4main.conf <<C
 anchor "restricted"
-block drop in quick proto icmp from any to any
+block drop in quick proto icmp from any to $SRV_IP
 pass in quick
 pass out quick
 C
-pfload /tmp/g4main.conf || { bad mainload; finish; }
+cat > /tmp/g4anc.conf <<C
-cat > /tmp/g4anc.conf <<'C'
+pass in quick proto icmp from $CLI_IP to any keep state
 pass in quick proto icmp from 10.0.1.10 to any keep state
 C
-r=$(pfput /tmp/g4anc.conf) || { bad anchorload; finish; }
+
-pf0 "/opt/pf/pfctl -a restricted -f $r" || { bad anchorload; finish; }
+pfload /tmp/g4main.conf || { bad "main ruleset load"; finish; }
-# The ICMP echo request is passed by the anchor's rule (evaluated before the
+ANC=$(pfput /tmp/g4anc.conf) || { bad "anchor conf upload"; finish; }
-# main ruleset's block line); the echo reply is carried by the state that the
+pf0 "/opt/pf/pfctl -a restricted -f $ANC" || { bad "anchor ruleset load"; finish; }
-# request created.  The first reply after the ruleset load races with that
+
-# state on the freshly booted pf0, so retry a few times: if the anchor's rule
+pf0 '/opt/pf/pfctl -sr' | grep -q 'anchor "restricted"' && ok "anchor rule in main ruleset" || bad "anchor rule missing"
-# is not honoured the *request* is dropped and no retry can succeed.
+pf0 '/opt/pf/pfctl -a restricted -sr' | grep -q 'proto icmp from' && ok "anchor rules listed" || bad "anchor rules missing"
-pass_ok=0
+
-for _ in 1 2 3 4 5; do
+# the request must pass through the anchor (the block rule would drop it)
-	if cli "ping -c1 -W2 $SRV_IP >/dev/null 2>&1"; then pass_ok=1; break; fi
+cli "ping -c2 -W2 $SRV_IP >/dev/null 2>&1" && ok "anchor pass honoured (request allowed)" \
-	sleep 1
+	|| bad "anchor pass not honored"
-done
+pf0 '/opt/pf/pfctl -ss' | grep -q "$SRV_IP" && ok "state created by anchor rule" || bad "no state from anchor"
-[ "$pass_ok" = 1 ] && ok "anchor pass evaluated before block" || bad "anchor pass not honored"
+
-# anchor rules are evaluated BEFORE the main-ruleset block line (top placement)
+# without the anchor rule the request hits the block rule -> dropped
-pf0 '/opt/pf/pfctl -s Anchors' | grep -q restricted && ok "anchors listed" || bad "anchor list"
+pf0 "/opt/pf/pfctl -a restricted -F all" >/dev/null 2>&1
-pf0 '/opt/pf/pfctl -d; /opt/pf/pfctl -f /dev/null' >/dev/null; finish
+cli "ping -c1 -W2 $SRV_IP >/dev/null 2>&1" && bad "request passed without anchor rule" \
 	|| ok "request dropped without anchor rule (block applies)"
 pf0 '/opt/pf/pfctl -d; /opt/pf/pfctl -f /dev/null' >/dev/null
 finish
@@ -1,35 +1,46 @@
 #!/usr/bin/env bash
 # M3 gate: nat-to — source NAT of transit traffic to the egress address.
 #
-# Scenario: cli (10.0.1.10) -> srv (10.0.2.20) through pf0, rule:
+# OpenBSD 8.0 unified the old "nat on ..." rule into the rule syntax:
-#   nat on egress proto icmp from 10.0.1.0/24 to any -> (egress)
+#   pass out on egress from <net> to any nat-to (egress)
-# Expected: srv sees src 192.168.122.8 (pf0 egress IP) instead of 10.0.1.10;
+#
-# the reply is de-NATed and delivered back to cli; `pfctl -s nat`,
+# Expected: the reply path is unaffected, srv sees the packet with
-# `pfctl -s states` show the translation.
+# src = pf0's egress address (not the client's), and pfctl shows the rule.
 set -uo pipefail; cd "$(dirname "$0")"; . ./lib.sh
 FAILED=0
-pf0 "insmod /tmp/pf.ko 2>/dev/null || insmod /opt/pf/pf.ko; true"
+# pf calls the interface "egress"; the OS names it ethX - resolve both
 EGRESS_DEV=$(pf0 "ip route get $SRV_IP | grep -o 'dev [a-z0-9]*' | cut -d' ' -f2")
 EGRESS_IP=$(pf0 "ip -4 addr show $EGRESS_DEV | grep -o 'inet [0-9.]*' | head -1 | cut -d' ' -f2")
 [ -n "$EGRESS_IP" ] || EGRESS_IP=192.168.122.8
 pf0 'insmod /opt/pf/pf.ko 2>/dev/null || insmod /tmp/pf.ko; true'
 pf0 '/opt/pf/pfctl -e; /opt/pf/pfctl -f /dev/null' >/dev/null
 # OpenBSD expands "(egress)" (an interface *group*) into the addresses of all
 # its members - on a multi-NIC host that is ambiguous, so the gate uses the
 # concrete transit interface (the OpenBSD-recommended form for NAT pools).
 cat > /tmp/g05.conf <<C
-nat on egress proto icmp from 10.0.1.0/24 to any -> (egress)
+pass out quick on egress from 10.0.1.0/24 to any nat-to ($EGRESS_DEV)
 pass in quick on ingress proto icmp keep state
-pass out quick proto icmp keep state
+pass out quick on egress proto icmp keep state
 C
 pfload /tmp/g05.conf || { bad "nat rule load"; finish; }
 pf0 '/opt/pf/pfctl -sr' | grep -q 'nat-to' && ok "nat-to rule listed" || bad "nat-to not listed"
-pf0 '/opt/pf/pfctl -sn' | grep -q 'nat on egress' && ok "nat rule listed" || bad "nat not listed"
+pf0 "tcpdump -ni $EGRESS_DEV -c 4 -e icmp > /tmp/nat.td 2>&1 &"
 # capture on pf0's egress while cli pings srv
 pf0 'tcpdump -ni egress -c 4 -e icmp > /tmp/nat.td 2>&1 &'
 sleep 1
-cli "ping -c 2 -W 2 $SRV_IP >/dev/null 2>&1" && ok "transit ping through NAT" || bad "transit ping"
+cli "ping -c2 -W2 $SRV_IP >/dev/null 2>&1" && ok "transit ping through NAT" || bad "transit ping"
 sleep 2
 TD=$(pf0 'cat /tmp/nat.td')
 echo "$TD" | grep -q "$CLI_IP" && bad "cli source address leaked to egress" || ok "source rewritten (no $CLI_IP on egress)"
 echo "$TD" | grep -q '192.168.122.8' && ok "nat applied to $(pf0 'ip -4 addr show egress | grep -o "192[0-9.]*"' | head -1)" || bad "egress src ip not seen"
-pf0 '/opt/pf/pfctl -s states' | grep -q "$SRV_IP" && ok "state present" || bad "no state"
+TD=$(pf0 'cat /tmp/nat.td')
 if echo "$TD" | grep -q "$CLI_IP"; then
 	bad "client address $CLI_IP still visible on egress"
 else
 	ok "source rewritten on egress (no $CLI_IP)"
 fi
 echo "$TD" | grep -q "$EGRESS_IP" && ok "egress source is $EGRESS_IP" || bad "egress source not seen"
 pf0 '/opt/pf/pfctl -ss' | grep -q "$SRV_IP" && ok "state present" || bad "no state"
 pf0 '/opt/pf/pfctl -d' >/dev/null
 finish
@@ -0,0 +1,69 @@
 #!/usr/bin/env bash
 # M3 gate: rdr-to — destination/port redirect.
 #
 # Lab: the client connects to the firewall's own address 10.0.1.1:8080; pf
 # redirects it to the server (10.0.2.20:80, busybox httpd).  This exercises
 # the inbound translation *and* the re-route of a packet whose destination pf
 # rewrote (the packet is no longer local to the firewall).
 set -uo pipefail; cd "$(dirname "$0")"; . ./lib.sh
 pf0 'insmod /opt/pf/pf.ko 2>/dev/null || insmod /tmp/pf.ko; true'
 pf0 '/opt/pf/pfctl -e; /opt/pf/pfctl -f /dev/null' >/dev/null
 # the runner only resets pf0 between gates - clear any responder a previous
 # run (or a manual session) left on the server, or it answers first and this
 # gate sees the wrong payload.
 srv 'pkill -f '[l]oop.sh' 2>/dev/null; pkill -f '[b]ig.sh' 2>/dev/null; pkill nc 2>/dev/null; true' >/dev/null
 sleep 1
 # a tiny responder on srv:80 (base64-encoded so it survives the two ssh hops
 # without nested quoting problems; detached so ssh returns)
 LOOP=$(printf 'while :; do nc -l -p 80 < /tmp/resp.txt >/dev/null 2>&1; done\n' | base64 | tr -d '\n')
 # NOTE: never pkill -f a pattern that also appears in this very command line -
 # it kills the invoking shell.  Use a pidfile and check the listening socket.
 srv "echo PF-RDR-OK > /tmp/resp.txt; echo $LOOP | base64 -d > /tmp/loop.sh; [ -f /tmp/loop.pid ] && kill \$(cat /tmp/loop.pid) 2>/dev/null; pkill nc 2>/dev/null; setsid sh /tmp/loop.sh >/dev/null 2>&1 </dev/null & echo \$! > /tmp/loop.pid; sleep 1; netstat -lnt 2>/dev/null | grep -q ':80 ' && echo RESPONDER_UP" | grep -q RESPONDER_UP && ok "responder up on srv:80" || bad "responder start"
 cat > /tmp/g06.conf <<C
 pass in quick proto tcp from 10.0.1.0/24 to 10.0.1.1 port 8080 rdr-to $SRV_IP port 80 keep state
 pass out quick proto tcp keep state
 C
 pfload /tmp/g06.conf || { bad "rdr rule load"; finish; }
 pf0 '/opt/pf/pfctl -sr' | grep -q 'rdr-to' && ok "rdr rule listed" || bad "rdr not listed"
 # the client talks to the firewall's address; pf redirects to srv:80
 RESP=$(cli 'printf "GET / HTTP/1.0\r\n\r\n" | nc -w3 10.0.1.1 8080' 2>/dev/null)
 if echo "$RESP" | grep -q 'PF-RDR-OK'; then
 	ok "redirect delivered the response"
 else
 	bad "no redirected response"
 	printf '%s\n' "$RESP" | head -4 | sed 's/^/    /'
 fi
 # the state must show the translated tuple (server:80 with the original in parens)
 pf0 '/opt/pf/pfctl -ss' | grep -E "$SRV_IP:80 .*$CLI_IP" >/dev/null && ok "state shows the translation" || bad "no translated state"
 # the reply path must survive the un-redirect
 pf0 '/opt/pf/pfctl -s info' | grep -qiE 'match' && ok "counters readable" || bad "counters"
 srv '[ -f /tmp/loop.pid ] && kill $(cat /tmp/loop.pid) 2>/dev/null; pkill nc 2>/dev/null; true' >/dev/null
 # bulk (GSO/TSO) through the redirect: 2 MB must arrive byte-identical.
 # This is the regression that "materialize the checksum in software" introduced:
 # computing the transport checksum once for a GSO skb corrupts every segment
 # after the first, and the peer silently stalls (sshd mute, transfers hang).
 B64=$(printf 'head -c 2000000 /dev/zero | tr "\0" A > /tmp/big.txt\n' | base64 | tr -d '\n')
 srv "echo $B64 | base64 -d > /tmp/mkbig.sh; sh /tmp/mkbig.sh; pkill nc 2>/dev/null; sleep 1; nohup setsid nc -l -p 9091 < /tmp/big.txt >/dev/null 2>&1 & sleep 1; netstat -lnt | grep -q 9091 && echo BULK_UP" | grep -q BULK_UP || bad "bulk listener"
 pf0 "printf 'pass in quick proto tcp from 10.0.1.0/24 to 10.0.1.1 port 9091 rdr-to $SRV_IP port 9091 keep state\npass out quick proto tcp keep state\n' > /tmp/g06b.conf; /opt/pf/pfctl -f /tmp/g06b.conf && echo BULK_RULE" >/dev/null
 GOT=$(cli 'nc -w10 10.0.1.1 9091 | wc -c' | tr -d '[:space:]')
 EXP=$(srv 'wc -c < /tmp/big.txt' | tr -d '[:space:]')
 if [ "$GOT" = "$EXP" ] && [ "${GOT:-0}" -ge 2000000 ]; then
 	ok "2 MB through the redirect intact ($GOT bytes)"
 else
 	bad "bulk through redirect: got $GOT want $EXP"
 fi
 # the firewall must still answer ssh (the GSO path must not wedge the box)
 pf0 'echo SSH_ALIVE' | grep -q SSH_ALIVE && ok "pf0 sshd still responsive" || bad "pf0 unreachable after bulk"
 srv 'pkill nc 2>/dev/null; true' >/dev/null
 pf0 '/opt/pf/pfctl -d' >/dev/null
 finish
@@ -0,0 +1,48 @@
 #!/usr/bin/env bash
 # M4 gate: pflog — logged packets must reach userspace.
 #
 # pflogd reads /dev/pflog (the port's chardev backend) and writes a pcap with
 # linktype DLT_PFLOG (117).  A `log` rule + ICMP traffic must produce records
 # that contain the real packet bytes (checked by searching the client address
 # in the capture) and a valid pcap header.
 set -uo pipefail; cd "$(dirname "$0")"; . ./lib.sh
 pf0 'insmod /opt/pf/pf.ko 2>/dev/null || insmod /tmp/pf.ko; true'
 pf0 '/opt/pf/pfctl -e; /opt/pf/pfctl -f /dev/null' >/dev/null
 pf0 'pkill -f "[p]flogd" 2>/dev/null; rm -f /tmp/pflog.pcap; true'
 LOGD=$(pfput ../../tools/out/pflogd) || { bad "pflogd deploy"; finish; }
 pf0 "test -c /dev/pflog && echo DEV_OK" | grep -q DEV_OK || { bad "/dev/pflog missing"; finish; }
 pf0 "chmod +x $LOGD; (setsid $LOGD -c 4 -f /dev/pflog /tmp/pflog.pcap >/tmp/pflogd.log 2>&1 </dev/null &); sleep 1; pgrep pflogd >/dev/null && echo DAEMON_UP" \
 	| grep -q DAEMON_UP || { bad "pflogd start"; finish; }
 cat > /tmp/g12.conf <<C
 pass in quick log proto icmp from $CLI_IP keep state
 pass out quick proto icmp keep state
 C
 pfload /tmp/g12.conf || { bad "log rule load"; finish; }
 cli "ping -c3 -W2 $SRV_IP >/dev/null 2>&1"
 sleep 2
 # pcap header: magic d4c3b2a1 (LE) and network = 117 at offset 20
 HDR=$(pf0 "od -An -tx1 -N24 /tmp/pflog.pcap 2>/dev/null | tr -d '[[:space:]]'")
 if [ "${HDR:0:8}" = "d4c3b2a1" ]; then ok "pcap magic ok"; else bad "pcap header: $HDR"; fi
 if [ "${HDR:40:4}" = "7500" ] || [ "${HDR:40:8}" = "75000000" ]; then
 	ok "linktype DLT_PFLOG (117)"
 else
 	bad "linktype: ${HDR:40:8}"
 fi
 SZ=$(pf0 'wc -c < /tmp/pflog.pcap' | tr -d '[:space:]')
 if [ "${SZ:-0}" -gt 24 ]; then ok "records captured ($SZ bytes)"; else bad "no records ($SZ)"; fi
 # the client address must appear inside a captured packet
 if pf0 "od -An -tx1 /tmp/pflog.pcap | tr -d '[[:space:]]' | grep -q 0a00010a"; then
 	ok "captured payload contains $CLI_IP"
 else
 	bad "client address not in capture"
 fi
 pf0 'pkill pflogd 2>/dev/null; /opt/pf/pfctl -d' >/dev/null
 finish
@@ -1,51 +1,51 @@
 #!/usr/bin/env bash
 # M6 gate: soak + unload/leak discipline.
 #
-# 1) load pf, ruleset with states/tables/anchors, drive mixed traffic ~20s
+# 1) load pf with a stateful ruleset, drive mixed traffic for ~20s
 # 2) assert no BUG/WARNING/Oops in dmesg
-# 3) disable, rmmod; assert module unloads cleanly (no refcount hang)
+# 3) disable + rmmod: the module must unload cleanly (no refcount leak)
-# 4) insmod/rmmod cycle x2; assert slab objects for pf caches return to the
+# 4) insmod/rmmod cycles: slab objects for the pf caches must return to the
-#    pre-load baseline (no leak), and that a reload works (timer/notifier
+#    pre-load baseline (no leak) and a reload must work
 #    re-registration is idempotent)
 set -uo pipefail; cd "$(dirname "$0")"; . ./lib.sh
-FAILED=0
+HELPER=$(pfput ./soak-helper.sh) || { bad "helper upload"; finish; }
-slab_snapshot() {
+pf0 "chmod +x $HELPER" >/dev/null
 	pf0 'grep -E "^(pf|pfk)[^ ]* " /proc/slabinfo | awk "{print \$1, \$3}"' \
 	    | sort
 }
-pf0 'insmod /tmp/pf.ko 2>/dev/null || insmod /opt/pf/pf.ko; true'
+pf0 'insmod /opt/pf/pf.ko 2>/dev/null || insmod /tmp/pf.ko; true'
 pf0 '/opt/pf/pfctl -e; /opt/pf/pfctl -f /dev/null' >/dev/null
-BEFORE=$(slab_snapshot)
+BEFORE=$(pf0 "$HELPER slab")
 cat > /tmp/g14.conf <<C
 table <peers> { $CLI_IP }
 anchor "s"
 pass in quick on ingress proto icmp from <peers> keep state
-pass out quick proto icmp keep state
+pass out quick on egress proto icmp keep state
 pass out quick proto tcp from any to any port 22 keep state
 C
-pfload /tmp/g14.conf || { bad soak ruleset; finish; }
+pfload /tmp/g14.conf || { bad "soak ruleset"; finish; }
-# mixed traffic: ICMP + TCP handshakes, 20s
+# mixed traffic: ICMP + TCP handshakes for ~20s
-cli "ping -q -c 30 -i 0.2 $SRV_IP >/dev/null 2>&1 &" 2>/dev/null
+cli "ping -q -c 40 -i 0.5 $SRV_IP >/dev/null 2>&1" 2>/dev/null
 cli "for i in 1 2 3 4 5; do nc -w1 -z $SRV_IP 22; done" 2>/dev/null
-sleep 22
+
-ST=$(pf0 '/opt/pf/pfctl -s states | wc -l')
+ST=$(pf0 "$HELPER states")
 [ "${ST:-0}" -ge 1 ] && ok "states under load ($ST)" || bad "no states under load"
-DMESG=$(pf0 'dmesg | grep -cE "BUG:|WARNING:|Oops|general protection"')
+ANOM=$(pf0 "$HELPER dmesg")
-[ "${DMESG:-1}" = 0 ] && ok "no kernel BUG/WARN/Oops" || bad "dmesg anomalies ($DMESG)"
+if [ -z "$ANOM" ]; then
 	ok "no kernel BUG/WARN/Oops"
 else
 	bad "dmesg anomalies:"
 	printf '%s\n' "$ANOM" | sed 's/^/    /'
 fi
 pf0 '/opt/pf/pfctl -d; /opt/pf/pfctl -f /dev/null' >/dev/null
 pf0 'rmmod pf && echo UNLOAD_OK' | grep -q UNLOAD_OK && ok "clean unload (refcounts released)" || bad "rmmod failed"
 sleep 1
-pf0 'insmod /tmp/pf.ko 2>/dev/null || insmod /opt/pf/pf.ko; true; (/opt/pf/pfctl -s info >/dev/null && echo RELOAD_OK)' | grep -q RELOAD_OK && ok "reload after unload" || bad "reload failed"
+pf0 'insmod /opt/pf/pf.ko 2>/dev/null || insmod /tmp/pf.ko; true; (/opt/pf/pfctl -s info >/dev/null && echo RELOAD_OK)' \
 	| grep -q RELOAD_OK && ok "reload after unload" || bad "reload failed"
 pf0 'rmmod pf && echo UNLOAD_OK2' | grep -q UNLOAD_OK2 && ok "second unload" || bad "second rmmod"
 sleep 1
-AFTER=$(slab_snapshot)
+AFTER=$(pf0 "$HELPER slab")
 # compare per-cache object counts if they exist on both sides
 LEAK=$(join <(echo "$BEFORE") <(echo "$AFTER") 2>/dev/null | awk '$3-$2 > 0 {print $1, $3-$2}')
 if [ -n "$LEAK" ]; then bad "slab growth after cycles: $LEAK"; else ok "no slab growth after unload cycles"; fi
 finish
@@ -0,0 +1,18 @@
 #!/usr/bin/env bash
 # Helper executed ON pf0 by 14-soak.sh (uploaded with pfput to avoid quoting
 # hell through two ssh hops).
 #   $1 = "slab" | "dmesg" | "states"
 case "${1:-}" in
 slab)
 	grep -E '^(pf|pfk)' /proc/slabinfo | tr -s ' ' | cut -d' ' -f1,3 | sort
 	;;
 dmesg)
 	# the CPU's Speculative Return Stack Overflow advisory is a kernel
 	# build/CPU feature notice, not a fault of this module
 	dmesg | grep -E 'BUG:|WARNING:|Oops|general protection|kernel BUG' \
 		| grep -v 'Speculative Return Stack Overflow' || true
 	;;
 states)
 	/opt/pf/pfctl -s states 2>/dev/null | grep -c 'icmp' || true
 	;;
 esac
@@ -57,7 +57,7 @@ GOODTESTS := tests/01_basic.conf tests/02_translations.conf \
 	tests/03_queues_scrub.conf
 .PHONY: all pfctl abitest test abi clean
-all: pfctl abitest pdivert
+all: pfctl abitest pdivert pflogd
 pfctl: $(OUT)/pfctl
@@ -190,3 +190,12 @@ $(PD): pdivert/pdivert.c
 	$(CC) $(CFLAGS) $(LDFLAGS) -o $@ pdivert/pdivert.c
 clean: 
 # ---------------------------------------------------------------------------
 # pflogd - reads /dev/pflog and writes DLT_PFLOG pcap (self-contained)
 LOGD := $(OUT)/pflogd
 pflogd: $(LOGD)
 $(LOGD): pflogd/pflogd.c
 	@mkdir -p $(OUT)
 	$(CC) $(CFLAGS) $(LDFLAGS) -o $@ pflogd/pflogd.c
@@ -1562,6 +1562,84 @@ ifa_exists(const char *ifa_name)
 	return (NULL);
 }
 /*
 * PFLINUX: interface group emulation.
 *
 * OpenBSD creates the "ingress" and "egress" interface groups (ifgroups)
 * and pfctl expands "(egress)" in a pool into the addresses of the group
 * members via the SIOCGIFGMEMB socket ioctl.  Linux has neither ifgroups
 * nor that ioctl, so resolve the two groups locally:
 *
 *   egress  -> the interface of the default route (the same interface the
 *              kernel module marks as the pf "egress" interface)
 *   ingress -> every non-loopback interface
 */
 static struct node_host *
 ifa_group_members(const char *group, int flags)
 {
 	struct node_host	*n, *h = NULL, *c;
 	struct sockaddr_in	 sin;
 	socklen_t		 slen = sizeof(sin);
 	int			 s;
 	if (iftab == NULL)
 		ifa_load();
 	if (!strcmp(group, "egress")) {
 		/* source address of the default route */
 		memset(&sin, 0, sizeof(sin));
 		sin.sin_family = AF_INET;
 		sin.sin_port = htons(9);
 		sin.sin_addr.s_addr = htonl(0x01010101);	/* discard route */
 		if ((s = socket(AF_INET, SOCK_DGRAM, 0)) != -1) {
 			if (connect(s, (struct sockaddr *)&sin,
 			    sizeof(sin)) == 0 &&
 			    getsockname(s, (struct sockaddr *)&sin, &slen) == 0) {
 				close(s);
 				for (n = iftab; n; n = n->next) {
 					if (n->af != AF_INET)
 						continue;
 					if (n->addr.v.a.addr.v4.s_addr !=
 					    sin.sin_addr.s_addr)
 						continue;
 					if ((c = calloc(1, sizeof(*c))) == NULL)
 						err(1, "calloc");
 					*c = *n;
 					c->next = NULL;
 					c->tail = c;
 					return (c);
 				}
 				return (NULL);
 			}
 			close(s);
 		}
 		return (NULL);
 	}
 	/* "ingress" and any other group: every non-loopback interface */
 	for (n = iftab; n; n = n->next) {
 		if (n->af == AF_LINK)
 			continue;
 		if (n->ifa_flags & IFF_LOOPBACK)
 			continue;
 		if ((flags & PFI_AFLAG_BROADCAST) && n->af != AF_INET)
 			continue;
 		if ((c = calloc(1, sizeof(*c))) == NULL)
 			err(1, "calloc");
 		*c = *n;
 		c->next = NULL;
 		c->tail = c;
 		if (h == NULL)
 			h = c;
 		else {
 			h->tail->next = c;
 			h->tail = c->tail;
 		}
 	}
 	return (h);
 }
 struct node_host *
 ifa_grouplookup(const char *ifa_name, int flags)
 {
@@ -1576,6 +1654,10 @@ ifa_grouplookup(const char *ifa_name, int flags)
 	strlcpy(ifgr.ifgr_name, ifa_name, sizeof(ifgr.ifgr_name));
 	if (ioctl(s, SIOCGIFGMEMB, (caddr_t)&ifgr) == -1) {
 		close(s);
 		/* PFLINUX: interface group emulation (see above) */
 		if (!strcmp(ifa_name, "ingress") ||
 		    !strcmp(ifa_name, "egress"))
 			return (ifa_group_members(ifa_name, flags));
 		return (NULL);
 	}
@@ -0,0 +1,184 @@
 /*
 * pflogd for PF-on-Linux (ISC 2026)
 *
 * Reads records from /dev/pflog (see module/pfsrc/pfk_pflog.c) and writes a
 * pcap file with linktype DLT_PFLOG (117) - the same format OpenBSD's pflogd
 * produces, so tcpdump -r / wireshark decode it.
 *
 *   pflogd [-i iface] [-f /dev/pflog] [-p pidfile] [-c count] <file>
 *
 * A "-" as <file> writes to stdout.  Omitting <file> prints a human-readable
 * one-line summary per record (useful in the lab gates).
 */
 #include <errno.h>
 #include <fcntl.h>
 #include <stdint.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
 #include <signal.h>
 #include <unistd.h>
 #include <sys/select.h>
 #include <sys/stat.h>
 #include <sys/ioctl.h>
 #include <netinet/in.h>
 #define PFLOG_IFNAMSIZ		16
 #define PFLOG_RULESET_NAME	16
 #define DLT_PFLOG		117
 struct pflog_hdr {
 	int64_t		ts_sec;
 	int64_t		ts_usec;
 	uint8_t		af;
 	uint8_t		reason;
 	uint8_t		ifname[PFLOG_IFNAMSIZ];
 	uint8_t		ruleset[PFLOG_RULESET_NAME];
 	uint8_t		nr;
 	uint8_t		anc;
 	uint8_t		flags;
 	uint8_t		pad[9];
 	uint16_t	pktlen;		/* network order, as in BSD */
 } __attribute__((packed));
 struct pcap_file_header {
 	uint32_t magic;
 	uint16_t major, minor;
 	int32_t thiszone;
 	uint32_t sigfigs, snaplen, network;
 };
 struct pcap_pkthdr {
 	uint32_t ts_sec, ts_usec, caplen, len;
 };
 static volatile int running = 1;
 static void onterm(int s) { (void)s; running = 0; }
 static const char *reason_name(uint8_t r)
 {
 	static const char *n[] = { "pass", "drop", "scrub", "noscrub",
 	    "nat", "nonat", "rdr", "nordr" };
 	return r < 8 ? n[r] : "?";
 }
 int
 main(int argc, char **argv)
 {
 	const char *dev = "/dev/pflog", *file = NULL, *pidfile = NULL;
 	long want = -1;
 	int fd, out = 1, ch, wrote_header = 0, nrec = 0;
 	unsigned char buf[sizeof(struct pflog_hdr) + 2048];
 	while ((ch = getopt(argc, argv, "f:c:p:")) != -1)
 		switch (ch) {
 		case 'f': dev = optarg; break;
 		case 'c': want = atol(optarg); break;
 		case 'p': pidfile = optarg; break;
 		default: goto usage;
 		}
 	if (optind < argc)
 		file = argv[optind];
 	signal(SIGINT, onterm);
 	signal(SIGTERM, onterm);
 	fd = open(dev, O_RDONLY);
 	if (fd < 0) {
 		fprintf(stderr, "pflogd: open %s: %s\n", dev, strerror(errno));
 		return 1;
 	}
 	if (file && strcmp(file, "-") != 0) {
 		out = open(file, O_WRONLY | O_CREAT | O_TRUNC, 0600);
 		if (out < 0) {
 			fprintf(stderr, "pflogd: open %s: %s\n", file,
 			    strerror(errno));
 			return 1;
 		}
 	}
 	if (pidfile) {
 		FILE *p = fopen(pidfile, "w");
 		if (p) { fprintf(p, "%d\n", (int)getpid()); fclose(p); }
 	}
 	while (running) {
 		ssize_t n, off = 0;
 		struct pflog_hdr *h;
 		/* read one full record: the chardev returns header+payload
 		 * consecutively; a short read just continues where we left */
 		while (off < (ssize_t)sizeof(*h)) {
 			n = read(fd, buf + off, sizeof(buf) - off);
 			if (n <= 0) {
 				if (n < 0 && errno == EINTR)
 					goto cont;
 				if (n < 0 && (errno == EAGAIN || errno == EWOULDBLOCK)) {
 					fd_set r;
 					struct timeval tv = { 0, 500000 };
 					FD_ZERO(&r); FD_SET(fd, &r);
 					if (select(fd + 1, &r, NULL, NULL, &tv) <= 0)
 						goto cont;
 					continue;
 				}
 				goto cont;
 			}
 			off += n;
 		}
 		h = (struct pflog_hdr *)buf;
 		{
 			uint16_t plen = ntohs(h->pktlen);
 			while (off < (ssize_t)(sizeof(*h) + plen)) {
 				n = read(fd, buf + off, sizeof(buf) - off);
 				if (n <= 0)
 					break;
 				off += n;
 			}
 			if (off < (ssize_t)(sizeof(*h) + plen))
 				goto cont;
 			if (file) {
 				struct pcap_file_header g = {
 					.magic = 0xa1b2c3d4, .major = 2,
 					.minor = 4, .thiszone = 0,
 					.sigfigs = 0, .snaplen = 65535,
 					.network = DLT_PFLOG,
 				};
 				struct pcap_pkthdr ph = {
 					.ts_sec = (uint32_t)h->ts_sec,
 					.ts_usec = (uint32_t)h->ts_usec,
 					.caplen = sizeof(*h) + plen,
 					.len = sizeof(*h) + plen,
 				};
 				if (!wrote_header) {
 					if (write(out, &g, sizeof(g)) != sizeof(g))
 						break;
 					wrote_header = 1;
 				}
 				if (write(out, &ph, sizeof(ph)) != sizeof(ph) ||
 				    write(out, buf, ph.caplen) != (ssize_t)ph.caplen)
 					break;
 			} else {
 				printf("%lld.%06lld %s if=%s ruleset=%s nr=%u "
 				    "len=%u\n", (long long)h->ts_sec,
 				    (long long)h->ts_usec, reason_name(h->reason),
 				    (char *)h->ifname, (char *)h->ruleset,
 				    h->nr, plen);
 				fflush(stdout);
 			}
 			nrec++;
 			if (want > 0 && nrec >= want)
 				break;
 		}
 cont:
 		if (want > 0 && nrec >= want)
 			break;
 	}
 	if (file && strcmp(file, "-") != 0)
 		close(out);
 	close(fd);
 	fprintf(stderr, "pflogd: %d records\n", nrec);
 	return 0;
 usage:
 	fprintf(stderr, "usage: pflogd [-f dev] [-c count] [-p pidfile] [file]\n");
 	return 2;
 }
Author	SHA1	Message	Date
lgcosta	bc5b8a4a1e	demo: pflog step in the asciinema recording (pflogd: 2 records, DLT_PFLOG pcap); re-recorded cast	2026-11-11 22:07:45 -03:00
lgcosta	06ad1baa3a	runner: deploy pdivert/pflogd alongside pf.ko and pfctl	2026-11-11 22:00:15 -03:00
lgcosta	f263b63b8e	docs: guides reflect verified NAT/redirect/divert/pflog and the fixed state listing; CHANGELOG for v0.4.0	2026-11-11 21:58:57 -03:00
lgcosta	92c015b97b	docs: pflog verified (matrix EN+PT-BR, porting notes, README status)	2026-11-11 21:53:55 -03:00
lgcosta	7a81076c6a	pflog: /dev/pflog chardev backend (BSD pflog_hdr layout, DLT_PFLOG) + tools/pflogd + gate 12	2026-11-11 21:52:36 -03:00
lgcosta	dbb21048e4	tests: 06 gate self-sufficient (bracketed pkill patterns, srv cleanup) - avoids cross-run listener contamination	2026-11-11 21:35:50 -03:00
lgcosta	0eb040fcc8	docs: rdr-to verified; porting notes on checksum strategy across translation paths (GSO regression)	2026-11-11 21:27:32 -03:00
lgcosta	5af7adb08e	checksums: GSO-aware recompute after translation (software for plain skbs, CHECKSUM_PARTIAL for GSO); 06 gate asserts 2MB integrity through rdr	2026-11-11 21:24:41 -03:00
lgcosta	d9fc3a7e3f	checksums: materialize CHECKSUM_PARTIAL before pf so address-rewrite fixups operate on real values; new 06-rdr gate	2026-11-11 20:53:03 -03:00
lgcosta	b554c9a6ef	demo: ship the asciinema recording in the repo (out/demo-pf-linux.cast)	2026-11-11 20:33:57 -03:00
lgcosta	6cc4e21ae2	docs+demo: asciinema recording of the lab (stateful filtering, tables, drop, NAT, divert); README/matrix reflect the verified state	2026-11-11 20:33:27 -03:00
lgcosta	2cd13eb806	tests: soak helper ignores the CPU SRSO advisory	2026-11-11 20:29:33 -03:00
lgcosta	e03df9adc8	docs: data-path semantics and NAT checklist	2026-11-11 20:25:57 -03:00
lgcosta	7d1e4b81e9	cleanup: drop debug instrumentation	2026-11-11 20:24:53 -03:00
lgcosta	f3143a54a8	nat: route the reverse-translated packet with an empty source (kernel rejects non-local saddr)	2026-11-11 20:23:44 -03:00
lgcosta	917d3e3fe0	nat: recompute IPv4 header checksum after pf rewrote the addresses (OpenBSD ip_output does this)	2026-11-11 20:20:52 -03:00
lgcosta	d520172753	nat: re-route inbound packets whose destination pf rewrote (reverse NAT) instead of delivering them locally	2026-11-11 20:17:27 -03:00
lgcosta	9c3c4b6bbc	fix: skb->mark masks were multi-bit (IN mark matched OUT test) - transit traffic never reached the out ruleset; ingress/egress interface groups; pfctl resolves (egress) pool locally	2026-11-11 20:15:35 -03:00
lgcosta	c449e01b9a	pf: emulate OpenBSD ingress/egress interface groups as any-interface (Linux has no ifgroups)	2026-11-11 20:07:17 -03:00
lgcosta	525fe7ee28	tests: gate assertions corrected (pfctl state format, transit egress device, CPU SRSO advisory excluded); Makefile demo target	2026-11-11 20:04:16 -03:00
lgcosta	18c0a69f74	tests: soak gate via uploaded helper (quoting), 04/05 assertions fixed (pf interface name vs OS device)	2026-11-11 19:59:27 -03:00
lgcosta	3aa0c2d0d0	docs: demo guide (asciinema recording/playback/upload) EN+PT-BR	2026-11-11 19:56:24 -03:00
lgcosta	66939c839a	tests: 04 gate rewritten to OpenBSD-correct anchor semantics (reply is evaluated by the in ruleset); 05 gate uses 8.0 nat-to syntax; 14 quoting fixed; demo session + asciinema recorder	2026-11-11 19:54:29 -03:00
lgcosta	444a0555c6	docs: divert verified (matrix + porting notes); FORWARD hook evaluates the out ruleset for transit traffic	2026-11-11 19:50:14 -03:00
lgcosta	bdf9db7a21	fix: remove leftover debug fragment in pfk_hooks	2026-11-11 19:45:43 -03:00
		`@@ -1 +1 @@`
			savedcmd_pf.mod := printf '%s\n' pfsrc/pfk_mod.o pfsrc/pfk_hooks.o compat/pfk_mbuf.o compat/pfk_kern.o compat/pfk_if.o compat/pfk_xmit.o compat/pfk_pcb.o compat/pfk_ip4.o compat/siphash.o compat/toeplitz.o compat/md5.o compat/sha2.o compat/radix.o compat/pfk_rtable.o compat/pfk_ip6.o compat/subr_tree.o pfsrc/pf_ruleset.o pfsrc/pf_table.o pfsrc/pf_norm.o pfsrc/pf_if.o pfsrc/pf_lb.o pfsrc/pf_osfp.o pfsrc/pf_syncookies.o pfsrc/pf_ioctl.o pfsrc/pf.o pfsrc/divert_sock.o compat/pfk_ifq.o \| awk '!x[$$0]++ { print("./"$$0) }' > pf.mod				savedcmd_pf.mod := printf '%s\n' pfsrc/pfk_mod.o pfsrc/pfk_hooks.o compat/pfk_mbuf.o compat/pfk_kern.o compat/pfk_if.o compat/pfk_xmit.o compat/pfk_pcb.o compat/pfk_ip4.o compat/siphash.o compat/toeplitz.o compat/md5.o compat/sha2.o compat/radix.o compat/pfk_rtable.o compat/pfk_ip6.o compat/subr_tree.o pfsrc/pf_ruleset.o pfsrc/pf_table.o pfsrc/pf_norm.o pfsrc/pf_if.o pfsrc/pf_lb.o pfsrc/pf_osfp.o pfsrc/pf_syncookies.o pfsrc/pf_ioctl.o pfsrc/pf.o pfsrc/divert_sock.o pfsrc/pfk_pflog.o compat/pfk_ifq.o \| awk '!x[$$0]++ { print("./"$$0) }' > pf.mod