pf-linux-port/docs/superpowers/specs/2026-11-11-pf-linux-port-design.md

PF on Linux — Port Design / Design do Port

Date/2026-11-11 · Status: awaiting approval/aguardando aprovação · Repo: https://git.labexposed.com/lgcosta/pf-linux-port (public)

1. Intent / Intenção

Run OpenBSD's PF packet filter inside Linux 7.3.4 — the real PF engine, driven by the real pfctl speaking the real /dev/pf ioctl ABI — as an out-of-tree module with a documented, minimal set of kernel core patches, validated end-to-end in a libvirt Alpine VM. Docs bilingual (EN + pt-BR).

Rodar o PF do OpenBSD no Linux 7.3.4 — engine PF real, controlado pelo pfctl real falando a ABI ioctl /dev/pf — como módulo out-of-tree com um conjunto mínimo e documentado de patches no core, validado numa VM Alpine via libvirt.

2. Facts found / Fatos levantados

• OpenBSD replaced classic ALTQ by ≥7.7 (verified: no sys/net/altq dir in 7.7/7.8/8.0) — but PF queueing lives on in 8.0, rearchitected into the core ifq framework: sys/net/ifq.c (priq ops), sys/net/hfsc.c, sys/net/fq_codel.c, ioctls DIOCADDQUEUE/DIOCGETQUEUES/DIOCGETQUEUE/ DIOCGETQSTATS (pfvar.h:1707+), pf_rule.queue glue all present; pfctl still parses/installs queue on … hfsc|fq_codel|default. Older ALTQ releases (≤7.6) are no longer mirrored anywhere (cdn/ftp/mirrors/GitHub-raw/Wayback all 404), making a 7.x base infeasible/undesirable anyway. ⇒ port base = OpenBSD 8.0 (exactly matches the user's provided userland tree, pfctl.c v1.406 2026-08-30; sources fetched from the user's C3SL mirror). The ALTQ requirement is met as OpenBSD-8.0 queueing (priq + HFSC + fq_codel) — the framework modern OpenBSD itself ships; classic cbq/wfq/red are non-goals because they don't exist in the base, honestly matrixed.
• Kernel side ≈ 24.6k LOC (pf.c 9.2k, pf_ioctl.c 4.6k, pf_norm/if/lb/table/ ruleset/osfp/syncookies, pfvar(_priv).h) + queueing core (ifq/hfsc/fq_codel ≈ 2.4k LOC) + ip_divert.c reference (~300 LOC).
• Userland: single pfctl (~20k LOC incl. parse.y) — libpfctl was folded back into pfctl in this era (Makefile LDADD+= -lm only; no libpfctl in tree).
• Linux 7.3.4 tree clean; toolchain gcc 16/flex/bison/bc; libvirt 12 + KVM usable (existing sms-vm untouched); sudo works; Alpine cloud images available on dl-cdn.
• Module-visible APIs confirmed exported: neigh_lookup, pskb_expand_head, skb_clone, register_qdisc, qdisc_tree_reduce_backlog, tcf_classify, __dev_queue_xmit, netif_rx, inet_addr_type, fib_table_lookup (GPL).
• Confirmed not exported in 7.3.4: neigh_output, dev_queue_xmit (inline wrapper over __dev_queue_xmit), ip_route_output_key, ip6_route_output, netlink_kernel_create ⇒ core patch P1 (mandatory): export symbols required by the PF module (additive EXPORT_SYMBOL_*, zero behavior change) — allowed by user, fully documented in patches/kernel/ + docs/*/patches.md.

3. Goals / Non-goals

Goals (G)

1. PF engine in-tree-equivalent quality, delivered as module pf.ko built from pf_port/ sources: rules (anchors, quick, match/block/pass/drop, return variants, keep state), states + syncookies, tables (pfr: radix/hash, pfctl -t), NAT (nat-to, rdr-to incl. sticky-port/bitmap basics, binat-to, route-to), scrub/normalize (fragment reassembly, DF/TTL actions, min-ttl, max-mss), PF queues/ALTQ semantics of modern OpenBSD 8.0 (queue, queue on with default(priq), hfsc, fq_codel schedulers from sys/net/ifq.c, hfsc.c, fq_codel.c; pfctl -s queue -v stats), divert (divert-to, divert tag; userspace channel modeled on sys/netinet/ip_divert.c), route-to (pools + rr/rr-source/rr-any, hash-source/state; reply-to, dup-to, fail-to).
2. Control plane = OpenBSD ioctl ABI on /dev/pf; Linux-built pfctl (8.0, single binary, no libpfctl) + pflogd-lite + pdivert tool.
3. Patchset + scripts in the repo; kernel core patches only when unavoidable, each documented (what/why/where/impact).
4. VM validation: kernel 7.3.4 built with patches, module + tools deployed into an Alpine VM booted with that kernel; a scripted suite exercises every goal-1 area with real traffic across a 4-VM libvirt lab (§7) and checks PASS/FAIL.
5. Full docs EN + pt-BR: apply patches, build, install, feature guide, VM validation how-to, feature matrix incl. exclusions.

Non-goals (NG, explicit to user): pfsync/carp (no failover), full IPsec/pfkey, classic-ALTQ-only schedulers (cbq/wfq/red — do not exist in the OpenBSD 8.0 base, matrixed as "removed upstream"), queues on interfaces whose root qdisc is not ours, per-netns PF instances (faithful port = single global instance like OpenBSD; netns isolation documented as out of scope), OS-fingerprinting beyond pfctl -o (osfp is bonus if cheap), ftp-proxy and other userland daemons.

Anti-goals: silently faking a feature (stub verdicts), nftables-as-substitute claims, tests that only check exit codes without inspecting packets/state.

4. Approach (chosen) / Abordagem

B′ — faithful port with a compat shim (alternative C rejected: user wants PF in kernel; A rejected: infeasible). Port the actual OpenBSD 8.0 pf*.c + ifq/hfsc/ fq_codel + ip_divert-referenced divert under a compatibility layer, hook into Linux via netfilter, keep upstream ABI for userland. (Rejected variant: rewrite of PF semantics on nf APIs = not a port.)

4.1 Compat layer (module/compat/)

• Packet object: struct mbuf shim over sk_buff (single-segment: linearize on entry to PF; m_copym/m_adj/m_pull/m_copydata/m_cat/m_clump/m_free → skb ops; m_pkthdr ↔ skb->protocol/tc/index, address family, ifindex via skb->dev). This mirrors how OpenBSD's own pf code treats mbufs and keeps diffs localized to packet I/O, not to matching logic.
• APIs: kern_* glue: M_* malloc→kmalloc/kfree (pf's malloc flags → GFP), timeout()→timer/workqueue (state purger), rwlock_*→RW-lock, pool→kmem_cache, epoch→RCU read side + refcounts on rulesets (matching how conntrack already models this), time_second→ktime_get_seconds, syncache untouched, radix.h/tree.h (BSD) vendored with Linux tweaks, imsg not needed in kernel.
• Netlink/ioctl: misc_register /dev/pf, all DIOC* handled exactly as pf_ioctl.c (port file with shim includes). ioctl number encoding: define _IOR/_IOW/_IOWR(PFRDR,'n',type) OpenBSD-style in our shared pfvar.h so kernel and Linux-pfctl agree bit-for-bit.
• IPv4/IPv6: in_proto/in6_proto glue for checksums (ip_sum_eq, pseudo-header sum via csum_tcpudp_magic, tcp_input.c's in_cksum ported to pf_compat_cksum.c); raw socket sosend paths replaced by icmp_send/icmp6_send (RST via tcp_v4_send_reset equivalents kept in-pf to preserve PF's own return logic).
• Interface/addresses: ifnet→net_device + in_ifaddr→net_ifa walks via net_if_FOREACH/in_ifa_lookup helpers over if_addr_list + fib (replaces ifa_ifwithroute for route-to); pf_getifaddr ported with Linux address lists.

4.2 Hooking

Register netfilter hooks per family:

• PREROUTING (NF_IP_PRI_FIRST+): pf input path = scrub/normalize → state lookup → nat/rdr (pf's own NAT, independent of conntrack; deployment docs warn about overlap zones), filter INPUT.
• FORWARD: filter + route-to/dup-to execution.
• LOCAL_OUT: filter (OUTPUT) + nat-to (locally generated).
• POSTROUTING (pri low): state insert, queue classify (skb→pfq class id), route-to final xmit. Verdict mapping: pass→NF_ACCEPT, block/normalization-drop→NF_DROP (with PF's return-RST/ICMP generated by pf itself), divert→NF_STOLEN + divert channel, nat/rdr rewrite → NF_ACCEPT after skb_cow + checksum fix. Coexistence with iptables/nft: PF hooks run before conntrack (unregister-safe ordering); documented.

4.3 Queues (OpenBSD 8.0 ifq) ⇒ pfq qdisc shim

Port sys/net/ifq.c/ifq.h + sys/net/hfsc.c + sys/net/fq_codel.c (the real 8.0 queueing framework: priq glue, HFSC, FQ-CoDel as ifq_ops) into module/ifq/. struct ifqueue instances are backed by a classful Linux qdisc pfq (register_qdisc): pfq_enqueue → ifq_enqueue (matched class via pf tags), pfq_dequeue → ifq_deq_begin/commit → __dev_queue_xmit. tc qdisc replace dev X root pfq attaches; scripts/pf-qdisc.sh auto-attaches when a loaded ruleset declares queue on (pfq handles both schedulers + default priq via pf_qblocks priority bits). DIOCADDQUEUE/DIOCGETQUEUES/DIOCGETQUEUE/DIOCGETQSTATS (the real 8.0 names) served against the pfq/ifq registry so pfctl -s queue -v and pfctl -s ifq -v work. Tested mandatory schedulers: default(priq), hfsc, fq_codel with rate gates (§7); class visibility via tc -s qdisc show is a stretch goal, counters via pfctl are the gate.

4.4 divert — real AF_DIVERT in the kernel (no netlink substitute)

OpenBSD's divert(4) socket family is ported as a runtime-registered socket family inside pf.ko: Linux 7.3 registers protocol families with sock_register() and NPROTO = AF_MAX = 45, so BSD's PF_DIVERT = 32 (the unused legacy PF_IUCV slot) fits with zero core changes. Implemented in module/pfsrc/divert_sock.c: socket(PF_DIVERT, SOCK_RAW, IPPROTO_DIVERT)

• bind(port) + recvmsg/sendmsg, packet metadata via cmsg (ifindex, dir, divert type, original dst). divert_packet()/divert6_packet() (called by the ported pf.c) queue to the bound socket; sendmsg re-injects with OpenBSD semantics: PF_DIVERT_TO → local delivery (netif_rx on the recorded interface), plain divert tag → re-route (dst->output), mirroring ip_divert.c:divert_output(). The BSD divert(8) workflow (upstream regress/sys/net/pf_divert) drives this socket unchanged; our shipped pdivert is a thin Linux port. Wire format documented in docs/en/divert-wireformat.md. pflog keeps its netlink multicast channel (slot PFLOG_PROTO=1 on NETLINK_USERSOCK).

4.5 route-to / dup-to / fail-to

pf_route6/pf_route port: pool selection via pf_lb.c (as-is), next-hop resolution ifa_ifwithroute→pf_route4gw(): build flowi4/flowi6, ip_route_output_key to gw, then L2-redirect via neigh_lookup(arp_tbl/nd_tbl, &gw, dev) + neigh_output (or dev_queue_xmit after dev_hard_header); TTL-- before xmit, state key = translated, reply-to via reverse state's stored answer route. dup-to = skb_clone to original path. If a needed route/neigh helper isn't exported → kernel core export patch, documented.

4.6 pflog (small, included)

if_pflog.c → netlink multicast channel (NETLINK_PF group) + pflogd userland that writes pcap file (OpenBSD pflogd logic), so tcpdump -r verifies passes; enables state/queue tests to assert headers (e.g., NAT rewrites, scrub).

4.7 Userland build (tools/)

Linux build of OpenBSD 8.0 sbin/pfctl (single binary — libpfctl is folded in), parse.y via flex/bison, plus pdivert + pflogd, with a compat-user/ shim (err/warn, strlcpy, pledge stub, arc4random, sysctl subset, OpenBSD-shaped _IOR/_IOW/_IOWR). Shared ABI header include/net/pfvar.h (ours, identical to the kernel's) guarantees ioctl bit-parity; static asserts on every ioctl struct.

5. Kernel core patches

Target zero behavior changes to Linux; the patchset ships:

• P1 (mandatory) net: export symbols needed by the PF module (neigh_output, ip_route_output_key, ip6_route_output, netlink_kernel_create, + any added during M1–M5 builds) — additive EXPORT_SYMBOL*() lines, no logic touched. Confirmed missing in 7.3.4.
• No other core changes by default: module builds out-of-tree (make -C linux-7.3.4 M=pf_port/module), tree otherwise untouched; if a future step needs more (e.g., a netlink protocol reservation), each lands as its own patches/kernel/000N-*.patch appended by scripts/apply-patches.sh, with a per-patch entry in docs/{en,pt-BR}/patches.md: what/why/impact/how to revert.

6. Repo layout

pf_port/            (= repo root, git → pf-linux-port)
  README.md / README.pt-BR.md
  docs/{en,pt-BR}/  guide, install, apply-patches, feature-matrix, validation,
                    divert-wireformat, patches/*.md
  docs/superpowers/specs/            this file
  patches/kernel/   000N-*.patch + series + README
  module/           pf.ko + pfq.ko sources: pf/, ifq/, compat/, include/
  tools/            pfctl/, pdivert/, pflogd/, compat-user/
  scripts/          fetch-sources.sh  apply-patches.sh  unpatch.sh
                    build-module.sh   build-tools.sh    vm-create.sh
                    vm-deploy.sh      vm-run-tests.sh   bench-basic.sh
  tests/vm/         01-state.sh … 14-altq.sh, scapy/python helpers, expect-files
  _vendor/          OpenBSD 7.8/8.0 tarballs (gitignored; refetch script)

7. Validation (libvirt on the 192.168.1.242 host, Alpine) — "done" definition

scripts/vm-create.sh clones the Alpine 3.22 cloud qcow2 (verified: 840 MiB, sha256 e83786b1…), NoCloud seed (user/network config), and defines 4 libvirt domains on an isolated 192.168.122.x-free bridge lab (pf-lan, pf-wan, pf-wan2 networks, no NAT to the outside needed for gates): pftest-pf0 (boots the self-built 7.3.4 kernel + initrd via direct kernel boot; 3 NICs: lan/wan/wan2), pftest-cli, pftest-srv, pftest-gw2. Decision (builder-check #1): PF stays a single global instance bound to init_net, exactly like OpenBSD — no per-netns rulesets; hence endpoints must be real hosts (VMs), not namespaces. The deb host (12 cores, libvirt 11.3 validated) runs the lab; the laptop keeps building sources. Suite (vm-run-tests.sh, ssh-driven, PASS/FAIL per feature): stateful icmp/tcp/udp (+IPv6 smoke), block drop/return, tables (pfctl -t -T add/del/test), anchors+quick, nat-to (asserted on srv: src=pf0's addr), rdr-to (srv httpd reached via pf0 DNAT), binat-to, scrub (crafted frags via scapy on cli: overlap drop, DF-set, max-mss clamp), route-to pools rr + hash-source (asserted: wan vs wan2 tcpdump counts; fail-to when gw2 down; reply-to symmetry; dup-to capture), divert (pdivert rewrites payload end-to-end; timeout drop without server), queues: queue on … { hfsc … } and fq_codel and default priq — iperf throughput within ±15% of rate on the bottleneck link + pfctl -s queue -v counter growth + pfq attaches/detaches cleanly, pflog capture sanity (passes appear in pcap), ruleset reload under traffic (soak loop), rmmod/load leak check (kmemleak + module refcounts). IPv6 policy (builder-check #4): engine is v6-complete by construction (single pf.c); gates are v4 for everything + v6 smoke for state/block/nat66/scrub; v6-only landmines (route-to6 pool failover) matrixed "ported, smoke-validated". Performance (builder-check #2): functional gates are the bar; bench-basic.sh is informational (reports pps vs iptables baseline, no numeric gate) — a perf contract would force a non-faithful rewrite-first shim. Artifacts: run log + per-test pcap attached to the Gitea release. All gates must pass; anything failing ships as explicit FAIL in the feature matrix, not silently.

8. Milestones & delivery

M0 infra: repo+CI-ish scripts, kernel baseline build OK, VM boots custom kernel. M1 engine-boot: compat layer + module loads, /dev/pf, pfctl -f empty + -s info. M2 filter core: rules/states/tables/anchors pass VM tests 1–4. M3 NAT suite (nat/rdr/binat) + scrub + pflog. M4 route-to/dup-to/fail-to + divert. M5 queues: ifq/hfsc/fq_codel port + pfq qdisc + queue semantics + counters. M6 docs bilingual complete, patchset tidy, full suite re-run green, tags v0.x in repo. Commits per milestone to Gitea (public), final release with validation artifacts.

9. Builder-decisions (my calls, reversible)

• Base OpenBSD 8.0 (matches the provided userland tree exactly; ALTQ delivered as 8.0's real queue framework ifq{priq,hfsc,fq_codel}; older ALTQ releases are unrecoverable from public mirrors anyway).
• Single global PF instance on init_net (faithful to OpenBSD); validation via multi-VM lab, not netns (builder-check #1).
• sk_buff-backed mbuf shim instead of rewriting pf.c to skb (keeps upstream logic line-proximate; enables future rebase to newer OpenBSD).
• divert via netlink channel instead of inventing AF_DIVERT (no core socket-family patch).
• pflog via netlink not pseudo-if tap (keeps "no extra interface" promise to users).
• Module name pf.ko (no Linux collision).
• No numeric perf gate (bench informational) and v4-gated + v6-smoked suite (builder-checks #2/#4).

10. Risks

• pf.c porting surface largest; queue scheduler coverage limited to what exists upstream in 8.0 (mitigation: 3 schedulers gated, matrix explicit).
• ioctl-ABI bit parity between two toolchains (mitigation: single shared header, struct-size static asserts both sides).
• Kernel 7.3 symbol drift (mitigation: P1 export patches allowed+documented).
• Netfilter ordering vs conntrack for transit flows (mitigation: docs + test with conntrack module loaded/unloaded).
• Multi-segment/GSO packets through mbuf shim (mitigation: linearize-on-entry for v1, document perf cost; bench script quantifies).