AF_DIVERT family restored & compiling: full socket family (PF_DIVERT=32, sock_register), blocking recv/poll, divert_output-compatible sendmsg reinject, mtag type passthrough; pfk_api decls
This commit is contained in:
1 parent
fe6f767561
commit
ac56534c3b
30 files changed
+2273
-62
No files matched your search
@@ -0,0 +1,36 @@
|
||||
# PF-on-Linux documentation (English)
|
||||
|
||||
Documentação em português: [`docs/pt-BR/README.md`](../pt-BR/README.md).
|
||||
|
||||
This is the user documentation set for the port of the **OpenBSD 8.0 PF
|
||||
packet filter** to **Linux 7.3.4** — the real PF engine (`pf.c`, `pf_norm.c`,
|
||||
`pf_table.c`, `pf_lb.c`, …) running as the out-of-tree module `pf.ko`, driven
|
||||
by the real OpenBSD `pfctl` over the `/dev/pf` ioctl ABI.
|
||||
|
||||
## Index
|
||||
|
||||
| Doc | What it covers |
|
||||
|---|---|
|
||||
| [guide.md](guide.md) | One-page architecture: components, data path, control plane, milestones |
|
||||
| [install.md](install.md) | Apply/build/install recipe (kernel, module, tools) |
|
||||
| [build.md](build.md) | Build pipeline in detail: fetch → vendorize → kernel → module → pfctl |
|
||||
| [vm-lab.md](vm-lab.md) | The libvirt validation lab: topology, recreate/run, what each gate proves |
|
||||
| [feature-matrix.md](feature-matrix.md) | Per-feature status: VERIFIED-in-VM / COMPILES-untested / PENDING-Mx |
|
||||
| [pf-guide.md](pf-guide.md) | Daily use: what works today in `/etc/pf.conf`, what to avoid until M3/M4 |
|
||||
| [patches.md](patches.md) | Linux core patch set (currently empty) + rules and template for future `000N-*.patch` |
|
||||
| [caveats.md](caveats.md) | Known divergences from OpenBSD behavior and current limitations |
|
||||
|
||||
## Sources of truth
|
||||
|
||||
The docs describe the code as it exists in this tree. When in doubt, the
|
||||
authoritative references are:
|
||||
|
||||
- `docs/superpowers/specs/2026-11-11-pf-linux-port-design.md` — design
|
||||
- `docs/superpowers/plans/2026-11-11-pf-linux-port.md` — milestone plan (tasks T1–T19)
|
||||
- `docs/porting-notes.md` — staging log + M1 deviation record (the truth about compat-layer divergences)
|
||||
|
||||
Feature-status vocabulary used across these docs:
|
||||
|
||||
- **VERIFIED-in-VM** — exercised end-to-end in the libvirt lab (see vm-lab.md).
|
||||
- **COMPILES-untested** — the code is built into `pf.ko` but no gate has proven the behavior yet.
|
||||
- **PENDING-Mx** — lands (or lands fully) in milestone Mx; current observable behavior is documented.
|
||||
@@ -0,0 +1,116 @@
|
||||
# Build pipeline in detail
|
||||
|
||||
Português: [docs/pt-BR/build.md](../pt-BR/build.md).
|
||||
Quick recipe: [install.md](install.md). This page explains *what each stage
|
||||
does* and *what has actually been exercised*.
|
||||
|
||||
Everything below runs on the build host (the laptop); the VM lab (separate
|
||||
host "deb") only receives artifacts — see [vm-lab.md](vm-lab.md).
|
||||
|
||||
## 0. Prerequisites
|
||||
|
||||
From `docs/en/install.md`: gcc ≥ 15, GNU make, bison (flex is not needed —
|
||||
OpenBSD `parse.y` carries a hand-written `yylex`), `bc`, rsync, `glibc-static`
|
||||
for the static pfctl, root for `insmod`. Kernel tree expected at
|
||||
`../linux-7.3.4` (override with `KERNELDIR=`; the root `Makefile` defaults to
|
||||
`$(HOME)/lab/linux_kernel/linux-7.3.4`).
|
||||
|
||||
## 1. Fetch + verify OpenBSD sources — `scripts/fetch-sources.sh`
|
||||
|
||||
- Downloads `sys.tar.gz` and `src.tar.gz` from the primary mirror
|
||||
`https://openbsd.c3sl.ufpr.br/pub/OpenBSD/8.0/`, falling back to
|
||||
`cdn.openbsd.org`.
|
||||
- Verifies sha256 against the pins in `VERSIONS` (sys: `964b1aed…`, src:
|
||||
`9984c74f…`); an already-cached tarball is skipped when its hash matches.
|
||||
- Landing zone: `_vendor/openbsd/` (gitignored).
|
||||
|
||||
## 2. Vendorize — `scripts/vendorize.sh`
|
||||
|
||||
Extracts **only the files the port consumes** into `_vendor/upstream/`
|
||||
(gitignored): the `sys/net/pf*.c|h`, `ifq/hfsc/fq_codel/ip_pflog/ip_divert`,
|
||||
`radix/rtable/toeplitz`, `crypto/{md5,sha2,siphash}`, `sys/sys/*.h` list and
|
||||
`sbin/pfctl/*` + headers (see the `SYS_K`/`SRC_K` arrays). It then records
|
||||
each file's `$OpenBSD: …$` tag into `_vendor/upstream/IDS.txt` — the forensic
|
||||
baseline for every ported line and for future rebases.
|
||||
|
||||
## 3. Staging into the port — `scripts/stage-port.sh` (deterministic porting)
|
||||
|
||||
`stage-port.sh` turns pristine upstream files into the files under `module/`:
|
||||
|
||||
1. every `#include` is replaced by `pfk.h` + the staged `pfvar` headers
|
||||
(kernel branches selected with `-D_KERNEL=1`);
|
||||
2. the shared rename program (`pf_sed`) resolves BSD↔Linux name collisions
|
||||
(`struct ip` → `struct pf_ip`, `ifnet` → `pfk_ifnet`, `ip_output` →
|
||||
`pfk_ip_output`, `sa_len` assignments stripped, …);
|
||||
3. **each run is appended to `docs/porting-notes.md`** — that file is the
|
||||
staging log, not prose.
|
||||
|
||||
Ported files are then committed and hand-edited in-tree; re-staging is only
|
||||
for forensics (`stage-port.sh --diff <file>` compares our copy against the
|
||||
pristine upstream text). Radix/rtable/siphash/toeplitz are deliberately not
|
||||
auto-staged (hand-ported).
|
||||
|
||||
## 4. Kernel — `scripts/build-kernel.sh` (+ `kernel-config.sh`)
|
||||
|
||||
- `kernel-config.sh`: starts from `defconfig` if there is no `.config`, then
|
||||
sets (idempotently, via `scripts/config` + `make olddefconfig`):
|
||||
`VIRTIO_BLK/PCI/NET=y`, `EXT4_FS=y`, `NETFILTER=y`, `NETFILTER_ADVANCED`,
|
||||
`NET_SCHED=y`, `INET=y`, `IPV6=y`, `SERIAL_8250(+CONSOLE)=y` — i.e.
|
||||
everything needed to direct-boot the pf0 VM with **no initrd**.
|
||||
`kernel-config.sh test` additionally enables `DEBUG_INFO` +
|
||||
`DEBUG_KMEMLEAK` for soak builds.
|
||||
- `build-kernel.sh`: `make -j(nproc-2) bzImage modules_prepare`, ensures
|
||||
`Module.symvers` (copies `vmlinux.symvers` — kbuild does not write the
|
||||
former) and `scripts/module.lds` exist, then drops boot artifacts in
|
||||
`out/`: `linux-<ver>-bzImage`, `System.map`, `.config`, `kernelversion`,
|
||||
`bzImage.sha256`. Prints `KERNEL_BUILD_OK <kernelrelease>`.
|
||||
- **Kernel core patches are currently none**: `patches/kernel/` holds no
|
||||
`000N-*.patch` (see [patches.md](patches.md)).
|
||||
|
||||
## 5. Module — `scripts/build-module.sh`
|
||||
|
||||
`make -C "$KERNELDIR" M=module` → `module/pf.ko` (single module today; the
|
||||
`pfq` qdisc arrives with M5). The script self-prepares `Module.symvers` /
|
||||
`modules_prepare` first. Object list (`module/Makefile`): ported engine
|
||||
(`pf.c`, `pf_ioctl.c`, `pf_norm.c`, `pf_if.c`, `pf_lb.c`, `pf_ruleset.c`,
|
||||
`pf_table.c`, `pf_osfp.c`, `pf_syncookies.c`, `divert_sock.c`) + compat layer
|
||||
(mbuf shim, kern glue, if/ip4/ip6/pcb/xmit/ifq, crypto, radix, subr_tree) +
|
||||
`pfk_mod.c`/`pfk_hooks.c`. Compile flags define `PFLINUX=1` and `_KERNEL=1`.
|
||||
Prints `MODULE_BUILD_OK`.
|
||||
|
||||
`insmod module/pf.ko` creates `/dev/pf` (0600) and registers the netfilter
|
||||
hooks; `dmesg` shows `pfk: …` lines (see [caveats.md](caveats.md) for what
|
||||
the data path does).
|
||||
|
||||
## 6. Userland — `make -C tools`
|
||||
|
||||
Targets (from `tools/Makefile`): `pfctl`, `abitest`, `abi`, `test`, `clean`.
|
||||
|
||||
- `pfctl` — the OpenBSD 8.0 sources compiled against the copied ABI headers;
|
||||
result is a **static binary** at `tools/out/pfctl`. Every deviation from
|
||||
upstream is listed in `tools/UPSTREAM.md` (7 marked edits; the rest is the
|
||||
compat header/source set).
|
||||
- `abi` — builds `abitest` (userland) and `abitest-module` (same table under
|
||||
kernel compat headers), dumps both struct-size/offset tables and `diff`s
|
||||
them; a mismatch means the ioctl ABI drifted. `out/abi-*.tbl` are the
|
||||
reference inputs.
|
||||
- `test` — offline parse checks: runs `pfctl -n -f` over `tools/tests/*.conf`
|
||||
(basic / translations / queues+scrub / anchor / negative `broken.conf`).
|
||||
This is pure userland; it needs no `/dev/pf`.
|
||||
- `pdivert` — divert relay source under `tools/pdivert/`; see
|
||||
[feature-matrix.md](feature-matrix.md) for its (pending) kernel side.
|
||||
|
||||
## 7. Root `Makefile`
|
||||
|
||||
`make module | tools | kernel | vm | vm-destroy | test | docs | clean` — thin
|
||||
wrappers over the scripts above. `make test` runs the VM suite
|
||||
([vm-lab.md](vm-lab.md)); `make docs` just lists the bilingual doc set.
|
||||
|
||||
## What has been exercised vs. not
|
||||
|
||||
- Stages 1–6 have each been run during M0–M2 development on the build host
|
||||
(artifacts exist in-tree: `out/`, `module/pf.ko`, `tools/out/pfctl`), and
|
||||
the scripts above were verified line-by-line for this document.
|
||||
- A **fresh-clone end-to-end run** (`quickstart-check.sh`, plan task T19) has
|
||||
*not* been executed yet; treat the exact command sequence in install.md as
|
||||
"as designed, last-run state per milestone" rather than CI-verified.
|
||||
@@ -0,0 +1,137 @@
|
||||
# Caveats — divergences from OpenBSD behavior
|
||||
|
||||
Português: [docs/pt-BR/caveats.md](../pt-BR/caveats.md).
|
||||
|
||||
These are the ways this port differs from running the same ruleset on
|
||||
OpenBSD 8.0, or is not yet production-shaped. Everything here is grounded in
|
||||
the code (`module/pfsrc/pfk_hooks.c`, `module/pfsrc/pfk_mod.c`,
|
||||
`docs/porting-notes.md`) and in lab observations; nothing is speculative.
|
||||
|
||||
## Data path
|
||||
|
||||
### 1. PF runs in process context via a workqueue (netisr model), not in the packet fast path
|
||||
|
||||
OpenBSD evaluates `pf_test()` on the CPU that received the packet, at
|
||||
`IPL_NET`, with a non-sleeping NET read lock. Linux gives us softirq context
|
||||
and *sleepable* compat locks (rw-semaphores), so the hook never calls
|
||||
`pf_test()` inline: it steals the skb (`NF_STOLEN`) onto an **ordered
|
||||
workqueue** (`pfk_netisr`, `WQ_MEM_RECLAIM`) and runs the engine in process
|
||||
context, then re-injects (`netif_rx()` for input, `dst->output()` for
|
||||
output/transit).
|
||||
|
||||
Consequences, all observable:
|
||||
|
||||
- **Verdicts are asynchronous.** Anything chained after our hook on the same
|
||||
netfilter chain (other iptables/nft rules at higher priorities) does not
|
||||
see the packet until the reinjected pass comes through. Counters and log
|
||||
interleaving differ accordingly.
|
||||
- **Latency + throughput**: every packet takes a work-queue round trip
|
||||
serialized on one ordered worker. Functional gates are the current bar;
|
||||
the planned `bench-basic.sh` is informational only.
|
||||
- **Reordering**: the ordered queue preserves arrival order of *queued*
|
||||
work items, but the stack around it (RPS, multi-NAPI) can still hand us
|
||||
packets whose original completion order differs from reinjection order.
|
||||
|
||||
### 2. Re-entry protection is a per-direction tag in `skb->mark`
|
||||
|
||||
The reinjected survivor carries `skb->mark |= PFK_DONE_IN (0x50460010)` or
|
||||
`PFK_DONE_OUT (0x50460020)`; our hooks accept any packet already tagged for
|
||||
that direction instead of re-queuing it (otherwise PF_IN → netif_rx →
|
||||
PF_IN… would loop). `skb->mark` is a **shared, writable 32-bit field**: any
|
||||
subsystem that *overwrites* it — nft `meta mark set`, `CONNMARK --restore`,
|
||||
tc filters, some VPN stacks — can either (a) clear our tag and cause the
|
||||
packet to be **re-queued through PF again**, or (b) coincidentally set those
|
||||
bits and let a packet **skip** a PF pass. Practical rule on a PF-on-Linux
|
||||
host: do not use fwmark mangling on the same traffic; if you must, mask the
|
||||
PF tag bits out of your mark space.
|
||||
|
||||
### 3. No NET-lock on the data path (M2 interim)
|
||||
|
||||
`pf_test()` currently runs **without** the compat NET read lock (comment in
|
||||
`pfk_hooks.c`): with the workqueue model, a parked control writer (ruleset
|
||||
change, purge ioctl) starved the packet side badly enough to be
|
||||
counter-productive. Rule-set mutation and packet walk therefore run
|
||||
concurrently, protected by what upstream `pf.c` itself uses (PF_LOCK/RCU-ish
|
||||
patterns) — a deliberate interim deviation being reworked with the lock
|
||||
model in M5+. Until then: expect correctness under steady state, not under
|
||||
aggressive concurrent reload; the reload-under-traffic soak is an open gate.
|
||||
|
||||
### 4. Fail-open under memory pressure
|
||||
|
||||
The queue item is `kmalloc(GFP_ATOMIC)` in softirq. If it fails, the hook
|
||||
returns `NF_ACCEPT` — the packet **bypasses PF**. This is a deliberate
|
||||
availability choice (mirrors dropping into a full queue bypassing the qdisc)
|
||||
but it is a security trade-off worth knowing: PF verdicts are best-effort
|
||||
exactly when the host is out of atomic memory.
|
||||
|
||||
### 5. Hook ordering vs conntrack differs by direction
|
||||
|
||||
- Ingress (`PREROUTING`, `NF_IP_PRI_FIRST`): PF runs **before** defrag and
|
||||
conntrack — as designed (spec §4.2).
|
||||
- Output (`LOCAL_OUT` at `NF_IP_PRI_CONNTRACK+1`): PF sees locally generated
|
||||
traffic **after** conntrack has already committed the entry.
|
||||
- Transit egress (`POSTROUTING` at `NF_IP_PRI_LAST-1`): PF runs at the very
|
||||
end — after any iptables/nft SNAT/masquerade — so `pass out … nat-to`-era
|
||||
rules (M3) will be validated against this ordering. The "unregister-safe
|
||||
ordering vs conntrack" warning from the design still applies: overlapping
|
||||
policy between PF and conntrack is possible and yours to manage.
|
||||
|
||||
## Control plane / lifecycle
|
||||
|
||||
### 6. Unload is hardened; the soak is still pending
|
||||
|
||||
`rmmod pf` deregisters `/dev/pf`, zeroes `pf_status.running` under the NET
|
||||
lock, deletes **and syncs** both purge timers (they self-rearm — a real rmmod
|
||||
hang was observed and fixed by stopping sources before flushing the
|
||||
workqueue), unregisters hooks and flushes `pfk_netisr`. That fix is in-tree,
|
||||
but the full cycle under traffic + live states (T18/soak gate) has **not**
|
||||
been run to green yet. For now treat unload as admin-safe, automation-
|
||||
unproven.
|
||||
|
||||
## Behavioral / rendering issues open today
|
||||
|
||||
### 7. `pfctl -s states` visibility anomaly (M2 gate work)
|
||||
|
||||
In the most recent captured run (gate 01): transit ping **passed** under
|
||||
`keep state`, but no icmp state was listed afterwards. Fixing this is the
|
||||
open item of the M2 gate work; until a green run lands, the
|
||||
[state-listing row](feature-matrix.md) is the honest status. Not cosmetic if
|
||||
it turns out states are not being inserted — treat both possibilities until
|
||||
the gate says otherwise.
|
||||
|
||||
### 8. Silent-but-warned stubs
|
||||
|
||||
`divert` verdicts drop with a rate-limited warning; `log` rules log nothing
|
||||
with a one-time warning. Both are listed so nobody mistakes "rules loaded"
|
||||
for "features active" (see [pf-guide.md](pf-guide.md)).
|
||||
|
||||
## Compat-layer deviations (from `docs/porting-notes.md`, M1 record)
|
||||
|
||||
These are deliberate, marked-in-code divergences of the compat layer; they
|
||||
matter when reasoning about rule semantics:
|
||||
|
||||
- **Socket-owner matching** (`user`/`pid` on rules): uid only — Linux `struct
|
||||
sock` carries no egid/cpid, so gid degrades to `(gid_t)-1`, pid to
|
||||
`NO_PID`.
|
||||
- **Checksums**: driver offload is not trusted on ingress (M1 decision) —
|
||||
TCP checksums are recomputed in software; output pseudo-checksums are done
|
||||
in the port's own path.
|
||||
- **IP options / record-route / timestamps**: not processed on the PF output
|
||||
path; **ICMP redirects are not generated** by PF's route code (the Linux
|
||||
FIB emits those).
|
||||
- **Routing labels**: non-goal — the ioctl layer rejects labeled rules;
|
||||
`rtable_iterate()` always returns NULL (no multipath uRPF iteration).
|
||||
- **Interface groups**: Linux has none; `group …` matches behave like a
|
||||
missing group on OpenBSD.
|
||||
- **GSO/multi-segment skbs**: handled by linearization on entry to the mbuf
|
||||
shim (perf cost documented in spec §10; gates will quantify).
|
||||
- **Single instance**: one PF world in `init_net`, like OpenBSD has one
|
||||
kernel. Containers/namespaces are not visible to PF and cannot be filtered
|
||||
with it in this design.
|
||||
|
||||
## What is *not* a caveat
|
||||
|
||||
ARP and other non-IPv4/IPv6 EtherTypes never enter the netfilter IPv4/IPv6
|
||||
hooks — they pass through the box untouched, exactly as PF on OpenBSD ignores
|
||||
non-IP. `/dev/pf` is 0600 root-only by design (matches OpenBSD's
|
||||
`securelevel`-era default for `pfctl` write access being root).
|
||||
@@ -0,0 +1,78 @@
|
||||
# Feature matrix
|
||||
|
||||
Português: [docs/pt-BR/feature-matrix.md](../pt-BR/feature-matrix.md).
|
||||
|
||||
Status vocabulary (used identically in both languages):
|
||||
|
||||
- **VERIFIED-in-VM** — passed end-to-end in the libvirt lab (see [vm-lab.md](vm-lab.md)).
|
||||
- **M2-in-progress** — implemented; VM gates 01–04 exist but the suite is **not all green yet**; no VERIFIED claim until a clean full run.
|
||||
- **COMPILES-untested** — built into `pf.ko`/`pfctl`; no gate has exercised it.
|
||||
- **PENDING-Mx** — lands in milestone Mx; today's observable behavior is stated.
|
||||
- **OUT-OF-SCOPE** — non-goal by design (spec §3).
|
||||
|
||||
As of 2026-11-11; source of per-row evidence: `out/test-logs/`,
|
||||
`docs/porting-notes.md`, milestone commits.
|
||||
|
||||
## Control plane & tools
|
||||
|
||||
| Feature | Upstream syntax | Milestone | Status |
|
||||
|---|---|---|---|
|
||||
| Module load, `/dev/pf` (0600) | `insmod pf.ko` | M1 | VERIFIED-in-VM |
|
||||
| Enable/disable | `pfctl -e`, `pfctl -d` | M1 | VERIFIED-in-VM |
|
||||
| Ruleset load | `pfctl -f file` | M2 | VERIFIED-in-VM (loads succeeded in every captured gate run; run status itself is mixed — see [vm-lab.md](vm-lab.md) §6) |
|
||||
| Parse-only check | `pfctl -n -f file` | M1 | VERIFIED (offline fixture suite, `make -C tools test`) |
|
||||
| Status/counters | `pfctl -s info [-v]` | M1 | VERIFIED-in-VM |
|
||||
| Rules listing | `pfctl -s rules [-v]` | M2 | COMPILES-untested |
|
||||
| States listing | `pfctl -s states [-v]` | M2 | M2-in-progress — gate 01 shows a listing anomaly (transit ping OK, no icmp state displayed) while state machinery is being fixed |
|
||||
| Tables listing/ops | `pfctl -s tables`, `-t t -T show/add/del/kill` | M2 | M2-in-progress (gate 03) |
|
||||
| Anchors | `pfctl -a anchor -f file`, `-s Anchors` | M2 | M2-in-progress (gate 04) |
|
||||
| Queue listing | `pfctl -s queue -v`, `-s ifq -v` | M5 | PENDING-M5 (parses fine; counters not wired to real back ends yet) |
|
||||
| ioctl ABI parity | `make -C tools abi` (abitest tables) | M1 | VERIFIED build-side (userland table); kernel-side diff is an open integrator step (`tools/UPSTREAM.md`) |
|
||||
|
||||
## Filtering core
|
||||
|
||||
| Feature | Upstream syntax | Milestone | Status |
|
||||
|---|---|---|---|
|
||||
| `pass`/`match`, quick, on-interface/proto/addr/port | `pass in quick on wan proto tcp from 10/8 to port 80` | M2 | M2-in-progress (data path runs the real `pf_test`; latest partial run: transit ping OK, icmp block OK) |
|
||||
| `block drop` | `block drop in quick …` | M2 | M2-in-progress (gate 02a) |
|
||||
| `block return` / `return-rst` / `return-rst after quarter` | `block return-rst in quick proto tcp …` | M2 | M2-in-progress (gate 02b) |
|
||||
| `keep state` (icmp/tcp/udp), state table, purger | `… keep state` | M2 | M2-in-progress |
|
||||
| `set state-policy`, `allow-opts`, etc. | `set …` options | M2 | COMPILES-untested |
|
||||
| Syncookies (state overflow path) | `set syncookies always` | M2 | COMPILES-untested |
|
||||
| pfr tables (radix/hash) | `table <blk> persist { 1.2.3.4 }`, `-T add …` | M2 | M2-in-progress (gate 03) |
|
||||
| Anchors + include + quick precedence | `anchor "x"`, `load anchor …` | M2 | M2-in-progress (gate 04) |
|
||||
| IPv6 for all of the above | same syntax, v6 | M2+ | COMPILES-untested (engine is v6-complete by construction; v6 smoke gates planned) |
|
||||
| Non-IPv4/IPv6 traffic (ARP…) | — | M2 | PFLINUX by construction: netfilter IPv4/IPv6 hooks only — ARP never enters PF path (no gate yet) |
|
||||
|
||||
## NAT / normalization (M3)
|
||||
|
||||
| Feature | Upstream syntax | Milestone | Status |
|
||||
|---|---|---|---|
|
||||
| `nat-to` (port pools, sticky-port, static-port) | `pass out … nat-to (eth1:0)` | M3 | PENDING-M3 (`pf.c`/`pf_lb.c` compiled; wiring + VM gate in M3) |
|
||||
| `rdr-to` | `rdr pass on wan proto tcp from any to any port 8080 -> srv port 80` | M3 | PENDING-M3 |
|
||||
| `binat-to` | `binat-to …` | M3 | PENDING-M3 |
|
||||
| Scrub/normalize: reassembly, `fragment drop-overlap`, `min-ttl`, `max-mss`, `df`/`no-df`, `random-id`, `set reassemble yes` | `match on all scrub (max-mss 1380 …)` | M3 | COMPILES-untested (`pf_norm.c` compiled into pf.ko and reached from `pf_test`; scapy gate 08 pending) |
|
||||
| OS fingerprinting in states | `pass … set os A` + `/etc/pf.os` | bonus | COMPILES-untested (`pf_osfp.c` compiled; no fingerprint packaging yet — missing `/etc/pf.os` is tolerated) |
|
||||
|
||||
## Routing / divert / queues / logging
|
||||
|
||||
| Feature | Upstream syntax | Milestone | Status |
|
||||
|---|---|---|---|
|
||||
| `route-to` pools (`rr`, `round-robin`, `source-hash`, `hash`), `reply-to`, `dup-to`, `fail-to` | `pass out route-to (GW 10.0.3.2)` | M4 | PENDING-M4 (Linux route/neigh glue `pf_route` not implemented yet) |
|
||||
| `divert-to` / `divert` tag | `pass in … divert-to 10.0.99.9 10000`, `divert tag X` | M4 e2e; socket family per code comments M6 | PENDING — today's observable behavior: PF's `PF_DIVERT` verdict calls `divert_packet()`, which **drops the packet and logs a rate-limited `pr_warn`**. The userspace relay `pdivert` exists and expects an in-module `AF_DIVERT` family (`socket(PF_DIVERT=32, SOCK_RAW, IPPROTO_DIVERT=254)`), which is not instantiated yet. **Do not ship rules using divert on a live host.** |
|
||||
| Queues, default scheduler (priq) | `queue root on wan`, `pfctl -s queue -v` | M5 | COMPILES-untested — `compat/pfk_ifq.c` implements the ifq API with **priq fallback semantics**; `queue` rules parse and load, but packets are not actually shunted through a classful qdisc until M5 |
|
||||
| HFSC | `queue on wan { hfsc … }` | M5 | COMPILES-untested — **priq fallback until M5**; `pfsrc/hfsc.c` staged (real OpenBSD 8.0 code) but **not in the module object list** |
|
||||
| FQ-CoDel | `queue on wan { fq_codel … }` | M5 | COMPILES-untested — same as HFSC: staged, not built |
|
||||
| `pfq` qdisc (`tc qdisc … root pfq`) | n/a (Linux glue) | M5 | PENDING-M5 (does not exist in the tree yet) |
|
||||
| pflog (`log`, `log all`, `block log`, `pfctl -s log`) + `pflogd` pcap (DLT_PFLOG) | `pass out quick log` | M4 | PENDING-M4 — `pflog_packet()` returns success **without logging** (one-time `pr_warn "packet not logged"`); `if_pflog.c` staged, not built; `pflogd` not ported yet |
|
||||
|
||||
## Non-goals (explicit)
|
||||
|
||||
| Item | Reason |
|
||||
|---|---|
|
||||
| `pfsync` / CARP failover | non-goal (spec §3) |
|
||||
| IPsec / pfkey | non-goal |
|
||||
| `ftp-proxy` and userland daemons | non-goal |
|
||||
| Per-netns PF instances | faithful port = single global instance on `init_net` |
|
||||
| Classic-ALTQ-only schedulers (cbq/wfq/red…) | removed upstream before OpenBSD 8.0 — cannot be ported from the base |
|
||||
| Routing labels (`route-label`) | non-goal; ioctl layer rejects labeled rules |
|
||||
@@ -0,0 +1,101 @@
|
||||
# Architecture guide — one page
|
||||
|
||||
Português: [docs/pt-BR/guide.md](../pt-BR/guide.md).
|
||||
|
||||
## What this port is
|
||||
|
||||
A faithful port of the **OpenBSD 8.0 PF engine** running inside **Linux
|
||||
7.3.4** as one out-of-tree module, `pf.ko`. Nothing is faked: the matching,
|
||||
state, table, normalization and queueing code *is* OpenBSD's, compiled under a
|
||||
compatibility layer; control comes from the real `pfctl` speaking the real
|
||||
OpenBSD ioctl ABI through `/dev/pf`. There is **one global PF instance bound
|
||||
to `init_net`** — exactly like OpenBSD has one kernel; per-netns instances are
|
||||
out of scope.
|
||||
|
||||
Build targets: OpenBSD 8.0 sources (sha256-pinned, `VERSIONS`), Linux 7.3.4
|
||||
(`linux-7.3.4`), `gcc >= 15`, bison. `pfvar.h` v1.548 is shared bit-for-bit
|
||||
between kernel and userland (ioctl encoding per `module/include/pfk.h`).
|
||||
|
||||
## Components
|
||||
|
||||
| Path | Role |
|
||||
|---|---|
|
||||
| `module/pfsrc/` | Ported OpenBSD files: `pf.c` (engine), `pf_ioctl.c` (DIOC* ABI), `pf_norm.c` (scrub/fragments), `pf_table.c`, `pf_ruleset.c`, `pf_if.c`, `pf_lb.c` (pools/NAT ports), `pf_osfp.c`, `pf_syncookies.c`, `ifq.c`/`hfsc.c`/`fq_codel.c` (staged for M5), `if_pflog.c` + `ip_divert.c` (staged references), `divert_sock.c` (divert entry stub), `pfk_mod.c` (load/exit, `/dev/pf`), `pfk_hooks.c` (netfilter data path) |
|
||||
| `module/compat/` | Linux plumbing under the BSD-shaped sources: mbuf-over-`sk_buff` shim (`pfk_mbuf`), kernel glue (malloc/pools/timeouts/locks, `pfk_kern`), interface+address views (`pfk_if`), BSD-semantics IP output/forward (`pfk_ip4`, `pfk_ip6`), socket lookups (`pfk_pcb`), ifq API with priq semantics (`pfk_ifq`), radix/route-table/crypto ports |
|
||||
| `module/include/` | `pfk.h` (master compat header + OpenBSD-style `_IO*` encoding), staged `net/pfvar.h`, `pfk_mbuf.h`, `pfk_net.h`, `pfk_if.h` |
|
||||
| `tools/` | Linux build of OpenBSD 8.0 `pfctl` (single binary), `pdivert` relay, ABI parity harness (`abitest` + `make abi`), offline parse tests (`make test`). Deltas vs upstream: `tools/UPSTREAM.md` |
|
||||
| `scripts/` | `fetch-sources.sh`, `vendorize.sh`, `stage-port.sh` (deterministic staging), `build-kernel.sh`, `build-module.sh`, `vm-*.sh` lab automation |
|
||||
| `tests/vm/` | VM suite gates `01-state.sh … 04-anchors.sh` + `lib.sh`; `tests/confs/` rulesets deployed to the lab |
|
||||
| `patches/kernel/` | Core patch set — currently **empty** (`p1-needs.txt` records why no exports were needed); see [patches.md](patches.md) |
|
||||
|
||||
## Data path (M2, as implemented today)
|
||||
|
||||
OpenBSD runs PF at `IPL_NET` in process-ish context; our compat locks are
|
||||
sleepable, so the hook **never calls `pf_test()` in softirq**. Packets are
|
||||
stolen onto an ordered workqueue (`pfk_netisr`) and the verdict is applied by
|
||||
re-injection, tagged per direction via `skb->mark` so the survivor is not
|
||||
re-queued by our own hook.
|
||||
|
||||
```
|
||||
Linux 7.3.4, module pf.ko
|
||||
NIC ──► netfilter (softirq — enqueue only)
|
||||
PRE_ROUTING PF_IN ─┐ ┌─ LOCAL_OUT PF_OUT
|
||||
POST_ROUTING PF_OUT ─┤ NF_STOLEN ├─ (runs AFTER conntrack)
|
||||
LOCAL_IN/FORWARD: pass-through ┘ POST_ROUTING: NF_IP_PRI_LAST-1
|
||||
│ (skb ref + kmalloc pfk_pkt;
|
||||
│ alloc failure ⇒ NF_ACCEPT = fail-open)
|
||||
▼
|
||||
pfk_netisr workqueue (process context)
|
||||
│
|
||||
pfk_m_wrap_skb(skb)──► struct mbuf shim
|
||||
│
|
||||
pf_test(af, dir, ifp, &m) ◄── the real OpenBSD 8.0 engine:
|
||||
│ scrub/normalize → state lookup
|
||||
│ → rules/anchors/tables/quick
|
||||
│ → state insert → verdict
|
||||
▼
|
||||
pass ──► mark |= PFK_DONE_IN/OUT ──► reinject: PF_IN: netif_rx() (survivor
|
||||
of reassembly allowed through)
|
||||
PF_OUT: skb_dst()->output()
|
||||
block/drop ──► kfree_skb() (+ PF's own return-rst/ICMP generation)
|
||||
fragment-buffered ──► our reference dropped; pf keeps the survivor (m == NULL)
|
||||
```
|
||||
|
||||
Control plane: `/dev/pf` (miscdev, mode 0600, write gated on `CAP_NET_ADMIN`)
|
||||
→ `pfk_ioctl_io()` copies the parameter struct sized from the OpenBSD `_IOR/
|
||||
_IOW` encoding, dispatches into the ported `pfioctl()` (`pf_ioctl.c`) — the
|
||||
full DIOC* ABI as upstream. `pfctl -e/-f/-s/-t` are that same OpenBSD binary,
|
||||
rebuilt.
|
||||
|
||||
## Queues, divert, pflog — today's observable behavior
|
||||
|
||||
- **Queues**: `pfctl` parses `queue`/`queue on` (8.0 syntax); kernel-side the
|
||||
ifq API is implemented by `compat/pfk_ifq.c` with **priq semantics only**;
|
||||
the `pfq` classful qdisc and the HFSC/FQ-CoDel back ends land in **M5**.
|
||||
`ifq.c`/`hfsc.c`/`fq_codel.c` are staged and compile-ready but are not in
|
||||
the module object list yet.
|
||||
- **Divert**: `divert-to`/`divert` tag evaluate to PF's `PF_DIVERT` verdict;
|
||||
`divert_packet()` (`pfsrc/divert_sock.c`) currently **drops the packet with
|
||||
a rate-limited kernel log** because the AF_DIVERT socket family is not yet
|
||||
instantiated. The `pdivert` relay (userspace) exists and expects
|
||||
`socket(PF_DIVERT=32, SOCK_RAW, IPPROTO_DIVERT=254)`.
|
||||
- **pflog**: `pflog_packet()` returns success but **logs nothing** (one-time
|
||||
`pr_warn "packet not logged"`); `if_pflog.c` is staged, netlink back end is
|
||||
M4/M5.
|
||||
|
||||
## Milestones (see feature-matrix.md for per-feature status)
|
||||
|
||||
| Milestone | Scope | Status (2026-11-11) |
|
||||
|---|---|---|
|
||||
| M0 | repo, scripts, kernel build, VM boots custom kernel | done |
|
||||
| M1 | compat layer, `pf.ko` loads, `/dev/pf`, `pfctl -f`/`-s info` | done |
|
||||
| M2 | filter core: rules/states/tables/anchors VM gates | **in progress** — data path implemented, gates 01–04 not all green yet |
|
||||
| M3 | NAT (nat/rdr/binat), scrub e2e, pflog | pending |
|
||||
| M4 | route-to/dup-to/fail-to, divert e2e | pending |
|
||||
| M5 | queues: ifq back ends (hfsc/fq_codel) + `pfq` qdisc | pending |
|
||||
| M6 | docs complete, patchset tidy, full suite green, tagged releases | in progress (this doc set) |
|
||||
|
||||
## Next reads
|
||||
|
||||
[build.md](build.md) → [vm-lab.md](vm-lab.md) → [pf-guide.md](pf-guide.md) →
|
||||
[feature-matrix.md](feature-matrix.md) → [caveats.md](caveats.md).
|
||||
@@ -74,3 +74,29 @@ Because the deliverable is an out-of-tree module, "applying the patch set"
|
||||
(virtio/EXT4 =y) — no source files are modified unless a future milestone
|
||||
adds a documented `patches/kernel/000N-*.patch` (all such patches: one
|
||||
section each in docs/en/patches.md with what/why/impact/revert).
|
||||
|
||||
## 6. Notes on the steps above (2026-11-11)
|
||||
|
||||
- Step **2.3 is a documented no-op today**: the patch set is empty and
|
||||
`scripts/apply-patches.sh` / `scripts/unpatch.sh` / `patches/kernel/series`
|
||||
are not yet in the tree (they land with the first real core patch — see
|
||||
[patches.md](patches.md)). You can safely skip 2.3.
|
||||
- Step 2.4 builds `pfctl` and the ABI harness only (`make -C tools` targets:
|
||||
`pfctl abitest abi test clean`). `pdivert` is present as a source file with
|
||||
no Makefile target yet; `pflogd` is not ported.
|
||||
- After 2.5, what you can actually *use* is the M2 filter core (pass/block/
|
||||
quick/keep state/tables/anchors); NAT/scrub/queues/divert/pflog are gated
|
||||
by later milestones — see the [feature matrix](feature-matrix.md) and read
|
||||
[pf-guide.md](pf-guide.md) before enabling PF on a path you need.
|
||||
|
||||
## 7. Where to go next
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Architecture, one page | [guide.md](guide.md) |
|
||||
| Build stages in detail | [build.md](build.md) |
|
||||
| VM validation lab | [vm-lab.md](vm-lab.md) |
|
||||
| Daily rulesets & warnings | [pf-guide.md](pf-guide.md) |
|
||||
| Known divergences | [caveats.md](caveats.md) |
|
||||
| Core patch mechanism | [patches.md](patches.md) |
|
||||
| Doc index | [README.md](README.md) |
|
||||
@@ -0,0 +1,73 @@
|
||||
# Linux core patch set
|
||||
|
||||
Português: [docs/pt-BR/patches.md](../pt-BR/patches.md).
|
||||
|
||||
## Current status: EMPTY
|
||||
|
||||
The port requires **zero modifications to the Linux 7.3.4 core**. The design
|
||||
(spec §5) anticipated one mandatory patch — **P1: export symbols needed by
|
||||
the PF module** — but the M1 compile-port proved it unnecessary: modpost
|
||||
reported **no undefined symbols** for `pf.ko`. The compat layer was written
|
||||
against APIs the stock kernel already exports (`__inet_lookup_established`,
|
||||
`__inet_lookup_listener`, `__inet6_lookup_established`,
|
||||
`inet6_lookup_listener`, `__udp4_lib_lookup`, `__udp6_lib_lookup`,
|
||||
`ip_route_output_flow`/`ip6_route_output_flags` via inline wrappers,
|
||||
`icmp_send`, …). The evidence and the exact list live in
|
||||
`patches/kernel/p1-needs.txt`, which also carries a **watch list**: e.g.
|
||||
`tcp_v4_send_reset` would need an export only if userland RST helpers are
|
||||
used.
|
||||
|
||||
`patches/kernel/` therefore contains no `000N-*.patch` today, and applying
|
||||
the (empty) set is a no-op.
|
||||
|
||||
> Honest note: install.md and the design mention `scripts/apply-patches.sh`,
|
||||
> `scripts/unpatch.sh` and `patches/kernel/series`. Those helpers are **not
|
||||
> in the tree yet**; until they land, "apply the patch set" is by definition
|
||||
> nothing to do. When a real patch is needed, the first commit that adds it
|
||||
> MUST also add the two scripts and the `series` file (see rules below).
|
||||
|
||||
## The mechanism (how the set is meant to work)
|
||||
|
||||
- Every kernel-core change is a standalone file
|
||||
`patches/kernel/000N-short-description.patch`, listed in order in
|
||||
`patches/kernel/series`, applied to the pristine kernel tree with
|
||||
`git apply`-style tooling and reverted the same way
|
||||
(`unpatch.sh` reverse-applies).
|
||||
- The tree must never be left dirty silently: builds that must be
|
||||
reproducible apply, build, and record the state (`out/.config`,
|
||||
`out/kernelversion`).
|
||||
- Hard rules for any future patch:
|
||||
1. **Additive, zero behavior change** (e.g. `EXPORT_SYMBOL_GPL()` lines
|
||||
adjacent to existing export clusters). Anything that alters logic needs
|
||||
a design-spec amendment first.
|
||||
2. One concern per patch, numbered sequentially — never edit a landed
|
||||
patch; supersede with a new number.
|
||||
3. A section in this file (both languages) **before** the patch lands:
|
||||
what/why/impact/revert + the VM evidence.
|
||||
4. The patch must survive `make -C <kerneldir> bzImage modules` from a
|
||||
clean tree (CI-of-the-lab: `vm-run-tests.sh` after a fresh boot).
|
||||
|
||||
## Template for a future `000N` entry
|
||||
|
||||
Copy this block into both `docs/en/patches.md` and `docs/pt-BR/patches.md`
|
||||
(EN above the pt-BR file's same anchor) when adding `000N-*.patch`:
|
||||
|
||||
```
|
||||
### 000N — <one-line title>
|
||||
|
||||
- File: `patches/kernel/000N-<slug>.patch`
|
||||
- Status: landed (kernel 7.3.4) / superseded by 000M
|
||||
- Touches: <file(s) in the kernel tree>, <n> added lines, 0 removed
|
||||
- What: <exact symbols / hooks / Kconfig knobs added>
|
||||
- Why: <which PF-on-Linux code needs it; why no exported API suffices>
|
||||
- Impact: <runtime/ABI/memory impact on plain Linux; expected: none>
|
||||
- Revert: `./scripts/unpatch.sh <kerneldir>` (reverse-apply); module
|
||||
then fails to link with the missing symbol named in modpost output.
|
||||
- Evidence: <VM run id / date / relevant gate output>
|
||||
```
|
||||
|
||||
## Change log of this directory
|
||||
|
||||
| Date | Change |
|
||||
|---|---|
|
||||
| 2026-11-11 | P1 evaluated after M1 compile-port → **not needed**; `p1-needs.txt` records the evidence and the watch list. Set remains empty. |
|
||||
@@ -0,0 +1,93 @@
|
||||
# Using PF-on-Linux day to day
|
||||
|
||||
Português: [docs/pt-BR/pf-guide.md](../pt-BR/pf-guide.md).
|
||||
|
||||
This is the "what can I actually run today" guide. It follows the
|
||||
[feature matrix](feature-matrix.md): **M2 scope (filtering core: pass/block/
|
||||
quick/keep state/tables/anchors + enable/disable)** has VM evidence behind it;
|
||||
everything beyond that is either COMPILES-untested or PENDING. Install/build
|
||||
is covered by [install.md](install.md) and [build.md](build.md).
|
||||
|
||||
## Daily loop
|
||||
|
||||
```sh
|
||||
sudo insmod module/pf.ko # creates /dev/pf
|
||||
sudo pfctl -nf /etc/pf.conf # parse check (userland only — always safe)
|
||||
sudo pfctl -f /etc/pf.conf # load ruleset
|
||||
sudo pfctl -e # ENABLE — traffic starts hitting the ruleset NOW
|
||||
sudo pfctl -s info # status + counters
|
||||
sudo pfctl -s states # states listing (see caveat below)
|
||||
sudo pfctl -t blk -T add 10.0.1.10 # live table update
|
||||
sudo pfctl -a admin -f /etc/pf.d/admin.conf
|
||||
sudo pfctl -d # disable
|
||||
sudo rmmod pf # unload (rule/state flush happens here)
|
||||
```
|
||||
|
||||
> **Warning — order matters**: `-f` then `-e`. Enabling with no matching `pass` rule
|
||||
> for your admin path blocks it — this port follows OpenBSD's default-block
|
||||
> policy, and the M2 gate runs have *actually cut the test VM's ssh* that
|
||||
> way (the lab keeps a serial console for exactly this). On a remote host,
|
||||
> pass your management interface before you enable anything.
|
||||
|
||||
## A ruleset that works in this port today
|
||||
|
||||
```pf
|
||||
# /etc/pf.conf — M2 scope only. Interface names here match the VM lab
|
||||
# (mgmt/lan/wan); substitute your own (eth0, enp1s0, …).
|
||||
|
||||
# 1) keep yourself reachable (do this first, always)
|
||||
pass in quick on mgmt
|
||||
pass out quick
|
||||
|
||||
# 2) filter core — proven today: quick, proto/port matching, state, drop
|
||||
table <blk> persist # live blocklist (gate 03 pattern)
|
||||
block in quick proto tcp from <blk> to any
|
||||
|
||||
pass in quick on lan proto tcp from 10.0.1.0/24 to any port { 80 443 } keep state
|
||||
block drop in quick on wan proto icmp from any to any
|
||||
|
||||
# 3) anchors with precedence (gate 04 pattern)
|
||||
anchor "admin"
|
||||
```
|
||||
|
||||
Macros (`xif = "lan"`), nested `anchor "in/*"`, `label`, `tag`, `set
|
||||
timeout/limit/state-policy/block-policy/skip` and the rest of the OpenBSD 8.0
|
||||
grammar **parse** fine — `make -C tools test` exercises them offline — but
|
||||
the *kernel behavior* of most `set` options has not been gate-verified yet;
|
||||
treat them as load-and-pray until the matrix moves them.
|
||||
|
||||
## What to avoid until the milestones land
|
||||
|
||||
| Syntax | Until | What happens today |
|
||||
|---|---|---|
|
||||
| `nat-to`, `rdr-to`, `binat-to` | M3 | Rules load (engine code compiled) but translation is unwired/ungated — **do not rely on NAT**; outbound masquerade still comes from iptables/nft during the lab. |
|
||||
| `scrub (…)`, `set reassemble` | M3 | `pf_norm.c` runs inside `pf_test` (compiled) — crafted-fragment behavior is untested; treat results as unvalidated. |
|
||||
| `divert-to`, `divert` tag | M4 (+ family per code comments) | Matching packets are **dropped** with a rate-limited kernel warning. Never ship this on a live host. |
|
||||
| `route-to`, `dup-to`, `fail-to`, `reply-to` | M4 | Not implemented as Linux routing glue yet; rules may load but the behavior is undefined — avoid. |
|
||||
| `queue` / `queue on` / `pfctl -s queue` | M5 | Parse and load. Queuing falls back to the priq-shaped compat `ifq` layer; **no bandwidth enforcement happens until the `pfq` qdisc lands**. |
|
||||
| `log`, `log all`, `block log`, `pflogd` | M4 | Nothing is logged (stub warns once at load: `pfk: … packet not logged`). |
|
||||
| `route-label …` | never (non-goal) | Rejected at the ioctl layer. |
|
||||
| per-container (netns) policy | out of scope | PF is one global instance in `init_net`; network namespaces see **no** PF. |
|
||||
|
||||
## Reading state correctly
|
||||
|
||||
`pfctl -s states` currently has a listing/visibility anomaly being fixed in
|
||||
the M2 gate work (transit flows can pass while no state is displayed). Until
|
||||
then, use counters (`pfctl -s info`) and external observation (ping/nc
|
||||
results, `tcpdump`) as the primary evidence, as the gates do.
|
||||
|
||||
## Kernel log signals you should know
|
||||
|
||||
```
|
||||
pfk: PF-on-Linux engine loaded (OpenBSD 8.0 base) ← insmod ok
|
||||
pfk: netfilter hooks registered (10, netisr wq data path)
|
||||
pfk: divert v4 port N: no divert socket layer (M6 pending), dropping
|
||||
pfk: … packet not logged ← pflog stub
|
||||
```
|
||||
|
||||
Unloading stops the purge timers and drains the work queue first (an rmmod
|
||||
hang observed in the lab was fixed that way) — but the *soak* run
|
||||
(rmmod/insmod cycle under traffic) is still an open gate, so expect unload to
|
||||
be correct for admin use and unproven for hot-reload automation.
|
||||
|
||||
Full list of behavioral divergences: [caveats.md](caveats.md).
|
||||
@@ -0,0 +1,116 @@
|
||||
# VM validation lab (libvirt / Alpine)
|
||||
|
||||
Português: [docs/pt-BR/vm-lab.md](../pt-BR/vm-lab.md).
|
||||
|
||||
All commands below were read from the scripts in `scripts/` (they are the
|
||||
source of truth); the lab has been created and run during M0–M2 — a full-suite
|
||||
green run is the M2+ gate and is still in progress (see §6).
|
||||
|
||||
## 1. Where it lives
|
||||
|
||||
| Host | Role |
|
||||
|---|---|
|
||||
| laptop (this repo) | builds kernel/module/tools; drives everything over ssh |
|
||||
| **deb** — `root@192.168.1.242` | libvirt hypervisor; runs the 4 `pftest-*` Alpine VMs |
|
||||
|
||||
The lab never needs NAT to the outside; the only external access is the
|
||||
libvirt `default` network (192.168.122.0/24, mgmt+DHCP) used for cloud-init
|
||||
package installs and test ssh. The existing `sms-vm` on deb is never touched.
|
||||
|
||||
## 2. Topology (single source: `lab/ips.env`)
|
||||
|
||||
| VM | Role | NICs | Kernel |
|
||||
|---|---|---|---|
|
||||
| `pftest-pf0` | filter under test (runs `pf.ko`) | `mgmt` (dhcp, default net), `lan` 10.0.1.1/24, `wan` 10.0.2.1/24, `wan2` 10.0.3.1/24 | stock Alpine via image-extracted direct boot; **or** the self-built 7.3.4 `bzImage` with `--pf0-kernel=` (`root=/dev/vda2 ro console=ttyS0,115200n8`, no initrd) |
|
||||
| `pftest-cli` | client | `mgmt` dhcp (routes disabled), `lan` 10.0.1.10/24, gw 10.0.1.1 (pf0) | Alpine stock |
|
||||
| `pftest-srv` | server | `mgmt` dhcp (routes disabled), `wan` 10.0.2.20/24, gw 10.0.2.1 (pf0), 10.0.3.0/24 via gw2 | Alpine stock |
|
||||
| `pftest-gw2` | second-path gateway (ip_forward=1) | `wan2` 10.0.3.2/24, `wan` 10.0.2.2/24 (no mgmt NIC) | Alpine stock |
|
||||
|
||||
All VMs get `net.ipv4.ip_forward=1` and `rp_filter=0` via cloud-init, root
|
||||
password `pftest` plus an injected lab keypair (`/root/.ssh/pflab_ed` on deb,
|
||||
copy on the laptop for ProxyJump), and packages
|
||||
`iperf3 tcpdump python3 scapy bash iptables` (Alpine name is `scapy`; the
|
||||
image also needs `iptables` for pf0 masquerade experiments).
|
||||
|
||||
libvirt networks: `default` (kept as-is, started+autostarted for packages) and
|
||||
isolated bridges `pf-lan` 10.0.1.0/24, `pf-wan` 10.0.2.0/24, `pf-wan2`
|
||||
10.0.3.0/24 — `forward="none"`, no DHCP, plus host-side bridge IPs
|
||||
`10.0.x.254` on deb so deb can ssh into every guest's data NIC. VM sizing:
|
||||
2 vCPU / 1 GB each. Disks are per-VM qcow2 **backing** a shared immutable
|
||||
`/var/lib/libvirt/images/alpine-base.qcow2` (synced from
|
||||
`_vendor/images/alpine-3.22-x86_64-cloud.qcow2` by sha256).
|
||||
|
||||
> Boot note: the Alpine cloud image is not BIOS/UEFI bootable in QEMU as
|
||||
> delivered (empty ESP), so *all* VMs use direct kernel boot from the image's
|
||||
> own extracted kernel; `--pf0-kernel=` only swaps pf0's kernel for the
|
||||
> built one.
|
||||
|
||||
## 3. Recreate / deploy / destroy
|
||||
|
||||
```sh
|
||||
./scripts/vm-create.sh # idempotent: tears down and rebuilds pftest-* domains + seeds
|
||||
./scripts/vm-create.sh --pf0-kernel=out/linux-7.3.4-bzImage # pf0 boots the built kernel (T6)
|
||||
./scripts/vm-create.sh --recreate-base # force base-image re-sync
|
||||
./scripts/vm-deploy.sh # laptop→deb ($DEPLOY_DIR)→pf0:/opt/pf:
|
||||
# out/*bzImage, module/*.ko, tools/out/*
|
||||
./scripts/vm-destroy.sh # domains, disks, seeds, pf-* nets, ssh fragments
|
||||
./scripts/vm-destroy.sh --purge-base --purge-keys # + base image + lab keypair
|
||||
```
|
||||
|
||||
`vm-create.sh` ends by validating `ssh pf0 'ip -br a'` and leaves all VMs
|
||||
running. Laptop ssh aliases (`pf0|cli|srv|gw2`, ProxyJump via deb) are
|
||||
generated into `~/.ssh/config.d/pflab`.
|
||||
|
||||
## 4. Running the suite
|
||||
|
||||
```sh
|
||||
./scripts/vm-run-tests.sh # or: make test
|
||||
```
|
||||
|
||||
Deterministic by design: **before every gate** the runner destroys and
|
||||
restarts `pftest-pf0` (fresh disk delta + no leftover module), waits for ssh
|
||||
on the mgmt DHCP address `192.168.122.8`, re-deploys `pf.ko`/`pfctl`/`*.conf`
|
||||
into the guest, then runs the gate script from the **laptop** (gates talk to
|
||||
the VMs through deb with nested ssh — `tests/vm/lib.sh`). Per-gate output is
|
||||
captured to `out/test-logs/<gate>.sh.log`; the runner prints
|
||||
`PASS <gate>` / `FAIL <gate> (rc=...)` per test, a summary `PASS=n FAIL=m`,
|
||||
and exits non-zero if anything failed.
|
||||
|
||||
## 5. What each gate proves today
|
||||
|
||||
| Gate | Scenario (asserted with real traffic, not exit codes) |
|
||||
|---|---|
|
||||
| `01-state.sh` | `pass in/out quick proto icmp keep state` → cli pings srv **through** pf0 (transit); pf0 then lists an icmp state via `pfctl -s states` |
|
||||
| `02-block.sh` | `block drop in quick proto icmp` → cli ping fails; `block return-rst in quick proto tcp … port 8080` → cli's `nc` to srv:8080 is refused (RST), checked via nc exit |
|
||||
| `03-tables.sh` | `table <blk> { 10.0.1.10 }` + `block in quick from <blk>` → `pfctl -t blk -T show` lists it; cli ping blocked; live `-T delete` unblocks the same flow |
|
||||
| `04-anchors.sh` | anchor rule (`-a restricted -f`) passing icmp is evaluated **before** a later main-ruleset `block` (quick/top-placement); `pfctl -s Anchors` lists the anchor |
|
||||
|
||||
Planned gates per the plan document (T11–T18): 05-nat, 06-rdr, 07-binat,
|
||||
08-scrub (scapy), 09-routeto, 10-dup-fail, 11-divert, 12-queues (iperf rate
|
||||
tolerance ±15%), 13-pflog, 14-soak (reload under traffic + rmmod cycle +
|
||||
kmemleak). IPv6 is smoke-only for state/block/nat66/scrub. `bench-basic.sh`
|
||||
(planned, not in the tree yet) is informational; there is no numeric perf gate.
|
||||
|
||||
## 6. Current suite status (2026-11-11, honestly)
|
||||
|
||||
The M2 gates 01–04 exist and run, but **are not all green yet** — the data
|
||||
path (netisr workqueue model) and state visibility are being fixed in the M2
|
||||
gate work. The latest captured run (`out/test-logs/`) shows the mixed state:
|
||||
transit ping OK / state listing failing on 01, icmp-blocked OK on 02, partial
|
||||
results on 03, 04 aborted. Treat "VERIFIED-in-VM" in the
|
||||
[feature matrix](feature-matrix.md) as reserved for gates that pass in a
|
||||
full clean run; nothing beyond M1 is claimed VERIFIED until then.
|
||||
|
||||
## 7. Manual poking
|
||||
|
||||
```sh
|
||||
ssh pf0 # via laptop alias (ProxyJump deb)
|
||||
insmod /opt/pf/pf.ko
|
||||
/opt/pf/pfctl -e
|
||||
/opt/pf/pfctl -f /tmp/g1.conf
|
||||
/opt/pf/pfctl -s info ; /opt/pf/pfctl -s states -v
|
||||
dmesg | grep pfk
|
||||
```
|
||||
|
||||
Artifacts of future full runs (log + pcaps) are meant to ship with the Gitea
|
||||
release (spec §7).
|
||||
Reference in new issue
Block a user