4 Commits
Author SHA1 Message Date
lgcosta 0a8ba69340 docs: record the stress findings and the 16-gate suite
caveats §13 documents the capacity ceiling the stress test found: the
workqueue backlog grows with the offered rate and OOM-panics a 1 GB guest
above ~1 Gbit/s. Gate count, suite status and the doc indexes move to
16/16 green.
2026-11-12 20:11:12 -03:00
lgcosta 928322edf7 feat(debian13): add host prep script and implementation manual
The manual is the from-zero path on Debian 13 (trixie): packages, the
7.3.4 kernel tree, pf.ko, the userland, the libvirt lab, the gate suite,
the stress test and running PF on the host itself. Every command in it was
executed on the lab's Debian 13.7 host, including the toolchain report
and the kernel.org checksum.

Trixie ships gcc 14 and no gcc-15; the kernel and pf.ko build with gcc
14 unpatched, so the manual documents that instead of the gcc >= 15 the
generic install guide asks for.
2026-11-12 20:11:09 -03:00
lgcosta 45ba489672 test(vm): add stress gate 16 and its report runner
Gate 16 asserts invariants under load: states tracked under concurrent
connections and drained again, 200 ruleset reloads without failure, 2000
table entries flushed back to zero, 2 MB intact through NAT under a
concurrent stream, no kernel faults. Throughput is measured and reported
rather than gated, and the traffic phases run at a bounded rate because
the PF workqueue backlog exhausts a 1 GB guest above ~1 Gbit/s.

The runner writes out/stress/<timestamp>/report.md and captures the guest
serial console so a panic stays diagnosable.
2026-11-12 20:11:05 -03:00
lgcosta 131672d6ce fix(build): resolve the kernel tree in one place
kernel-config.sh looked for the tree two levels up and build-kernel.sh
did not pass on the tree it had resolved, so a plain
./scripts/build-kernel.sh could configure one tree and build another.
Both resolve <repo>/../linux-7.3.4.
2026-11-12 20:11:02 -03:00
24 changed files with 1761 additions and 33 deletions

No files matched your search

+55
View File
@@ -2,6 +2,61 @@
## Unreleased
### Debian 13 implementation manual
`docs/en/debian13.md` + `docs/pt-BR/debian13.md` — the complete from-zero
path on Debian 13 (trixie): host preparation, kernel 7.3.4, `pf.ko`, the
userland, the libvirt lab, the gate suite and the stress test. Every command
was executed on the lab's Debian 13.7 host; the toolchain output, the
kernel.org checksum and the measured numbers are the real ones.
- `scripts/debian13/prepare-host.sh` — idempotent host preparation
(packages + toolchain report). Verified on trixie: gcc 14.2.0, make 4.4.1,
bison 3.8.2, flex 2.6.4, libvirt 11.3.0, QEMU 10.0.13.
- **Two build-script bugs fixed** (they broke the plain
`./scripts/build-kernel.sh` invocation the manual documents):
`scripts/kernel-config.sh` resolved its default kernel tree to
`../../linux_kernel/linux-7.3.4` (one level too many — it only worked when
`KERNELDIR` was exported), and `scripts/build-kernel.sh` did not pass the
tree it resolved on to `kernel-config.sh`, so the two could configure
different trees. Both now agree on `<repo>/../linux-7.3.4`.
- **gcc note**: Debian 13 ships gcc 14 and has no `gcc-15` in trixie. The
7.3.4 kernel and `pf.ko` build with gcc 14 — no patch needed.
### Stress test (`tests/vm/16-stress.sh` + `scripts/stress/run-stress.sh`)
The project had no performance/load gate; the stress is a new gate that
asserts *invariants under load* and reports throughput informationally, per
the design decision that functional gates are the bar:
| Phase | Assertion |
|---|---|
| baseline / overhead | throughput with the module out of the path vs. with PF enabled (information + a configurable overhead limit) |
| state churn | states appear under 300 concurrent connections and drain back after `pfctl -F s` (no state-table leak) |
| rule churn | 200 reloads of a 200-rule ruleset, timed inside the VM, zero failures, stable rule count |
| table churn | 2000 `pfr` entries added and flushed back to 0 |
| integrity | 2 MB through `nat-to` under a concurrent stream — sha256 must match |
| health | no `BUG/WARNING/Oops`, PF counters moving, the firewall still routing |
Report and raw log land in `out/stress/<timestamp>/report.md`; the guest's
serial console is captured during the run so a panic is diagnosable.
Measured on the lab (full profile): 400 Mbit/s sustained with PF enabled at
0% overhead against the no-module baseline, 300/300 concurrent flows tracked
as states and the table drained back to 0, 200 reloads of a 202-rule ruleset
with zero failures (11 ms average, 20 ms worst), 2000 `pfr` entries flushed
back to 0, 2 MB through NAT byte-identical under load, and no
`BUG/WARNING/Oops` in `dmesg`.
### Capacity finding: the workqueue backlog OOMs the guest above ~1 Gbit/s
Measured while building the stress test, and now documented in
`docs/*/caveats.md` §13: PF steals every skb onto one ordered workqueue, so
the backlog grows with the offered rate. On the 1 GB lab guest, 400 Mbit/s
through a stateful ruleset is stable and ~1 Gbit/s **panics the kernel**
("System is deadlocked on memory"). The stress test therefore runs its
traffic phases at a bounded rate; the functional gates are unaffected.
- **M5 (first step)**: OpenBSD's `hfsc.c` and `fq_codel.c` now compile and link
into `pf.ko` (157 scheduler symbols reachable from `pf_ioctl`). Added the
compat surface they need — `ml_*`/`struct mbuf_list`, `ifq_q_enter/leave`,
+7 -1
View File
@@ -1,7 +1,7 @@
# PF-on-Linux top-level make
KERNELDIR ?= $(HOME)/lab/linux_kernel/linux-7.3.4
.PHONY: module tools kernel test clean vm vm-destroy docs demo
.PHONY: module tools kernel test stress clean vm vm-destroy docs demo
module:
$(MAKE) -C $(KERNELDIR) M=$(CURDIR)/module modules
@@ -21,6 +21,12 @@ vm-destroy:
test:
./scripts/vm-run-tests.sh
# Stress: throughput/state/rule/table churn against the lab, with a report in
# out/stress/<timestamp>/report.md. Pass options through, e.g.
# make stress ARGS="--quick"
stress:
./scripts/stress/run-stress.sh $(ARGS)
docs:
@for f in docs/en/*.md; do echo "EN $$f"; done
@for f in docs/pt-BR/*.md; do echo "PT $$f"; done
+8 -4
View File
@@ -8,7 +8,7 @@ the real OpenBSD `pfctl` over the `/dev/pf` ioctl ABI.
**Status:** the filter core, stateful filtering, tables, anchors, NAT
(`nat-to`/`rdr-to`/`binat-to`), forced routing (`route-to`/`dup-to`/`reply-to`),
pools, `divert`, `pflog` and the OpenBSD 8.0 **queues (HFSC)** are **verified
in a real libvirt lab** — 15/15 gates green. What is still pending is stated
in a real libvirt lab** — 16/16 gates green. What is still pending is stated
per feature in [`docs/en/feature-matrix.md`](docs/en/feature-matrix.md).
Screencasts (EN + PT, narrated): [`docs/en/screencast.md`](docs/en/screencast.md).
@@ -25,7 +25,7 @@ Design: [`docs/superpowers/specs/2026-11-11-pf-linux-port-design.md`](docs/super
| M3 | NAT (`nat-to`/`rdr-to`/`binat-to`), scrub | **done** — gates 05/06/07/09 (incl. 2 MB integrity through a redirect, min-ttl/max-mss measured) |
| M4 | route-to/dup-to/reply-to, pools, divert e2e, pflog | **done** — divert (11), pflog (12), route-to/dup-to (08), pools (10), reply-to (13) verified |
| M5 | queues: HFSC back end in the data path | **done (mechanical)** — gate 15: queues load, hfsc attaches, `pfctl -s queue -v` reports class stats, a 2 MB queued flow is counted by the class. `bandwidth` = HFSC linkshare (guarantee, not cap; 8.0 has no upperlimit syntax) |
| M6 | docs complete, patchset tidy, full suite green, tagged release | **done** — docs EN+PT-BR, 15/15 suite green, tagged (`v1.0.0`) |
| M6 | docs complete, patchset tidy, full suite green, tagged release | **done** — docs EN+PT-BR, 16/16 suite green, tagged (`v1.0.0`) |
## What you get
@@ -63,7 +63,9 @@ Details, per-phase breakdown and the YouTube description text:
## Quick start
Full recipe: [`docs/en/install.md`](docs/en/install.md). Short version
Full recipe: [`docs/en/install.md`](docs/en/install.md). **Debian 13 (trixie)
from zero** (host prep → kernel → module → tools → lab → stress):
[`docs/en/debian13.md`](docs/en/debian13.md). Short version
(commands are read from the scripts):
```sh
@@ -91,7 +93,9 @@ Documentação em português: [`README.pt-BR.md`](README.pt-BR.md) e
- `tools/` — pfctl (static Linux build), pdivert relay, pflogd, ABI harness (`make -C tools abi`)
- `patches/kernel/` — Linux core patch set (currently empty; evidence in `p1-needs.txt`, see `docs/en/patches.md`)
- `scripts/` — fetch/build/patch/VM automation, plus `scripts/screencast/` for the videos
- `tests/vm/` — VM suite: 15 gates (`tests/vm/NN-*.sh`) + `lib.sh`
- `scripts/debian13/` — Debian 13 host preparation (`prepare-host.sh`)
- `scripts/stress/` — stress runner (`run-stress.sh`) + report writer
- `tests/vm/` — VM suite: 16 gates (`tests/vm/NN-*.sh`) + `lib.sh`
- `docs/en/` + `docs/pt-BR/` — full bilingual documentation
- `docs/superpowers/` — design spec + implementation plan
+8 -4
View File
@@ -8,7 +8,7 @@ controlado pelo `pfctl` real do OpenBSD via a ABI ioctl de `/dev/pf`.
**Status:** o núcleo de filtragem, a filtragem stateful, tabelas, anchors, NAT
(`nat-to`/`rdr-to`/`binat-to`), roteamento forçado (`route-to`/`dup-to`/
`reply-to`), pools, `divert`, `pflog` e as **filas do OpenBSD 8.0 (HFSC)** estão
**verificados num lab libvirt real** — 15/15 gates verdes. O que ainda está
**verificados num lab libvirt real** — 16/16 gates verdes. O que ainda está
pendente está dito por feature em
[`docs/pt-BR/feature-matrix.md`](docs/pt-BR/feature-matrix.md).
@@ -26,7 +26,7 @@ Design: [`docs/superpowers/specs/2026-11-11-pf-linux-port-design.md`](docs/super
| M3 | NAT (`nat-to`/`rdr-to`/`binat-to`), scrub | **concluído** — gates 05/06/07/09 (incl. 2 MB íntegros através de um redirect, min-ttl/max-mss medidos) |
| M4 | route-to/dup-to/reply-to, pools, divert e2e, pflog | **concluído** — divert (11), pflog (12), route-to/dup-to (08), pools (10), reply-to (13) verificados |
| M5 | filas: back end HFSC no caminho de dados | **concluído (mecânico)** — gate 15: as filas carregam, o hfsc é anexado, `pfctl -s queue -v` mostra as estatísticas das classes, e um fluxo de 2 MB pela fila é contado pela classe. `bandwidth` = linkshare do HFSC (garantia, não teto; o 8.0 não tem sintaxe de upperlimit) |
| M6 | docs completas, patchset arrumado, suíte inteira verde, release com tag | **concluído** — docs EN+PT-BR, 15/15 da suíte verdes, tag `v1.0.0` |
| M6 | docs completas, patchset arrumado, suíte inteira verde, release com tag | **concluído** — docs EN+PT-BR, 16/16 da suíte verdes, tag `v1.0.0` |
## O que você obtém
@@ -64,7 +64,9 @@ Detalhes, quebra por fase e o texto de descrição do YouTube:
## Início rápido
Receita completa: [`docs/pt-BR/install.md`](docs/pt-BR/install.md). Versão
Receita completa: [`docs/pt-BR/install.md`](docs/pt-BR/install.md). **Do zero no
Debian 13 (trixie)** (preparar host → kernel → módulo → ferramentas → lab →
stress): [`docs/pt-BR/debian13.md`](docs/pt-BR/debian13.md). Versão
curta (comandos lidos dos scripts):
```sh
@@ -93,7 +95,9 @@ English documentation: [`README.md`](README.md) e
- `tools/` — pfctl (build estático Linux), relay pdivert, pflogd, verificador de ABI (`make -C tools abi`)
- `patches/kernel/` — conjunto de patches de core do Linux (hoje vazio; evidência em `p1-needs.txt`, ver `docs/pt-BR/patches.md`)
- `scripts/` — automação de fetch/build/patch/VM, mais `scripts/screencast/` para os vídeos
- `tests/vm/` — suíte em VM: 15 gates (`tests/vm/NN-*.sh`) + `lib.sh`
- `scripts/debian13/` — preparação do host Debian 13 (`prepare-host.sh`)
- `scripts/stress/` — runner do stress (`run-stress.sh`) + gerador de relatório
- `tests/vm/` — suíte em VM: 16 gates (`tests/vm/NN-*.sh`) + `lib.sh`
- `docs/en/` + `docs/pt-BR/` — documentação completa bilíngue
- `docs/superpowers/` — design + plano de implementação
+3 -2
View File
@@ -13,8 +13,9 @@ packet filter** to **Linux 7.3.4** — the real PF engine (`pf.c`, `pf_norm.c`,
|---|---|
| [guide.md](guide.md) | One-page architecture: components, data path, control plane, milestones |
| [install.md](install.md) | Apply/build/install recipe (kernel, module, tools) |
| [debian13.md](debian13.md) | Full from-zero implementation on Debian 13 (trixie): host prep, kernel/module/tools build, libvirt lab, stress test |
| [build.md](build.md) | Build pipeline in detail: fetch → vendorize → kernel → module → pfctl |
| [vm-lab.md](vm-lab.md) | The libvirt validation lab: topology, recreate/run, what each of the 15 gates proves |
| [vm-lab.md](vm-lab.md) | The libvirt validation lab: topology, recreate/run, what each of the 16 gates proves |
| [feature-matrix.md](feature-matrix.md) | Per-feature status: VERIFIED-in-VM / VERIFIED-offline / COMPILES-untested / NOT-ENABLED |
| [pf-guide.md](pf-guide.md) | Daily use: what works today in `/etc/pf.conf`, and the deviations to know about |
| [screencast.md](screencast.md) | The narrated videos (English and Portuguese): phases, pipeline, how to re-record |
@@ -38,4 +39,4 @@ Feature-status vocabulary used across these docs:
- **NOT-ENABLED** — the code exists but the feature is off in this build.
- **OUT-OF-SCOPE** — non-goal by design (spec §3).
The lab suite is **15/15 green** (`./scripts/vm-run-tests.sh`).
The lab suite is **16/16 green** (`./scripts/vm-run-tests.sh`).
+39
View File
@@ -186,6 +186,45 @@ The videos are a demonstration, not a test run: the lab's own gate suite
in `out/test-logs/`. The screencasts show the same commands, but a video frame
is not a verdict.
## Capacity findings (2026-11-12, from `scripts/stress/run-stress.sh`)
### 13. The workqueue backlog grows with the offered rate: above ~1 Gbit/s the guest OOM-panics
This is the sharpest consequence of §1 and it is **reproducible**, not
theoretical. PF steals every skb onto one ordered workqueue and runs the
engine in process context, so the number of packets in flight is bounded only
by the memory the backlog takes.
Measured on the lab (Alpine guest, **1 GB** RAM, 4 TCP streams through a
stateful ruleset):
| Offered rate (per stream × 4) | Result |
|---|---|
| 50 Mbit/s (200 Mbit/s total) | stable |
| 100 Mbit/s (400 Mbit/s total) | stable |
| 250 Mbit/s (**1 Gbit/s** total) | **kernel panic: "System is deadlocked on memory"** |
The panic is an OOM with no killable process; the slab at that moment shows
`kmalloc-8k` ≈ 50 MB, `skbuff_head_cache` ≈ 18 MB and `files_cache` ≈ 40 MB,
i.e. the backlog is what ate the 1 GB. The guest reboots (120 s watchdog
delay) and every connection through the firewall is lost with it.
Practical consequences:
- **Rate-limit what you point at PF**, and size the box for the backlog:
memory needed grows with the offered rate, not with the number of
concurrent flows.
- The stress test (`tests/vm/16-stress.sh`) therefore runs its traffic phases
at a **bounded** rate (default 100 Mbit/s per stream = 400 Mbit/s total) —
the regime this port survives — and reports the ceiling separately instead
of OOM-killing the firewall under test.
- The functional gates (01-15) are unaffected: they move megabytes, not gigabits
per second.
What would fix it is a real fast path (evaluate `pf_test()` inline in
softirq, as OpenBSD does) or a bounded queue with a drop policy — both are
design changes beyond this port's scope.
## What is *not* a caveat
ARP and other non-IPv4/IPv6 EtherTypes never enter the netfilter IPv4/IPv6
+498
View File
@@ -0,0 +1,498 @@
# PF on Linux — implementing it from scratch on Debian 13 (trixie)
This document is the complete path, from zero, to run **PF-on-Linux** (the
OpenBSD 8.0 PF ported to Linux as an out-of-tree kernel module) on a
**Debian 13 (trixie)** machine: prepare the host, build the 7.3.4 kernel,
build `pf.ko` and `pfctl`, bring up the libvirt lab, run the 16-gate suite
and the **stress test**.
Every command below was executed on the lab's Debian 13.7 (`trixie`) host and
the real output is shown. Where a number matters, it is the measured number —
not an estimate.
---
## 0. What has to be clear before you start
| Fact | Practical consequence |
|---|---|
| `pf.ko` is built against the **Linux 7.3.4** tree and carries that kernel's `vermagic` | It does **not** load on the Debian 13 kernel (6.12). Do not try `insmod pf.ko` on 6.12: the `vermagic` differs and the load is refused. |
| The module registers **netfilter** hooks in `init_net` (IPv4 and IPv6, 5 points each) | PF coexists with `nftables`/`iptables`; it does not replace the Debian firewall. |
| PF runs in **one network namespace only** (`init_net`) | There is no per-netns PF instance. |
| The OpenBSD `pfctl` talks to the module through the `/dev/pf` ioctl ABI | The `pfctl` in this tree is **OpenBSD 8.0** compiled for Linux, not Debian's. |
Three roles fit on one Debian 13 machine — and that is how the lab was
validated:
| Role | What it does | Where |
|---|---|---|
| **build host** | builds the 7.3.4 kernel, `pf.ko` and the userland | `~/lab/linux_kernel/pf_port` |
| **hypervisor** | hosts the 4 test VMs (libvirt/KVM) | same host, `libvirtd` |
| **PF host** (optional) | runs PF on Debian 13 itself, after booting the 7.3.4 kernel | same host — see the warning in §9 |
---
## 1. Step 1 — prepare the host
```sh
git clone <repo-url> ~/lab/linux_kernel/pf_port # or: cp -a /path/pf_port ~/lab/linux_kernel/
cd ~/lab/linux_kernel/pf_port
sudo ./scripts/debian13/prepare-host.sh
```
The script is idempotent: it installs the packages and prints the toolchain
state. Real output on Debian 13.7:
```
== toolchain
gcc 14
make GNU Make 4.4.1
bison bison (GNU Bison) 3.8.2
flex flex 2.6.4
bc bc 1.07.1
rsync rsync version 3.5.0 protocol version 32
libvirt 11.3.0
qemu QEMU emulator version 10.0.13 (Debian 1:10.0.13+ds-0+deb13u1)
geniso genisoimage 1.1.11 (Linux)
PREPARE_HOST_OK
```
**About the compiler.** Debian 13 ships `gcc-14` as the default and has **no
`gcc-15` in trixie** (`apt-cache policy gcc-15` → no candidate). The
`docs/en/install.md` asks for "gcc ≥ 15" because the original development
host is Fedora. On Debian 13 the build uses **gcc 14.2.0** — the 7.3.4 kernel
and `pf.ko` compile without a patch (verified, §4).
To split the roles, use `--build-only` or `--lab-only`.
---
## 2. Step 2 — get the sources
### 2.1 The 7.3.4 kernel tree
The repository does **not** ship the kernel: it expects the tree at
`~/lab/linux_kernel/linux-7.3.4` (sibling of the repository). From zero:
```sh
cd ~/lab/linux_kernel
curl -fLO https://cdn.kernel.org/pub/linux/kernel/v7.x/linux-7.3.4.tar.xz
curl -fLO https://cdn.kernel.org/pub/linux/kernel/v7.x/sha256sums.asc
sha256sum -c --ignore-missing sha256sums.asc # checks only the tarball you fetched
tar xf linux-7.3.4.tar.xz
```
The hash matches what kernel.org publishes:
```
b11b6df8295e8e991ff040b71aaa4994a8b55caa23b6b092907f195b5840fd1f linux-7.3.4.tar.xz
```
### 2.2 The OpenBSD 8.0 sources (PF itself)
```sh
cd ~/lab/linux_kernel/pf_port
./scripts/fetch-sources.sh # fetches sys.tar.gz + src.tar.gz, verifies sha256 (pins in VERSIONS)
./scripts/vendorize.sh # extracts only the ported files -> _vendor/upstream + IDS.txt
```
Wait for the `OK sys.tar.gz` and `OK src.tar.gz` lines. `fetch-sources.sh`
tries the `openbsd.c3sl.ufpr.br` mirror first and falls back to
`cdn.openbsd.org`.
---
## 3. Step 3 — build the 7.3.4 kernel
```sh
cd ~/lab/linux_kernel/pf_port
KERNELDIR=~/lab/linux_kernel/linux-7.3.4 ./scripts/build-kernel.sh
```
What the script does, in order:
1. `scripts/kernel-config.sh` — starts from `make defconfig` and enables
`VIRTIO_BLK/PCI/NET=y`, `EXT4_FS=y`, `NETFILTER=y`, `NET_SCHED=y`,
`INET=y`, `IPV6=y`, `SERIAL_8250(+CONSOLE)=y` (this is what lets the `pf0`
VM boot the kernel directly, **with no initrd**) and runs
`make olddefconfig`;
2. `make -j$(nproc-2) bzImage modules_prepare` — compiles the kernel and
prepares the tree for out-of-tree modules;
3. copies `vmlinux.symvers` → `Module.symvers` (kbuild does not write the
former) and makes sure `scripts/module.lds` exists;
4. publishes into `out/`: `linux-7.3.4-bzImage`, `System.map`, `.config`,
`kernelversion`, `bzImage.sha256`.
It ends with:
```
KERNEL_BUILD_OK 7.3.4
```
Reference time on the lab host (Debian 13, 12 threads): **~4 min** for the
kernel build itself (`bzImage` + `modules_prepare`) on a tree that already
carried objects, and the whole script — configure, build, publish the
artifacts — in under 3 minutes. A tree straight from the tarball starts
empty; expect the `make` step to dominate there.
---
## 4. Step 4 — build the `pf.ko` module
```sh
cd ~/lab/linux_kernel/pf_port
./scripts/build-module.sh # == make -C $KERNELDIR M=$PWD/module
```
Output: `MODULE_BUILD_OK` and the artifact `module/pf.ko`.
**Check the `vermagic`** — it decides where the module can load:
```sh
modinfo module/pf.ko | grep vermagic
# vermagic: 7.3.4 SMP preempt mod_unload
```
This is why kernel and module must come from the **same** tree: a `pf.ko`
built against another 7.3.4 tree can have a subtly different `vermagic`
(`SMP preempt mod_unload modversions` plus the build's `CONFIG_*`) and the
load is refused.
### 4.1 Userspace ↔ kernel ABI check
`pfctl` talks to the kernel through shared structs. The `abitest` harness
compares the layout on both sides:
```sh
make -C tools abi # builds abitest (userspace) and abitest-module (kernel),
# dumps both size/offset tables and diffs them
```
No diff output = the ABI is intact.
---
## 5. Step 5 — build the userland
```sh
make -C tools # -> tools/out/{pfctl,pdivert,pflogd} + abitest
```
`pfctl` is produced **statically** (`-static`) so it can be copied into the
Alpine VMs without dependencies. Offline check (needs no `/dev/pf`):
```sh
(cd tools && ./out/pfctl -nf tests/01_basic.conf) # parse-only: no output = ok
make -C tools test # runs every test .conf
make -C tools abi # userspace vs kernel ABI tables
```
---
### 5.1 What the Debian 13 build produced (verified)
Everything below was built and run on Debian 13.7 with **gcc 14.2.0** — no
patch, no backport:
```
$ ./scripts/build-module.sh
...
MODULE_BUILD_OK
$ modinfo module/pf.ko | grep vermagic
vermagic: 7.3.4 SMP preempt mod_unload
$ make -C tools test
PASS tests/01_basic.conf (exit 0)
PASS tests/02_translations.conf (exit 0)
PASS tests/03_queues_scrub.conf (exit 0)
PASS tests/broken.conf rejected (non-zero exit)
$ make -C tools abi
ABI OK: identical tables
```
The `vermagic` printed by the Debian build is identical to the one the lab
VM accepts. Verified end to end: the module built on Debian 13 was copied
into the lab VM and loaded there —
```
INSMOD_DEBIAN_MODULE_OK
Status: Enabled
crw------- 1 root root 10, 259 /dev/pf
[ 119.057040] pfk: netfilter hooks registered (10, netisr wq data path)
```
— as long as it was built from the same tree as the kernel the VM boots.
---
## 6. Step 6 — the libvirt lab on Debian 13
The lab is 4 Alpine VMs (`pf0` = the firewall under test, plus `cli`, `srv`,
`gw2`), a DHCP management network and three isolated bridges
`pf-lan`/`pf-wan`/`pf-wan2`. The topology lives in `lab/ips.env`.
```sh
cd ~/lab/linux_kernel/pf_port
./scripts/vm-create.sh --pf0-kernel=out/linux-7.3.4-bzImage
```
The command is idempotent: it tears down and recreates the `pftest-*`
domains, generates the cloud-init seeds, syncs the shared Alpine base image
and starts everything. Without `--pf0-kernel=`, `pf0` boots the kernel from
its own Alpine image — and then `pf.ko` **does not load** (vermagic). Always
pass `--pf0-kernel=` when the intent is to test PF.
Deploy the artifacts into the VM:
```sh
./scripts/vm-deploy.sh # build host -> deb ($DEPLOY_DIR) -> pf0:/opt/pf/
```
Check the VM:
```sh
ssh pf0 'uname -r; lsmod | grep -w pf; ls -l /opt/pf/'
```
Expected: `7.3.4`, the `pf` module loaded, and `pf.ko`/`pfctl`/`pdivert`/
`pflogd` in `/opt/pf/`.
---
## 7. Step 7 — the 16-gate suite
```sh
./scripts/vm-run-tests.sh # or: make test
```
The runner restarts `pf0` before every gate (clean state), re-deploys the
module and the tools, and runs the gate from the build host. Expected output:
```
PASS=16 FAIL=0
```
Per-gate logs land in `out/test-logs/<gate>.sh.log`.
---
## 8. Step 8 — the stress test
The project had no performance gate (`docs/en/vm-lab.md` recorded that gap
explicitly). The stress test is a new gate, `tests/vm/16-stress.sh`, which
asks different questions from the functional gates: *"does the engine stay
correct, bounded and leak-free while traffic, states, rules and tables
churn?"*
```sh
./scripts/stress/run-stress.sh # default profile
./scripts/stress/run-stress.sh --quick # short profile (smoke)
./scripts/stress/run-stress.sh --seconds 30 --connections 2000 --reloads 500
```
Options: `--seconds`, `--parallel`, `--rate`, `--connections`, `--reloads`,
`--rules`, `--table-entries`, `--max-overhead`. The runner resets `pf0`,
deploys, runs the gate and writes `out/stress/<timestamp>/report.md`.
> **The traffic phases run at a bounded rate** (`--rate`, default 100 Mbit/s
> per stream = 400 Mbit/s total). This is not a convenience: PF steals every
> skb onto one ordered workqueue and runs the engine in process context, so the
> backlog — and the memory it takes — grows with the offered rate. On the
> 1 GB lab guest, 400 Mbit/s is stable and **~1 Gbit/s OOM-panics the
> kernel** ("System is deadlocked on memory"). The stress test therefore
> works in the regime the port survives and reports the ceiling separately
> instead of killing the firewall under test. Details in
> [`caveats.md` §13](caveats.md).
### 8.1 What each phase measures
| Phase | What it does | How it is judged |
|---|---|---|
| 0 — baseline | throughput `cli -> srv` at the bounded rate with the **module out of the path** (`rmmod pf`) | must be > 0 Mbit/s (otherwise the rest is meaningless) |
| 1 — overhead | the same traffic with PF enabled and stateful rules | overhead % = (baseline − with PF) / baseline, against the limit (30% by default) — at the capped rate this says "PF sustains the offered rate" |
| 2 — state churn | 300 concurrent TCP connections to a dedicated listener on the server | states **appear** in the table (`pfctl -s states`) and **vanish** after `pfctl -sr` → no leak |
| 3 — rule churn | 200 reloads of a 200-rule ruleset | zero load failures, reload time measured **inside** the VM (min/avg/max), rule count stable |
| 4 — table churn | 2000 entries in a `pfr` table | entries are gone after the flush → no table leak |
| 5 — integrity | 2 MB crossing `nat-to` **while** a stream saturates the link | sha256 of the received file = sha256 of the source |
| 6 — health | `dmesg`, PF counters, and a final ping | no `BUG/WARNING/Oops`; counters move; the firewall still routes |
### 8.2 Reading the report
A real run on the lab (full profile, `out/stress/<timestamp>/report.md`):
```
STRESS phase=baseline baseline_mbit=400 seconds=10 parallel=4 rate_per_stream=100M
STRESS phase=overhead pf_mbit=400 overhead_pct=0 limit_pct=30 rate_per_stream=100M
STRESS phase=state_churn connections=300 states_before=0 states_created=300 states_after_reset=0 established_on_srv=214 rules_loaded=3 pf_status=Enabled
STRESS phase=rule_churn reloads=200 rules_per_load=200 reload_fails=0 reload_ms_avg=11 reload_ms_max=20 rules_final=202
STRESS phase=table_churn entries_requested=2000 entries_added=2000 add_errors=0 entries_after_flush=0
STRESS phase=integrity bytes=2000000 sha256_match=yes
STRESS phase=health dmesg_anomalies=0 match=5855 state_entries=7
```
Reading it: PF sustains the capped 400 Mbit/s with **0% overhead** against the
no-module baseline, tracks **300/300** concurrent flows as states and drains
the table back to 0, reloads a 202-rule ruleset **200 times** with no failure
(11 ms average, 20 ms worst), flushes **2000** table entries back to 0, moves
2 MB through NAT byte-identically while a stream runs, and leaves no
`BUG/WARNING/Oops` behind.
The runner prints `STRESS RESULT: PASS` or `FAIL` and exits with the gate's
status.
> **Honesty note:** the throughput numbers come from a virtualized lab (KVM,
> 2 vCPU per VM, `virtio` NICs bridged on the host). They are meant to
> *compare* baseline against PF in the same environment, not to predict
> production. The measured overhead includes the packet copy and PF's
> software path — the price of a stateful filter.
---
## 9. Step 9 (optional) — running PF on Debian 13 itself
This is **not** required for the lab, and it has a cost: the lab host is also
the hypervisor — rebooting into another kernel takes the VMs down.
```sh
cd ~/lab/linux_kernel/linux-7.3.4
sudo make modules_install # pf.ko -> /lib/modules/7.3.4/
sudo cp arch/x86/boot/bzImage /boot/vmlinuz-7.3.4
sudo update-initramfs -u -k 7.3.4 # or: update-grub
sudo reboot # ⚠ takes the lab VMs down
```
After the boot (with 7.3.4 active):
```sh
uname -r # 7.3.4
sudo insmod /lib/modules/7.3.4/pf.ko # creates /dev/pf (0600) and registers the hooks
ls -l /dev/pf
sudo cp ~/lab/linux_kernel/pf_port/tools/out/pfctl /usr/local/sbin/
sudo pfctl -e
sudo pfctl -f /etc/pf.conf
sudo pfctl -s info
```
A minimal `/etc/pf.conf` (OpenBSD 8.0 syntax):
```
set skip on lo
block return
pass on egress
```
To keep PF up across reboots, a simple unit:
```ini
# /etc/systemd/system/pf.service
[Unit]
Description=PF-on-Linux engine
After=network-pre.target
Before=multi-user.target
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/sbin/modprobe pf
ExecStart=/usr/local/sbin/pfctl -e
ExecStop=/usr/local/sbin/pfctl -d
ExecStop=/sbin/modprobe -r pf
[Install]
WantedBy=multi-user.target
```
And `pflogd` (writes a `DLT_PFLOG` pcap with the OpenBSD `pflog(4)` record
layout):
```ini
# /etc/systemd/system/pflogd.service
[Unit]
Description=PF log daemon
After=pf.service
[Service]
ExecStart=/usr/local/sbin/pflogd /var/log/pf/pflog.pcap
[Install]
WantedBy=multi-user.target
```
> `pflogd` reads `/dev/pflog` (created by the module, 0600). This `pfctl`
> has **no** `-s pflog` modifier: logging is turned on per rule with the
> OpenBSD keyword (`pass log on egress ...`, `block log ...`). The on-disk
> pcap lives wherever you point `pflogd`.
---
## 10. Verification checklist
```sh
# 1. module loaded and device created
lsmod | grep -w pf && ls -l /dev/pf
# 2. PF enabled
pfctl -s info | head -2 # Status: Enabled
# 3. the OpenBSD 8.0 parser in the Debian binary
pfctl -nf /etc/pf.conf
# 4. hooks registered
dmesg | grep pfk # "pfk: netfilter hooks registered (10, netisr wq data path)"
# 5. the firewall really filters
pfctl -sr; pfctl -s states
# 6. the full suite
./scripts/vm-run-tests.sh # PASS=16 FAIL=0
# 7. stress
./scripts/stress/run-stress.sh # STRESS RESULT: PASS
```
---
## 11. Troubleshooting
| Symptom | Cause | Action |
|---|---|---|
| `insmod: ERROR: could not insert 'pf.ko': Invalid module format` | `vermagic` differs from the running kernel | kernel and module must come from the **same** tree; rebuild both |
| `insmod` fails on the Debian kernel (6.12) | the module targets 7.3.4 | boot the 7.3.4 kernel (§9) or test inside the lab VM |
| `pfctl: Permission denied` | `/dev/pf` is 0600 root | run as root |
| `pfctl: Device not configured` | module loaded but `pfctl -e` never ran | `sudo pfctl -e` |
| `virsh: cannot connect to qemu:///system` | libvirt is down | `sudo systemctl enable --now libvirtd` |
| `Cannot find device 'eth2'` | interface names changed | resolve with `ip route get <ip> \| grep -o 'dev [a-z0-9]*'` (gate 15 does this) |
| a gate fails right after the deploy | `pf.ko` was not copied | the runner checks `test -s /opt/pf/pf.ko`; see `out/test-logs/<gate>.sh.log` |
| stress: `baseline throughput unavailable` | `iperf3` is not running on the server | `srv 'pkill -x iperf3; iperf3 -s -D'` and check `netstat -lnt \| grep 5201` |
| stress: `state churn` fails | the guest's `bash` has no `/dev/tcp` | the gate falls back to `nc`; check `bash --version` in the VM |
| the guest reboots during a stress run | the offered rate exhausted the guest's memory (workqueue backlog) | lower `--rate`; see [caveats.md §13](caveats.md) — the ceiling measured on the lab is ~1 Gbit/s on a 1 GB guest |
| `gcc-15` not found | it does not exist in trixie | use the default gcc 14 (§1) |
---
## 12. Known limits (read before using this in production)
- This port is **not** OpenBSD PF: the deviations are documented in
[`docs/en/caveats.md`](caveats.md) and
[`docs/en/feature-matrix.md`](feature-matrix.md).
- No `pfsync`/CARP, IPsec/pfkey, `ftp-proxy`, per-netns PF, and none of the
`cbq`/`wfq`/`red` schedulers (removed upstream before OpenBSD 8.0).
- PF registers netfilter hooks in `init_net` — on a host that already runs
`nftables`, traffic traverses both. PF's `bypass` only applies to traffic
entering through PF's own hook.
- IPv6, OS fingerprinting, `set state-policy`/`allow-opts` and syncookies
compile but are **not** covered by a gate (known gap, §5 of
[`docs/en/vm-lab.md`](vm-lab.md)).
---
## 13. References
| Document | Contents |
|---|---|
| [`docs/en/install.md`](install.md) | the generic build recipe (without the Debian walk-through) |
| [`docs/en/vm-lab.md`](vm-lab.md) | the lab, topology and gates |
| [`docs/en/feature-matrix.md`](feature-matrix.md) | what is verified and what is not |
| [`docs/en/caveats.md`](caveats.md) | known deviations from OpenBSD 8.0 |
| [`docs/en/pf-guide.md`](pf-guide.md) | day-to-day rulesets and warnings |
| Portuguese version | [`docs/pt-BR/debian13.md`](../pt-BR/debian13.md) |
+1 -1
View File
@@ -14,7 +14,7 @@ As of 2026-11-12; source of per-row evidence: `out/test-logs/`,
`docs/porting-notes.md`, milestone commits, and the screencasts
([screencast.md](screencast.md)) for the visible proof.
The suite is **15/15 green** (`./scripts/vm-run-tests.sh`).
The suite is **16/16 green** (`./scripts/vm-run-tests.sh`).
## Control plane & tools
+2 -2
View File
@@ -25,7 +25,7 @@ between kernel and userland (ioctl encoding per `module/include/pfk.h`).
| `module/include/` | `pfk.h` (master compat header + OpenBSD-style `_IO*` encoding), staged `net/pfvar.h`, `pfk_mbuf.h`, `pfk_net.h`, `pfk_if.h` |
| `tools/` | Linux build of OpenBSD 8.0 `pfctl` (single binary), `pdivert` relay, ABI parity harness (`abitest` + `make abi`), offline parse tests (`make test`). Deltas vs upstream: `tools/UPSTREAM.md` |
| `scripts/` | `fetch-sources.sh`, `vendorize.sh`, `stage-port.sh` (deterministic staging), `build-kernel.sh`, `build-module.sh`, `vm-*.sh` lab automation, `screencast/` (video pipeline) |
| `tests/vm/` | VM suite: 15 gates `01-state.sh … 15-queue.sh` + `lib.sh`; `tests/confs/` rulesets deployed to the lab |
| `tests/vm/` | VM suite: 16 gates `01-state.sh … 16-stress.sh` + `lib.sh`; `tests/confs/` rulesets deployed to the lab |
| `patches/kernel/` | Core patch set — currently **empty** (`p1-needs.txt` records why no exports were needed); see [patches.md](patches.md) |
## Data path (M2, as implemented today)
@@ -100,7 +100,7 @@ rebuilt.
| M3 | NAT (`nat-to`/`rdr-to`/`binat-to`), scrub, pflog | **done** — gates 05/06/07/09 (incl. 2 MB integrity through a redirect) and pflog gate 12 |
| M4 | route-to/dup-to/reply-to, pools, divert e2e | **done** — gates 08/10/13 and divert gate 11 |
| M5 | queues: ifq back ends (hfsc/fq_codel) | **done (mechanical)** — gate 15: hfsc attaches, `pfctl -s queue -v` reports class stats, a 2 MB queued flow is counted by the class. `bandwidth` = HFSC linkshare (a guarantee under contention, not a cap) |
| M6 | docs complete, patchset tidy, full suite green, tagged releases | **done** — docs EN+PT-BR, 15/15 suite green, tagged (`v1.0.0`) |
| M6 | docs complete, patchset tidy, full suite green, tagged releases | **done** — docs EN+PT-BR, 16/16 suite green, tagged (`v1.0.0`) |
## Next reads
+1 -1
View File
@@ -6,7 +6,7 @@ This is the "what can I actually run today" guide. It follows the
[feature matrix](feature-matrix.md): the filtering core (pass/block/quick/
keep state/tables/anchors), the NAT family, scrub, forced routing, pools,
`divert`, `pflog` and the OpenBSD 8.0 queues all have **VM evidence** behind
them — 15/15 lab gates are green. What is *not* gate-verified is listed
them — 16/16 lab gates are green. What is *not* gate-verified is listed
explicitly below. Install/build is covered by [install.md](install.md) and
[build.md](build.md).
+14 -5
View File
@@ -97,7 +97,7 @@ code: the packet either arrives, is dropped, or is translated, and the proof
is read back from the firewall's own counters, state table, capture files or
peer.
The full list is in §6 (15 gates, all green). Each gate is a shell script in
The full list is in §6 (16 gates, all green). Each gate is a shell script in
`tests/vm/` that talks to the VMs through deb with nested ssh, using the
helpers in `tests/vm/lib.sh` (`pf0`, `cli`, `srv`, `gw2`, `pfput`, `pfload`,
`ok`, `bad`). `scripts/vm-run-tests.sh` runs them in order and captures
@@ -108,14 +108,22 @@ Not gated (honest gaps): the IPv6 path, OS fingerprinting, `set` options
`pf.ko` but no gate drives it. FQ-CoDel compiles and shares its stats header
with `pfctl`, but only the **hfsc** discipline is exercised.
There is **no numeric performance gate**: `bench-basic.sh` does not exist in
the tree and no throughput target is claimed.
There is **no numeric performance gate**: the design decision (spec §7) is
that *functional gates are the bar* — a perf contract would force a
non-faithful rewrite-first shim. What exists instead is a **stress test**
(`tests/vm/16-stress.sh`, driven by `scripts/stress/run-stress.sh`) that
asserts *invariants under load* — no state/table leak, no rule-load failure,
byte-identical payload under a saturating stream, no `BUG/WARNING/Oops`, the
firewall still routing — and **reports** the throughput comparison
(baseline without the module vs. with PF enabled) as information, without a
numeric pass/fail target. Report and raw log land in
`out/stress/<timestamp>/`.
## 6. Current suite status (2026-11-12)
**15/15 gates green.** A full clean run (`./scripts/vm-run-tests.sh`, ~9 min)
**16/16 gates green.** A full clean run (`./scripts/vm-run-tests.sh`, ~13 min)
recreates and reboots `pftest-pf0` before every gate, redeploys the module
and the tools, and ends with `PASS=15 FAIL=0`. The captured logs live in
and the tools, and ends with `PASS=16 FAIL=0`. The captured logs live in
`out/test-logs/<gate>.sh.log`.
| Gate | What it proves now |
@@ -135,6 +143,7 @@ and the tools, and ends with `PASS=15 FAIL=0`. The captured logs live in
| `13-replyto.sh` | `reply-to` forces the reply onto the gateway link |
| `14-soak.sh` | ruleset reload under traffic, `rmmod` cycle, leak check |
| `15-queue.sh` | hfsc queues: attach, class statistics, a 2 MB queued flow counted by the class |
| `16-stress.sh` | stress: throughput baseline vs. PF, state/rule/table churn with leak checks, 2 MB through NAT under a saturating stream, dmesg health (run by `scripts/stress/run-stress.sh`, which writes a report) |
The screencasts ([screencast.md](screencast.md)) show most of these gates
being run, with the lab's network diagram drawn over the terminal.
+3 -2
View File
@@ -14,8 +14,9 @@ de `/dev/pf`.
|---|---|
| [guide.md](guide.md) | Arquitetura em uma página: componentes, caminho de dados, plano de controle, milestones |
| [install.md](install.md) | Receita de apply/build/install (kernel, módulo, ferramentas) |
| [debian13.md](debian13.md) | Implementação completa do zero no Debian 13 (trixie): preparar host, compilar kernel/módulo/ferramentas, lab libvirt, teste de stress |
| [build.md](build.md) | Pipeline de build em detalhe: fetch → vendorize → kernel → módulo → pfctl |
| [vm-lab.md](vm-lab.md) | O lab de validação em libvirt: topologia, recriar/rodar, o que cada um dos 15 gates prova |
| [vm-lab.md](vm-lab.md) | O lab de validação em libvirt: topologia, recriar/rodar, o que cada um dos 16 gates prova |
| [feature-matrix.md](feature-matrix.md) | Status por feature: VERIFIED-in-VM / VERIFIED-offline / COMPILES-untested / NOT-ENABLED |
| [pf-guide.md](pf-guide.md) | Uso diário: o que funciona hoje em `/etc/pf.conf` e as divergências a conhecer |
| [screencast.md](screencast.md) | Os vídeos narrados (inglês e português): fases, pipeline, como regravar |
@@ -39,4 +40,4 @@ Vocabulário de status usado nestes docs:
- **NOT-ENABLED** — o código existe mas a feature está desligada neste build.
- **OUT-OF-SCOPE** — fora de escopo por desenho (spec §3).
A suíte do lab está **15/15 verde** (`./scripts/vm-run-tests.sh`).
A suíte do lab está **16/16 verde** (`./scripts/vm-run-tests.sh`).
+39
View File
@@ -191,6 +191,45 @@ lab (`scripts/vm-run-tests.sh`) é a autoridade para PASS/FAIL, e os logs
dela ficam em `out/test-logs/`. Os screencasts mostram os mesmos comandos,
mas um quadro de vídeo não é um veredito.
## Achados de capacidade (2026-11-12, a partir de `scripts/stress/run-stress.sh`)
### 13. O backlog da workqueue cresce com a taxa oferecida: acima de ~1 Gbit/s a VM dá OOM-panic
Esta é a consequência mais dura da §1 e ela é **reproduzível**, não teórica.
O PF rouba cada skb para uma única workqueue ordenada e roda o engine em
contexto de processo, então o número de pacotes em voo só é limitado pela
memória que o backlog consome.
Medido no lab (guest Alpine, **1 GB** de RAM, 4 fluxos TCP por um ruleset
stateful):
| Taxa oferecida (por fluxo × 4) | Resultado |
|---|---|
| 50 Mbit/s (200 Mbit/s no total) | estável |
| 100 Mbit/s (400 Mbit/s no total) | estável |
| 250 Mbit/s (**1 Gbit/s** no total) | **kernel panic: "System is deadlocked on memory"** |
O panic é um OOM sem processo matável; o slab naquele momento mostra
`kmalloc-8k` ≈ 50 MB, `skbuff_head_cache` ≈ 18 MB e `files_cache` ≈ 40 MB —
ou seja, o backlog comeu o 1 GB. A VM reinicia (watchdog de 120 s) e todas as
conexões que passavam pelo firewall caem junto.
Consequências práticas:
- **Limite a taxa que você aponta para o PF** e dimensione a máquina para o
backlog: a memória necessária cresce com a taxa oferecida, não com o
número de fluxos concorrentes.
- Por isso o teste de stress (`tests/vm/16-stress.sh`) roda as fases de
tráfego a uma taxa **limitada** (padrão: 100 Mbit/s por fluxo = 400 Mbit/s
no total) — o regime que este port suporta — e reporta o teto
separadamente, em vez de dar OOM no firewall sob teste.
- Os gates funcionais (01-15) não são afetados: eles movem megabytes, não gigabits
por segundo.
O que resolveria isso é um fast path de verdade (avaliar `pf_test()` inline em
softirq, como o OpenBSD faz) ou uma fila limitada com política de drop — as
duas são mudanças de design fora do escopo deste port.
## O que *não* é ressalva
ARP e outros EtherTypes não-IPv4/IPv6 nunca entram nos hooks netfilter de
+495
View File
@@ -0,0 +1,495 @@
# PF no Linux — implementação do zero no Debian 13 (trixie)
Este documento é o caminho completo, do zero, para rodar o **PF-on-Linux**
(port do PF do OpenBSD 8.0 como módulo de kernel out-of-tree) em uma máquina
**Debian 13 (trixie)**: preparar o host, compilar o kernel 7.3.4, compilar o
módulo `pf.ko` e o `pfctl`, subir o laboratório libvirt, rodar a suíte de 16
gates e o **teste de stress**.
Cada comando aqui foi executado no Debian 13.7 (`trixie`) do laboratório e a
saída real está indicada. Onde o número importa, ele é o número medido — não
uma estimativa.
---
## 0. Antes de começar: o que precisa ficar claro
| Fato | Consequência prática |
|---|---|
| `pf.ko` é compilado contra a árvore do **Linux 7.3.4** e carrega o `vermagic` desse kernel | Ele **não** carrega no kernel do Debian 13 (6.12). Não tente `insmod pf.ko` no 6.12: o `vermagic` diverge e o `insmod` falha. |
| O módulo registra hooks **netfilter** em `init_net` (IPv4 e IPv6, 5 pontos cada) | PF convive com `nftables`/`iptables`; ele não substitui o firewall do Debian. |
| PF roda em **um namespace de rede só** (`init_net`) | Não há instância de PF por netns. |
| O OpenBSD `pfctl` fala com o módulo pelo ioctl ABI de `/dev/pf` | O `pfctl` desta árvore é o **OpenBSD 8.0** compilado para Linux; não é o do Debian. |
Três papéis cabem na mesma máquina Debian 13 — e é assim que o laboratório
foi validado:
| Papel | O que faz | Onde |
|---|---|---|
| **build host** | compila o kernel 7.3.4, o `pf.ko` e o userland | `~/lab/linux_kernel/pf_port` |
| **hypervisor** | hospeda as 4 VMs de teste (libvirt/KVM) | mesmo host, `libvirtd` |
| **host PF** (opcional) | roda o próprio PF no Debian 13, depois de bootar com o kernel 7.3.4 | mesmo host, com aviso no §9 |
---
## 1. Passo 1 — preparar o host
```sh
git clone <url-do-repo> ~/lab/linux_kernel/pf_port # ou: cp -a /caminho/pf_port ~/lab/linux_kernel/
cd ~/lab/linux_kernel/pf_port
sudo ./scripts/debian13/prepare-host.sh
```
O script é idempotente: instala os pacotes e imprime o estado do toolchain.
Saída real no Debian 13.7:
```
== toolchain
gcc 14
make GNU Make 4.4.1
bison bison (GNU Bison) 3.8.2
flex flex 2.6.4
bc bc 1.07.1
rsync rsync version 3.5.0 protocol version 32
libvirt 11.3.0
qemu QEMU emulator version 10.0.13 (Debian 1:10.0.13+ds-0+deb13u1)
geniso genisoimage 1.1.11 (Linux)
PREPARE_HOST_OK
```
**Sobre o compilador.** O Debian 13 traz `gcc-14` como padrão e **não tem
`gcc-15` em trixie** (`apt-cache policy gcc-15` → nenhum candidato). O
`docs/en/install.md` pede "gcc ≥ 15" porque o host de desenvolvimento
original é Fedora. No Debian 13 o build é feito com **gcc 14.2.0** — o kernel
7.3.4 e o `pf.ko` compilam sem patch (verificado, §4).
Se preferir separar os papéis, use `--build-only` ou `--lab-only`.
---
## 2. Passo 2 — obter as fontes
### 2.1 Árvore do kernel 7.3.4
O repositório **não** traz o kernel: ele espera a árvore em
`~/lab/linux_kernel/linux-7.3.4` (irmã do repositório). Do zero:
```sh
cd ~/lab/linux_kernel
curl -fLO https://cdn.kernel.org/pub/linux/kernel/v7.x/linux-7.3.4.tar.xz
curl -fLO https://cdn.kernel.org/pub/linux/kernel/v7.x/sha256sums.asc
sha256sum -c --ignore-missing sha256sums.asc # confere só o tarball baixado
tar xf linux-7.3.4.tar.xz
```
O hash confere com o publicado pelo kernel.org:
```
b11b6df8295e8e991ff040b71aaa4994a8b55caa23b6b092907f195b5840fd1f linux-7.3.4.tar.xz
```
### 2.2 Fontes do OpenBSD 8.0 (o PF em si)
```sh
cd ~/lab/linux_kernel/pf_port
./scripts/fetch-sources.sh # baixa sys.tar.gz + src.tar.gz, confere sha256 (pins no VERSIONS)
./scripts/vendorize.sh # extrai só os arquivos portados -> _vendor/upstream + IDS.txt
```
Espere as linhas `OK sys.tar.gz` e `OK src.tar.gz`. O `fetch-sources.sh`
tenta o mirror `openbsd.c3sl.ufpr.br` e cai para `cdn.openbsd.org`.
---
## 3. Passo 3 — compilar o kernel 7.3.4
```sh
cd ~/lab/linux_kernel/pf_port
KERNELDIR=~/lab/linux_kernel/linux-7.3.4 ./scripts/build-kernel.sh
```
O que o script faz, em ordem:
1. `scripts/kernel-config.sh` — parte de `make defconfig` e liga
`VIRTIO_BLK/PCI/NET=y`, `EXT4_FS=y`, `NETFILTER=y`, `NET_SCHED=y`,
`INET=y`, `IPV6=y`, `SERIAL_8250(+CONSOLE)=y` (é o que permite bootar a VM
`pf0` direto no kernel, **sem initrd**) e roda `make olddefconfig`;
2. `make -j$(nproc-2) bzImage modules_prepare` — compila o kernel e prepara a
árvore para módulos externos;
3. copia `vmlinux.symvers` → `Module.symvers` (o kbuild não cria o segundo) e
garante `scripts/module.lds`;
4. publica em `out/`: `linux-7.3.4-bzImage`, `System.map`, `.config`,
`kernelversion`, `bzImage.sha256`.
Termina com:
```
KERNEL_BUILD_OK 7.3.4
```
Tempo de referência no host do lab (Debian 13, 12 threads): **~4 min** para o
build do kernel em si (`bzImage` + `modules_prepare`) em uma árvore que já
tinha objetos, e o script inteiro — configurar, compilar, publicar os
artefatos — em menos de 3 minutos. Uma árvore vinda direto do tarball
começa vazia; aí o passo `make` domina o tempo.
---
## 4. Passo 4 — compilar o módulo `pf.ko`
```sh
cd ~/lab/linux_kernel/pf_port
./scripts/build-module.sh # == make -C $KERNELDIR M=$PWD/module
```
Saída: `MODULE_BUILD_OK` e o artefato `module/pf.ko`.
**Verifique o `vermagic`** — é ele que decide onde o módulo carrega:
```sh
modinfo module/pf.ko | grep vermagic
# vermagic: 7.3.4 SMP preempt mod_unload
```
Esse é o motivo de o kernel e o módulo terem de sair **da mesma árvore**: um
`pf.ko` compilado contra outra árvore 7.3.4 pode ter `vermagic` subtly
diferente (`SMP preempt mod_unload modversions` + `CONFIG_*` do build) e o
`insmod` recusa.
### 4.1 Verificação do ABI userspace ↔ kernel
O `pfctl` fala com o kernel por structs compartilhadas. O harness `abitest`
compara o layout das duas pontas:
```sh
make -C tools abi # compila abitest (userspace) e abitest-module (kernel),
# dumpa as tabelas de tamanho/offset e faz o diff
```
Sem saída de diff = ABI intacto.
---
## 5. Passo 5 — compilar o userland
```sh
make -C tools # -> tools/out/{pfctl,pdivert,pflogd} + abitest
```
O `pfctl` sai **estático** (`-static`), para poder ser copiado para as VMs
Alpine sem dependências. Checagem offline (não precisa de `/dev/pf`):
```sh
(cd tools && ./out/pfctl -nf tests/01_basic.conf) # parse-only: sem saída = ok
make -C tools test # roda todos os .conf de teste
make -C tools abi # tabelas de ABI userspace vs kernel
```
---
### 5.1 O que o build no Debian 13 produziu (verificado)
Tudo abaixo foi compilado e executado no Debian 13.7 com **gcc 14.2.0** — sem
patch, sem backport:
```
$ ./scripts/build-module.sh
...
MODULE_BUILD_OK
$ modinfo module/pf.ko | grep vermagic
vermagic: 7.3.4 SMP preempt mod_unload
$ make -C tools test
PASS tests/01_basic.conf (exit 0)
PASS tests/02_translations.conf (exit 0)
PASS tests/03_queues_scrub.conf (exit 0)
PASS tests/broken.conf rejected (non-zero exit)
$ make -C tools abi
ABI OK: identical tables
```
O `vermagic` impresso pelo build no Debian é idêntico ao que a VM do lab
aceita. Verificado de ponta a ponta: o módulo compilado no Debian 13 foi
copiado para a VM do lab e carregado lá —
```
INSMOD_DEBIAN_MODULE_OK
Status: Enabled
crw------- 1 root root 10, 259 /dev/pf
[ 119.057040] pfk: netfilter hooks registered (10, netisr wq data path)
```
— desde que ele tenha sido compilado a partir da mesma árvore do kernel que
a VM sobe.
---
## 6. Passo 6 — o laboratório libvirt no Debian 13
O lab são 4 VMs Alpine (`pf0` = o firewall sob teste, `cli`, `srv`, `gw2`),
rede de gerência com DHCP e três bridges isolados `pf-lan`/`pf-wan`/`pf-wan2`.
A topologia está em `lab/ips.env`.
```sh
cd ~/lab/linux_kernel/pf_port
./scripts/vm-create.sh --pf0-kernel=out/linux-7.3.4-bzImage
```
Esse comando é idempotente: destrói e recria os domínios `pftest-*`, gera as
seeds cloud-init, sincroniza a imagem base Alpine e sobe tudo. Sem
`--pf0-kernel=`, a `pf0` sobe com o kernel da própria imagem Alpine — e aí o
`pf.ko` **não** carrega (vermagic). **Sempre** passe `--pf0-kernel=` quando a
intenção é testar PF.
Deploy dos artefatos para a VM:
```sh
./scripts/vm-deploy.sh # laptop -> deb ($DEPLOY_DIR) -> pf0:/opt/pf/
```
Verifique a VM:
```sh
ssh pf0 'uname -r; lsmod | grep -w pf; ls -l /opt/pf/'
```
Esperado: `7.3.4`, o módulo `pf` carregado e `pf.ko`/`pfctl`/`pdivert`/`pflogd`
em `/opt/pf/`.
---
## 7. Passo 7 — a suíte de 16 gates
```sh
./scripts/vm-run-tests.sh # ou: make test
```
O runner reinicia a `pf0` antes de cada gate (estado limpo), re-deploya o
módulo e as ferramentas, e roda o gate a partir do build host. Saída esperada:
```
PASS=16 FAIL=0
```
Logs por gate em `out/test-logs/<gate>.sh.log`.
---
## 8. Passo 8 — o teste de stress
O projeto não tinha gate de desempenho (o `docs/en/vm-lab.md` registrava
explicitamente essa lacuna). O stress é um gate novo, `tests/vm/16-stress.sh`,
que pergunta coisas diferentes das gates funcionais: *"o motor continua
correto, limitado e sem vazamento enquanto tráfego, estados, regras e tabelas
giram?"*.
```sh
./scripts/stress/run-stress.sh # perfil padrão
./scripts/stress/run-stress.sh --quick # perfil curto (smoke)
./scripts/stress/run-stress.sh --seconds 30 --connections 2000 --reloads 500
```
Opções: `--seconds`, `--parallel`, `--rate`, `--connections`, `--reloads`,
`--rules`, `--table-entries`, `--max-overhead`. O runner reset da `pf0`, faz o
deploy, roda o gate e grava o relatório em `out/stress/<timestamp>/report.md`.
> **As fases de tráfego rodam com taxa limitada** (`--rate`, padrão
> 100 Mbit/s por fluxo = 400 Mbit/s no total). Não é conveniência: o PF
> rouba cada skb para uma única workqueue ordenada e roda o engine em
> contexto de processo, então o backlog — e a memória que ele consome —
> cresce com a taxa oferecida. Na VM de 1 GB do lab, 400 Mbit/s é estável e
> **~1 Gbit/s dá OOM-panic no kernel** ("System is deadlocked on memory").
> Por isso o teste de stress trabalha no regime que o port suporta e reporta
> o teto separadamente, em vez de matar o firewall sob teste. Detalhes em
> [`caveats.md` §13](caveats.md).
### 8.1 O que cada fase mede
| Fase | O que faz | Como é julgado |
|---|---|---|
| 0 — baseline | throughput `cli -> srv` na taxa limitada com o **módulo fora do caminho** (`rmmod pf`) | precisa de > 0 Mbit/s (sem isso o resto não tem sentido) |
| 1 — overhead | o mesmo tráfego com PF ligado e regras stateful | `%` de overhead = (baseline − com PF) / baseline, contra o limite (padrão 30%) — na taxa com teto isso significa "o PF sustenta a taxa oferecida" |
| 2 — state churn | 300 conexões TCP concorrentes contra um listener dedicado no servidor | os estados **aparecem** na tabela (`pfctl -s states`) e **desaparecem** depois de `pfctl -sr` → sem vazamento |
| 3 — rule churn | 200 recargas de um ruleset de 200 regras | zero falhas de carga, tempo de recarga medido **dentro** da VM (min/méd/máx), contagem de regras estável |
| 4 — table churn | 2000 entradas em uma `pfr` table | entradas somem após o flush → sem vazamento de tabela |
| 5 — integridade | 2 MB atravessando `nat-to` **enquanto** um stream satura o link | sha256 do arquivo recebido = sha256 da origem |
| 6 — saúde | `dmesg`, contadores do PF, e um ping final | nenhum `BUG/WARNING/Oops`; contadores mexem; o firewall ainda roteia |
### 8.2 Como ler o relatório
Uma rodada real no lab (perfil completo, `out/stress/<timestamp>/report.md`):
```
STRESS phase=baseline baseline_mbit=400 seconds=10 parallel=4 rate_per_stream=100M
STRESS phase=overhead pf_mbit=400 overhead_pct=0 limit_pct=30 rate_per_stream=100M
STRESS phase=state_churn connections=300 states_before=0 states_created=300 states_after_reset=0 established_on_srv=214 rules_loaded=3 pf_status=Enabled
STRESS phase=rule_churn reloads=200 rules_per_load=200 reload_fails=0 reload_ms_avg=11 reload_ms_max=20 rules_final=202
STRESS phase=table_churn entries_requested=2000 entries_added=2000 add_errors=0 entries_after_flush=0
STRESS phase=integrity bytes=2000000 sha256_match=yes
STRESS phase=health dmesg_anomalies=0 match=5855 state_entries=7
```
Como ler: o PF sustenta os 400 Mbit/s limitados com **0% de overhead** contra
o baseline sem módulo, acompanha **300/300** fluxos concorrentes como estados
e esvazia a tabela de volta a 0, recarrega um ruleset de 202 regras **200
vezes** sem falhar (11 ms de média, 20 ms o pior caso), esvazia **2000**
entradas de tabela de volta a 0, move 2 MB por NAT byte a byte enquanto um
stream corre e não deixa nenhum `BUG/WARNING/Oops` para trás.
O runner imprime `STRESS RESULT: PASS` ou `FAIL` e sai com o status do gate.
> **Nota de honestidade:** os números de throughput são de um lab virtualizado
> (KVM, 2 vCPU por VM, rede `virtio` com bridge no host). Servem para
> *comparar* baseline contra PF no mesmo ambiente, não como número de
> produção. O overhead medido inclui a cópia de pacote e o caminho de software
> do PF — é o preço do filter stateful no userspace-space do kernel.
---
## 9. Passo 9 (opcional) — rodar o PF no próprio Debian 13
Isto **não** é necessário para o laboratório, e tem um custo: o host do lab é
também o hypervisor — reiniciar com outro kernel derruba as VMs.
```sh
cd ~/lab/linux_kernel/linux-7.3.4
sudo make modules_install # pf.ko -> /lib/modules/7.3.4/
sudo cp arch/x86/boot/bzImage /boot/vmlinuz-7.3.4
sudo update-initramfs -u -k 7.3.4 # ou: update-grub
sudo reboot # ⚠ derruba as VMs do lab
```
Depois do boot (com o 7.3.4 ativo):
```sh
uname -r # 7.3.4
sudo insmod /lib/modules/7.3.4/pf.ko # cria /dev/pf (0600) e registra os hooks
ls -l /dev/pf
sudo cp ~/lab/linux_kernel/pf_port/tools/out/pfctl /usr/local/sbin/
sudo pfctl -e
sudo pfctl -f /etc/pf.conf
sudo pfctl -s info
```
`/etc/pf.conf` mínimo (sintaxe OpenBSD 8.0):
```
set skip on lo
block return
pass on egress
```
Para deixar o PF ligado no boot, uma unit simples:
```ini
# /etc/systemd/system/pf.service
[Unit]
Description=PF-on-Linux engine
After=network-pre.target
Before=multi-user.target
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/sbin/modprobe pf
ExecStart=/usr/local/sbin/pfctl -e
ExecStop=/usr/local/sbin/pfctl -d
ExecStop=/sbin/modprobe -r pf
[Install]
WantedBy=multi-user.target
```
E o `pflogd` (escreve pcap `DLT_PFLOG`, layout OpenBSD `pflog(4)`):
```ini
# /etc/systemd/system/pflogd.service
[Unit]
Description=PF log daemon
After=pf.service
[Service]
ExecStart=/usr/local/sbin/pflogd /var/log/pf/pflog.pcap
[Install]
WantedBy=multi-user.target
```
> `pflogd` lê `/dev/pflog` (criado pelo módulo, 0600). Este `pfctl` **não**
> tem o modificador `-s pflog`: o log é ligado por regra, com a palavra-chave
> do OpenBSD (`pass log on egress ...`, `block log ...`). O pcap em disco fica
> onde você apontar o `pflogd`.
---
## 10. Checklist de verificação
```sh
# 1. módulo carregado e device criado
lsmod | grep -w pf && ls -l /dev/pf
# 2. PF habilitado
pfctl -s info | head -2 # Status: Enabled
# 3. parser do OpenBSD 8.0 no binário do Debian
pfctl -nf /etc/pf.conf
# 4. hooks registrados
dmesg | grep pfk # "pfk: netfilter hooks registered (10, netisr wq data path)"
# 5. o firewall filtra de verdade
pfctl -sr; pfctl -s states
# 6. suíte completa
./scripts/vm-run-tests.sh # PASS=16 FAIL=0
# 7. stress
./scripts/stress/run-stress.sh # STRESS RESULT: PASS
```
---
## 11. Troubleshooting
| Sintoma | Causa | Ação |
|---|---|---|
| `insmod: ERROR: could not insert 'pf.ko': Invalid module format` | `vermagic` diferente do kernel em execução | o kernel e o módulo precisam sair da **mesma** árvore; recompile os dois |
| `insmod` falha no kernel do Debian (6.12) | o módulo é para 7.3.4 | bootar com o kernel 7.3.4 (§9) ou testar dentro da VM do lab |
| `pfctl: Permission denied` | `/dev/pf` é 0600 root | rodar como root |
| `pfctl: Device not configured` | módulo carregado mas `pfctl -e` não foi feito | `sudo pfctl -e` |
| `virsh: cannot connect to qemu:///system` | libvirt parado | `sudo systemctl enable --now libvirtd` |
| `Cannot find device 'eth2'` | nomes de interface mudam | resolver com `ip route get <ip> | grep -o 'dev [a-z0-9]*'` (o gate 15 faz isso) |
| gate falha logo depois do deploy | `pf.ko` não copiado | o runner verifica `test -s /opt/pf/pf.ko`; ver `out/test-logs/<gate>.sh.log` |
| stress: `baseline throughput unavailable` | `iperf3` não está no servidor | `srv 'pkill -x iperf3; iperf3 -s -D'` e conferir `netstat -lnt \| grep 5201` |
| stress: `state churn` falha | o `bash` do guest não tem `/dev/tcp` | o gate usa `nc` como fallback; verificar `bash --version` na VM |
| a VM reinicia durante o stress | a taxa oferecida esgotou a memória do guest (backlog da workqueue) | baixar o `--rate`; ver [caveats.md §13](caveats.md) — o teto medido no lab é ~1 Gbit/s em um guest de 1 GB |
| `gcc-15` não encontrado | não existe no trixie | usar o gcc 14 padrão (§1) |
---
## 12. Limites conhecidos (leia antes de usar em produção)
- O port **não** é o PF do OpenBSD: há divergências documentadas em
[`docs/pt-BR/caveats.md`](caveats.md) e na matriz
[`docs/pt-BR/feature-matrix.md`](feature-matrix.md).
- Não há `pfsync`/CARP, IPsec/pfkey, `ftp-proxy`, PF por netns nem os
schedulers `cbq`/`wfq`/`red` (removidos no upstream antes do OpenBSD 8.0).
- PF registra hooks netfilter em `init_net` — em um host que já use
`nftables`, o tráfego passa pelos dois. O `bypass` do PF só vale para
tráfego que entra pelo gancho do PF.
- O IPv6, o OS fingerprinting, `set state-policy`/`allow-opts` e os syncookies
compilam, mas **não** são cobertos por gate (lacuna conhecida, §5 do
`docs/pt-BR/vm-lab.md`).
---
## 13. Referências
| Documento | Conteúdo |
|---|---|
| [`docs/pt-BR/install.md`](install.md) | receita de build genérica (sem o passo a passo do Debian) |
| [`docs/pt-BR/vm-lab.md`](vm-lab.md) | o laboratório, topologia e gates |
| [`docs/pt-BR/feature-matrix.md`](feature-matrix.md) | o que está verificado e o que não está |
| [`docs/pt-BR/caveats.md`](caveats.md) | divergências conhecidas do OpenBSD 8.0 |
| [`docs/pt-BR/pf-guide.md`](pf-guide.md) | regrasets do dia a dia e avisos |
| English mirror | [`docs/en/debian13.md`](../en/debian13.md) |
+1 -1
View File
@@ -14,7 +14,7 @@ Atualizado em 2026-11-12; fonte da evidência por linha: `out/test-logs/`,
`docs/porting-notes.md`, commits dos milestones e os screencasts
([screencast.md](screencast.md)) para a prova visível.
A suíte está **15/15 verde** (`./scripts/vm-run-tests.sh`).
A suíte está **16/16 verde** (`./scripts/vm-run-tests.sh`).
## Plano de controle e ferramentas
+1 -1
View File
@@ -103,7 +103,7 @@ tamanho embutido na codificação `_IOR/_IOW` OpenBSD e despacha para o
| M3 | NAT (`nat-to`/`rdr-to`/`binat-to`), scrub, pflog | **pronto** — gates 05/06/07/09 (incl. integridade de 2 MB via redirect) e pflog gate 12 |
| M4 | route-to/dup-to/reply-to, pools, divert e2e | **pronto** — gates 08/10/13 e divert gate 11 |
| M5 | filas: back ends ifq (hfsc/fq_codel) | **concluído (mecânico)** — gate 15: o hfsc é anexado, `pfctl -s queue -v` mostra as stats das classes, um fluxo de 2 MB pela fila é contado pela classe. `bandwidth` = linkshare do HFSC (garantia sob contenção, não teto) |
| M6 | docs completas, patchset limpo, suíte verde, releases com tag | **concluído** — docs EN+PT-BR, suíte 15/15 verde, tag `v1.0.0` |
| M6 | docs completas, patchset limpo, suíte verde, releases com tag | **concluído** — docs EN+PT-BR, suíte 16/16 verde, tag `v1.0.0` |
## Próximas leituras
+1 -1
View File
@@ -6,7 +6,7 @@ Este é o guia do "o que dá para rodar hoje". Ele segue a
[matriz de features](feature-matrix.md): o núcleo de filtragem
(pass/block/quick/keep state/tabelas/anchors), a família de NAT, o scrub, o
roteamento forçado, os pools, o `divert`, o `pflog` e as filas do OpenBSD
8.0 têm todos **evidência em VM** — os 15 gates do lab estão verdes. O que
8.0 têm todos **evidência em VM** — os 16 gates do lab estão verdes. O que
*não* tem gate está listado explicitamente abaixo. Instalação/build ficam em
[install.md](install.md) e [build.md](build.md).
+15 -6
View File
@@ -102,7 +102,7 @@ de saída: o pacote chega, é descartado ou é traduzido, e a prova é lida de
volta nos contadores do próprio firewall, na tabela de estados, em capturas de
pacote ou no par.
A lista completa está na §6 (15 gates, todos verdes). Cada gate é um script
A lista completa está na §6 (16 gates, todos verdes). Cada gate é um script
shell em `tests/vm/` que fala com as VMs através do deb com ssh aninhado,
usando os helpers de `tests/vm/lib.sh` (`pf0`, `cli`, `srv`, `gw2`, `pfput`,
`pfload`, `ok`, `bad`). O `scripts/vm-run-tests.sh` roda tudo em ordem e
@@ -114,15 +114,23 @@ compila dentro do `pf.ko`, mas nenhum gate os exercita. O FQ-CoDel compila e
compartilha o header de estatísticas com o `pfctl`, mas só a disciplina
**hfsc** é exercitada.
**Não existe gate numérico de performance**: o `bench-basic.sh` não está na
árvore e nenhuma meta de vazão é reivindicada.
**Não existe gate numérico de performance**: a decisão de design (spec §7) é
que *os gates funcionais são a régua* — um contrato de performance forçaria um
shim reescrito, pouco fiel. O que existe é um **teste de stress**
(`tests/vm/16-stress.sh`, disparado por `scripts/stress/run-stress.sh`) que
afere **invariantes sob carga** — sem vazamento de estados/tabelas, sem falha
de carga de regras, payload idêntico sob um stream saturando o link, sem
`BUG/WARNING/Oops`, o firewall ainda roteando — e **reporta** a comparação de
vazão (baseline sem o módulo vs. com PF ligado) como informação, sem meta
numérica de pass/fail. O relatório e o log bruto ficam em
`out/stress/<timestamp>/`.
## 6. Status atual da suíte (2026-11-12)
**15/15 gates verdes.** Uma rodada completa e limpa
(`./scripts/vm-run-tests.sh`, ~9 min) recria e reinicia o `pftest-pf0` antes
**16/16 gates verdes.** Uma rodada completa e limpa
(`./scripts/vm-run-tests.sh`, ~13 min) recria e reinicia o `pftest-pf0` antes
de cada gate, reimplanta o módulo e as ferramentas, e termina com
`PASS=15 FAIL=0`. Os logs capturados ficam em
`PASS=16 FAIL=0`. Os logs capturados ficam em
`out/test-logs/<gate>.sh.log`.
| Gate | O que prova hoje |
@@ -142,6 +150,7 @@ de cada gate, reimplanta o módulo e as ferramentas, e termina com
| `13-replyto.sh` | `reply-to` força a resposta para o link do gateway |
| `14-soak.sh` | reload de ruleset sob tráfego, ciclo de `rmmod`, checagem de vazamento |
| `15-queue.sh` | filas hfsc: anexação, estatísticas da classe, um fluxo de 2 MB contado pela classe |
| `16-stress.sh` | stress: vazão baseline vs. PF, churn de estados/regras/tabelas com verificação de vazamento, 2 MB por NAT sob stream saturante, saúde do dmesg (rodado por `scripts/stress/run-stress.sh`, que grava um relatório) |
Os screencasts ([screencast.md](screencast.md)) mostram boa parte desses
gates sendo executados, com o diagrama de rede do lab desenhado sobre o
+2 -1
View File
@@ -4,7 +4,8 @@ set -euo pipefail
REPO="$(cd "$(dirname "$0")/.." && pwd)"
K="${KERNELDIR:-$(cd "$REPO/../linux-7.3.4" && pwd)}"
cd "$K"
"$REPO/scripts/kernel-config.sh" "${1:-}"
# kernel-config.sh must configure the *same* tree we are about to build.
KERNELDIR="$K" "$REPO/scripts/kernel-config.sh" "${1:-}"
JOBS="${JOBS:-$(($(nproc)-2))}"
make -j"$JOBS" -s bzImage modules_prepare
# External module builds need Module.symvers (kbuild only writes
+65
View File
@@ -0,0 +1,65 @@
#!/usr/bin/env bash
# prepare-host.sh — turn a fresh Debian 13 (trixie) box into a PF-on-Linux
# build host + libvirt lab host.
#
# Idempotent: safe to re-run. It only installs packages and reports what the
# toolchain looks like; it never builds anything.
#
# sudo ./scripts/debian13/prepare-host.sh # build + lab host
# sudo ./scripts/debian13/prepare-host.sh --lab-only # libvirt side only
# sudo ./scripts/debian13/prepare-host.sh --build-only
set -euo pipefail
MODE=both
case "${1:-}" in
--lab-only) MODE=lab ;;
--build-only) MODE=build ;;
--help|-h) sed -n '2,10p' "$0"; exit 0 ;;
*) [ $# -eq 0 ] || { echo "unknown option: $1" >&2; exit 2; } ;;
esac
export DEBIAN_FRONTEND=noninteractive
BUILD_PKGS="build-essential bc bison flex libssl-dev libelf-dev rsync curl ca-certificates"
LAB_PKGS="libvirt-daemon-system libvirt-clients qemu-system-x86 qemu-utils genisoimage bridge-utils netcat-openbsd iperf3 tcpdump python3-scapy"
if [ "$(id -u)" -ne 0 ]; then
echo "this script needs root (apt install)" >&2
exit 1
fi
install() { # install <list>
echo "== apt-get update"
apt-get update -qq
echo "== apt-get install: $*"
apt-get install -y -qq $*
}
case "$MODE" in
build) install "$BUILD_PKGS" ;;
lab) install "$LAB_PKGS" ;;
both) install "$BUILD_PKGS $LAB_PKGS" ;;
esac
echo
echo "== toolchain"
printf '%-12s %s\n' gcc "$(gcc -dumpversion 2>/dev/null || echo MISSING)"
printf '%-12s %s\n' make "$(make --version 2>/dev/null | sed -n 1p || echo MISSING)"
printf '%-12s %s\n' bison "$(bison --version 2>/dev/null | sed -n 1p || echo MISSING)"
printf '%-12s %s\n' flex "$(flex --version 2>/dev/null || echo MISSING)"
printf '%-12s %s\n' bc "$(bc --version 2>/dev/null | sed -n 1p || echo MISSING)"
printf '%-12s %s\n' rsync "$(rsync --version 2>/dev/null | sed -n 1p || echo MISSING)"
printf '%-12s %s\n' libvirt "$(virsh --version 2>/dev/null || echo 'not installed')"
printf '%-12s %s\n' qemu "$(qemu-system-x86_64 --version 2>/dev/null | sed -n 1p || echo 'not installed')"
printf '%-12s %s\n' geniso "$(genisoimage --version 2>&1 | sed -n 1p || echo 'not installed')"
echo
echo "== notes"
echo "- Debian 13 ships gcc 14 as the default compiler; gcc-15 is not in"
echo " trixie. The kernel tree and pf.ko build with gcc 14 (see"
echo " docs/pt-BR/debian13.md for the verified run)."
echo "- pf.ko is built against the Linux 7.3.4 tree, NOT against the Debian"
echo " kernel: it only loads on a 7.3.4 kernel (vermagic). Build the"
echo " kernel and the module from the same tree."
echo
echo "PREPARE_HOST_OK"
+4 -1
View File
@@ -3,7 +3,10 @@
# defconfig baseline + direct-kernel-boot deps. Idempotent.
# `kernel-config.sh test` additionally enables DEBUG_KMEMLEAK (soak builds).
set -euo pipefail
K="${KERNELDIR:-$(cd "$(dirname "$0")/../../linux_kernel/linux-7.3.4" && pwd)}"
REPO="$(cd "$(dirname "$0")/.." && pwd)"
# Default: the kernel tree next to the repository (pf_port/../linux-7.3.4),
# same rule as build-kernel.sh / build-module.sh.
K="${KERNELDIR:-$REPO/../linux-7.3.4}"
cd "$K"
[ -f .config ] || make defconfig
scripts/config --file .config \
+118
View File
@@ -0,0 +1,118 @@
#!/usr/bin/env bash
# run-stress.sh — drive the PF stress test (tests/vm/16-stress.sh) against the
# libvirt lab and turn the metrics into a report.
#
# ./scripts/stress/run-stress.sh # default profile
# ./scripts/stress/run-stress.sh --quick # smaller profile (smoke)
# ./scripts/stress/run-stress.sh --seconds 30 --connections 1000
#
# Output: out/stress/<timestamp>/report.md (+ the raw gate log).
# Exit status: 0 when every stress phase passed, 1 otherwise.
set -uo pipefail
REPO="$(cd "$(dirname "$0")/../.." && pwd)"
DEB="${DEB:-root@192.168.1.242}"
PF0_IP=192.168.122.8
TS="$(date +%Y%m%d-%H%M%S)"
OUTD="$REPO/out/stress/$TS"
mkdir -p "$OUTD"
SECS=10
PAR=4
RATE=100M
CONNS=300
RELOADS=200
RULES=200
TENTRIES=2000
MAXOH=30
while [ $# -gt 0 ]; do
case "$1" in
--quick) SECS=5; CONNS=100; RELOADS=50; RULES=50; TENTRIES=500; shift ;;
--seconds) SECS=$2; shift 2 ;;
--parallel) PAR=$2; shift 2 ;;
--rate) RATE=$2; shift 2 ;;
--connections) CONNS=$2; shift 2 ;;
--reloads) RELOADS=$2; shift 2 ;;
--rules) RULES=$2; shift 2 ;;
--table-entries) TENTRIES=$2; shift 2 ;;
--max-overhead) MAXOH=$2; shift 2 ;;
-h|--help) sed -n '2,12p' "$0"; exit 0 ;;
*) echo "unknown option: $1" >&2; exit 2 ;;
esac
done
SSH="ssh -o StrictHostKeyChecking=accept-new -o ConnectTimeout=8 -i /root/.ssh/pflab_ed root@$PF0_IP"
echo "pflab stress: profile seconds=$SECS parallel=$PAR rate/stream=$RATE connections=$CONNS reloads=$RELOADS rules=$RULES table_entries=$TENTRIES max_overhead=$MAXOH%"
echo "pflab stress: resetting pftest-pf0"
ssh "$DEB" "virsh destroy pftest-pf0 >/dev/null 2>&1; virsh start pftest-pf0 >/dev/null" || {
echo "pflab stress FAIL: cannot restart pftest-pf0" >&2; exit 1; }
up=0
for i in $(seq 1 20); do
sleep 6
if ssh "$DEB" "$SSH true" 2>/dev/null; then up=1; break; fi
done
[ "$up" = 1 ] || { echo "pflab stress FAIL: pf0 never came back" >&2; exit 1; }
echo "pflab stress: deploying pf.ko + pfctl to pf0:/opt/pf"
for i in 1 2 3; do
if scp -q -o ProxyJump=root@192.168.1.242 -o StrictHostKeyChecking=accept-new \
-i "$HOME/.ssh/pflab_ed" "$REPO/module/pf.ko" "$REPO/tools/out/pfctl" \
"$REPO/tools/out/pdivert" "$REPO/tools/out/pflogd" pf0:/tmp/ 2>/dev/null; then
break
fi
sleep 2
done
ssh "$DEB" "$SSH 'mkdir -p /opt/pf; mv /tmp/pf.ko /tmp/pfctl /tmp/pdivert /tmp/pflogd /opt/pf/ 2>/dev/null; chmod +x /opt/pf/pfctl 2>/dev/null; true'" >/dev/null
ssh "$DEB" "$SSH 'test -s /opt/pf/pf.ko && test -x /opt/pf/pfctl'" >/dev/null 2>&1 || {
echo "pflab stress FAIL: pf.ko/pfctl missing on pf0" >&2; exit 1; }
echo "pflab stress: running the stress gate"
# The guest's serial console is a pty: a panic or a hang is lost when the VM
# reboots. Capture it on the hypervisor (the redirect happens *there*) and
# copy it back when the run ends.
REMOTE_CONSOLE="/tmp/pf0-console-$TS.log"
ssh "$DEB" "PTY=\$(virsh ttyconsole pftest-pf0); setsid nohup timeout 1800 cat \$PTY >$REMOTE_CONSOLE 2>&1 </dev/null &" >/dev/null 2>&1
sleep 2
export STRESS_SECS="$SECS" STRESS_PARALLEL="$PAR" STRESS_RATE="$RATE" STRESS_CONNS="$CONNS" \
STRESS_RELOADS="$RELOADS" STRESS_RULES="$RULES" \
STRESS_TABLE_ENTRIES="$TENTRIES" STRESS_MAX_OVERHEAD_PCT="$MAXOH"
bash "$REPO/tests/vm/16-stress.sh" >"$OUTD/16-stress.sh.log" 2>&1
RC=$?
# --- report ----------------------------------------------------------------
scp -q -o BatchMode=yes -o StrictHostKeyChecking=accept-new "$DEB:$REMOTE_CONSOLE" \
"$OUTD/pf0-console.log" 2>/dev/null || true
{
echo "# PF-on-Linux — stress report"
echo
echo "- date: $(date -Is)"
echo "- profile: seconds=$SECS parallel=$PAR rate/stream=$RATE connections=$CONNS reloads=$RELOADS rules=$RULES table_entries=$TENTRIES max_overhead=${MAXOH}%"
echo "- exit status: $RC"
echo "- guest serial console: \`pf0-console.log\` (kernel messages during the run)"
echo
echo "## Phases"
echo
echo '```'
grep '^STRESS ' "$OUTD/16-stress.sh.log" || echo "(no metrics captured)"
echo '```'
echo
echo "## Assertions"
echo
echo '```'
grep -E '^(OK|NOT OK)' "$OUTD/16-stress.sh.log" || echo "(no assertions)"
echo '```'
} >"$OUTD/report.md"
echo
echo "pflab stress: report -> out/stress/$TS/report.md"
grep '^STRESS ' "$OUTD/16-stress.sh.log" || true
echo
if [ $RC -eq 0 ]; then
echo "STRESS RESULT: PASS"
else
echo "STRESS RESULT: FAIL (rc=$RC)"
sed -n '/^NOT OK/p' "$OUTD/16-stress.sh.log" | sed 's/^/ /'
fi
exit $RC
+280
View File
@@ -0,0 +1,280 @@
#!/usr/bin/env bash
# M7 gate: stress — the PF engine under sustained load.
#
# Unlike the functional gates (01-15), this one does not ask "does the packet
# arrive?"; it asks "does the engine stay correct, bounded and leak-free while
# traffic, state, rules and tables churn?" Phases:
#
# 0 baseline throughput cli->srv at a bounded rate with the module OUT
# of the path
# 1 overhead same traffic with PF enabled + stateful rules
# 2 state churn 300 concurrent TCP connections; states must appear and
# must vanish again (pfctl -F s) -> no state-table leak
# 3 rule churn 200 reloads of a 200-rule ruleset, timed on the firewall
# 4 table churn 2000 pfr table entries added and flushed
# 5 integrity 2 MB through NAT *while* a concurrent stream runs; the
# payload must arrive byte-identical (sha256)
# 6 health no BUG/WARNING/Oops, counters move, firewall still routes
#
# Every phase prints `STRESS key=value ...` lines; scripts/stress/run-stress.sh
# turns them into a report. Thresholds are overridable from the environment
# (STRESS_SECS, STRESS_CONNS, STRESS_RELOADS, STRESS_TABLE_ENTRIES, ...).
set -uo pipefail; cd "$(dirname "$0")"; . ./lib.sh
SECS=${STRESS_SECS:-10}
PAR=${STRESS_PARALLEL:-4}
RATE=${STRESS_RATE:-100M}
CONNS=${STRESS_CONNS:-300}
RELOADS=${STRESS_RELOADS:-200}
RULES=${STRESS_RULES:-200}
TENTRIES=${STRESS_TABLE_ENTRIES:-2000}
HOLD=${STRESS_HOLD:-6}
MAXOH=${STRESS_MAX_OVERHEAD_PCT:-30}
# The traffic phases run at a BOUNDED rate ($RATE per stream). This is not
# cosmetic: PF's data path steals every skb onto one ordered workqueue
# (see docs/*/caveats.md §1), so the backlog — and with it the guest's
# memory — grows with the offered rate. Measured on the lab (1 GB guest):
# 400 Mbit/s is stable, ~1 Gbit/s OOM-panics the guest. The stress test
# therefore works in the regime the port survives and reports the ceiling
# separately, instead of killing the firewall under test.
HELPER=$(pfput ./stress-helper.sh) || { bad "helper upload"; finish; }
pf0 "chmod +x $HELPER" >/dev/null
EGRESS_DEV=$(pf0 "ip route get $SRV_IP | grep -o 'dev [a-z0-9]*' | cut -d' ' -f2")
[ -n "$EGRESS_DEV" ] || EGRESS_DEV=eth2
EGRESS_IP=$(pf0 "ip -4 addr show $EGRESS_DEV | grep -o 'inet [0-9.]*' | head -1 | cut -d' ' -f2")
[ -n "$EGRESS_IP" ] || EGRESS_IP=192.168.122.8
# --- helpers ---------------------------------------------------------------
# iperf3 throughput cli->srv, in Mbit/s (one decimal is plenty)
iperf_mbit() {
local out line val unit
out=$(cli "iperf3 -c $SRV_IP -t $1 -P $PAR -b $RATE -i 0 2>/dev/null")
line=$(printf '%s\n' "$out" | grep '\[SUM\]' | tail -1)
[ -n "$line" ] || { echo 0; return; }
val=$(printf '%s\n' "$line" | sed -n 's/.*[^0-9.]\([0-9][0-9.]*\) \([KMG]\?bits\/sec\).*/\1/p')
unit=$(printf '%s\n' "$line" | sed -n 's/.* \([KMG]\?bits\/sec\).*/\1/p')
[ -n "$val" ] || { echo 0; return; }
awk -v v="$val" -v u="$unit" 'BEGIN{
if (u ~ /^G/) printf "%.1f\n", v * 1000;
else if (u ~ /^M/) printf "%.1f\n", v;
else if (u ~ /^K/) printf "%.1f\n", v / 1000;
else printf "%.3f\n", v / 1000000; }'
}
# `pkill -x iperf3` matches the process *name* only. Never use `pkill -f`
# with a pattern that also appears in this very command line: it kills the
# invoking shell (see the note in 06-rdr.sh).
iperf_up() { srv 'pkill -x iperf3 2>/dev/null; iperf3 -s -D >/dev/null 2>&1; sleep 1; pgrep -x iperf3 >/dev/null || echo IPERF_SERVER_DOWN' >/dev/null; }
# --- phase 0: baseline (module out of the path) ----------------------------
pf0 'rmmod pf 2>/dev/null; true' >/dev/null
iperf_up
BASE=$(iperf_mbit "$SECS")
BASE=$(printf '%s' "$BASE" | cut -d. -f1)
if [ "${BASE:-0}" -gt 0 ]; then
ok "baseline throughput ${BASE} Mbit/s (module unloaded)"
else
bad "baseline throughput unavailable (iperf3 cli->srv failed)"
echo "STRESS phase=baseline baseline_mbit=0 seconds=$SECS parallel=$PAR rate_per_stream=$RATE"
finish
fi
echo "STRESS phase=baseline baseline_mbit=$BASE seconds=$SECS parallel=$PAR rate_per_stream=$RATE"
# --- phase 1: same traffic with PF enabled ---------------------------------
pf0 'insmod /opt/pf/pf.ko 2>/dev/null || insmod /tmp/pf.ko; true' >/dev/null
pf0 '/opt/pf/pfctl -e; /opt/pf/pfctl -f /dev/null' >/dev/null
cat > /tmp/g16.conf <<C
pass in quick proto tcp from $CLI_IP to $SRV_IP keep state
pass in quick proto icmp from $CLI_IP keep state
pass out quick keep state
C
pfload /tmp/g16.conf || { bad "stress ruleset"; finish; }
iperf_up
PFM=$(iperf_mbit "$SECS")
PFM=$(printf '%s' "$PFM" | cut -d. -f1)
[ "${PFM:-0}" -gt 0 ] && ok "throughput with PF enabled: ${PFM} Mbit/s" || bad "no throughput with PF enabled"
OH=0
[ "${BASE:-0}" -gt 0 ] && OH=$(awk -v b="$BASE" -v p="$PFM" 'BEGIN{printf "%d", (b-p)*100/b}')
if [ "${OH:-999}" -le "$MAXOH" ]; then
ok "PF overhead ${OH}% (<= ${MAXOH}%)"
else
bad "PF overhead ${OH}% exceeds ${MAXOH}%"
fi
echo "STRESS phase=overhead pf_mbit=$PFM overhead_pct=$OH limit_pct=$MAXOH rate_per_stream=$RATE"
# --- phase 2: state churn ---------------------------------------------------
# The churn client cannot use sshd: MaxStartups (10:30:100 by default) drops
# most concurrent unauthenticated connections. The server therefore gets a
# dedicated listener (python's http.server forks per connection and holds it
# open until the client closes).
#
# The setup and cleanup scripts are shipped base64-encoded: a plain `$(...)`
# would be expanded by the *intermediate* host (deb), not by the guest, and
# would silently do nothing. The listener also outlives the ssh session, so
# the setup kills leftovers from an earlier run (an orphan holding the port
# would silently break the churn).
SETUP=$(cat <<'EOS' | base64 | tr -d '\n'
mkdir -p /tmp/stress-web
cd /tmp/stress-web || exit 0
echo ok > index.html
for pid in $(ls /proc 2>/dev/null | grep -E '^[0-9]+$'); do
[ "$pid" = "$$" ] && continue
cmd=$(tr '\0' ' ' < /proc/$pid/cmdline 2>/dev/null) || continue
case "$cmd" in
*stress-loop*|*http.server*) kill "$pid" 2>/dev/null ;;
esac
done
if [ -f /tmp/stress-httpd.pid ]; then kill $(cat /tmp/stress-httpd.pid) 2>/dev/null; rm -f /tmp/stress-httpd.pid; fi
setsid sh -c 'echo $$ > /tmp/stress-httpd.pid; exec python3 -m http.server 9097 --bind 0.0.0.0' >/dev/null 2>&1 </dev/null &
sleep 1
EOS
)
srv "echo $SETUP | base64 -d > /tmp/stress-setup.sh; sh /tmp/stress-setup.sh" >/dev/null
if srv 'netstat -lnt 2>/dev/null | grep -q ":9097 "' >/dev/null 2>&1; then
ok "churn listener up on srv:9097"
else
bad "churn listener NOT listening on srv:9097 (stale listener holding the port?)"
finish
fi
RULESET=$(pf0 '/opt/pf/pfctl -s rules | wc -l')
PFSTAT=$(pf0 '/opt/pf/pfctl -s info | head -1')
pf0 "/opt/pf/pfctl -F s" >/dev/null
S0=$(pf0 "$HELPER states")
# The connections must stay open while the states are counted: a peer that
# closes right away tears the state down before it can be observed.
# The iteration list is expanded HERE, on the build host: a `$(seq ...)`
# left in the command string would be expanded by the *intermediate* host
# (deb), whose newline-separated output breaks the `for` loop one hop later.
RANGE=$(seq -s' ' 1 "$CONNS")
cli "bash -c 'for i in $RANGE; do (exec 3<>/dev/tcp/$SRV_IP/9097; sleep $HOLD) 2>/dev/null & done; wait'" >/dev/null 2>&1 &
CLIPID=$!
sleep 4
S1=$(pf0 "$HELPER states")
EST=$(srv 'netstat -tn 2>/dev/null | grep -c ":9097 "')
wait $CLIPID 2>/dev/null
if [ "${S1:-0}" -ge $((CONNS / 2)) ]; then
ok "state churn: $S1 states tracked for $CONNS connections"
else
bad "state churn: only $S1 states for $CONNS connections"
fi
S2=$(pf0 "$HELPER state-reset")
sleep 1
S3=$(pf0 "$HELPER states")
if [ "${S3:-0}" -le "${S0:-0}" ]; then
ok "state table drained after reset ($S0 -> $S3)"
else
bad "state table did not drain ($S0 -> $S3)"
fi
echo "STRESS phase=state_churn connections=$CONNS states_before=$S0 states_created=$S1 states_after_reset=$S3 established_on_srv=${EST:-0} rules_loaded=$RULESET pf_status=$(printf '%s' "$PFSTAT" | tr -s ' ' | cut -d' ' -f2-)"
# --- phase 3: rule churn ---------------------------------------------------
{
echo "table <stress> { }"
echo "pass in quick proto tcp from $CLI_IP to $SRV_IP keep state"
echo "pass out quick keep state"
i=1
while [ "$i" -le "$RULES" ]; do
# a distinct destination port per rule: rules with the same shape
# are folded by pfctl's optimizer into one automatic table, which
# would make the rule count meaningless
echo "block drop in quick proto tcp from 10.100.$((i % 250)).$((i % 250 + 1)) to any port $((1000 + i))"
i=$((i + 1))
done
} > /tmp/g16-churn.conf
CH=$(pfput /tmp/g16-churn.conf) || { bad "churn ruleset upload"; finish; }
pf0 "/opt/pf/pfctl -f $CH" >/dev/null || { bad "churn ruleset load"; finish; }
R0=$(pf0 "$HELPER rules")
RL=$(pf0 "$HELPER reload-loop $RELOADS $CH")
eval "$RL"
if [ "${reload_fails:-1}" -eq 0 ]; then
ok "rule churn: $RELOADS reloads, no failure (avg ${reload_ms_avg:-?} ms, max ${reload_ms_max:-?} ms)"
else
bad "rule churn: ${reload_fails:-?} of $RELOADS reloads failed"
fi
R1=$(pf0 "$HELPER rules")
if [ "${R1:-0}" -eq "${R0:-0}" ] && [ "${R1:-0}" -eq "$((RULES + 2))" ]; then
ok "rule count stable across churn ($R1 rules)"
else
bad "rule count drifted: $R0 -> $R1 (expected $((RULES + 2)))"
fi
echo "STRESS phase=rule_churn reloads=$RELOADS rules_per_load=$RULES reload_fails=${reload_fails:-0} reload_ms_avg=${reload_ms_avg:-0} reload_ms_max=${reload_ms_max:-0} rules_final=$R1"
# --- phase 4: table churn --------------------------------------------------
TC=$(pf0 "$HELPER table-churn $TENTRIES stress")
eval "$TC"
if [ "${table_add_errors:-1}" -eq 0 ] && [ "${table_entries_after_add:-0}" -ge $((TENTRIES / 2)) ]; then
ok "table churn: ${table_entries_after_add} entries added"
else
bad "table churn: ${table_entries_after_add:-0} entries, ${table_add_errors:-?} errors"
fi
if [ "${table_entries_after_flush:-1}" -eq 0 ]; then
ok "table flushed back to 0 entries"
else
bad "table not empty after flush (${table_entries_after_flush})"
fi
echo "STRESS phase=table_churn entries_requested=$TENTRIES entries_added=${table_entries_after_add:-0} add_errors=${table_add_errors:-0} entries_after_flush=${table_entries_after_flush:-0}"
# --- phase 5: integrity under load -----------------------------------------
cat > /tmp/g16-nat.conf <<C
pass out quick on $EGRESS_DEV from $CLI_IP to any nat-to ($EGRESS_DEV)
pass in quick proto tcp from $CLI_IP to $SRV_IP port 9099 keep state
pass out quick keep state
C
pfload /tmp/g16-nat.conf || { bad "nat ruleset"; finish; }
SZ=2000000
srv 'pkill nc 2>/dev/null; head -c 2000000 /dev/zero | tr "\0" A > /tmp/stress.bin; sleep 1; nohup setsid nc -l -p 9099 < /tmp/stress.bin >/dev/null 2>&1 & sleep 1' >/dev/null
SRC_SUM=$(srv 'sha256sum /tmp/stress.bin | cut -c1-64' | tr -d '[:space:]')
# saturating background stream while the payload crosses PF
cli "iperf3 -c $SRV_IP -t 15 -P $PAR -b $RATE >/dev/null 2>&1 &" >/dev/null 2>&1
sleep 1
GOT=$(cli 'nc -w20 10.0.2.20 9099 | sha256sum | cut -c1-64' | tr -d '[:space:]')
if [ -n "$SRC_SUM" ] && [ "$GOT" = "$SRC_SUM" ]; then
ok "2 MB through NAT intact under load (sha256 match)"
INTEG=yes
else
bad "payload corrupted under load (src=$SRC_SUM got=$GOT)"
INTEG=no
fi
echo "STRESS phase=integrity bytes=$SZ sha256_match=$INTEG"
# --- phase 6: health -------------------------------------------------------
ANOM=$(pf0 "$HELPER dmesg")
if [ -z "$ANOM" ]; then
ok "no kernel BUG/WARN/Oops after the stress run"
else
bad "dmesg anomalies:"
printf '%s\n' "$ANOM" | sed 's/^/ /'
fi
MATCHED=$(pf0 "$HELPER counter match")
ENTRIES=$(pf0 "$HELPER state-entries")
[ "${MATCHED:-0}" -gt 0 ] && ok "PF counters moved (match=${MATCHED:-0} state_entries=${ENTRIES:-0})" \
|| bad "PF counters did not move (match=${MATCHED:-0})"
cli "ping -c 3 -W 2 $SRV_IP >/dev/null 2>&1" && ok "firewall still routes traffic after stress" \
|| bad "firewall stopped routing traffic after stress"
echo "STRESS phase=health dmesg_anomalies=$(printf '%s' "$ANOM" | grep -c . ) match=${MATCHED:-0} state_entries=${ENTRIES:-0}"
# --- cleanup ---------------------------------------------------------------
CLEAN=$(cat <<'EOS' | base64 | tr -d '\n'
if [ -f /tmp/stress-httpd.pid ]; then kill $(cat /tmp/stress-httpd.pid) 2>/dev/null; rm -f /tmp/stress-httpd.pid; fi
for pid in $(ls /proc 2>/dev/null | grep -E '^[0-9]+$'); do
[ "$pid" = "$$" ] && continue
cmd=$(tr '\0' ' ' < /proc/$pid/cmdline 2>/dev/null) || continue
case "$cmd" in
*http.server*) kill "$pid" 2>/dev/null ;;
esac
done
pkill -x nc 2>/dev/null
pkill -x iperf3 2>/dev/null
true
EOS
)
srv "echo $CLEAN | base64 -d > /tmp/stress-clean.sh; sh /tmp/stress-clean.sh" >/dev/null
cli 'pkill -x iperf3 2>/dev/null; true' >/dev/null
pf0 '/opt/pf/pfctl -d; /opt/pf/pfctl -f /dev/null' >/dev/null
finish
+101
View File
@@ -0,0 +1,101 @@
#!/bin/sh
# Helper executed ON pf0 by 16-stress.sh (uploaded with pfput to avoid quoting
# hell through two ssh hops). POSIX sh: the guest is Alpine (busybox ash).
#
# $1 = "slab" | "dmesg" | "states" | "state-reset" | "rules" | "info"
# | "reload-loop <n> <ruleset>" | "table-churn <n> <table>"
#
# Every subcommand prints machine-readable KEY=VALUE pairs on stdout so the
# gate can parse them without another ssh round trip.
PFCTL=/opt/pf/pfctl
now_ms() { awk '{printf "%d", $1 * 1000}' /proc/uptime; }
case "${1:-}" in
slab)
grep -E '^(pf|pfk)' /proc/slabinfo | tr -s ' ' | cut -d' ' -f1,3 | sort
;;
dmesg)
# the CPU's Speculative Return Stack Overflow advisory is a kernel
# build/CPU feature notice, not a fault of this module
dmesg | grep -E 'BUG:|WARNING:|Oops|general protection|kernel BUG' \
| grep -v 'Speculative Return Stack Overflow' || true
;;
states)
# `pfctl -s states` prints one line per direction, so count the
# initiator lines ("->") to get the number of states
$PFCTL -s states 2>/dev/null | grep -c ' -> ' || true
;;
state-reset)
# -F s clears the state table (-s s only *shows* it)
$PFCTL -F s >/dev/null 2>&1 || true
$PFCTL -s states 2>/dev/null | grep -c ' -> ' || true
;;
rules)
$PFCTL -s rules 2>/dev/null | grep -c . || true
;;
info)
$PFCTL -s info 2>/dev/null
;;
counter)
# $2 = counter name as printed by `pfctl -s info` (match, short,
# state-mismatch, ...). Prints the value, or 0 when absent.
$PFCTL -s info 2>/dev/null | awk -v k="${2:-match}" '$1 == k { print $2; exit }'
;;
state-entries)
# the number of entries the state table currently holds
$PFCTL -s info 2>/dev/null | awk '/current entries/ { print $NF; exit }'
;;
reload-loop)
# $2 = iterations, $3 = ruleset file. Times each `pfctl -f` on the
# firewall itself (no ssh overhead in the measurement).
n="${2:-0}"; f="${3:-/tmp/g16-churn.conf}"
i=0; fails=0; min=999999999; max=0; sum=0
while [ "$i" -lt "$n" ]; do
s=$(now_ms)
if ! $PFCTL -f "$f" >/dev/null 2>&1; then
fails=$((fails + 1))
fi
e=$(now_ms)
d=$((e - s)); [ "$d" -lt 0 ] && d=0
sum=$((sum + d))
[ "$d" -lt "$min" ] && min=$d
[ "$d" -gt "$max" ] && max=$d
i=$((i + 1))
done
[ "$min" = 999999999 ] && min=0
avg=0; [ "$n" -gt 0 ] && avg=$((sum / n))
echo "reload_iterations=$n reload_fails=$fails reload_ms_min=$min reload_ms_avg=$avg reload_ms_max=$max"
;;
table-churn)
# $2 = number of addresses, $3 = table name. Adds N distinct addresses,
# counts the table, flushes it and counts again.
n="${2:-0}"; t="${3:-stress}"
$PFCTL -t "$t" -T flush >/dev/null 2>&1 || true
i=1; err=0
while [ "$i" -le "$n" ]; do
# 10.0.<a>.<b> keeps every entry distinct without touching the
# lab's real segments (10.0.1.0/24 lan, 10.0.2.0/24 wan)
a=$(( (i - 1) / 250 + 100 ))
b=$(( (i - 1) % 250 + 1 ))
if ! $PFCTL -t "$t" -T add "10.$a.$b" >/dev/null 2>&1; then
err=$((err + 1))
fi
i=$((i + 1))
done
added=$($PFCTL -t "$t" -T show 2>/dev/null | grep -c . || true)
$PFCTL -t "$t" -T flush >/dev/null 2>&1 || true
left=$($PFCTL -t "$t" -T show 2>/dev/null | grep -c . || true)
if [ "${left:-0}" -ne 0 ]; then
# flush did not empty the table: fall back to deleting every entry
$PFCTL -t "$t" -T show 2>/dev/null | awk '{print $1}' | \
while read -r a; do $PFCTL -t "$t" -T delete "$a" >/dev/null 2>&1 || true; done
left=$($PFCTL -t "$t" -T show 2>/dev/null | grep -c . || true)
fi
echo "table_entries_requested=$n table_add_errors=$err table_entries_after_add=$added table_entries_after_flush=$left"
;;
*)
echo "unknown helper command: ${1:-}" >&2
exit 2
;;
esac