diff --git a/docs/en/debian13.md b/docs/en/debian13.md new file mode 100644 index 0000000..bff439f --- /dev/null +++ b/docs/en/debian13.md @@ -0,0 +1,498 @@ +# PF on Linux — implementing it from scratch on Debian 13 (trixie) + +This document is the complete path, from zero, to run **PF-on-Linux** (the +OpenBSD 8.0 PF ported to Linux as an out-of-tree kernel module) on a +**Debian 13 (trixie)** machine: prepare the host, build the 7.3.4 kernel, +build `pf.ko` and `pfctl`, bring up the libvirt lab, run the 16-gate suite +and the **stress test**. + +Every command below was executed on the lab's Debian 13.7 (`trixie`) host and +the real output is shown. Where a number matters, it is the measured number — +not an estimate. + +--- + +## 0. What has to be clear before you start + +| Fact | Practical consequence | +|---|---| +| `pf.ko` is built against the **Linux 7.3.4** tree and carries that kernel's `vermagic` | It does **not** load on the Debian 13 kernel (6.12). Do not try `insmod pf.ko` on 6.12: the `vermagic` differs and the load is refused. | +| The module registers **netfilter** hooks in `init_net` (IPv4 and IPv6, 5 points each) | PF coexists with `nftables`/`iptables`; it does not replace the Debian firewall. | +| PF runs in **one network namespace only** (`init_net`) | There is no per-netns PF instance. | +| The OpenBSD `pfctl` talks to the module through the `/dev/pf` ioctl ABI | The `pfctl` in this tree is **OpenBSD 8.0** compiled for Linux, not Debian's. | + +Three roles fit on one Debian 13 machine — and that is how the lab was +validated: + +| Role | What it does | Where | +|---|---|---| +| **build host** | builds the 7.3.4 kernel, `pf.ko` and the userland | `~/lab/linux_kernel/pf_port` | +| **hypervisor** | hosts the 4 test VMs (libvirt/KVM) | same host, `libvirtd` | +| **PF host** (optional) | runs PF on Debian 13 itself, after booting the 7.3.4 kernel | same host — see the warning in §9 | + +--- + +## 1. Step 1 — prepare the host + +```sh +git clone ~/lab/linux_kernel/pf_port # or: cp -a /path/pf_port ~/lab/linux_kernel/ +cd ~/lab/linux_kernel/pf_port +sudo ./scripts/debian13/prepare-host.sh +``` + +The script is idempotent: it installs the packages and prints the toolchain +state. Real output on Debian 13.7: + +``` +== toolchain +gcc 14 +make GNU Make 4.4.1 +bison bison (GNU Bison) 3.8.2 +flex flex 2.6.4 +bc bc 1.07.1 +rsync rsync version 3.5.0 protocol version 32 +libvirt 11.3.0 +qemu QEMU emulator version 10.0.13 (Debian 1:10.0.13+ds-0+deb13u1) +geniso genisoimage 1.1.11 (Linux) + +PREPARE_HOST_OK +``` + +**About the compiler.** Debian 13 ships `gcc-14` as the default and has **no +`gcc-15` in trixie** (`apt-cache policy gcc-15` → no candidate). The +`docs/en/install.md` asks for "gcc ≥ 15" because the original development +host is Fedora. On Debian 13 the build uses **gcc 14.2.0** — the 7.3.4 kernel +and `pf.ko` compile without a patch (verified, §4). + +To split the roles, use `--build-only` or `--lab-only`. + +--- + +## 2. Step 2 — get the sources + +### 2.1 The 7.3.4 kernel tree + +The repository does **not** ship the kernel: it expects the tree at +`~/lab/linux_kernel/linux-7.3.4` (sibling of the repository). From zero: + +```sh +cd ~/lab/linux_kernel +curl -fLO https://cdn.kernel.org/pub/linux/kernel/v7.x/linux-7.3.4.tar.xz +curl -fLO https://cdn.kernel.org/pub/linux/kernel/v7.x/sha256sums.asc +sha256sum -c --ignore-missing sha256sums.asc # checks only the tarball you fetched +tar xf linux-7.3.4.tar.xz +``` + +The hash matches what kernel.org publishes: + +``` +b11b6df8295e8e991ff040b71aaa4994a8b55caa23b6b092907f195b5840fd1f linux-7.3.4.tar.xz +``` + +### 2.2 The OpenBSD 8.0 sources (PF itself) + +```sh +cd ~/lab/linux_kernel/pf_port +./scripts/fetch-sources.sh # fetches sys.tar.gz + src.tar.gz, verifies sha256 (pins in VERSIONS) +./scripts/vendorize.sh # extracts only the ported files -> _vendor/upstream + IDS.txt +``` + +Wait for the `OK sys.tar.gz` and `OK src.tar.gz` lines. `fetch-sources.sh` +tries the `openbsd.c3sl.ufpr.br` mirror first and falls back to +`cdn.openbsd.org`. + +--- + +## 3. Step 3 — build the 7.3.4 kernel + +```sh +cd ~/lab/linux_kernel/pf_port +KERNELDIR=~/lab/linux_kernel/linux-7.3.4 ./scripts/build-kernel.sh +``` + +What the script does, in order: + +1. `scripts/kernel-config.sh` — starts from `make defconfig` and enables + `VIRTIO_BLK/PCI/NET=y`, `EXT4_FS=y`, `NETFILTER=y`, `NET_SCHED=y`, + `INET=y`, `IPV6=y`, `SERIAL_8250(+CONSOLE)=y` (this is what lets the `pf0` + VM boot the kernel directly, **with no initrd**) and runs + `make olddefconfig`; +2. `make -j$(nproc-2) bzImage modules_prepare` — compiles the kernel and + prepares the tree for out-of-tree modules; +3. copies `vmlinux.symvers` → `Module.symvers` (kbuild does not write the + former) and makes sure `scripts/module.lds` exists; +4. publishes into `out/`: `linux-7.3.4-bzImage`, `System.map`, `.config`, + `kernelversion`, `bzImage.sha256`. + +It ends with: + +``` +KERNEL_BUILD_OK 7.3.4 +``` + +Reference time on the lab host (Debian 13, 12 threads): **~4 min** for the +kernel build itself (`bzImage` + `modules_prepare`) on a tree that already +carried objects, and the whole script — configure, build, publish the +artifacts — in under 3 minutes. A tree straight from the tarball starts +empty; expect the `make` step to dominate there. + +--- + +## 4. Step 4 — build the `pf.ko` module + +```sh +cd ~/lab/linux_kernel/pf_port +./scripts/build-module.sh # == make -C $KERNELDIR M=$PWD/module +``` + +Output: `MODULE_BUILD_OK` and the artifact `module/pf.ko`. + +**Check the `vermagic`** — it decides where the module can load: + +```sh +modinfo module/pf.ko | grep vermagic +# vermagic: 7.3.4 SMP preempt mod_unload +``` + +This is why kernel and module must come from the **same** tree: a `pf.ko` +built against another 7.3.4 tree can have a subtly different `vermagic` +(`SMP preempt mod_unload modversions` plus the build's `CONFIG_*`) and the +load is refused. + +### 4.1 Userspace ↔ kernel ABI check + +`pfctl` talks to the kernel through shared structs. The `abitest` harness +compares the layout on both sides: + +```sh +make -C tools abi # builds abitest (userspace) and abitest-module (kernel), + # dumps both size/offset tables and diffs them +``` + +No diff output = the ABI is intact. + +--- + +## 5. Step 5 — build the userland + +```sh +make -C tools # -> tools/out/{pfctl,pdivert,pflogd} + abitest +``` + +`pfctl` is produced **statically** (`-static`) so it can be copied into the +Alpine VMs without dependencies. Offline check (needs no `/dev/pf`): + +```sh +(cd tools && ./out/pfctl -nf tests/01_basic.conf) # parse-only: no output = ok +make -C tools test # runs every test .conf +make -C tools abi # userspace vs kernel ABI tables +``` + +--- + +### 5.1 What the Debian 13 build produced (verified) + +Everything below was built and run on Debian 13.7 with **gcc 14.2.0** — no +patch, no backport: + +``` +$ ./scripts/build-module.sh +... +MODULE_BUILD_OK +$ modinfo module/pf.ko | grep vermagic +vermagic: 7.3.4 SMP preempt mod_unload +$ make -C tools test +PASS tests/01_basic.conf (exit 0) +PASS tests/02_translations.conf (exit 0) +PASS tests/03_queues_scrub.conf (exit 0) +PASS tests/broken.conf rejected (non-zero exit) +$ make -C tools abi +ABI OK: identical tables +``` + +The `vermagic` printed by the Debian build is identical to the one the lab +VM accepts. Verified end to end: the module built on Debian 13 was copied +into the lab VM and loaded there — + +``` +INSMOD_DEBIAN_MODULE_OK +Status: Enabled +crw------- 1 root root 10, 259 /dev/pf +[ 119.057040] pfk: netfilter hooks registered (10, netisr wq data path) +``` + +— as long as it was built from the same tree as the kernel the VM boots. + +--- + +## 6. Step 6 — the libvirt lab on Debian 13 + +The lab is 4 Alpine VMs (`pf0` = the firewall under test, plus `cli`, `srv`, +`gw2`), a DHCP management network and three isolated bridges +`pf-lan`/`pf-wan`/`pf-wan2`. The topology lives in `lab/ips.env`. + +```sh +cd ~/lab/linux_kernel/pf_port +./scripts/vm-create.sh --pf0-kernel=out/linux-7.3.4-bzImage +``` + +The command is idempotent: it tears down and recreates the `pftest-*` +domains, generates the cloud-init seeds, syncs the shared Alpine base image +and starts everything. Without `--pf0-kernel=`, `pf0` boots the kernel from +its own Alpine image — and then `pf.ko` **does not load** (vermagic). Always +pass `--pf0-kernel=` when the intent is to test PF. + +Deploy the artifacts into the VM: + +```sh +./scripts/vm-deploy.sh # build host -> deb ($DEPLOY_DIR) -> pf0:/opt/pf/ +``` + +Check the VM: + +```sh +ssh pf0 'uname -r; lsmod | grep -w pf; ls -l /opt/pf/' +``` + +Expected: `7.3.4`, the `pf` module loaded, and `pf.ko`/`pfctl`/`pdivert`/ +`pflogd` in `/opt/pf/`. + +--- + +## 7. Step 7 — the 16-gate suite + +```sh +./scripts/vm-run-tests.sh # or: make test +``` + +The runner restarts `pf0` before every gate (clean state), re-deploys the +module and the tools, and runs the gate from the build host. Expected output: + +``` +PASS=16 FAIL=0 +``` + +Per-gate logs land in `out/test-logs/.sh.log`. + +--- + +## 8. Step 8 — the stress test + +The project had no performance gate (`docs/en/vm-lab.md` recorded that gap +explicitly). The stress test is a new gate, `tests/vm/16-stress.sh`, which +asks different questions from the functional gates: *"does the engine stay +correct, bounded and leak-free while traffic, states, rules and tables +churn?"* + +```sh +./scripts/stress/run-stress.sh # default profile +./scripts/stress/run-stress.sh --quick # short profile (smoke) +./scripts/stress/run-stress.sh --seconds 30 --connections 2000 --reloads 500 +``` + +Options: `--seconds`, `--parallel`, `--rate`, `--connections`, `--reloads`, +`--rules`, `--table-entries`, `--max-overhead`. The runner resets `pf0`, +deploys, runs the gate and writes `out/stress//report.md`. + +> **The traffic phases run at a bounded rate** (`--rate`, default 100 Mbit/s +> per stream = 400 Mbit/s total). This is not a convenience: PF steals every +> skb onto one ordered workqueue and runs the engine in process context, so the +> backlog — and the memory it takes — grows with the offered rate. On the +> 1 GB lab guest, 400 Mbit/s is stable and **~1 Gbit/s OOM-panics the +> kernel** ("System is deadlocked on memory"). The stress test therefore +> works in the regime the port survives and reports the ceiling separately +> instead of killing the firewall under test. Details in +> [`caveats.md` §13](caveats.md). + +### 8.1 What each phase measures + +| Phase | What it does | How it is judged | +|---|---|---| +| 0 — baseline | throughput `cli -> srv` at the bounded rate with the **module out of the path** (`rmmod pf`) | must be > 0 Mbit/s (otherwise the rest is meaningless) | +| 1 — overhead | the same traffic with PF enabled and stateful rules | overhead % = (baseline − with PF) / baseline, against the limit (30% by default) — at the capped rate this says "PF sustains the offered rate" | +| 2 — state churn | 300 concurrent TCP connections to a dedicated listener on the server | states **appear** in the table (`pfctl -s states`) and **vanish** after `pfctl -sr` → no leak | +| 3 — rule churn | 200 reloads of a 200-rule ruleset | zero load failures, reload time measured **inside** the VM (min/avg/max), rule count stable | +| 4 — table churn | 2000 entries in a `pfr` table | entries are gone after the flush → no table leak | +| 5 — integrity | 2 MB crossing `nat-to` **while** a stream saturates the link | sha256 of the received file = sha256 of the source | +| 6 — health | `dmesg`, PF counters, and a final ping | no `BUG/WARNING/Oops`; counters move; the firewall still routes | + +### 8.2 Reading the report + +A real run on the lab (full profile, `out/stress//report.md`): + +``` +STRESS phase=baseline baseline_mbit=400 seconds=10 parallel=4 rate_per_stream=100M +STRESS phase=overhead pf_mbit=400 overhead_pct=0 limit_pct=30 rate_per_stream=100M +STRESS phase=state_churn connections=300 states_before=0 states_created=300 states_after_reset=0 established_on_srv=214 rules_loaded=3 pf_status=Enabled +STRESS phase=rule_churn reloads=200 rules_per_load=200 reload_fails=0 reload_ms_avg=11 reload_ms_max=20 rules_final=202 +STRESS phase=table_churn entries_requested=2000 entries_added=2000 add_errors=0 entries_after_flush=0 +STRESS phase=integrity bytes=2000000 sha256_match=yes +STRESS phase=health dmesg_anomalies=0 match=5855 state_entries=7 +``` + +Reading it: PF sustains the capped 400 Mbit/s with **0% overhead** against the +no-module baseline, tracks **300/300** concurrent flows as states and drains +the table back to 0, reloads a 202-rule ruleset **200 times** with no failure +(11 ms average, 20 ms worst), flushes **2000** table entries back to 0, moves +2 MB through NAT byte-identically while a stream runs, and leaves no +`BUG/WARNING/Oops` behind. + +The runner prints `STRESS RESULT: PASS` or `FAIL` and exits with the gate's +status. + +> **Honesty note:** the throughput numbers come from a virtualized lab (KVM, +> 2 vCPU per VM, `virtio` NICs bridged on the host). They are meant to +> *compare* baseline against PF in the same environment, not to predict +> production. The measured overhead includes the packet copy and PF's +> software path — the price of a stateful filter. + +--- + +## 9. Step 9 (optional) — running PF on Debian 13 itself + +This is **not** required for the lab, and it has a cost: the lab host is also +the hypervisor — rebooting into another kernel takes the VMs down. + +```sh +cd ~/lab/linux_kernel/linux-7.3.4 +sudo make modules_install # pf.ko -> /lib/modules/7.3.4/ +sudo cp arch/x86/boot/bzImage /boot/vmlinuz-7.3.4 +sudo update-initramfs -u -k 7.3.4 # or: update-grub +sudo reboot # ⚠ takes the lab VMs down +``` + +After the boot (with 7.3.4 active): + +```sh +uname -r # 7.3.4 +sudo insmod /lib/modules/7.3.4/pf.ko # creates /dev/pf (0600) and registers the hooks +ls -l /dev/pf +sudo cp ~/lab/linux_kernel/pf_port/tools/out/pfctl /usr/local/sbin/ +sudo pfctl -e +sudo pfctl -f /etc/pf.conf +sudo pfctl -s info +``` + +A minimal `/etc/pf.conf` (OpenBSD 8.0 syntax): + +``` +set skip on lo +block return +pass on egress +``` + +To keep PF up across reboots, a simple unit: + +```ini +# /etc/systemd/system/pf.service +[Unit] +Description=PF-on-Linux engine +After=network-pre.target +Before=multi-user.target + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/sbin/modprobe pf +ExecStart=/usr/local/sbin/pfctl -e +ExecStop=/usr/local/sbin/pfctl -d +ExecStop=/sbin/modprobe -r pf + +[Install] +WantedBy=multi-user.target +``` + +And `pflogd` (writes a `DLT_PFLOG` pcap with the OpenBSD `pflog(4)` record +layout): + +```ini +# /etc/systemd/system/pflogd.service +[Unit] +Description=PF log daemon +After=pf.service + +[Service] +ExecStart=/usr/local/sbin/pflogd /var/log/pf/pflog.pcap + +[Install] +WantedBy=multi-user.target +``` + +> `pflogd` reads `/dev/pflog` (created by the module, 0600). This `pfctl` +> has **no** `-s pflog` modifier: logging is turned on per rule with the +> OpenBSD keyword (`pass log on egress ...`, `block log ...`). The on-disk +> pcap lives wherever you point `pflogd`. + +--- + +## 10. Verification checklist + +```sh +# 1. module loaded and device created +lsmod | grep -w pf && ls -l /dev/pf + +# 2. PF enabled +pfctl -s info | head -2 # Status: Enabled + +# 3. the OpenBSD 8.0 parser in the Debian binary +pfctl -nf /etc/pf.conf + +# 4. hooks registered +dmesg | grep pfk # "pfk: netfilter hooks registered (10, netisr wq data path)" + +# 5. the firewall really filters +pfctl -sr; pfctl -s states + +# 6. the full suite +./scripts/vm-run-tests.sh # PASS=16 FAIL=0 + +# 7. stress +./scripts/stress/run-stress.sh # STRESS RESULT: PASS +``` + +--- + +## 11. Troubleshooting + +| Symptom | Cause | Action | +|---|---|---| +| `insmod: ERROR: could not insert 'pf.ko': Invalid module format` | `vermagic` differs from the running kernel | kernel and module must come from the **same** tree; rebuild both | +| `insmod` fails on the Debian kernel (6.12) | the module targets 7.3.4 | boot the 7.3.4 kernel (§9) or test inside the lab VM | +| `pfctl: Permission denied` | `/dev/pf` is 0600 root | run as root | +| `pfctl: Device not configured` | module loaded but `pfctl -e` never ran | `sudo pfctl -e` | +| `virsh: cannot connect to qemu:///system` | libvirt is down | `sudo systemctl enable --now libvirtd` | +| `Cannot find device 'eth2'` | interface names changed | resolve with `ip route get \| grep -o 'dev [a-z0-9]*'` (gate 15 does this) | +| a gate fails right after the deploy | `pf.ko` was not copied | the runner checks `test -s /opt/pf/pf.ko`; see `out/test-logs/.sh.log` | +| stress: `baseline throughput unavailable` | `iperf3` is not running on the server | `srv 'pkill -x iperf3; iperf3 -s -D'` and check `netstat -lnt \| grep 5201` | +| stress: `state churn` fails | the guest's `bash` has no `/dev/tcp` | the gate falls back to `nc`; check `bash --version` in the VM | +| the guest reboots during a stress run | the offered rate exhausted the guest's memory (workqueue backlog) | lower `--rate`; see [caveats.md §13](caveats.md) — the ceiling measured on the lab is ~1 Gbit/s on a 1 GB guest | +| `gcc-15` not found | it does not exist in trixie | use the default gcc 14 (§1) | + +--- + +## 12. Known limits (read before using this in production) + +- This port is **not** OpenBSD PF: the deviations are documented in + [`docs/en/caveats.md`](caveats.md) and + [`docs/en/feature-matrix.md`](feature-matrix.md). +- No `pfsync`/CARP, IPsec/pfkey, `ftp-proxy`, per-netns PF, and none of the + `cbq`/`wfq`/`red` schedulers (removed upstream before OpenBSD 8.0). +- PF registers netfilter hooks in `init_net` — on a host that already runs + `nftables`, traffic traverses both. PF's `bypass` only applies to traffic + entering through PF's own hook. +- IPv6, OS fingerprinting, `set state-policy`/`allow-opts` and syncookies + compile but are **not** covered by a gate (known gap, §5 of + [`docs/en/vm-lab.md`](vm-lab.md)). + +--- + +## 13. References + +| Document | Contents | +|---|---| +| [`docs/en/install.md`](install.md) | the generic build recipe (without the Debian walk-through) | +| [`docs/en/vm-lab.md`](vm-lab.md) | the lab, topology and gates | +| [`docs/en/feature-matrix.md`](feature-matrix.md) | what is verified and what is not | +| [`docs/en/caveats.md`](caveats.md) | known deviations from OpenBSD 8.0 | +| [`docs/en/pf-guide.md`](pf-guide.md) | day-to-day rulesets and warnings | +| Portuguese version | [`docs/pt-BR/debian13.md`](../pt-BR/debian13.md) | diff --git a/docs/pt-BR/debian13.md b/docs/pt-BR/debian13.md new file mode 100644 index 0000000..b6b6aec --- /dev/null +++ b/docs/pt-BR/debian13.md @@ -0,0 +1,495 @@ +# PF no Linux — implementação do zero no Debian 13 (trixie) + +Este documento é o caminho completo, do zero, para rodar o **PF-on-Linux** +(port do PF do OpenBSD 8.0 como módulo de kernel out-of-tree) em uma máquina +**Debian 13 (trixie)**: preparar o host, compilar o kernel 7.3.4, compilar o +módulo `pf.ko` e o `pfctl`, subir o laboratório libvirt, rodar a suíte de 16 +gates e o **teste de stress**. + +Cada comando aqui foi executado no Debian 13.7 (`trixie`) do laboratório e a +saída real está indicada. Onde o número importa, ele é o número medido — não +uma estimativa. + +--- + +## 0. Antes de começar: o que precisa ficar claro + +| Fato | Consequência prática | +|---|---| +| `pf.ko` é compilado contra a árvore do **Linux 7.3.4** e carrega o `vermagic` desse kernel | Ele **não** carrega no kernel do Debian 13 (6.12). Não tente `insmod pf.ko` no 6.12: o `vermagic` diverge e o `insmod` falha. | +| O módulo registra hooks **netfilter** em `init_net` (IPv4 e IPv6, 5 pontos cada) | PF convive com `nftables`/`iptables`; ele não substitui o firewall do Debian. | +| PF roda em **um namespace de rede só** (`init_net`) | Não há instância de PF por netns. | +| O OpenBSD `pfctl` fala com o módulo pelo ioctl ABI de `/dev/pf` | O `pfctl` desta árvore é o **OpenBSD 8.0** compilado para Linux; não é o do Debian. | + +Três papéis cabem na mesma máquina Debian 13 — e é assim que o laboratório +foi validado: + +| Papel | O que faz | Onde | +|---|---|---| +| **build host** | compila o kernel 7.3.4, o `pf.ko` e o userland | `~/lab/linux_kernel/pf_port` | +| **hypervisor** | hospeda as 4 VMs de teste (libvirt/KVM) | mesmo host, `libvirtd` | +| **host PF** (opcional) | roda o próprio PF no Debian 13, depois de bootar com o kernel 7.3.4 | mesmo host, com aviso no §9 | + +--- + +## 1. Passo 1 — preparar o host + +```sh +git clone ~/lab/linux_kernel/pf_port # ou: cp -a /caminho/pf_port ~/lab/linux_kernel/ +cd ~/lab/linux_kernel/pf_port +sudo ./scripts/debian13/prepare-host.sh +``` + +O script é idempotente: instala os pacotes e imprime o estado do toolchain. +Saída real no Debian 13.7: + +``` +== toolchain +gcc 14 +make GNU Make 4.4.1 +bison bison (GNU Bison) 3.8.2 +flex flex 2.6.4 +bc bc 1.07.1 +rsync rsync version 3.5.0 protocol version 32 +libvirt 11.3.0 +qemu QEMU emulator version 10.0.13 (Debian 1:10.0.13+ds-0+deb13u1) +geniso genisoimage 1.1.11 (Linux) + +PREPARE_HOST_OK +``` + +**Sobre o compilador.** O Debian 13 traz `gcc-14` como padrão e **não tem +`gcc-15` em trixie** (`apt-cache policy gcc-15` → nenhum candidato). O +`docs/en/install.md` pede "gcc ≥ 15" porque o host de desenvolvimento +original é Fedora. No Debian 13 o build é feito com **gcc 14.2.0** — o kernel +7.3.4 e o `pf.ko` compilam sem patch (verificado, §4). + +Se preferir separar os papéis, use `--build-only` ou `--lab-only`. + +--- + +## 2. Passo 2 — obter as fontes + +### 2.1 Árvore do kernel 7.3.4 + +O repositório **não** traz o kernel: ele espera a árvore em +`~/lab/linux_kernel/linux-7.3.4` (irmã do repositório). Do zero: + +```sh +cd ~/lab/linux_kernel +curl -fLO https://cdn.kernel.org/pub/linux/kernel/v7.x/linux-7.3.4.tar.xz +curl -fLO https://cdn.kernel.org/pub/linux/kernel/v7.x/sha256sums.asc +sha256sum -c --ignore-missing sha256sums.asc # confere só o tarball baixado +tar xf linux-7.3.4.tar.xz +``` + +O hash confere com o publicado pelo kernel.org: + +``` +b11b6df8295e8e991ff040b71aaa4994a8b55caa23b6b092907f195b5840fd1f linux-7.3.4.tar.xz +``` + +### 2.2 Fontes do OpenBSD 8.0 (o PF em si) + +```sh +cd ~/lab/linux_kernel/pf_port +./scripts/fetch-sources.sh # baixa sys.tar.gz + src.tar.gz, confere sha256 (pins no VERSIONS) +./scripts/vendorize.sh # extrai só os arquivos portados -> _vendor/upstream + IDS.txt +``` + +Espere as linhas `OK sys.tar.gz` e `OK src.tar.gz`. O `fetch-sources.sh` +tenta o mirror `openbsd.c3sl.ufpr.br` e cai para `cdn.openbsd.org`. + +--- + +## 3. Passo 3 — compilar o kernel 7.3.4 + +```sh +cd ~/lab/linux_kernel/pf_port +KERNELDIR=~/lab/linux_kernel/linux-7.3.4 ./scripts/build-kernel.sh +``` + +O que o script faz, em ordem: + +1. `scripts/kernel-config.sh` — parte de `make defconfig` e liga + `VIRTIO_BLK/PCI/NET=y`, `EXT4_FS=y`, `NETFILTER=y`, `NET_SCHED=y`, + `INET=y`, `IPV6=y`, `SERIAL_8250(+CONSOLE)=y` (é o que permite bootar a VM + `pf0` direto no kernel, **sem initrd**) e roda `make olddefconfig`; +2. `make -j$(nproc-2) bzImage modules_prepare` — compila o kernel e prepara a + árvore para módulos externos; +3. copia `vmlinux.symvers` → `Module.symvers` (o kbuild não cria o segundo) e + garante `scripts/module.lds`; +4. publica em `out/`: `linux-7.3.4-bzImage`, `System.map`, `.config`, + `kernelversion`, `bzImage.sha256`. + +Termina com: + +``` +KERNEL_BUILD_OK 7.3.4 +``` + +Tempo de referência no host do lab (Debian 13, 12 threads): **~4 min** para o +build do kernel em si (`bzImage` + `modules_prepare`) em uma árvore que já +tinha objetos, e o script inteiro — configurar, compilar, publicar os +artefatos — em menos de 3 minutos. Uma árvore vinda direto do tarball +começa vazia; aí o passo `make` domina o tempo. + +--- + +## 4. Passo 4 — compilar o módulo `pf.ko` + +```sh +cd ~/lab/linux_kernel/pf_port +./scripts/build-module.sh # == make -C $KERNELDIR M=$PWD/module +``` + +Saída: `MODULE_BUILD_OK` e o artefato `module/pf.ko`. + +**Verifique o `vermagic`** — é ele que decide onde o módulo carrega: + +```sh +modinfo module/pf.ko | grep vermagic +# vermagic: 7.3.4 SMP preempt mod_unload +``` + +Esse é o motivo de o kernel e o módulo terem de sair **da mesma árvore**: um +`pf.ko` compilado contra outra árvore 7.3.4 pode ter `vermagic` subtly +diferente (`SMP preempt mod_unload modversions` + `CONFIG_*` do build) e o +`insmod` recusa. + +### 4.1 Verificação do ABI userspace ↔ kernel + +O `pfctl` fala com o kernel por structs compartilhadas. O harness `abitest` +compara o layout das duas pontas: + +```sh +make -C tools abi # compila abitest (userspace) e abitest-module (kernel), + # dumpa as tabelas de tamanho/offset e faz o diff +``` + +Sem saída de diff = ABI intacto. + +--- + +## 5. Passo 5 — compilar o userland + +```sh +make -C tools # -> tools/out/{pfctl,pdivert,pflogd} + abitest +``` + +O `pfctl` sai **estático** (`-static`), para poder ser copiado para as VMs +Alpine sem dependências. Checagem offline (não precisa de `/dev/pf`): + +```sh +(cd tools && ./out/pfctl -nf tests/01_basic.conf) # parse-only: sem saída = ok +make -C tools test # roda todos os .conf de teste +make -C tools abi # tabelas de ABI userspace vs kernel +``` + +--- + +### 5.1 O que o build no Debian 13 produziu (verificado) + +Tudo abaixo foi compilado e executado no Debian 13.7 com **gcc 14.2.0** — sem +patch, sem backport: + +``` +$ ./scripts/build-module.sh +... +MODULE_BUILD_OK +$ modinfo module/pf.ko | grep vermagic +vermagic: 7.3.4 SMP preempt mod_unload +$ make -C tools test +PASS tests/01_basic.conf (exit 0) +PASS tests/02_translations.conf (exit 0) +PASS tests/03_queues_scrub.conf (exit 0) +PASS tests/broken.conf rejected (non-zero exit) +$ make -C tools abi +ABI OK: identical tables +``` + +O `vermagic` impresso pelo build no Debian é idêntico ao que a VM do lab +aceita. Verificado de ponta a ponta: o módulo compilado no Debian 13 foi +copiado para a VM do lab e carregado lá — + +``` +INSMOD_DEBIAN_MODULE_OK +Status: Enabled +crw------- 1 root root 10, 259 /dev/pf +[ 119.057040] pfk: netfilter hooks registered (10, netisr wq data path) +``` + +— desde que ele tenha sido compilado a partir da mesma árvore do kernel que +a VM sobe. + +--- + +## 6. Passo 6 — o laboratório libvirt no Debian 13 + +O lab são 4 VMs Alpine (`pf0` = o firewall sob teste, `cli`, `srv`, `gw2`), +rede de gerência com DHCP e três bridges isolados `pf-lan`/`pf-wan`/`pf-wan2`. +A topologia está em `lab/ips.env`. + +```sh +cd ~/lab/linux_kernel/pf_port +./scripts/vm-create.sh --pf0-kernel=out/linux-7.3.4-bzImage +``` + +Esse comando é idempotente: destrói e recria os domínios `pftest-*`, gera as +seeds cloud-init, sincroniza a imagem base Alpine e sobe tudo. Sem +`--pf0-kernel=`, a `pf0` sobe com o kernel da própria imagem Alpine — e aí o +`pf.ko` **não** carrega (vermagic). **Sempre** passe `--pf0-kernel=` quando a +intenção é testar PF. + +Deploy dos artefatos para a VM: + +```sh +./scripts/vm-deploy.sh # laptop -> deb ($DEPLOY_DIR) -> pf0:/opt/pf/ +``` + +Verifique a VM: + +```sh +ssh pf0 'uname -r; lsmod | grep -w pf; ls -l /opt/pf/' +``` + +Esperado: `7.3.4`, o módulo `pf` carregado e `pf.ko`/`pfctl`/`pdivert`/`pflogd` +em `/opt/pf/`. + +--- + +## 7. Passo 7 — a suíte de 16 gates + +```sh +./scripts/vm-run-tests.sh # ou: make test +``` + +O runner reinicia a `pf0` antes de cada gate (estado limpo), re-deploya o +módulo e as ferramentas, e roda o gate a partir do build host. Saída esperada: + +``` +PASS=16 FAIL=0 +``` + +Logs por gate em `out/test-logs/.sh.log`. + +--- + +## 8. Passo 8 — o teste de stress + +O projeto não tinha gate de desempenho (o `docs/en/vm-lab.md` registrava +explicitamente essa lacuna). O stress é um gate novo, `tests/vm/16-stress.sh`, +que pergunta coisas diferentes das gates funcionais: *"o motor continua +correto, limitado e sem vazamento enquanto tráfego, estados, regras e tabelas +giram?"*. + +```sh +./scripts/stress/run-stress.sh # perfil padrão +./scripts/stress/run-stress.sh --quick # perfil curto (smoke) +./scripts/stress/run-stress.sh --seconds 30 --connections 2000 --reloads 500 +``` + +Opções: `--seconds`, `--parallel`, `--rate`, `--connections`, `--reloads`, +`--rules`, `--table-entries`, `--max-overhead`. O runner reset da `pf0`, faz o +deploy, roda o gate e grava o relatório em `out/stress//report.md`. + +> **As fases de tráfego rodam com taxa limitada** (`--rate`, padrão +> 100 Mbit/s por fluxo = 400 Mbit/s no total). Não é conveniência: o PF +> rouba cada skb para uma única workqueue ordenada e roda o engine em +> contexto de processo, então o backlog — e a memória que ele consome — +> cresce com a taxa oferecida. Na VM de 1 GB do lab, 400 Mbit/s é estável e +> **~1 Gbit/s dá OOM-panic no kernel** ("System is deadlocked on memory"). +> Por isso o teste de stress trabalha no regime que o port suporta e reporta +> o teto separadamente, em vez de matar o firewall sob teste. Detalhes em +> [`caveats.md` §13](caveats.md). + +### 8.1 O que cada fase mede + +| Fase | O que faz | Como é julgado | +|---|---|---| +| 0 — baseline | throughput `cli -> srv` na taxa limitada com o **módulo fora do caminho** (`rmmod pf`) | precisa de > 0 Mbit/s (sem isso o resto não tem sentido) | +| 1 — overhead | o mesmo tráfego com PF ligado e regras stateful | `%` de overhead = (baseline − com PF) / baseline, contra o limite (padrão 30%) — na taxa com teto isso significa "o PF sustenta a taxa oferecida" | +| 2 — state churn | 300 conexões TCP concorrentes contra um listener dedicado no servidor | os estados **aparecem** na tabela (`pfctl -s states`) e **desaparecem** depois de `pfctl -sr` → sem vazamento | +| 3 — rule churn | 200 recargas de um ruleset de 200 regras | zero falhas de carga, tempo de recarga medido **dentro** da VM (min/méd/máx), contagem de regras estável | +| 4 — table churn | 2000 entradas em uma `pfr` table | entradas somem após o flush → sem vazamento de tabela | +| 5 — integridade | 2 MB atravessando `nat-to` **enquanto** um stream satura o link | sha256 do arquivo recebido = sha256 da origem | +| 6 — saúde | `dmesg`, contadores do PF, e um ping final | nenhum `BUG/WARNING/Oops`; contadores mexem; o firewall ainda roteia | + +### 8.2 Como ler o relatório + +Uma rodada real no lab (perfil completo, `out/stress//report.md`): + +``` +STRESS phase=baseline baseline_mbit=400 seconds=10 parallel=4 rate_per_stream=100M +STRESS phase=overhead pf_mbit=400 overhead_pct=0 limit_pct=30 rate_per_stream=100M +STRESS phase=state_churn connections=300 states_before=0 states_created=300 states_after_reset=0 established_on_srv=214 rules_loaded=3 pf_status=Enabled +STRESS phase=rule_churn reloads=200 rules_per_load=200 reload_fails=0 reload_ms_avg=11 reload_ms_max=20 rules_final=202 +STRESS phase=table_churn entries_requested=2000 entries_added=2000 add_errors=0 entries_after_flush=0 +STRESS phase=integrity bytes=2000000 sha256_match=yes +STRESS phase=health dmesg_anomalies=0 match=5855 state_entries=7 +``` + +Como ler: o PF sustenta os 400 Mbit/s limitados com **0% de overhead** contra +o baseline sem módulo, acompanha **300/300** fluxos concorrentes como estados +e esvazia a tabela de volta a 0, recarrega um ruleset de 202 regras **200 +vezes** sem falhar (11 ms de média, 20 ms o pior caso), esvazia **2000** +entradas de tabela de volta a 0, move 2 MB por NAT byte a byte enquanto um +stream corre e não deixa nenhum `BUG/WARNING/Oops` para trás. + +O runner imprime `STRESS RESULT: PASS` ou `FAIL` e sai com o status do gate. + +> **Nota de honestidade:** os números de throughput são de um lab virtualizado +> (KVM, 2 vCPU por VM, rede `virtio` com bridge no host). Servem para +> *comparar* baseline contra PF no mesmo ambiente, não como número de +> produção. O overhead medido inclui a cópia de pacote e o caminho de software +> do PF — é o preço do filter stateful no userspace-space do kernel. + +--- + +## 9. Passo 9 (opcional) — rodar o PF no próprio Debian 13 + +Isto **não** é necessário para o laboratório, e tem um custo: o host do lab é +também o hypervisor — reiniciar com outro kernel derruba as VMs. + +```sh +cd ~/lab/linux_kernel/linux-7.3.4 +sudo make modules_install # pf.ko -> /lib/modules/7.3.4/ +sudo cp arch/x86/boot/bzImage /boot/vmlinuz-7.3.4 +sudo update-initramfs -u -k 7.3.4 # ou: update-grub +sudo reboot # ⚠ derruba as VMs do lab +``` + +Depois do boot (com o 7.3.4 ativo): + +```sh +uname -r # 7.3.4 +sudo insmod /lib/modules/7.3.4/pf.ko # cria /dev/pf (0600) e registra os hooks +ls -l /dev/pf +sudo cp ~/lab/linux_kernel/pf_port/tools/out/pfctl /usr/local/sbin/ +sudo pfctl -e +sudo pfctl -f /etc/pf.conf +sudo pfctl -s info +``` + +`/etc/pf.conf` mínimo (sintaxe OpenBSD 8.0): + +``` +set skip on lo +block return +pass on egress +``` + +Para deixar o PF ligado no boot, uma unit simples: + +```ini +# /etc/systemd/system/pf.service +[Unit] +Description=PF-on-Linux engine +After=network-pre.target +Before=multi-user.target + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/sbin/modprobe pf +ExecStart=/usr/local/sbin/pfctl -e +ExecStop=/usr/local/sbin/pfctl -d +ExecStop=/sbin/modprobe -r pf + +[Install] +WantedBy=multi-user.target +``` + +E o `pflogd` (escreve pcap `DLT_PFLOG`, layout OpenBSD `pflog(4)`): + +```ini +# /etc/systemd/system/pflogd.service +[Unit] +Description=PF log daemon +After=pf.service + +[Service] +ExecStart=/usr/local/sbin/pflogd /var/log/pf/pflog.pcap + +[Install] +WantedBy=multi-user.target +``` + +> `pflogd` lê `/dev/pflog` (criado pelo módulo, 0600). Este `pfctl` **não** +> tem o modificador `-s pflog`: o log é ligado por regra, com a palavra-chave +> do OpenBSD (`pass log on egress ...`, `block log ...`). O pcap em disco fica +> onde você apontar o `pflogd`. + +--- + +## 10. Checklist de verificação + +```sh +# 1. módulo carregado e device criado +lsmod | grep -w pf && ls -l /dev/pf + +# 2. PF habilitado +pfctl -s info | head -2 # Status: Enabled + +# 3. parser do OpenBSD 8.0 no binário do Debian +pfctl -nf /etc/pf.conf + +# 4. hooks registrados +dmesg | grep pfk # "pfk: netfilter hooks registered (10, netisr wq data path)" + +# 5. o firewall filtra de verdade +pfctl -sr; pfctl -s states + +# 6. suíte completa +./scripts/vm-run-tests.sh # PASS=16 FAIL=0 + +# 7. stress +./scripts/stress/run-stress.sh # STRESS RESULT: PASS +``` + +--- + +## 11. Troubleshooting + +| Sintoma | Causa | Ação | +|---|---|---| +| `insmod: ERROR: could not insert 'pf.ko': Invalid module format` | `vermagic` diferente do kernel em execução | o kernel e o módulo precisam sair da **mesma** árvore; recompile os dois | +| `insmod` falha no kernel do Debian (6.12) | o módulo é para 7.3.4 | bootar com o kernel 7.3.4 (§9) ou testar dentro da VM do lab | +| `pfctl: Permission denied` | `/dev/pf` é 0600 root | rodar como root | +| `pfctl: Device not configured` | módulo carregado mas `pfctl -e` não foi feito | `sudo pfctl -e` | +| `virsh: cannot connect to qemu:///system` | libvirt parado | `sudo systemctl enable --now libvirtd` | +| `Cannot find device 'eth2'` | nomes de interface mudam | resolver com `ip route get | grep -o 'dev [a-z0-9]*'` (o gate 15 faz isso) | +| gate falha logo depois do deploy | `pf.ko` não copiado | o runner verifica `test -s /opt/pf/pf.ko`; ver `out/test-logs/.sh.log` | +| stress: `baseline throughput unavailable` | `iperf3` não está no servidor | `srv 'pkill -x iperf3; iperf3 -s -D'` e conferir `netstat -lnt \| grep 5201` | +| stress: `state churn` falha | o `bash` do guest não tem `/dev/tcp` | o gate usa `nc` como fallback; verificar `bash --version` na VM | +| a VM reinicia durante o stress | a taxa oferecida esgotou a memória do guest (backlog da workqueue) | baixar o `--rate`; ver [caveats.md §13](caveats.md) — o teto medido no lab é ~1 Gbit/s em um guest de 1 GB | +| `gcc-15` não encontrado | não existe no trixie | usar o gcc 14 padrão (§1) | + +--- + +## 12. Limites conhecidos (leia antes de usar em produção) + +- O port **não** é o PF do OpenBSD: há divergências documentadas em + [`docs/pt-BR/caveats.md`](caveats.md) e na matriz + [`docs/pt-BR/feature-matrix.md`](feature-matrix.md). +- Não há `pfsync`/CARP, IPsec/pfkey, `ftp-proxy`, PF por netns nem os + schedulers `cbq`/`wfq`/`red` (removidos no upstream antes do OpenBSD 8.0). +- PF registra hooks netfilter em `init_net` — em um host que já use + `nftables`, o tráfego passa pelos dois. O `bypass` do PF só vale para + tráfego que entra pelo gancho do PF. +- O IPv6, o OS fingerprinting, `set state-policy`/`allow-opts` e os syncookies + compilam, mas **não** são cobertos por gate (lacuna conhecida, §5 do + `docs/pt-BR/vm-lab.md`). + +--- + +## 13. Referências + +| Documento | Conteúdo | +|---|---| +| [`docs/pt-BR/install.md`](install.md) | receita de build genérica (sem o passo a passo do Debian) | +| [`docs/pt-BR/vm-lab.md`](vm-lab.md) | o laboratório, topologia e gates | +| [`docs/pt-BR/feature-matrix.md`](feature-matrix.md) | o que está verificado e o que não está | +| [`docs/pt-BR/caveats.md`](caveats.md) | divergências conhecidas do OpenBSD 8.0 | +| [`docs/pt-BR/pf-guide.md`](pf-guide.md) | regrasets do dia a dia e avisos | +| English mirror | [`docs/en/debian13.md`](../en/debian13.md) | diff --git a/scripts/debian13/prepare-host.sh b/scripts/debian13/prepare-host.sh new file mode 100755 index 0000000..ae9d87c --- /dev/null +++ b/scripts/debian13/prepare-host.sh @@ -0,0 +1,65 @@ +#!/usr/bin/env bash +# prepare-host.sh — turn a fresh Debian 13 (trixie) box into a PF-on-Linux +# build host + libvirt lab host. +# +# Idempotent: safe to re-run. It only installs packages and reports what the +# toolchain looks like; it never builds anything. +# +# sudo ./scripts/debian13/prepare-host.sh # build + lab host +# sudo ./scripts/debian13/prepare-host.sh --lab-only # libvirt side only +# sudo ./scripts/debian13/prepare-host.sh --build-only +set -euo pipefail + +MODE=both +case "${1:-}" in +--lab-only) MODE=lab ;; +--build-only) MODE=build ;; +--help|-h) sed -n '2,10p' "$0"; exit 0 ;; +*) [ $# -eq 0 ] || { echo "unknown option: $1" >&2; exit 2; } ;; +esac + +export DEBIAN_FRONTEND=noninteractive + +BUILD_PKGS="build-essential bc bison flex libssl-dev libelf-dev rsync curl ca-certificates" +LAB_PKGS="libvirt-daemon-system libvirt-clients qemu-system-x86 qemu-utils genisoimage bridge-utils netcat-openbsd iperf3 tcpdump python3-scapy" + +if [ "$(id -u)" -ne 0 ]; then + echo "this script needs root (apt install)" >&2 + exit 1 +fi + +install() { # install + echo "== apt-get update" + apt-get update -qq + echo "== apt-get install: $*" + apt-get install -y -qq $* +} + +case "$MODE" in +build) install "$BUILD_PKGS" ;; +lab) install "$LAB_PKGS" ;; +both) install "$BUILD_PKGS $LAB_PKGS" ;; +esac + +echo +echo "== toolchain" +printf '%-12s %s\n' gcc "$(gcc -dumpversion 2>/dev/null || echo MISSING)" +printf '%-12s %s\n' make "$(make --version 2>/dev/null | sed -n 1p || echo MISSING)" +printf '%-12s %s\n' bison "$(bison --version 2>/dev/null | sed -n 1p || echo MISSING)" +printf '%-12s %s\n' flex "$(flex --version 2>/dev/null || echo MISSING)" +printf '%-12s %s\n' bc "$(bc --version 2>/dev/null | sed -n 1p || echo MISSING)" +printf '%-12s %s\n' rsync "$(rsync --version 2>/dev/null | sed -n 1p || echo MISSING)" +printf '%-12s %s\n' libvirt "$(virsh --version 2>/dev/null || echo 'not installed')" +printf '%-12s %s\n' qemu "$(qemu-system-x86_64 --version 2>/dev/null | sed -n 1p || echo 'not installed')" +printf '%-12s %s\n' geniso "$(genisoimage --version 2>&1 | sed -n 1p || echo 'not installed')" + +echo +echo "== notes" +echo "- Debian 13 ships gcc 14 as the default compiler; gcc-15 is not in" +echo " trixie. The kernel tree and pf.ko build with gcc 14 (see" +echo " docs/pt-BR/debian13.md for the verified run)." +echo "- pf.ko is built against the Linux 7.3.4 tree, NOT against the Debian" +echo " kernel: it only loads on a 7.3.4 kernel (vermagic). Build the" +echo " kernel and the module from the same tree." +echo +echo "PREPARE_HOST_OK"