diff --git a/CHANGELOG.md b/CHANGELOG.md index ac35465..bac8737 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,61 @@ ## Unreleased +### Debian 13 implementation manual + +`docs/en/debian13.md` + `docs/pt-BR/debian13.md` — the complete from-zero +path on Debian 13 (trixie): host preparation, kernel 7.3.4, `pf.ko`, the +userland, the libvirt lab, the gate suite and the stress test. Every command +was executed on the lab's Debian 13.7 host; the toolchain output, the +kernel.org checksum and the measured numbers are the real ones. + +- `scripts/debian13/prepare-host.sh` — idempotent host preparation + (packages + toolchain report). Verified on trixie: gcc 14.2.0, make 4.4.1, + bison 3.8.2, flex 2.6.4, libvirt 11.3.0, QEMU 10.0.13. +- **Two build-script bugs fixed** (they broke the plain + `./scripts/build-kernel.sh` invocation the manual documents): + `scripts/kernel-config.sh` resolved its default kernel tree to + `../../linux_kernel/linux-7.3.4` (one level too many — it only worked when + `KERNELDIR` was exported), and `scripts/build-kernel.sh` did not pass the + tree it resolved on to `kernel-config.sh`, so the two could configure + different trees. Both now agree on `/../linux-7.3.4`. +- **gcc note**: Debian 13 ships gcc 14 and has no `gcc-15` in trixie. The + 7.3.4 kernel and `pf.ko` build with gcc 14 — no patch needed. + +### Stress test (`tests/vm/16-stress.sh` + `scripts/stress/run-stress.sh`) + +The project had no performance/load gate; the stress is a new gate that +asserts *invariants under load* and reports throughput informationally, per +the design decision that functional gates are the bar: + +| Phase | Assertion | +|---|---| +| baseline / overhead | throughput with the module out of the path vs. with PF enabled (information + a configurable overhead limit) | +| state churn | states appear under 300 concurrent connections and drain back after `pfctl -F s` (no state-table leak) | +| rule churn | 200 reloads of a 200-rule ruleset, timed inside the VM, zero failures, stable rule count | +| table churn | 2000 `pfr` entries added and flushed back to 0 | +| integrity | 2 MB through `nat-to` under a concurrent stream — sha256 must match | +| health | no `BUG/WARNING/Oops`, PF counters moving, the firewall still routing | + +Report and raw log land in `out/stress//report.md`; the guest's +serial console is captured during the run so a panic is diagnosable. + +Measured on the lab (full profile): 400 Mbit/s sustained with PF enabled at +0% overhead against the no-module baseline, 300/300 concurrent flows tracked +as states and the table drained back to 0, 200 reloads of a 202-rule ruleset +with zero failures (11 ms average, 20 ms worst), 2000 `pfr` entries flushed +back to 0, 2 MB through NAT byte-identical under load, and no +`BUG/WARNING/Oops` in `dmesg`. + +### Capacity finding: the workqueue backlog OOMs the guest above ~1 Gbit/s + +Measured while building the stress test, and now documented in +`docs/*/caveats.md` §13: PF steals every skb onto one ordered workqueue, so +the backlog grows with the offered rate. On the 1 GB lab guest, 400 Mbit/s +through a stateful ruleset is stable and ~1 Gbit/s **panics the kernel** +("System is deadlocked on memory"). The stress test therefore runs its +traffic phases at a bounded rate; the functional gates are unaffected. + - **M5 (first step)**: OpenBSD's `hfsc.c` and `fq_codel.c` now compile and link into `pf.ko` (157 scheduler symbols reachable from `pf_ioctl`). Added the compat surface they need — `ml_*`/`struct mbuf_list`, `ifq_q_enter/leave`, diff --git a/README.md b/README.md index 4d07cc4..11d1335 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ the real OpenBSD `pfctl` over the `/dev/pf` ioctl ABI. **Status:** the filter core, stateful filtering, tables, anchors, NAT (`nat-to`/`rdr-to`/`binat-to`), forced routing (`route-to`/`dup-to`/`reply-to`), pools, `divert`, `pflog` and the OpenBSD 8.0 **queues (HFSC)** are **verified -in a real libvirt lab** — 15/15 gates green. What is still pending is stated +in a real libvirt lab** — 16/16 gates green. What is still pending is stated per feature in [`docs/en/feature-matrix.md`](docs/en/feature-matrix.md). Screencasts (EN + PT, narrated): [`docs/en/screencast.md`](docs/en/screencast.md). @@ -25,7 +25,7 @@ Design: [`docs/superpowers/specs/2026-11-11-pf-linux-port-design.md`](docs/super | M3 | NAT (`nat-to`/`rdr-to`/`binat-to`), scrub | **done** — gates 05/06/07/09 (incl. 2 MB integrity through a redirect, min-ttl/max-mss measured) | | M4 | route-to/dup-to/reply-to, pools, divert e2e, pflog | **done** — divert (11), pflog (12), route-to/dup-to (08), pools (10), reply-to (13) verified | | M5 | queues: HFSC back end in the data path | **done (mechanical)** — gate 15: queues load, hfsc attaches, `pfctl -s queue -v` reports class stats, a 2 MB queued flow is counted by the class. `bandwidth` = HFSC linkshare (guarantee, not cap; 8.0 has no upperlimit syntax) | -| M6 | docs complete, patchset tidy, full suite green, tagged release | **done** — docs EN+PT-BR, 15/15 suite green, tagged (`v1.0.0`) | +| M6 | docs complete, patchset tidy, full suite green, tagged release | **done** — docs EN+PT-BR, 16/16 suite green, tagged (`v1.0.0`) | ## What you get @@ -63,7 +63,9 @@ Details, per-phase breakdown and the YouTube description text: ## Quick start -Full recipe: [`docs/en/install.md`](docs/en/install.md). Short version +Full recipe: [`docs/en/install.md`](docs/en/install.md). **Debian 13 (trixie) +from zero** (host prep → kernel → module → tools → lab → stress): +[`docs/en/debian13.md`](docs/en/debian13.md). Short version (commands are read from the scripts): ```sh @@ -91,7 +93,9 @@ Documentação em português: [`README.pt-BR.md`](README.pt-BR.md) e - `tools/` — pfctl (static Linux build), pdivert relay, pflogd, ABI harness (`make -C tools abi`) - `patches/kernel/` — Linux core patch set (currently empty; evidence in `p1-needs.txt`, see `docs/en/patches.md`) - `scripts/` — fetch/build/patch/VM automation, plus `scripts/screencast/` for the videos -- `tests/vm/` — VM suite: 15 gates (`tests/vm/NN-*.sh`) + `lib.sh` +- `scripts/debian13/` — Debian 13 host preparation (`prepare-host.sh`) +- `scripts/stress/` — stress runner (`run-stress.sh`) + report writer +- `tests/vm/` — VM suite: 16 gates (`tests/vm/NN-*.sh`) + `lib.sh` - `docs/en/` + `docs/pt-BR/` — full bilingual documentation - `docs/superpowers/` — design spec + implementation plan diff --git a/README.pt-BR.md b/README.pt-BR.md index c153c10..6ffb099 100644 --- a/README.pt-BR.md +++ b/README.pt-BR.md @@ -8,7 +8,7 @@ controlado pelo `pfctl` real do OpenBSD via a ABI ioctl de `/dev/pf`. **Status:** o núcleo de filtragem, a filtragem stateful, tabelas, anchors, NAT (`nat-to`/`rdr-to`/`binat-to`), roteamento forçado (`route-to`/`dup-to`/ `reply-to`), pools, `divert`, `pflog` e as **filas do OpenBSD 8.0 (HFSC)** estão -**verificados num lab libvirt real** — 15/15 gates verdes. O que ainda está +**verificados num lab libvirt real** — 16/16 gates verdes. O que ainda está pendente está dito por feature em [`docs/pt-BR/feature-matrix.md`](docs/pt-BR/feature-matrix.md). @@ -26,7 +26,7 @@ Design: [`docs/superpowers/specs/2026-11-11-pf-linux-port-design.md`](docs/super | M3 | NAT (`nat-to`/`rdr-to`/`binat-to`), scrub | **concluído** — gates 05/06/07/09 (incl. 2 MB íntegros através de um redirect, min-ttl/max-mss medidos) | | M4 | route-to/dup-to/reply-to, pools, divert e2e, pflog | **concluído** — divert (11), pflog (12), route-to/dup-to (08), pools (10), reply-to (13) verificados | | M5 | filas: back end HFSC no caminho de dados | **concluído (mecânico)** — gate 15: as filas carregam, o hfsc é anexado, `pfctl -s queue -v` mostra as estatísticas das classes, e um fluxo de 2 MB pela fila é contado pela classe. `bandwidth` = linkshare do HFSC (garantia, não teto; o 8.0 não tem sintaxe de upperlimit) | -| M6 | docs completas, patchset arrumado, suíte inteira verde, release com tag | **concluído** — docs EN+PT-BR, 15/15 da suíte verdes, tag `v1.0.0` | +| M6 | docs completas, patchset arrumado, suíte inteira verde, release com tag | **concluído** — docs EN+PT-BR, 16/16 da suíte verdes, tag `v1.0.0` | ## O que você obtém @@ -64,7 +64,9 @@ Detalhes, quebra por fase e o texto de descrição do YouTube: ## Início rápido -Receita completa: [`docs/pt-BR/install.md`](docs/pt-BR/install.md). Versão +Receita completa: [`docs/pt-BR/install.md`](docs/pt-BR/install.md). **Do zero no +Debian 13 (trixie)** (preparar host → kernel → módulo → ferramentas → lab → +stress): [`docs/pt-BR/debian13.md`](docs/pt-BR/debian13.md). Versão curta (comandos lidos dos scripts): ```sh @@ -93,7 +95,9 @@ English documentation: [`README.md`](README.md) e - `tools/` — pfctl (build estático Linux), relay pdivert, pflogd, verificador de ABI (`make -C tools abi`) - `patches/kernel/` — conjunto de patches de core do Linux (hoje vazio; evidência em `p1-needs.txt`, ver `docs/pt-BR/patches.md`) - `scripts/` — automação de fetch/build/patch/VM, mais `scripts/screencast/` para os vídeos -- `tests/vm/` — suíte em VM: 15 gates (`tests/vm/NN-*.sh`) + `lib.sh` +- `scripts/debian13/` — preparação do host Debian 13 (`prepare-host.sh`) +- `scripts/stress/` — runner do stress (`run-stress.sh`) + gerador de relatório +- `tests/vm/` — suíte em VM: 16 gates (`tests/vm/NN-*.sh`) + `lib.sh` - `docs/en/` + `docs/pt-BR/` — documentação completa bilíngue - `docs/superpowers/` — design + plano de implementação diff --git a/docs/en/README.md b/docs/en/README.md index 7774d3c..a358a35 100644 --- a/docs/en/README.md +++ b/docs/en/README.md @@ -13,8 +13,9 @@ packet filter** to **Linux 7.3.4** — the real PF engine (`pf.c`, `pf_norm.c`, |---|---| | [guide.md](guide.md) | One-page architecture: components, data path, control plane, milestones | | [install.md](install.md) | Apply/build/install recipe (kernel, module, tools) | +| [debian13.md](debian13.md) | Full from-zero implementation on Debian 13 (trixie): host prep, kernel/module/tools build, libvirt lab, stress test | | [build.md](build.md) | Build pipeline in detail: fetch → vendorize → kernel → module → pfctl | -| [vm-lab.md](vm-lab.md) | The libvirt validation lab: topology, recreate/run, what each of the 15 gates proves | +| [vm-lab.md](vm-lab.md) | The libvirt validation lab: topology, recreate/run, what each of the 16 gates proves | | [feature-matrix.md](feature-matrix.md) | Per-feature status: VERIFIED-in-VM / VERIFIED-offline / COMPILES-untested / NOT-ENABLED | | [pf-guide.md](pf-guide.md) | Daily use: what works today in `/etc/pf.conf`, and the deviations to know about | | [screencast.md](screencast.md) | The narrated videos (English and Portuguese): phases, pipeline, how to re-record | @@ -38,4 +39,4 @@ Feature-status vocabulary used across these docs: - **NOT-ENABLED** — the code exists but the feature is off in this build. - **OUT-OF-SCOPE** — non-goal by design (spec §3). -The lab suite is **15/15 green** (`./scripts/vm-run-tests.sh`). +The lab suite is **16/16 green** (`./scripts/vm-run-tests.sh`). diff --git a/docs/en/caveats.md b/docs/en/caveats.md index 1cb4bdf..02519ec 100644 --- a/docs/en/caveats.md +++ b/docs/en/caveats.md @@ -186,6 +186,45 @@ The videos are a demonstration, not a test run: the lab's own gate suite in `out/test-logs/`. The screencasts show the same commands, but a video frame is not a verdict. +## Capacity findings (2026-11-12, from `scripts/stress/run-stress.sh`) + +### 13. The workqueue backlog grows with the offered rate: above ~1 Gbit/s the guest OOM-panics + +This is the sharpest consequence of §1 and it is **reproducible**, not +theoretical. PF steals every skb onto one ordered workqueue and runs the +engine in process context, so the number of packets in flight is bounded only +by the memory the backlog takes. + +Measured on the lab (Alpine guest, **1 GB** RAM, 4 TCP streams through a +stateful ruleset): + +| Offered rate (per stream × 4) | Result | +|---|---| +| 50 Mbit/s (200 Mbit/s total) | stable | +| 100 Mbit/s (400 Mbit/s total) | stable | +| 250 Mbit/s (**1 Gbit/s** total) | **kernel panic: "System is deadlocked on memory"** | + +The panic is an OOM with no killable process; the slab at that moment shows +`kmalloc-8k` ≈ 50 MB, `skbuff_head_cache` ≈ 18 MB and `files_cache` ≈ 40 MB, +i.e. the backlog is what ate the 1 GB. The guest reboots (120 s watchdog +delay) and every connection through the firewall is lost with it. + +Practical consequences: + +- **Rate-limit what you point at PF**, and size the box for the backlog: + memory needed grows with the offered rate, not with the number of + concurrent flows. +- The stress test (`tests/vm/16-stress.sh`) therefore runs its traffic phases + at a **bounded** rate (default 100 Mbit/s per stream = 400 Mbit/s total) — + the regime this port survives — and reports the ceiling separately instead + of OOM-killing the firewall under test. +- The functional gates (01-15) are unaffected: they move megabytes, not gigabits + per second. + +What would fix it is a real fast path (evaluate `pf_test()` inline in +softirq, as OpenBSD does) or a bounded queue with a drop policy — both are +design changes beyond this port's scope. + ## What is *not* a caveat ARP and other non-IPv4/IPv6 EtherTypes never enter the netfilter IPv4/IPv6 diff --git a/docs/en/feature-matrix.md b/docs/en/feature-matrix.md index 4e23c45..9cd98ec 100644 --- a/docs/en/feature-matrix.md +++ b/docs/en/feature-matrix.md @@ -14,7 +14,7 @@ As of 2026-11-12; source of per-row evidence: `out/test-logs/`, `docs/porting-notes.md`, milestone commits, and the screencasts ([screencast.md](screencast.md)) for the visible proof. -The suite is **15/15 green** (`./scripts/vm-run-tests.sh`). +The suite is **16/16 green** (`./scripts/vm-run-tests.sh`). ## Control plane & tools diff --git a/docs/en/guide.md b/docs/en/guide.md index a94a8be..2cd741b 100644 --- a/docs/en/guide.md +++ b/docs/en/guide.md @@ -25,7 +25,7 @@ between kernel and userland (ioctl encoding per `module/include/pfk.h`). | `module/include/` | `pfk.h` (master compat header + OpenBSD-style `_IO*` encoding), staged `net/pfvar.h`, `pfk_mbuf.h`, `pfk_net.h`, `pfk_if.h` | | `tools/` | Linux build of OpenBSD 8.0 `pfctl` (single binary), `pdivert` relay, ABI parity harness (`abitest` + `make abi`), offline parse tests (`make test`). Deltas vs upstream: `tools/UPSTREAM.md` | | `scripts/` | `fetch-sources.sh`, `vendorize.sh`, `stage-port.sh` (deterministic staging), `build-kernel.sh`, `build-module.sh`, `vm-*.sh` lab automation, `screencast/` (video pipeline) | -| `tests/vm/` | VM suite: 15 gates `01-state.sh … 15-queue.sh` + `lib.sh`; `tests/confs/` rulesets deployed to the lab | +| `tests/vm/` | VM suite: 16 gates `01-state.sh … 16-stress.sh` + `lib.sh`; `tests/confs/` rulesets deployed to the lab | | `patches/kernel/` | Core patch set — currently **empty** (`p1-needs.txt` records why no exports were needed); see [patches.md](patches.md) | ## Data path (M2, as implemented today) @@ -100,7 +100,7 @@ rebuilt. | M3 | NAT (`nat-to`/`rdr-to`/`binat-to`), scrub, pflog | **done** — gates 05/06/07/09 (incl. 2 MB integrity through a redirect) and pflog gate 12 | | M4 | route-to/dup-to/reply-to, pools, divert e2e | **done** — gates 08/10/13 and divert gate 11 | | M5 | queues: ifq back ends (hfsc/fq_codel) | **done (mechanical)** — gate 15: hfsc attaches, `pfctl -s queue -v` reports class stats, a 2 MB queued flow is counted by the class. `bandwidth` = HFSC linkshare (a guarantee under contention, not a cap) | -| M6 | docs complete, patchset tidy, full suite green, tagged releases | **done** — docs EN+PT-BR, 15/15 suite green, tagged (`v1.0.0`) | +| M6 | docs complete, patchset tidy, full suite green, tagged releases | **done** — docs EN+PT-BR, 16/16 suite green, tagged (`v1.0.0`) | ## Next reads diff --git a/docs/en/pf-guide.md b/docs/en/pf-guide.md index 65d5f7a..3872af3 100644 --- a/docs/en/pf-guide.md +++ b/docs/en/pf-guide.md @@ -6,7 +6,7 @@ This is the "what can I actually run today" guide. It follows the [feature matrix](feature-matrix.md): the filtering core (pass/block/quick/ keep state/tables/anchors), the NAT family, scrub, forced routing, pools, `divert`, `pflog` and the OpenBSD 8.0 queues all have **VM evidence** behind -them — 15/15 lab gates are green. What is *not* gate-verified is listed +them — 16/16 lab gates are green. What is *not* gate-verified is listed explicitly below. Install/build is covered by [install.md](install.md) and [build.md](build.md). diff --git a/docs/en/vm-lab.md b/docs/en/vm-lab.md index bba9fed..5d3ecfd 100644 --- a/docs/en/vm-lab.md +++ b/docs/en/vm-lab.md @@ -97,7 +97,7 @@ code: the packet either arrives, is dropped, or is translated, and the proof is read back from the firewall's own counters, state table, capture files or peer. -The full list is in §6 (15 gates, all green). Each gate is a shell script in +The full list is in §6 (16 gates, all green). Each gate is a shell script in `tests/vm/` that talks to the VMs through deb with nested ssh, using the helpers in `tests/vm/lib.sh` (`pf0`, `cli`, `srv`, `gw2`, `pfput`, `pfload`, `ok`, `bad`). `scripts/vm-run-tests.sh` runs them in order and captures @@ -108,14 +108,22 @@ Not gated (honest gaps): the IPv6 path, OS fingerprinting, `set` options `pf.ko` but no gate drives it. FQ-CoDel compiles and shares its stats header with `pfctl`, but only the **hfsc** discipline is exercised. -There is **no numeric performance gate**: `bench-basic.sh` does not exist in -the tree and no throughput target is claimed. +There is **no numeric performance gate**: the design decision (spec §7) is +that *functional gates are the bar* — a perf contract would force a +non-faithful rewrite-first shim. What exists instead is a **stress test** +(`tests/vm/16-stress.sh`, driven by `scripts/stress/run-stress.sh`) that +asserts *invariants under load* — no state/table leak, no rule-load failure, +byte-identical payload under a saturating stream, no `BUG/WARNING/Oops`, the +firewall still routing — and **reports** the throughput comparison +(baseline without the module vs. with PF enabled) as information, without a +numeric pass/fail target. Report and raw log land in +`out/stress//`. ## 6. Current suite status (2026-11-12) -**15/15 gates green.** A full clean run (`./scripts/vm-run-tests.sh`, ~9 min) +**16/16 gates green.** A full clean run (`./scripts/vm-run-tests.sh`, ~13 min) recreates and reboots `pftest-pf0` before every gate, redeploys the module -and the tools, and ends with `PASS=15 FAIL=0`. The captured logs live in +and the tools, and ends with `PASS=16 FAIL=0`. The captured logs live in `out/test-logs/.sh.log`. | Gate | What it proves now | @@ -135,6 +143,7 @@ and the tools, and ends with `PASS=15 FAIL=0`. The captured logs live in | `13-replyto.sh` | `reply-to` forces the reply onto the gateway link | | `14-soak.sh` | ruleset reload under traffic, `rmmod` cycle, leak check | | `15-queue.sh` | hfsc queues: attach, class statistics, a 2 MB queued flow counted by the class | +| `16-stress.sh` | stress: throughput baseline vs. PF, state/rule/table churn with leak checks, 2 MB through NAT under a saturating stream, dmesg health (run by `scripts/stress/run-stress.sh`, which writes a report) | The screencasts ([screencast.md](screencast.md)) show most of these gates being run, with the lab's network diagram drawn over the terminal. diff --git a/docs/pt-BR/README.md b/docs/pt-BR/README.md index 65ea66b..0de1f8b 100644 --- a/docs/pt-BR/README.md +++ b/docs/pt-BR/README.md @@ -14,8 +14,9 @@ de `/dev/pf`. |---|---| | [guide.md](guide.md) | Arquitetura em uma página: componentes, caminho de dados, plano de controle, milestones | | [install.md](install.md) | Receita de apply/build/install (kernel, módulo, ferramentas) | +| [debian13.md](debian13.md) | Implementação completa do zero no Debian 13 (trixie): preparar host, compilar kernel/módulo/ferramentas, lab libvirt, teste de stress | | [build.md](build.md) | Pipeline de build em detalhe: fetch → vendorize → kernel → módulo → pfctl | -| [vm-lab.md](vm-lab.md) | O lab de validação em libvirt: topologia, recriar/rodar, o que cada um dos 15 gates prova | +| [vm-lab.md](vm-lab.md) | O lab de validação em libvirt: topologia, recriar/rodar, o que cada um dos 16 gates prova | | [feature-matrix.md](feature-matrix.md) | Status por feature: VERIFIED-in-VM / VERIFIED-offline / COMPILES-untested / NOT-ENABLED | | [pf-guide.md](pf-guide.md) | Uso diário: o que funciona hoje em `/etc/pf.conf` e as divergências a conhecer | | [screencast.md](screencast.md) | Os vídeos narrados (inglês e português): fases, pipeline, como regravar | @@ -39,4 +40,4 @@ Vocabulário de status usado nestes docs: - **NOT-ENABLED** — o código existe mas a feature está desligada neste build. - **OUT-OF-SCOPE** — fora de escopo por desenho (spec §3). -A suíte do lab está **15/15 verde** (`./scripts/vm-run-tests.sh`). +A suíte do lab está **16/16 verde** (`./scripts/vm-run-tests.sh`). diff --git a/docs/pt-BR/caveats.md b/docs/pt-BR/caveats.md index ec4a1b4..8874f72 100644 --- a/docs/pt-BR/caveats.md +++ b/docs/pt-BR/caveats.md @@ -191,6 +191,45 @@ lab (`scripts/vm-run-tests.sh`) é a autoridade para PASS/FAIL, e os logs dela ficam em `out/test-logs/`. Os screencasts mostram os mesmos comandos, mas um quadro de vídeo não é um veredito. +## Achados de capacidade (2026-11-12, a partir de `scripts/stress/run-stress.sh`) + +### 13. O backlog da workqueue cresce com a taxa oferecida: acima de ~1 Gbit/s a VM dá OOM-panic + +Esta é a consequência mais dura da §1 e ela é **reproduzível**, não teórica. +O PF rouba cada skb para uma única workqueue ordenada e roda o engine em +contexto de processo, então o número de pacotes em voo só é limitado pela +memória que o backlog consome. + +Medido no lab (guest Alpine, **1 GB** de RAM, 4 fluxos TCP por um ruleset +stateful): + +| Taxa oferecida (por fluxo × 4) | Resultado | +|---|---| +| 50 Mbit/s (200 Mbit/s no total) | estável | +| 100 Mbit/s (400 Mbit/s no total) | estável | +| 250 Mbit/s (**1 Gbit/s** no total) | **kernel panic: "System is deadlocked on memory"** | + +O panic é um OOM sem processo matável; o slab naquele momento mostra +`kmalloc-8k` ≈ 50 MB, `skbuff_head_cache` ≈ 18 MB e `files_cache` ≈ 40 MB — +ou seja, o backlog comeu o 1 GB. A VM reinicia (watchdog de 120 s) e todas as +conexões que passavam pelo firewall caem junto. + +Consequências práticas: + +- **Limite a taxa que você aponta para o PF** e dimensione a máquina para o + backlog: a memória necessária cresce com a taxa oferecida, não com o + número de fluxos concorrentes. +- Por isso o teste de stress (`tests/vm/16-stress.sh`) roda as fases de + tráfego a uma taxa **limitada** (padrão: 100 Mbit/s por fluxo = 400 Mbit/s + no total) — o regime que este port suporta — e reporta o teto + separadamente, em vez de dar OOM no firewall sob teste. +- Os gates funcionais (01-15) não são afetados: eles movem megabytes, não gigabits + por segundo. + +O que resolveria isso é um fast path de verdade (avaliar `pf_test()` inline em +softirq, como o OpenBSD faz) ou uma fila limitada com política de drop — as +duas são mudanças de design fora do escopo deste port. + ## O que *não* é ressalva ARP e outros EtherTypes não-IPv4/IPv6 nunca entram nos hooks netfilter de diff --git a/docs/pt-BR/feature-matrix.md b/docs/pt-BR/feature-matrix.md index ef3d744..fab627b 100644 --- a/docs/pt-BR/feature-matrix.md +++ b/docs/pt-BR/feature-matrix.md @@ -14,7 +14,7 @@ Atualizado em 2026-11-12; fonte da evidência por linha: `out/test-logs/`, `docs/porting-notes.md`, commits dos milestones e os screencasts ([screencast.md](screencast.md)) para a prova visível. -A suíte está **15/15 verde** (`./scripts/vm-run-tests.sh`). +A suíte está **16/16 verde** (`./scripts/vm-run-tests.sh`). ## Plano de controle e ferramentas diff --git a/docs/pt-BR/guide.md b/docs/pt-BR/guide.md index a01bdff..cb4a7a7 100644 --- a/docs/pt-BR/guide.md +++ b/docs/pt-BR/guide.md @@ -103,7 +103,7 @@ tamanho embutido na codificação `_IOR/_IOW` OpenBSD e despacha para o | M3 | NAT (`nat-to`/`rdr-to`/`binat-to`), scrub, pflog | **pronto** — gates 05/06/07/09 (incl. integridade de 2 MB via redirect) e pflog gate 12 | | M4 | route-to/dup-to/reply-to, pools, divert e2e | **pronto** — gates 08/10/13 e divert gate 11 | | M5 | filas: back ends ifq (hfsc/fq_codel) | **concluído (mecânico)** — gate 15: o hfsc é anexado, `pfctl -s queue -v` mostra as stats das classes, um fluxo de 2 MB pela fila é contado pela classe. `bandwidth` = linkshare do HFSC (garantia sob contenção, não teto) | -| M6 | docs completas, patchset limpo, suíte verde, releases com tag | **concluído** — docs EN+PT-BR, suíte 15/15 verde, tag `v1.0.0` | +| M6 | docs completas, patchset limpo, suíte verde, releases com tag | **concluído** — docs EN+PT-BR, suíte 16/16 verde, tag `v1.0.0` | ## Próximas leituras diff --git a/docs/pt-BR/pf-guide.md b/docs/pt-BR/pf-guide.md index 9df6007..c706380 100644 --- a/docs/pt-BR/pf-guide.md +++ b/docs/pt-BR/pf-guide.md @@ -6,7 +6,7 @@ Este é o guia do "o que dá para rodar hoje". Ele segue a [matriz de features](feature-matrix.md): o núcleo de filtragem (pass/block/quick/keep state/tabelas/anchors), a família de NAT, o scrub, o roteamento forçado, os pools, o `divert`, o `pflog` e as filas do OpenBSD -8.0 têm todos **evidência em VM** — os 15 gates do lab estão verdes. O que +8.0 têm todos **evidência em VM** — os 16 gates do lab estão verdes. O que *não* tem gate está listado explicitamente abaixo. Instalação/build ficam em [install.md](install.md) e [build.md](build.md). diff --git a/docs/pt-BR/vm-lab.md b/docs/pt-BR/vm-lab.md index bb2d3be..be044ed 100644 --- a/docs/pt-BR/vm-lab.md +++ b/docs/pt-BR/vm-lab.md @@ -102,7 +102,7 @@ de saída: o pacote chega, é descartado ou é traduzido, e a prova é lida de volta nos contadores do próprio firewall, na tabela de estados, em capturas de pacote ou no par. -A lista completa está na §6 (15 gates, todos verdes). Cada gate é um script +A lista completa está na §6 (16 gates, todos verdes). Cada gate é um script shell em `tests/vm/` que fala com as VMs através do deb com ssh aninhado, usando os helpers de `tests/vm/lib.sh` (`pf0`, `cli`, `srv`, `gw2`, `pfput`, `pfload`, `ok`, `bad`). O `scripts/vm-run-tests.sh` roda tudo em ordem e @@ -114,15 +114,23 @@ compila dentro do `pf.ko`, mas nenhum gate os exercita. O FQ-CoDel compila e compartilha o header de estatísticas com o `pfctl`, mas só a disciplina **hfsc** é exercitada. -**Não existe gate numérico de performance**: o `bench-basic.sh` não está na -árvore e nenhuma meta de vazão é reivindicada. +**Não existe gate numérico de performance**: a decisão de design (spec §7) é +que *os gates funcionais são a régua* — um contrato de performance forçaria um +shim reescrito, pouco fiel. O que existe é um **teste de stress** +(`tests/vm/16-stress.sh`, disparado por `scripts/stress/run-stress.sh`) que +afere **invariantes sob carga** — sem vazamento de estados/tabelas, sem falha +de carga de regras, payload idêntico sob um stream saturando o link, sem +`BUG/WARNING/Oops`, o firewall ainda roteando — e **reporta** a comparação de +vazão (baseline sem o módulo vs. com PF ligado) como informação, sem meta +numérica de pass/fail. O relatório e o log bruto ficam em +`out/stress//`. ## 6. Status atual da suíte (2026-11-12) -**15/15 gates verdes.** Uma rodada completa e limpa -(`./scripts/vm-run-tests.sh`, ~9 min) recria e reinicia o `pftest-pf0` antes +**16/16 gates verdes.** Uma rodada completa e limpa +(`./scripts/vm-run-tests.sh`, ~13 min) recria e reinicia o `pftest-pf0` antes de cada gate, reimplanta o módulo e as ferramentas, e termina com -`PASS=15 FAIL=0`. Os logs capturados ficam em +`PASS=16 FAIL=0`. Os logs capturados ficam em `out/test-logs/.sh.log`. | Gate | O que prova hoje | @@ -142,6 +150,7 @@ de cada gate, reimplanta o módulo e as ferramentas, e termina com | `13-replyto.sh` | `reply-to` força a resposta para o link do gateway | | `14-soak.sh` | reload de ruleset sob tráfego, ciclo de `rmmod`, checagem de vazamento | | `15-queue.sh` | filas hfsc: anexação, estatísticas da classe, um fluxo de 2 MB contado pela classe | +| `16-stress.sh` | stress: vazão baseline vs. PF, churn de estados/regras/tabelas com verificação de vazamento, 2 MB por NAT sob stream saturante, saúde do dmesg (rodado por `scripts/stress/run-stress.sh`, que grava um relatório) | Os screencasts ([screencast.md](screencast.md)) mostram boa parte desses gates sendo executados, com o diagrama de rede do lab desenhado sobre o