Docs: bring the documentation in line with the code
- Demo docs replaced by the screencast docs (single source of truth): the legacy demo-session.sh/demo-record.py and the v0.9 renderer were removed instead of being kept as a second convention. - feature-matrix: stale 'M2-in-progress' vocabulary gone; the rows the 15 gates prove now say VERIFIED-in-VM and name the gate; af-to is now an explicit non-goal row (INET6 0 in this build). - guide: the queues/divert/pflog sections described a state that no longer exists (priq-only, divert dropping packets, pflog logging nothing); they now describe the shipped behaviour. - vm-lab: the 15 real gates with what each proves, 15/15 green, and the pf0 -> cli/srv/gw2 ssh setup the screencasts rely on. - install/build: pdivert and pflogd are built and ported (docs said otherwise); module object list now includes hfsc.c, fq_codel.c, pfk_pflog.c; the screencast pipeline is an optional build stage. - pf-guide: the 'M2 scope only' framing is gone; honest gaps listed. - caveats: three new sections about the screencast pipeline (MiniMax video generation blocked by the account tier, narration as a separate track, a video frame is not a verdict). - porting-notes: what the recording taught us. - README/README.pt-BR: status, milestones, layout and video section updated. All internal doc links verified.
This commit is contained in:
1 parent
cc67531602
commit
dff2185b20
37 files changed
+677
-1616
No files matched your search
+14
-9
@@ -4,8 +4,8 @@ Documentação em português: [`docs/pt-BR/README.md`](../pt-BR/README.md).
|
||||
|
||||
This is the user documentation set for the port of the **OpenBSD 8.0 PF
|
||||
packet filter** to **Linux 7.3.4** — the real PF engine (`pf.c`, `pf_norm.c`,
|
||||
`pf_table.c`, `pf_lb.c`, …) running as the out-of-tree module `pf.ko`, driven
|
||||
by the real OpenBSD `pfctl` over the `/dev/pf` ioctl ABI.
|
||||
`pf_table.c`, `pf_lb.c`, `hfsc.c`, …) running as the out-of-tree module
|
||||
`pf.ko`, driven by the real OpenBSD `pfctl` over the `/dev/pf` ioctl ABI.
|
||||
|
||||
## Index
|
||||
|
||||
@@ -14,9 +14,10 @@ by the real OpenBSD `pfctl` over the `/dev/pf` ioctl ABI.
|
||||
| [guide.md](guide.md) | One-page architecture: components, data path, control plane, milestones |
|
||||
| [install.md](install.md) | Apply/build/install recipe (kernel, module, tools) |
|
||||
| [build.md](build.md) | Build pipeline in detail: fetch → vendorize → kernel → module → pfctl |
|
||||
| [vm-lab.md](vm-lab.md) | The libvirt validation lab: topology, recreate/run, what each gate proves |
|
||||
| [feature-matrix.md](feature-matrix.md) | Per-feature status: VERIFIED-in-VM / COMPILES-untested / PENDING-Mx |
|
||||
| [pf-guide.md](pf-guide.md) | Daily use: what works today in `/etc/pf.conf`, what to avoid until M3/M4 |
|
||||
| [vm-lab.md](vm-lab.md) | The libvirt validation lab: topology, recreate/run, what each of the 15 gates proves |
|
||||
| [feature-matrix.md](feature-matrix.md) | Per-feature status: VERIFIED-in-VM / VERIFIED-offline / COMPILES-untested / NOT-ENABLED |
|
||||
| [pf-guide.md](pf-guide.md) | Daily use: what works today in `/etc/pf.conf`, and the deviations to know about |
|
||||
| [screencast.md](screencast.md) | The narrated videos (English and Portuguese): phases, pipeline, how to re-record |
|
||||
| [patches.md](patches.md) | Linux core patch set (currently empty) + rules and template for future `000N-*.patch` |
|
||||
| [caveats.md](caveats.md) | Known divergences from OpenBSD behavior and current limitations |
|
||||
|
||||
@@ -27,10 +28,14 @@ authoritative references are:
|
||||
|
||||
- `docs/superpowers/specs/2026-11-11-pf-linux-port-design.md` — design
|
||||
- `docs/superpowers/plans/2026-11-11-pf-linux-port.md` — milestone plan (tasks T1–T19)
|
||||
- `docs/porting-notes.md` — staging log + M1 deviation record (the truth about compat-layer divergences)
|
||||
- `docs/porting-notes.md` — staging log + the deviation record (the truth about compat-layer divergences)
|
||||
|
||||
Feature-status vocabulary used across these docs:
|
||||
|
||||
- **VERIFIED-in-VM** — exercised end-to-end in the libvirt lab (see vm-lab.md).
|
||||
- **COMPILES-untested** — the code is built into `pf.ko` but no gate has proven the behavior yet.
|
||||
- **PENDING-Mx** — lands (or lands fully) in milestone Mx; current observable behavior is documented.
|
||||
- **VERIFIED-in-VM** — exercised end-to-end in the libvirt lab; the gate that proves it is named (see [vm-lab.md](vm-lab.md)).
|
||||
- **VERIFIED-offline** — proven by a fixture or ABI test, not by traffic.
|
||||
- **COMPILES-untested** — the code is built into `pf.ko`/`pfctl` but no gate has exercised it.
|
||||
- **NOT-ENABLED** — the code exists but the feature is off in this build.
|
||||
- **OUT-OF-SCOPE** — non-goal by design (spec §3).
|
||||
|
||||
The lab suite is **15/15 green** (`./scripts/vm-run-tests.sh`).
|
||||
+42
-17
@@ -69,14 +69,15 @@ auto-staged (hand-ported).
|
||||
|
||||
## 5. Module — `scripts/build-module.sh`
|
||||
|
||||
`make -C "$KERNELDIR" M=module` → `module/pf.ko` (single module today; the
|
||||
`pfq` qdisc arrives with M5). The script self-prepares `Module.symvers` /
|
||||
`modules_prepare` first. Object list (`module/Makefile`): ported engine
|
||||
(`pf.c`, `pf_ioctl.c`, `pf_norm.c`, `pf_if.c`, `pf_lb.c`, `pf_ruleset.c`,
|
||||
`pf_table.c`, `pf_osfp.c`, `pf_syncookies.c`, `divert_sock.c`) + compat layer
|
||||
(mbuf shim, kern glue, if/ip4/ip6/pcb/xmit/ifq, crypto, radix, subr_tree) +
|
||||
`pfk_mod.c`/`pfk_hooks.c`. Compile flags define `PFLINUX=1` and `_KERNEL=1`.
|
||||
Prints `MODULE_BUILD_OK`.
|
||||
`make -C "$KERNELDIR" M=module` → `module/pf.ko` (a single module: the BSD
|
||||
`ifq`/`pfq` interface lives inside it, it is not a separate `tc` qdisc). The
|
||||
script self-prepares `Module.symvers` / `modules_prepare` first. Object list
|
||||
(`module/Makefile`): ported engine (`pf.c`, `pf_ioctl.c`, `pf_norm.c`,
|
||||
`pf_if.c`, `pf_lb.c`, `pf_ruleset.c`, `pf_table.c`, `pf_osfp.c`,
|
||||
`pf_syncookies.c`, `divert_sock.c`, `pfk_pflog.c`, `hfsc.c`, `fq_codel.c`) +
|
||||
compat layer (mbuf shim, kern glue, if/ip4/ip6/pcb/xmit/ifq, crypto, radix,
|
||||
subr_tree) + `pfk_mod.c`/`pfk_hooks.c`. Compile flags define `PFLINUX=1` and
|
||||
`_KERNEL=1`. Prints `MODULE_BUILD_OK`.
|
||||
|
||||
`insmod module/pf.ko` creates `/dev/pf` (0600) and registers the netfilter
|
||||
hooks; `dmesg` shows `pfk: …` lines (see [caveats.md](caveats.md) for what
|
||||
@@ -84,7 +85,8 @@ the data path does).
|
||||
|
||||
## 6. Userland — `make -C tools`
|
||||
|
||||
Targets (from `tools/Makefile`): `pfctl`, `abitest`, `abi`, `test`, `clean`.
|
||||
Targets (from `tools/Makefile`): `pfctl`, `pdivert`, `pflogd`, `abitest`,
|
||||
`abi`, `test`, `clean`.
|
||||
|
||||
- `pfctl` — the OpenBSD 8.0 sources compiled against the copied ABI headers;
|
||||
result is a **static binary** at `tools/out/pfctl`. Every deviation from
|
||||
@@ -97,20 +99,43 @@ Targets (from `tools/Makefile`): `pfctl`, `abitest`, `abi`, `test`, `clean`.
|
||||
- `test` — offline parse checks: runs `pfctl -n -f` over `tools/tests/*.conf`
|
||||
(basic / translations / queues+scrub / anchor / negative `broken.conf`).
|
||||
This is pure userland; it needs no `/dev/pf`.
|
||||
- `pdivert` — divert relay source under `tools/pdivert/`; see
|
||||
[feature-matrix.md](feature-matrix.md) for its (pending) kernel side.
|
||||
- `pdivert` — the divert relay, built from `tools/pdivert/`. It talks to the
|
||||
`AF_DIVERT` socket family the module registers at runtime
|
||||
(`socket(PF_DIVERT=32, SOCK_RAW, IPPROTO_DIVERT=254)`); gate `11-divert.sh`
|
||||
proves the round trip.
|
||||
- `pflogd` — reads `/dev/pflog` and writes a DLT_PFLOG pcap whose record
|
||||
layout is byte-compatible with OpenBSD `pflog(4)`; gate `12-pflog.sh`.
|
||||
|
||||
## 7. Root `Makefile`
|
||||
## 7. Screencasts — `scripts/screencast/build-all.sh`
|
||||
|
||||
`make module | tools | kernel | vm | vm-destroy | test | docs | clean` — thin
|
||||
Not part of the kernel build, but part of the deliverable. The pipeline:
|
||||
|
||||
| Stage | Script | What it does |
|
||||
|---|---|---|
|
||||
| narration text | `make_narration.py` | writes `scripts/screencast/narration/<phase>.{en,pt}.json` (one segment per beat of the session) |
|
||||
| voice | `make_audio.py` | MiniMax TTS, one MP3 per segment, female voice per language |
|
||||
| session | `record.py` | real SSH session on the lab: commands typed key by key, captured as asciicast v2 |
|
||||
| video | `render_video.py` | replays the capture through a terminal emulator, draws the lab diagram, burns the subtitles |
|
||||
| assembly | `assemble.py` | mixes the narration onto the timeline |
|
||||
| full film | `join.py` | crossfades the phases through chapter cards |
|
||||
|
||||
`make screencast` (or the script) runs all of it; it needs
|
||||
`MINIMAX_API_KEY` and the lab up. Details: [screencast.md](screencast.md).
|
||||
|
||||
## 8. Root `Makefile`
|
||||
|
||||
`make module | tools | kernel | vm | vm-destroy | test | screencast | docs | clean` — thin
|
||||
wrappers over the scripts above. `make test` runs the VM suite
|
||||
([vm-lab.md](vm-lab.md)); `make docs` just lists the bilingual doc set.
|
||||
([vm-lab.md](vm-lab.md)); `make screencast` renders the videos;
|
||||
`make docs` just lists the bilingual doc set.
|
||||
|
||||
## What has been exercised vs. not
|
||||
|
||||
- Stages 1–6 have each been run during M0–M2 development on the build host
|
||||
(artifacts exist in-tree: `out/`, `module/pf.ko`, `tools/out/pfctl`), and
|
||||
the scripts above were verified line-by-line for this document.
|
||||
- Stages 1–6 have each been run repeatedly on the build host (artifacts exist
|
||||
in-tree: `out/`, `module/pf.ko`, `tools/out/{pfctl,pdivert,pflogd}`), and the
|
||||
scripts above were verified line-by-line for this document.
|
||||
- Stage 7 (the screencasts) has been run end to end for all seven phases in
|
||||
both languages; the videos are in `out/screencast/{en,pt}/`.
|
||||
- A **fresh-clone end-to-end run** (`quickstart-check.sh`, plan task T19) has
|
||||
*not* been executed yet; treat the exact command sequence in install.md as
|
||||
"as designed, last-run state per milestone" rather than CI-verified.
|
||||
@@ -159,6 +159,33 @@ matter when reasoning about rule semantics:
|
||||
kernel. Containers/namespaces are not visible to PF and cannot be filtered
|
||||
with it in this design.
|
||||
|
||||
## Screencast pipeline (2026-11-12)
|
||||
|
||||
### 10. MiniMax video generation is not enabled on this account
|
||||
|
||||
Every MiniMax video model (`T2V-01`, `T2V-01-Director`, `MiniMax-Hailuo-02`)
|
||||
answers with API error **2068 — "The current Token Plan tier does not support
|
||||
this model"**. The narration (TTS) and the title artwork work fine; only the
|
||||
video models are blocked. The phase transitions are therefore rendered
|
||||
locally with ffmpeg (a crossfade over a chapter card) instead of AI-generated
|
||||
clips. Upgrading the plan is the only thing needed to change that — the
|
||||
extension point is `scripts/screencast/join.py`, which builds the cards.
|
||||
|
||||
### 11. The narration is a separate track, not a dubbing of the video
|
||||
|
||||
Each phase is recorded **once** and serves both languages: the terminal
|
||||
session is language-neutral, and the two narrated versions differ only in the
|
||||
audio track and the burned-in subtitles. The recorder holds each beat on
|
||||
screen for as long as the longest narration segment (EN or PT) needs, so the
|
||||
voice and the terminal action stay in step in both.
|
||||
|
||||
### 12. What the videos do not show
|
||||
|
||||
The videos are a demonstration, not a test run: the lab's own gate suite
|
||||
(`scripts/vm-run-tests.sh`) is the authority for PASS/FAIL, and its logs live
|
||||
in `out/test-logs/`. The screencasts show the same commands, but a video frame
|
||||
is not a verdict.
|
||||
|
||||
## What is *not* a caveat
|
||||
|
||||
ARP and other non-IPv4/IPv6 EtherTypes never enter the netfilter IPv4/IPv6
|
||||
|
||||
@@ -1,41 +0,0 @@
|
||||
# Demo (asciinema)
|
||||
|
||||
The lab can be recorded as an [asciinema](https://asciinema.org) cast. Every
|
||||
byte in the recording is the real output of the lab VMs (Alpine under libvirt
|
||||
running the kernel built from source with `pf.ko`); only the pacing is
|
||||
scripted.
|
||||
|
||||
## Record
|
||||
|
||||
```sh
|
||||
pip install asciinema # once
|
||||
bash scripts/vm-run-tests.sh # or: scripts/vm-create.sh + deploy
|
||||
asciinema rec out/demo-pf-linux.cast -c "bash scripts/demo-session.sh"
|
||||
```
|
||||
|
||||
The session walks through:
|
||||
|
||||
1. the lab topology and the freshly loaded `pf.ko` (`dmesg` proof),
|
||||
2. `pfctl -s info` over the OpenBSD ioctl ABI,
|
||||
3. a real `pf.conf` (table, stateful rules, explicit drop) and `pfctl -sr`,
|
||||
4. transit ping + `pfctl -ss` (stateful),
|
||||
5. `pfctl -t <table> -T show` (live tables),
|
||||
6. a blocked ping (counters with `pfctl -sr -vv`),
|
||||
7. `divert-to` with the `pdivert` relay (packet to userspace and back),
|
||||
8. shutdown.
|
||||
|
||||
## Play back
|
||||
|
||||
```sh
|
||||
asciinema play out/demo-pf-linux.cast
|
||||
```
|
||||
|
||||
## Upload
|
||||
|
||||
```sh
|
||||
asciinema upload out/demo-pf-linux.cast
|
||||
```
|
||||
|
||||
`scripts/demo-record.py` is an alternative recorder that produces the same
|
||||
asciicast v2 format without the asciinema binary (it drives the lab over
|
||||
ssh and writes the cast itself).
|
||||
+39
-34
@@ -4,67 +4,71 @@ Português: [docs/pt-BR/feature-matrix.md](../pt-BR/feature-matrix.md).
|
||||
|
||||
Status vocabulary (used identically in both languages):
|
||||
|
||||
- **VERIFIED-in-VM** — passed end-to-end in the libvirt lab (see [vm-lab.md](vm-lab.md)).
|
||||
- **M2-in-progress** — implemented; VM gates 01–04 exist but the suite is **not all green yet**; no VERIFIED claim until a clean full run.
|
||||
- **VERIFIED-in-VM** — passed end-to-end in the libvirt lab (see [vm-lab.md](vm-lab.md)); the gate that proves it is named.
|
||||
- **VERIFIED-offline** — proven by a fixture/ABI test, not by traffic (e.g. `make -C tools test`).
|
||||
- **COMPILES-untested** — built into `pf.ko`/`pfctl`; no gate has exercised it.
|
||||
- **PENDING-Mx** — lands in milestone Mx; today's observable behavior is stated.
|
||||
- **NOT-ENABLED** — the code exists but the feature is off in this build (e.g. `af-to`, which needs the `INET6` paths).
|
||||
- **OUT-OF-SCOPE** — non-goal by design (spec §3).
|
||||
|
||||
As of 2026-11-11; source of per-row evidence: `out/test-logs/`,
|
||||
`docs/porting-notes.md`, milestone commits.
|
||||
As of 2026-11-12; source of per-row evidence: `out/test-logs/`,
|
||||
`docs/porting-notes.md`, milestone commits, and the screencasts
|
||||
([screencast.md](screencast.md)) for the visible proof.
|
||||
|
||||
The suite is **15/15 green** (`./scripts/vm-run-tests.sh`).
|
||||
|
||||
## Control plane & tools
|
||||
|
||||
| Feature | Upstream syntax | Milestone | Status |
|
||||
|---|---|---|---|
|
||||
| Module load, `/dev/pf` (0600) | `insmod pf.ko` | M1 | VERIFIED-in-VM |
|
||||
| Module load, `/dev/pf` (0600) | `insmod pf.ko` | M1 | VERIFIED-in-VM (gate `02-ativacao.sh`) |
|
||||
| Enable/disable | `pfctl -e`, `pfctl -d` | M1 | VERIFIED-in-VM |
|
||||
| Ruleset load | `pfctl -f file` | M2 | VERIFIED-in-VM (loads succeeded in every captured gate run; run status itself is mixed — see [vm-lab.md](vm-lab.md) §6) |
|
||||
| Parse-only check | `pfctl -n -f file` | M1 | VERIFIED (offline fixture suite, `make -C tools test`) |
|
||||
| Ruleset load | `pfctl -f file` | M2 | VERIFIED-in-VM (every gate loads a real ruleset) |
|
||||
| Parse-only check | `pfctl -n -f file` | M1 | VERIFIED-offline (`make -C tools test`) |
|
||||
| Status/counters | `pfctl -s info [-v]` | M1 | VERIFIED-in-VM |
|
||||
| Rules listing | `pfctl -s rules [-v]` | M2 | COMPILES-untested |
|
||||
| States listing | `pfctl -s states [-v]` | M2 | M2-in-progress — gate 01 shows a listing anomaly (transit ping OK, no icmp state displayed) while state machinery is being fixed |
|
||||
| Tables listing/ops | `pfctl -s tables`, `-t t -T show/add/del/kill` | M2 | M2-in-progress (gate 03) |
|
||||
| Anchors | `pfctl -a anchor -f file`, `-s Anchors` | M2 | M2-in-progress (gate 04) |
|
||||
| Queue listing | `pfctl -s queue -v`, `-s ifq -v` | M5 | PENDING-M5 (parses fine; counters not wired to real back ends yet) |
|
||||
| ioctl ABI parity | `make -C tools abi` (abitest tables) | M1 | VERIFIED build-side (userland table); kernel-side diff is an open integrator step (`tools/UPSTREAM.md`) |
|
||||
| Rules listing | `pfctl -s rules [-v]`, `pfctl -sr` | M2 | VERIFIED-in-VM (gates 02–06) |
|
||||
| States listing | `pfctl -s states [-v]`, `pfctl -ss` | M2 | VERIFIED-in-VM (gate `01-state.sh`) |
|
||||
| Tables listing/ops | `pfctl -s tables`, `-t t -T show/add/del/kill` | M2 | VERIFIED-in-VM (gate `03-tables.sh`, incl. a live `-T delete` unblocking the flow) |
|
||||
| Anchors | `pfctl -a anchor -f file`, `-s anchors` | M2 | VERIFIED-in-VM (gate `04-anchors.sh`) |
|
||||
| Queue listing | `pfctl -s queue -v` | M5 | VERIFIED-in-VM (gate `15-queue.sh`: both classes with statistics) |
|
||||
| ioctl ABI parity | `make -C tools abi` (abitest tables) | M1 | VERIFIED-offline (userland table); the kernel-side diff is an open integrator step (`tools/UPSTREAM.md`) |
|
||||
|
||||
## Filtering core
|
||||
|
||||
| Feature | Upstream syntax | Milestone | Status |
|
||||
|---|---|---|---|
|
||||
| `pass`/`match`, quick, on-interface/proto/addr/port | `pass in quick on wan proto tcp from 10/8 to port 80` | M2 | M2-in-progress (data path runs the real `pf_test`; latest partial run: transit ping OK, icmp block OK) |
|
||||
| `block drop` | `block drop in quick …` | M2 | M2-in-progress (gate 02a) |
|
||||
| `block return` / `return-rst` / `return-rst after quarter` | `block return-rst in quick proto tcp …` | M2 | M2-in-progress (gate 02b) |
|
||||
| `keep state` (icmp/tcp/udp), state table, purger | `… keep state` | M2 | M2-in-progress |
|
||||
| `pass`/`match`, quick, on-interface/proto/addr/port | `pass in quick on wan proto tcp from 10/8 to port 80` | M2 | VERIFIED-in-VM (gate `01-state.sh`: transit ping through the firewall) |
|
||||
| `block drop` | `block drop in quick …` | M2 | VERIFIED-in-VM (gate `02-block.sh`: ping dies, counters prove it) |
|
||||
| `block return` / `return-rst` / `return-rst after quarter` | `block return-rst in quick proto tcp …` | M2 | VERIFIED-in-VM (gate `02-block.sh`: the client's `nc` gets an RST) |
|
||||
| `keep state` (icmp/tcp/udp), state table, purger | `… keep state` | M2 | VERIFIED-in-VM (gate `01-state.sh`; state visible in `pfctl -ss`) |
|
||||
| `set state-policy`, `allow-opts`, etc. | `set …` options | M2 | COMPILES-untested |
|
||||
| Syncookies (state overflow path) | `set syncookies always` | M2 | COMPILES-untested |
|
||||
| pfr tables (radix/hash) | `table <blk> persist { 1.2.3.4 }`, `-T add …` | M2 | M2-in-progress (gate 03) |
|
||||
| Anchors + include + quick precedence | `anchor "x"`, `load anchor …` | M2 | M2-in-progress (gate 04) |
|
||||
| IPv6 for all of the above | same syntax, v6 | M2+ | COMPILES-untested (engine is v6-complete by construction; v6 smoke gates planned) |
|
||||
| Non-IPv4/IPv6 traffic (ARP…) | — | M2 | PFLINUX by construction: netfilter IPv4/IPv6 hooks only — ARP never enters PF path (no gate yet) |
|
||||
| pfr tables (radix/hash) | `table <blk> persist { 1.2.3.4 }`, `-T add …` | M2 | VERIFIED-in-VM (gate `03-tables.sh`) |
|
||||
| Anchors + include + quick precedence | `anchor "x"`, `load anchor …` | M2 | VERIFIED-in-VM (gate `04-anchors.sh`) |
|
||||
| IPv6 for all of the above | same syntax, v6 | M2+ | COMPILES-untested — the engine is v6-complete by construction, but the v6 path is not exercised by a gate |
|
||||
| Non-IPv4/IPv6 traffic (ARP…) | — | M2 | PFLINUX by construction: netfilter IPv4/IPv6 hooks only — ARP never enters the PF path (no gate) |
|
||||
|
||||
## NAT / normalization (M3)
|
||||
|
||||
| Feature | Upstream syntax | Milestone | Status |
|
||||
|---|---|---|---|
|
||||
| `nat-to` (port pools, sticky-port, static-port) | `pass out … nat-to (eth1:0)` | M3 | **VERIFIED in lab** — source NAT works end to end: the packet leaves with the pool address (tcpdump proof), the reply is de-NATed and re-routed to the client, and the state shows both tuples (`10.0.2.1:59335 (10.0.1.10:4380) -> 10.0.2.20:8`). Gate: `tests/vm/05-nat.sh`. |
|
||||
| `rdr-to` | `pass in … to <fw> port 8080 rdr-to <srv> port 80 keep state` | M3 | **VERIFIED in lab** — port redirect works end to end (gate `06-rdr.sh`): the client talks to the firewall's address, pf rewrites the destination, re-routes the packet, and the reply is un-redirected. The gate also pulls 2 MB through the redirect and checks byte-for-byte integrity (GSO/TSO regression guard). |
|
||||
| `binat-to` | `pass quick proto icmp from 10.0.2.20 to 10.0.1.10 binat-to 10.0.2.50 keep state` | M3 | **VERIFIED in lab** — gate `07-binat.sh`. pfctl expands one `binat-to` rule into the documented pair (`pass out … nat-to <mapped> static-port` + `pass in … rdr-to <real>`), and both halves work: the peer sees the mapped address, the original never leaks, and the reply is translated back. |
|
||||
| Scrub/normalize: reassembly, `fragment drop-overlap`, `min-ttl`, `max-mss`, `df`/`no-df`, `random-id`, `set reassemble yes` | `match in all scrub (min-ttl 100 max-mss 500)` | M3 | **VERIFIED in lab** — gate `09-scrub.sh`: `min-ttl` raises the TTL (egress shows 99 after the forward decrement) and `max-mss` clamps a SYN's MSS option, both measured on the firewall's own egress. Reassembly/overlap paths are compiled and reached but not yet gated. |
|
||||
| OS fingerprinting in states | `pass … set os A` + `/etc/pf.os` | bonus | COMPILES-untested (`pf_osfp.c` compiled; no fingerprint packaging yet — missing `/etc/pf.os` is tolerated) |
|
||||
| `nat-to` (port pools, sticky-port, static-port) | `pass out … nat-to (eth1:0)` | M3 | **VERIFIED-in-VM** — gate `05-nat.sh`: source NAT works end to end (tcpdump proof), the reply is de-NATed and re-routed to the client, and the state shows both tuples (`10.0.2.1:59335 (10.0.1.10:4380) -> 10.0.2.20:8`). |
|
||||
| `rdr-to` | `pass in … to <fw> port 8080 rdr-to <srv> port 80 keep state` | M3 | **VERIFIED-in-VM** — gate `06-rdr.sh`: the client talks to the firewall's address, pf rewrites the destination, re-routes the packet, and the reply is un-redirected. The gate also pulls 2 MB through the redirect and checks byte-for-byte integrity (GSO/TSO regression guard). |
|
||||
| `binat-to` | `pass quick proto icmp from 10.0.2.20 to 10.0.1.10 binat-to 10.0.2.50 keep state` | M3 | **VERIFIED-in-VM** — gate `07-binat.sh`. pfctl expands one `binat-to` rule into the documented pair (`pass out … nat-to <mapped> static-port` + `pass in … rdr-to <real>`), and both halves work: the peer sees the mapped address, the original never leaks, and the reply is translated back. |
|
||||
| NAT pools (`round-robin`, `source-hash`) | `nat-to { 10.0.2.1, 10.0.3.1 } round-robin` | M4 | **VERIFIED-in-VM** — gate `10-pools.sh`, and visible in the screencast: round-robin alternates the source address between the members (3 packets each), source-hash keeps one origin on one member. |
|
||||
| Scrub/normalize: reassembly, `fragment drop-overlap`, `min-ttl`, `max-mss`, `df`/`no-df`, `random-id`, `set reassemble yes` | `match in all scrub (min-ttl 100 max-mss 500)` | M3 | **VERIFIED-in-VM** — gate `09-scrub.sh`: `min-ttl` raises the TTL (egress shows 99 after the forward decrement) and `max-mss` clamps a SYN's MSS option, both measured on the firewall's own egress. Reassembly/overlap paths are compiled and reached but not gated. |
|
||||
| OS fingerprinting in states | `pass … set os A` + `/etc/pf.os` | bonus | COMPILES-untested (`pf_osfp.c` compiled; no fingerprint packaging yet — a missing `/etc/pf.os` is tolerated) |
|
||||
|
||||
## Routing / divert / queues / logging
|
||||
|
||||
| Feature | Upstream syntax | Milestone | Status |
|
||||
|---|---|---|---|
|
||||
| `route-to` / `dup-to` / `reply-to` | `pass in … route-to { 10.0.3.2 } keep state`, `reply-to { gw }` | M4 | **VERIFIED in lab** — gates `08-routeto.sh` (route-to leaves via the gateway's link, dup-to copies while keeping the flow) and `13-replyto.sh` (the reply of a flow is forced onto the gateway link instead of the client's). Deviation: the forced nexthop is installed by setting `rt_gw4`/`rt_gw_family` on the looked-up route (Linux only ARPs the gateway when the route declares one). `fail-to` is **not a keyword in OpenBSD 8.0** (removed upstream), so it is out of scope. |
|
||||
| `divert-to` / `divert` tag | `pass in … divert-to <host> port <n>` | VERIFIED in lab | **Works end-to-end.** The module registers an `AF_DIVERT` socket family (`PF_DIVERT=32`) at runtime — no kernel-tree patch. A `divert-to` rule hands the packet to that family; with no listener the packet is dropped (`divs_noport`, BSD parity); with `tools/pdivert` running the packet is delivered to userspace, rewritten and reinjected (verified with a payload mutation + checksum fix, ping 0% loss). |
|
||||
| Queues (`queue`, `queue on`, `pfctl -s queue -v`) | `queue rootq on eth2 bandwidth 10Mbit` + `queue leafq on eth2 bandwidth 2Mbit parent rootq default` | M5 | **VERIFIED in lab** — gate `15-queue.sh`: the queue config loads, the hfsc discipline attaches to the interface, `pfctl -s queue -v` lists both classes with statistics, a 2 MB queued flow completes, and the class's packet counter advances (the scheduler is in the data path). Note `bandwidth` is HFSC *linkshare* - a guarantee under contention, not a cap: OpenBSD 8.0's pf.conf has no `minimum`/`maximum`, so a lone flow is not rate-limited, on Linux or on OpenBSD. |
|
||||
| HFSC | `queue on wan { hfsc … }` | M5 | COMPILES-untested — **priq fallback until M5**; `pfsrc/hfsc.c` staged (real OpenBSD 8.0 code) but **not in the module object list** |
|
||||
| FQ-CoDel | `queue on wan { fq_codel … }` | M5 | COMPILES-untested — same as HFSC: staged, not built |
|
||||
| `pfq` qdisc (`tc qdisc … root pfq`) | n/a (Linux glue) | M5 | PENDING-M5 (does not exist in the tree yet) |
|
||||
| pflog (`log`, `log all`, `block log`) + `pflogd` pcap (DLT_PFLOG) | `pass out quick log` + `pflogd /tmp/x.pcap` | M4 | **VERIFIED in lab** — gate `12-pflog.sh`: logged packets are queued by the engine, read through `/dev/pflog` and written as a pcap with linktype DLT_PFLOG (117); the dump contains the real packet bytes. Deviation: a chardev instead of an if_pflog/bpf interface (Linux has no bpf over such a virtual interface here); record layout is byte-compatible with OpenBSD `pflog(4)`. `if_pflog.c` stays unbuilt. |
|
||||
| `route-to` / `dup-to` / `reply-to` | `pass in … route-to { 10.0.3.2 } keep state`, `reply-to { gw }` | M4 | **VERIFIED-in-VM** — gates `08-routeto.sh` (route-to leaves via the gateway's link, dup-to copies while keeping the flow) and `13-replyto.sh` (the reply of a flow is forced onto the gateway link instead of the client's). Deviation: the forced nexthop is installed by setting `rt_gw4`/`rt_gw_family` on the looked-up route (Linux only ARPs the gateway when the route declares one). `fail-to` is **not a keyword in OpenBSD 8.0** (removed upstream), so it is out of scope. |
|
||||
| `divert-to` / `divert` tag | `pass in … divert-to <host> port <n>` | M4 | **VERIFIED-in-VM** — gate `11-divert.sh`. The module registers an `AF_DIVERT` socket family (`PF_DIVERT=32`) at runtime — no kernel-tree patch. A `divert-to` rule hands the packet to that family; with no listener the packet is dropped (`divs_noport`, BSD parity); with `tools/pdivert` running the packet is delivered to userspace, rewritten and reinjected (verified with a payload mutation + checksum fix, ping 0% loss). |
|
||||
| Queues (`queue`, `queue on`, `pfctl -s queue -v`) | `queue rootq on eth2 bandwidth 10Mbit` + `queue leafq on eth2 bandwidth 2Mbit parent rootq default` | M5 | **VERIFIED-in-VM** — gate `15-queue.sh`: the queue config loads, the hfsc discipline attaches to the interface, `pfctl -s queue -v` lists both classes with statistics, a 2 MB queued flow completes, and the class's packet counter advances (the scheduler is in the data path). Note `bandwidth` is HFSC *linkshare* — a guarantee under contention, not a cap: OpenBSD 8.0's pf.conf has no `minimum`/`maximum`, so a lone flow is not rate-limited, on Linux or on OpenBSD. |
|
||||
| HFSC | `queue on wan { hfsc … }` | M5 | **VERIFIED-in-VM** — `pfsrc/hfsc.c` (real OpenBSD 8.0 code) is in the module object list and is the discipline the gate exercises. The class handles use the vendored `net/hfsc.h` (root class `0x10000`, default class `0x1000`) shared with `pfctl`, so the stats ABI matches on both sides. |
|
||||
| FQ-CoDel | `queue on wan { fq_codel … }` | M5 | COMPILES-untested — `pfsrc/fq_codel.c` is built into `pf.ko` and the vendored `net/fq_codel.h` is shared with `pfctl`, but no gate drives it. |
|
||||
| `pfq` interface | n/a (BSD `ifq`/`pfq` interface, not a Linux `tc` qdisc) | M5 | **VERIFIED-in-VM** — the port keeps the BSD shape: `compat/pfk_ifq.c` implements the `ifq_ops`/`pfq_ops` interfaces and the OpenBSD back ends plug into them (`pfq` is not a Linux `tc` qdisc and never was meant to be). priq is the default discipline. |
|
||||
| pflog (`log`, `log all`, `block log`) + `pflogd` pcap (DLT_PFLOG) | `pass out quick log` + `pflogd /tmp/x.pcap` | M4 | **VERIFIED-in-VM** — gate `12-pflog.sh`: logged packets are queued by the engine, read through `/dev/pflog` and written as a pcap with linktype DLT_PFLOG (117); the dump contains the real packet bytes. Deviation: a chardev instead of an if_pflog/bpf interface (Linux has no bpf over such a virtual interface here); record layout is byte-compatible with OpenBSD `pflog(4)`. `if_pflog.c` stays unbuilt. |
|
||||
|
||||
## Non-goals (explicit)
|
||||
|
||||
@@ -76,3 +80,4 @@ As of 2026-11-11; source of per-row evidence: `out/test-logs/`,
|
||||
| Per-netns PF instances | faithful port = single global instance on `init_net` |
|
||||
| Classic-ALTQ-only schedulers (cbq/wfq/red…) | removed upstream before OpenBSD 8.0 — cannot be ported from the base |
|
||||
| Routing labels (`route-label`) | non-goal; ioctl layer rejects labeled rules |
|
||||
| `af-to` (IPv4↔IPv6 translation) | the code sits under `#ifdef INET6` and this build has `INET6 0`; enabling it is a separate milestone, not a missing feature in the base |
|
||||
+27
-19
@@ -24,8 +24,8 @@ between kernel and userland (ioctl encoding per `module/include/pfk.h`).
|
||||
| `module/compat/` | Linux plumbing under the BSD-shaped sources: mbuf-over-`sk_buff` shim (`pfk_mbuf`), kernel glue (malloc/pools/timeouts/locks, `pfk_kern`), interface+address views (`pfk_if`), BSD-semantics IP output/forward (`pfk_ip4`, `pfk_ip6`), socket lookups (`pfk_pcb`), ifq API with priq semantics (`pfk_ifq`), radix/route-table/crypto ports |
|
||||
| `module/include/` | `pfk.h` (master compat header + OpenBSD-style `_IO*` encoding), staged `net/pfvar.h`, `pfk_mbuf.h`, `pfk_net.h`, `pfk_if.h` |
|
||||
| `tools/` | Linux build of OpenBSD 8.0 `pfctl` (single binary), `pdivert` relay, ABI parity harness (`abitest` + `make abi`), offline parse tests (`make test`). Deltas vs upstream: `tools/UPSTREAM.md` |
|
||||
| `scripts/` | `fetch-sources.sh`, `vendorize.sh`, `stage-port.sh` (deterministic staging), `build-kernel.sh`, `build-module.sh`, `vm-*.sh` lab automation |
|
||||
| `tests/vm/` | VM suite gates `01-state.sh … 04-anchors.sh` + `lib.sh`; `tests/confs/` rulesets deployed to the lab |
|
||||
| `scripts/` | `fetch-sources.sh`, `vendorize.sh`, `stage-port.sh` (deterministic staging), `build-kernel.sh`, `build-module.sh`, `vm-*.sh` lab automation, `screencast/` (video pipeline) |
|
||||
| `tests/vm/` | VM suite: 15 gates `01-state.sh … 15-queue.sh` + `lib.sh`; `tests/confs/` rulesets deployed to the lab |
|
||||
| `patches/kernel/` | Core patch set — currently **empty** (`p1-needs.txt` records why no exports were needed); see [patches.md](patches.md) |
|
||||
|
||||
## Data path (M2, as implemented today)
|
||||
@@ -69,33 +69,41 @@ rebuilt.
|
||||
|
||||
## Queues, divert, pflog — today's observable behavior
|
||||
|
||||
- **Queues**: `pfctl` parses `queue`/`queue on` (8.0 syntax); kernel-side the
|
||||
ifq API is implemented by `compat/pfk_ifq.c` with **priq semantics only**;
|
||||
the `pfq` classful qdisc and the HFSC/FQ-CoDel back ends land in **M5**.
|
||||
`ifq.c`/`hfsc.c`/`fq_codel.c` are staged and compile-ready but are not in
|
||||
the module object list yet.
|
||||
- **Divert**: `divert-to`/`divert` tag evaluate to PF's `PF_DIVERT` verdict;
|
||||
`divert_packet()` (`pfsrc/divert_sock.c`) currently **drops the packet with
|
||||
a rate-limited kernel log** because the AF_DIVERT socket family is not yet
|
||||
instantiated. The `pdivert` relay (userspace) exists and expects
|
||||
`socket(PF_DIVERT=32, SOCK_RAW, IPPROTO_DIVERT=254)`.
|
||||
- **pflog**: `pflog_packet()` returns success but **logs nothing** (one-time
|
||||
`pr_warn "packet not logged"`); `if_pflog.c` is staged, netlink back end is
|
||||
M4/M5.
|
||||
- **Queues**: `pfctl` parses the OpenBSD 8.0 `queue`/`queue on` syntax and the
|
||||
kernel side implements it in the BSD shape: `compat/pfk_ifq.c` provides the
|
||||
`ifq_ops`/`pfq_ops` interfaces and the OpenBSD back ends (`pfsrc/hfsc.c`,
|
||||
`pfsrc/fq_codel.c`) plug into them; **priq** is the default discipline.
|
||||
Gate `15-queue.sh` proves the **hfsc** discipline end to end: the queue
|
||||
config loads, the discipline attaches to the interface, `pfctl -s queue -v`
|
||||
lists both classes with statistics, a 2 MB queued flow completes and the
|
||||
class's packet counter advances. The class handles and the stats structs
|
||||
come from OpenBSD's own `net/hfsc.h`/`net/fq_codel.h`, vendored so the
|
||||
kernel and `pfctl` cannot drift apart.
|
||||
- **Divert**: `divert-to`/`divert` tag evaluate to PF's `PF_DIVERT` verdict
|
||||
and the module registers an `AF_DIVERT` socket family (`PF_DIVERT=32`) at
|
||||
runtime — no kernel-tree patch. With no listener the packet is dropped
|
||||
(`divs_noport`, BSD parity); with the `pdivert` relay running the packet is
|
||||
delivered to userspace, rewritten and reinjected. Gate `11-divert.sh`.
|
||||
- **pflog**: the engine queues logged packets, `pfsrc/pfk_pflog.c` exposes
|
||||
them through `/dev/pflog`, and `tools/pflogd` writes a DLT_PFLOG pcap whose
|
||||
record layout is byte-compatible with OpenBSD `pflog(4)`. Gate
|
||||
`12-pflog.sh`. The deviation from upstream is the transport (a chardev
|
||||
instead of an `if_pflog`/bpf interface), not the format.
|
||||
|
||||
## Milestones (see feature-matrix.md for per-feature status)
|
||||
|
||||
| Milestone | Scope | Status (2026-11-11) |
|
||||
| Milestone | Scope | Status |
|
||||
|---|---|---|
|
||||
| M0 | repo, scripts, kernel build, VM boots custom kernel | done |
|
||||
| M1 | compat layer, `pf.ko` loads, `/dev/pf`, `pfctl -f`/`-s info` | done |
|
||||
| M2 | filter core: rules/states/tables/anchors VM gates | **done** — gates 01–04 green |
|
||||
| M3 | NAT (`nat-to`/`rdr-to`/`binat-to`), scrub, pflog | **done** — gates 05/06/07/09 (incl. 2 MB integrity through a redirect) and pflog gate 12 |
|
||||
| M4 | route-to/dup-to/reply-to, pools, divert e2e | **done** — gates 08/10/13 and divert gate 11 |
|
||||
| M5 | queues: ifq back ends (hfsc/fq_codel) | **partial** — back ends compiled into `pf.ko`; queue attachment/pacing pending |
|
||||
| M6 | docs complete, patchset tidy, full suite green, tagged releases | in progress — docs EN+PT-BR complete, 14/14 suite green, tagged |
|
||||
| M5 | queues: ifq back ends (hfsc/fq_codel) | **done (mechanical)** — gate 15: hfsc attaches, `pfctl -s queue -v` reports class stats, a 2 MB queued flow is counted by the class. `bandwidth` = HFSC linkshare (a guarantee under contention, not a cap) |
|
||||
| M6 | docs complete, patchset tidy, full suite green, tagged releases | **done** — docs EN+PT-BR, 15/15 suite green, tagged (`v1.0.0`) |
|
||||
|
||||
## Next reads
|
||||
|
||||
[build.md](build.md) → [vm-lab.md](vm-lab.md) → [pf-guide.md](pf-guide.md) →
|
||||
[feature-matrix.md](feature-matrix.md) → [caveats.md](caveats.md).
|
||||
[feature-matrix.md](feature-matrix.md) → [screencast.md](screencast.md) →
|
||||
[caveats.md](caveats.md).
|
||||
+24
-8
@@ -75,19 +75,34 @@ Because the deliverable is an out-of-tree module, "applying the patch set"
|
||||
adds a documented `patches/kernel/000N-*.patch` (all such patches: one
|
||||
section each in docs/en/patches.md with what/why/impact/revert).
|
||||
|
||||
## 6. Notes on the steps above (2026-11-11)
|
||||
## 6. Notes on the steps above (2026-11-12)
|
||||
|
||||
- Step **2.3 is a documented no-op today**: the patch set is empty and
|
||||
`scripts/apply-patches.sh` / `scripts/unpatch.sh` / `patches/kernel/series`
|
||||
are not yet in the tree (they land with the first real core patch — see
|
||||
[patches.md](patches.md)). You can safely skip 2.3.
|
||||
- Step 2.4 builds `pfctl` and the ABI harness only (`make -C tools` targets:
|
||||
`pfctl abitest abi test clean`). `pdivert` is present as a source file with
|
||||
no Makefile target yet; `pflogd` is not ported.
|
||||
- After 2.5, what you can actually *use* is the M2 filter core (pass/block/
|
||||
quick/keep state/tables/anchors); NAT/scrub/queues/divert/pflog are gated
|
||||
by later milestones — see the [feature matrix](feature-matrix.md) and read
|
||||
[pf-guide.md](pf-guide.md) before enabling PF on a path you need.
|
||||
- Step 2.4 builds every userland tool: `make -C tools` targets are `pfctl`,
|
||||
`pdivert`, `pflogd` and the ABI harness (`abitest`, `abi`, `test`, `clean`).
|
||||
- After 2.5, the whole feature set is usable and gated by the lab suite —
|
||||
filtering with state, tables and anchors, NAT (`nat-to`/`rdr-to`/`binat-to`
|
||||
plus pools), scrub, forced routing (`route-to`/`dup-to`/`reply-to`),
|
||||
`divert`, `pflog` and the OpenBSD 8.0 queues (HFSC). The
|
||||
[feature matrix](feature-matrix.md) says which gate proves what; read
|
||||
[pf-guide.md](pf-guide.md) for the deviations before enabling PF on a path
|
||||
you need.
|
||||
|
||||
## 6.1 Optional: the narrated screencasts
|
||||
|
||||
```sh
|
||||
export MINIMAX_API_KEY=... # MiniMax TTS for the narration
|
||||
./scripts/vm-create.sh # the lab must be up
|
||||
./scripts/screencast/build-all.sh # -> out/screencast/{en,pt}/*.mp4
|
||||
```
|
||||
|
||||
It records real shell sessions against the lab, generates the narration with
|
||||
a MiniMax female voice (English and Portuguese) and renders 1080p30 MP4s
|
||||
with burned-in subtitles and the lab's network diagram. Details:
|
||||
[screencast.md](screencast.md).
|
||||
|
||||
## 7. Where to go next
|
||||
|
||||
@@ -99,4 +114,5 @@ section each in docs/en/patches.md with what/why/impact/revert).
|
||||
| Daily rulesets & warnings | [pf-guide.md](pf-guide.md) |
|
||||
| Known divergences | [caveats.md](caveats.md) |
|
||||
| Core patch mechanism | [patches.md](patches.md) |
|
||||
| Narrated screencasts | [screencast.md](screencast.md) |
|
||||
| Doc index | [README.md](README.md) |
|
||||
+19
-12
@@ -3,10 +3,12 @@
|
||||
Português: [docs/pt-BR/pf-guide.md](../pt-BR/pf-guide.md).
|
||||
|
||||
This is the "what can I actually run today" guide. It follows the
|
||||
[feature matrix](feature-matrix.md): **M2 scope (filtering core: pass/block/
|
||||
quick/keep state/tables/anchors + enable/disable)** has VM evidence behind it;
|
||||
everything beyond that is either COMPILES-untested or PENDING. Install/build
|
||||
is covered by [install.md](install.md) and [build.md](build.md).
|
||||
[feature matrix](feature-matrix.md): the filtering core (pass/block/quick/
|
||||
keep state/tables/anchors), the NAT family, scrub, forced routing, pools,
|
||||
`divert`, `pflog` and the OpenBSD 8.0 queues all have **VM evidence** behind
|
||||
them — 15/15 lab gates are green. What is *not* gate-verified is listed
|
||||
explicitly below. Install/build is covered by [install.md](install.md) and
|
||||
[build.md](build.md).
|
||||
|
||||
## Daily loop
|
||||
|
||||
@@ -25,21 +27,21 @@ sudo rmmod pf # unload (rule/state flush happens here)
|
||||
|
||||
> **Warning — order matters**: `-f` then `-e`. Enabling with no matching `pass` rule
|
||||
> for your admin path blocks it — this port follows OpenBSD's default-block
|
||||
> policy, and the M2 gate runs have *actually cut the test VM's ssh* that
|
||||
> way (the lab keeps a serial console for exactly this). On a remote host,
|
||||
> pass your management interface before you enable anything.
|
||||
> policy, and the lab runs have *actually cut the test VM's ssh* that way
|
||||
> (the lab keeps a serial console for exactly this). On a remote host, pass
|
||||
> your management interface before you enable anything.
|
||||
|
||||
## A ruleset that works in this port today
|
||||
|
||||
```pf
|
||||
# /etc/pf.conf — M2 scope only. Interface names here match the VM lab
|
||||
# (mgmt/lan/wan); substitute your own (eth0, enp1s0, …).
|
||||
# /etc/pf.conf. Interface names here match the VM lab (mgmt/lan/wan);
|
||||
# substitute your own (eth0, enp1s0, …).
|
||||
|
||||
# 1) keep yourself reachable (do this first, always)
|
||||
pass in quick on mgmt
|
||||
pass out quick
|
||||
|
||||
# 2) filter core — proven today: quick, proto/port matching, state, drop
|
||||
# 2) filter core — proven in the lab: quick, proto/port matching, state, drop
|
||||
table <blk> persist # live blocklist (gate 03 pattern)
|
||||
block in quick proto tcp from <blk> to any
|
||||
|
||||
@@ -56,12 +58,17 @@ grammar **parse** fine — `make -C tools test` exercises them offline — but
|
||||
the *kernel behavior* of most `set` options has not been gate-verified yet;
|
||||
treat them as load-and-pray until the matrix moves them.
|
||||
|
||||
## What to avoid until the milestones land
|
||||
**Not gate-verified** (honest gaps): the IPv6 path, OS fingerprinting, the
|
||||
`set` options above, syncookies and the FQ-CoDel discipline. `af-to` is
|
||||
compiled out in this build (`INET6 0`).
|
||||
|
||||
## Feature notes — what each syntax does today
|
||||
|
||||
| Syntax | Until | What happens today |
|
||||
|---|---|---|
|
||||
| `nat-to`, `rdr-to`, `binat-to` | — | **Verified in the lab** (gates `05-nat`, `06-rdr`, the latter pulls 2 MB through a redirect and compares it byte for byte). `nat-to (egress)` expands the OpenBSD interface *group*; on a multi-NIC host prefer the explicit `nat-to (<iface>)` form. `binat-to` is expanded by pfctl into a `nat-to … static-port` + `rdr-to` pair (gate `07-binat`). |
|
||||
| `scrub (min-ttl, max-mss, df, random-id)`, `set reassemble` | — | **Verified in the lab** (gate `09-scrub.sh`): `min-ttl` and `max-mss` measurably rewrite transit packets. || `divert-to`, `divert` tag | — | **Verified in the lab** (gate `11-divert`): the packet is delivered to an `AF_DIVERT` socket (`PF_DIVERT=32`) served by `tools/pdivert`; with no listener it is dropped, exactly like BSD's `divs_noport`. |
|
||||
| `scrub (min-ttl, max-mss, df, random-id)`, `set reassemble` | — | **Verified in the lab** (gate `09-scrub.sh`): `min-ttl` and `max-mss` measurably rewrite transit packets. |
|
||||
| `divert-to`, `divert` tag | — | **Verified in the lab** (gate `11-divert`): the packet is delivered to an `AF_DIVERT` socket (`PF_DIVERT=32`) served by `tools/pdivert`; with no listener it is dropped, exactly like BSD's `divs_noport`. |
|
||||
| `route-to`, `dup-to`, `reply-to` | — | **Verified in the lab** (gates `08-routeto.sh`, `13-replyto.sh`): the forced gateway is used instead of the normal egress, `dup-to` copies while keeping the flow, and `reply-to` pushes a flow's answers out through the gateway link. `fail-to` is not a keyword in OpenBSD 8.0 (removed upstream). |
|
||||
| `queue` / `queue on` / `pfctl -s queue` | — | **Verified in the lab** (gate `15-queue.sh`): queues load, hfsc attaches, `pfctl -s queue -v` reports class statistics and a queued 2 MB flow is counted by the class. `bandwidth` is a *guarantee under contention*, not a cap - 8.0's pf.conf has no upperlimit keyword. |
|
||||
| `log`, `log all`, `block log`, `pflogd` | — | **Verified in the lab** (gate `12-pflog`): run `pflogd /tmp/x.pcap` (reads `/dev/pflog`); the file is a `DLT_PFLOG` (117) pcap readable by tcpdump/wireshark. |
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Screencasts / Videos da demonstracao
|
||||
|
||||
> PT-BR: [docs/pt-BR/screencast.md](pt-BR/screencast.md) · EN: this page
|
||||
> PT-BR: [docs/pt-BR/screencast.md](../pt-BR/screencast.md) · EN: this page
|
||||
|
||||
Sete videos por idioma, com **narracao com voz feminina** (MiniMax),
|
||||
**legendas queimadas**, **diagrama de rede do lab** e um **filme completo**
|
||||
@@ -52,7 +52,7 @@ Etapas individuais:
|
||||
```sh
|
||||
python3 scripts/screencast/make_audio.py --lang pt # audio MiniMax
|
||||
python3 scripts/screencast/record.py --plan scripts/screencast/plans/04-roteamento.json \\
|
||||
--out out/screencast/caps/04-roteamento.cast --pace
|
||||
--out out/screencast/caps/04-roteamento.cast --pace --ensure-pf
|
||||
python3 scripts/screencast/assemble.py --phase 04-roteamento --lang pt
|
||||
python3 scripts/screencast/join.py --lang pt
|
||||
```
|
||||
|
||||
+60
-22
@@ -3,8 +3,8 @@
|
||||
Português: [docs/pt-BR/vm-lab.md](../pt-BR/vm-lab.md).
|
||||
|
||||
All commands below were read from the scripts in `scripts/` (they are the
|
||||
source of truth); the lab has been created and run during M0–M2 — a full-suite
|
||||
green run is the M2+ gate and is still in progress (see §6).
|
||||
source of truth). The lab has been created and run through M0–M6; the full
|
||||
suite is green (§6).
|
||||
|
||||
## 1. Where it lives
|
||||
|
||||
@@ -61,6 +61,20 @@ isolated bridges `pf-lan` 10.0.1.0/24, `pf-wan` 10.0.2.0/24, `pf-wan2`
|
||||
running. Laptop ssh aliases (`pf0|cli|srv|gw2`, ProxyJump via deb) are
|
||||
generated into `~/.ssh/config.d/pflab`.
|
||||
|
||||
**From pf0 to the other VMs.** The firewall has its own keypair and the
|
||||
peers trust it, so a session on pf0 can drive the client and the server
|
||||
directly — this is what the screencasts do (`ssh cli 'ping …'`):
|
||||
|
||||
```sh
|
||||
# once per lab, from the hypervisor
|
||||
ssh pf0 'ssh-keygen -t ed25519 -N "" -C pf0 -f /root/.ssh/id_ed25519'
|
||||
# then authorize that public key on cli, srv and gw2
|
||||
```
|
||||
|
||||
`/root/.ssh/config` on pf0 maps the short names `cli`, `srv` and `gw2` to
|
||||
10.0.1.10, 10.0.2.20 and 10.0.3.2, and `/etc/hosts` carries the same
|
||||
mapping.
|
||||
|
||||
## 4. Running the suite
|
||||
|
||||
```sh
|
||||
@@ -78,28 +92,52 @@ and exits non-zero if anything failed.
|
||||
|
||||
## 5. What each gate proves today
|
||||
|
||||
| Gate | Scenario (asserted with real traffic, not exit codes) |
|
||||
Every gate asserts **real traffic and observable state**, never just an exit
|
||||
code: the packet either arrives, is dropped, or is translated, and the proof
|
||||
is read back from the firewall's own counters, state table, capture files or
|
||||
peer.
|
||||
|
||||
The full list is in §6 (15 gates, all green). Each gate is a shell script in
|
||||
`tests/vm/` that talks to the VMs through deb with nested ssh, using the
|
||||
helpers in `tests/vm/lib.sh` (`pf0`, `cli`, `srv`, `gw2`, `pfput`, `pfload`,
|
||||
`ok`, `bad`). `scripts/vm-run-tests.sh` runs them in order and captures
|
||||
`out/test-logs/<gate>.sh.log`.
|
||||
|
||||
Not gated (honest gaps): the IPv6 path, OS fingerprinting, `set` options
|
||||
(`state-policy`, `allow-opts`) and syncookies — the code is compiled into
|
||||
`pf.ko` but no gate drives it. FQ-CoDel compiles and shares its stats header
|
||||
with `pfctl`, but only the **hfsc** discipline is exercised.
|
||||
|
||||
There is **no numeric performance gate**: `bench-basic.sh` does not exist in
|
||||
the tree and no throughput target is claimed.
|
||||
|
||||
## 6. Current suite status (2026-11-12)
|
||||
|
||||
**15/15 gates green.** A full clean run (`./scripts/vm-run-tests.sh`, ~9 min)
|
||||
recreates and reboots `pftest-pf0` before every gate, redeploys the module
|
||||
and the tools, and ends with `PASS=15 FAIL=0`. The captured logs live in
|
||||
`out/test-logs/<gate>.sh.log`.
|
||||
|
||||
| Gate | What it proves now |
|
||||
|---|---|
|
||||
| `01-state.sh` | `pass in/out quick proto icmp keep state` → cli pings srv **through** pf0 (transit); pf0 then lists an icmp state via `pfctl -s states` |
|
||||
| `02-block.sh` | `block drop in quick proto icmp` → cli ping fails; `block return-rst in quick proto tcp … port 8080` → cli's `nc` to srv:8080 is refused (RST), checked via nc exit |
|
||||
| `03-tables.sh` | `table <blk> { 10.0.1.10 }` + `block in quick from <blk>` → `pfctl -t blk -T show` lists it; cli ping blocked; live `-T delete` unblocks the same flow |
|
||||
| `04-anchors.sh` | anchor rule (`-a restricted -f`) passing icmp is evaluated **before** a later main-ruleset `block` (quick/top-placement); `pfctl -s Anchors` lists the anchor |
|
||||
| `01-state.sh` | `keep state` across a transit ping; the state shows up in `pfctl -ss` |
|
||||
| `02-block.sh` | `block drop` kills the ping; `block return-rst` gets the client an RST |
|
||||
| `03-tables.sh` | a `pfr` table blocks a host; a live `-T delete` unblocks the same flow |
|
||||
| `04-anchors.sh` | an anchor rule is evaluated before a later main-ruleset `block` |
|
||||
| `05-nat.sh` | source NAT: the packet leaves with the pool address, the reply is de-NATed |
|
||||
| `06-rdr.sh` | port redirect end to end, plus 2 MB through it with byte-for-byte integrity |
|
||||
| `07-binat.sh` | `binat-to`: the peer sees the mapped address, the original never leaks |
|
||||
| `08-routeto.sh` | `route-to` leaves via the gateway's link; `dup-to` copies and keeps the flow |
|
||||
| `09-scrub.sh` | `min-ttl` and `max-mss` measured on the firewall's own egress |
|
||||
| `10-pools.sh` | NAT pool selection (`round-robin`, `source-hash`) |
|
||||
| `11-divert.sh` | `divert-to` to the `pdivert` relay: packet to userspace and back |
|
||||
| `12-pflog.sh` | logged packets reach `/dev/pflog` and land in a DLT_PFLOG pcap |
|
||||
| `13-replyto.sh` | `reply-to` forces the reply onto the gateway link |
|
||||
| `14-soak.sh` | ruleset reload under traffic, `rmmod` cycle, leak check |
|
||||
| `15-queue.sh` | hfsc queues: attach, class statistics, a 2 MB queued flow counted by the class |
|
||||
|
||||
Planned gates per the plan document (T11–T18): 05-nat, 06-rdr, 07-binat,
|
||||
08-scrub (scapy), 09-routeto, 10-dup-fail, 11-divert, 12-queues (iperf rate
|
||||
tolerance ±15%), 13-pflog, 14-soak (reload under traffic + rmmod cycle +
|
||||
kmemleak). IPv6 is smoke-only for state/block/nat66/scrub. `bench-basic.sh`
|
||||
(planned, not in the tree yet) is informational; there is no numeric perf gate.
|
||||
|
||||
## 6. Current suite status (2026-11-11, honestly)
|
||||
|
||||
The M2 gates 01–04 exist and run, but **are not all green yet** — the data
|
||||
path (netisr workqueue model) and state visibility are being fixed in the M2
|
||||
gate work. The latest captured run (`out/test-logs/`) shows the mixed state:
|
||||
transit ping OK / state listing failing on 01, icmp-blocked OK on 02, partial
|
||||
results on 03, 04 aborted. Treat "VERIFIED-in-VM" in the
|
||||
[feature matrix](feature-matrix.md) as reserved for gates that pass in a
|
||||
full clean run; nothing beyond M1 is claimed VERIFIED until then.
|
||||
The screencasts ([screencast.md](screencast.md)) show most of these gates
|
||||
being run, with the lab's network diagram drawn over the terminal.
|
||||
|
||||
## 7. Manual poking
|
||||
|
||||
|
||||
Reference in new issue
Block a user