AF_DIVERT family restored & compiling: full socket family (PF_DIVERT=32, sock_register), blocking recv/poll, divert_output-compatible sendmsg reinject, mtag type passthrough; pfk_api decls

docs/en/build.md

@@ -0,0 +1,116 @@
+# Build pipeline in detail
+
+Português: [docs/pt-BR/build.md](../pt-BR/build.md).
+Quick recipe: [install.md](install.md). This page explains *what each stage
+does* and *what has actually been exercised*.
+
+Everything below runs on the build host (the laptop); the VM lab (separate
+host "deb") only receives artifacts — see [vm-lab.md](vm-lab.md).
+
+## 0. Prerequisites
+
+From `docs/en/install.md`: gcc ≥ 15, GNU make, bison (flex is not needed —
+OpenBSD `parse.y` carries a hand-written `yylex`), `bc`, rsync, `glibc-static`
+for the static pfctl, root for `insmod`. Kernel tree expected at
+`../linux-7.3.4` (override with `KERNELDIR=`; the root `Makefile` defaults to
+`$(HOME)/lab/linux_kernel/linux-7.3.4`).
+
+## 1. Fetch + verify OpenBSD sources — `scripts/fetch-sources.sh`
+
+- Downloads `sys.tar.gz` and `src.tar.gz` from the primary mirror
+  `https://openbsd.c3sl.ufpr.br/pub/OpenBSD/8.0/`, falling back to
+  `cdn.openbsd.org`.
+- Verifies sha256 against the pins in `VERSIONS` (sys: `964b1aed…`, src:
+  `9984c74f…`); an already-cached tarball is skipped when its hash matches.
+- Landing zone: `_vendor/openbsd/` (gitignored).
+
+## 2. Vendorize — `scripts/vendorize.sh`
+
+Extracts **only the files the port consumes** into `_vendor/upstream/`
+(gitignored): the `sys/net/pf*.c|h`, `ifq/hfsc/fq_codel/ip_pflog/ip_divert`,
+`radix/rtable/toeplitz`, `crypto/{md5,sha2,siphash}`, `sys/sys/*.h` list and
+`sbin/pfctl/*` + headers (see the `SYS_K`/`SRC_K` arrays). It then records
+each file's `$OpenBSD: …$` tag into `_vendor/upstream/IDS.txt` — the forensic
+baseline for every ported line and for future rebases.
+
+## 3. Staging into the port — `scripts/stage-port.sh` (deterministic porting)
+
+`stage-port.sh` turns pristine upstream files into the files under `module/`:
+
+1. every `#include` is replaced by `pfk.h` + the staged `pfvar` headers
+   (kernel branches selected with `-D_KERNEL=1`);
+2. the shared rename program (`pf_sed`) resolves BSD↔Linux name collisions
+   (`struct ip` → `struct pf_ip`, `ifnet` → `pfk_ifnet`, `ip_output` →
+   `pfk_ip_output`, `sa_len` assignments stripped, …);
+3. **each run is appended to `docs/porting-notes.md`** — that file is the
+   staging log, not prose.
+
+Ported files are then committed and hand-edited in-tree; re-staging is only
+for forensics (`stage-port.sh --diff <file>` compares our copy against the
+pristine upstream text). Radix/rtable/siphash/toeplitz are deliberately not
+auto-staged (hand-ported).
+
+## 4. Kernel — `scripts/build-kernel.sh` (+ `kernel-config.sh`)
+
+- `kernel-config.sh`: starts from `defconfig` if there is no `.config`, then
+  sets (idempotently, via `scripts/config` + `make olddefconfig`):
+  `VIRTIO_BLK/PCI/NET=y`, `EXT4_FS=y`, `NETFILTER=y`, `NETFILTER_ADVANCED`,
+  `NET_SCHED=y`, `INET=y`, `IPV6=y`, `SERIAL_8250(+CONSOLE)=y` — i.e.
+  everything needed to direct-boot the pf0 VM with **no initrd**.
+  `kernel-config.sh test` additionally enables `DEBUG_INFO` +
+  `DEBUG_KMEMLEAK` for soak builds.
+- `build-kernel.sh`: `make -j(nproc-2) bzImage modules_prepare`, ensures
+  `Module.symvers` (copies `vmlinux.symvers` — kbuild does not write the
+  former) and `scripts/module.lds` exist, then drops boot artifacts in
+  `out/`: `linux-<ver>-bzImage`, `System.map`, `.config`, `kernelversion`,
+  `bzImage.sha256`. Prints `KERNEL_BUILD_OK <kernelrelease>`.
+- **Kernel core patches are currently none**: `patches/kernel/` holds no
+  `000N-*.patch` (see [patches.md](patches.md)).
+
+## 5. Module — `scripts/build-module.sh`
+
+`make -C "$KERNELDIR" M=module` → `module/pf.ko` (single module today; the
+`pfq` qdisc arrives with M5). The script self-prepares `Module.symvers` /
+`modules_prepare` first. Object list (`module/Makefile`): ported engine
+(`pf.c`, `pf_ioctl.c`, `pf_norm.c`, `pf_if.c`, `pf_lb.c`, `pf_ruleset.c`,
+`pf_table.c`, `pf_osfp.c`, `pf_syncookies.c`, `divert_sock.c`) + compat layer
+(mbuf shim, kern glue, if/ip4/ip6/pcb/xmit/ifq, crypto, radix, subr_tree) +
+`pfk_mod.c`/`pfk_hooks.c`. Compile flags define `PFLINUX=1` and `_KERNEL=1`.
+Prints `MODULE_BUILD_OK`.
+
+`insmod module/pf.ko` creates `/dev/pf` (0600) and registers the netfilter
+hooks; `dmesg` shows `pfk: …` lines (see [caveats.md](caveats.md) for what
+the data path does).
+
+## 6. Userland — `make -C tools`
+
+Targets (from `tools/Makefile`): `pfctl`, `abitest`, `abi`, `test`, `clean`.
+
+- `pfctl` — the OpenBSD 8.0 sources compiled against the copied ABI headers;
+  result is a **static binary** at `tools/out/pfctl`. Every deviation from
+  upstream is listed in `tools/UPSTREAM.md` (7 marked edits; the rest is the
+  compat header/source set).
+- `abi` — builds `abitest` (userland) and `abitest-module` (same table under
+  kernel compat headers), dumps both struct-size/offset tables and `diff`s
+  them; a mismatch means the ioctl ABI drifted. `out/abi-*.tbl` are the
+  reference inputs.
+- `test` — offline parse checks: runs `pfctl -n -f` over `tools/tests/*.conf`
+  (basic / translations / queues+scrub / anchor / negative `broken.conf`).
+  This is pure userland; it needs no `/dev/pf`.
+- `pdivert` — divert relay source under `tools/pdivert/`; see
+  [feature-matrix.md](feature-matrix.md) for its (pending) kernel side.
+
+## 7. Root `Makefile`
+
+`make module | tools | kernel | vm | vm-destroy | test | docs | clean` — thin
+wrappers over the scripts above. `make test` runs the VM suite
+([vm-lab.md](vm-lab.md)); `make docs` just lists the bilingual doc set.
+
+## What has been exercised vs. not
+
+- Stages 1–6 have each been run during M0–M2 development on the build host
+  (artifacts exist in-tree: `out/`, `module/pf.ko`, `tools/out/pfctl`), and
+  the scripts above were verified line-by-line for this document.
+- A **fresh-clone end-to-end run** (`quickstart-check.sh`, plan task T19) has
+  *not* been executed yet; treat the exact command sequence in install.md as
+  "as designed, last-run state per milestone" rather than CI-verified.