68 Commits
Author SHA1 Message Date
lgcosta dff2185b20 Docs: bring the documentation in line with the code
- Demo docs replaced by the screencast docs (single source of truth): the
  legacy demo-session.sh/demo-record.py and the v0.9 renderer were removed
  instead of being kept as a second convention.
- feature-matrix: stale 'M2-in-progress' vocabulary gone; the rows the 15
  gates prove now say VERIFIED-in-VM and name the gate; af-to is now an
  explicit non-goal row (INET6 0 in this build).
- guide: the queues/divert/pflog sections described a state that no longer
  exists (priq-only, divert dropping packets, pflog logging nothing); they
  now describe the shipped behaviour.
- vm-lab: the 15 real gates with what each proves, 15/15 green, and the
  pf0 -> cli/srv/gw2 ssh setup the screencasts rely on.
- install/build: pdivert and pflogd are built and ported (docs said
  otherwise); module object list now includes hfsc.c, fq_codel.c, pfk_pflog.c;
  the screencast pipeline is an optional build stage.
- pf-guide: the 'M2 scope only' framing is gone; honest gaps listed.
- caveats: three new sections about the screencast pipeline (MiniMax video
  generation blocked by the account tier, narration as a separate track, a
  video frame is not a verdict).
- porting-notes: what the recording taught us.
- README/README.pt-BR: status, milestones, layout and video section updated.

All internal doc links verified.
v1.0.1
2026-11-12 11:51:25 -03:00
lgcosta cc67531602 Screencasts v2: real shell sessions, MiniMax narration (EN/PT), lab diagram
O problema da primeira serie era a sensacao de 'quadro a quadro': a captura
era a saida de um script despejada em blocos.  Agora cada video e uma sessao
shell de verdade.

- record.py: abre um pty, faz SSH no firewall e digita os comandos tecla a
  tecla (~34 c/s, com jitter).  O tty remoto ecoa, o kernel responde ao vivo;
  a captura e asciicast v2 com timestamps.  'bash --noediting' evita que o
  editor de linha redesenhe o comando e o tty o repita (a linha aparecia
  duplicada).
- record.py --pace: segura a tela enquanto a voz fala - cada batida dura o
  trecho de audio que a acompanha, de modo que uma unica captura serve as
  duas versoes de idioma.
- record.py --ensure-pf: garante pf.ko carregado antes de gravar.  Sem isso
  uma fase que roda 'rmmod' (ativacao) pode derrubar o /dev/pf de outra
  gravacao concorrentes - foi exatamente o que quebrou a fase de roteamento
  na primeira tentativa.
- make_audio.py: geracao da narracao pelo MiniMax TTS, uma voz feminina por
  idioma (English_ConfidentWoman / Portuguese_ConfidentWoman), um arquivo
  por segmento.  A API devolve o audio em hexadecimal, nao base64.
- render_video.py: reproduz a captura por um emulador de terminal e escreve
  os frames direto no ffmpeg como RGB bruto (um PNG por frame era uma ordem
  de grandeza mais lento).  Desenha o diagrama de rede do lab por cima do
  terminal nas fases de roteamento e balanceamento, destacando o caminho em
  discussao, e queima a legenda da narracao.
- assemble.py: mixa os segmentos de audio na linha do tempo.
- join.py: emenda as fases com crossfade sobre um cartao de capitulo e
  sincroniza o audio na mesma linha do tempo.
- build-all.sh: roda o pipeline inteiro.

Arte de fundo e audio vem do MiniMax; a geracao de video do MiniMax nao esta
habilitada nesta conta (erro 2068 em todos os modelos), entao as transicoes
sao renderizadas localmente com ffmpeg.

16 videos: 7 fases + filme completo, em ingles e portugues, com legenda
queimada.
v1.0.0
2026-11-12 10:57:29 -03:00
lgcosta fe324259c0 Screencasts: seis videos silenciosos 1080p30 da demonstracao
Novo pipeline de video: scripts/screencast/render.py roda uma fase em um pty,
emula o terminal com pyte, gera frames PNG e mux com ffmpeg (H.264,
yuv420p, +faststart, sem trilha de audio). Fallback de encoder: libx264 ->
libopenh264 -> VP9.

Fases (scripts/screencast/phases/, uma sessao real contra o lab por video):
  1. implementacao no kernel - fontes OpenBSD + compat + build limpo do pf.ko
  2. ativacao - insmod, /dev/pf, pfctl -e, pfctl -s info
  3. regras basicas - tabela, anchor, keep state, block drop
  4. roteamento - route-to e reply-to (provado com tcpdump nas 2 interfaces)
  5. balanceamento - pool round-robin, pool source-hash, dup-to
  6. QOS - filas hfsc, pfctl -s queue -v, transferencia de 2 MB pela fila

Os videos ficam versionados em out/screencast/ (.gitignore ajustado para
nao ignorar *.mp4) e o indice/texto de descricao do YouTube em
docs/{en,pt-BR}/screencast.md.

Correcoes na propria demo (encontradas gravando): a acao de uma regra no pf
e pass OU block (nunca os dois); as regras de um anchor vivem em um ruleset
proprio (pfctl -a <anchor> -f); o nc do QOS precisa rodar na VM do servidor,
nao no firewall; o rotulo 'FASE x/6' vem do numero da fase, nao da posicao
na lista de gravacao.
v0.9.0
2026-11-12 08:41:41 -03:00
lgcosta 90c51f43fc M5: queue gate 15 green (hfsc in the data path, pfctl -s queue works); vendor OpenBSD net/hfsc.h + net/fq_codel.h so kernel and pfctl share the ABI; pflog payload anchored; splassert no longer BUGs v0.8.0 2026-11-12 03:10:46 -03:00
lgcosta 4bade69cf9 docs: queue status honest - attach path reached, purge lock assert documented as the blocker (caveats #9) v0.7.0 2026-11-12 01:42:51 -03:00
lgcosta ccad35b247 hfsc: fix class-handle collision (root class used bit 0x1, colliding with odd pf qids -> pfq_addqueue EBUSY); guard against NULL discipline; drop debug prints 2026-11-12 01:40:49 -03:00
lgcosta 14ff7d1c39 ifq: real ifq_start (drain+transmit), dequeue cookie fix, pf_kifq_input hook in the data path; 14/14 suite unchanged 2026-11-12 01:15:26 -03:00
lgcosta f34c4f92a9 M5 step 1: build OpenBSD hfsc/fq_codel back ends into pf.ko (ml_* API, ifq_q handshake, scheduler types); priq still attached, 14/14 suite unchanged 2026-11-12 00:48:56 -03:00
lgcosta ca75fc49cc docs: guide milestone tables synced with the verified state (EN+PT-BR) v0.6.0 2026-11-11 23:59:03 -03:00
lgcosta 12e9c1a2f3 reply-to: gate 13 green (reply forced onto the gateway link); docs/README/CHANGELOG for 14 gates 2026-11-11 23:56:01 -03:00
lgcosta c2d1fc4c53 docs: fail-to is not an OpenBSD 8.0 keyword (removed upstream) - correct all references; reply-to gate added 2026-11-11 23:52:11 -03:00
lgcosta 3235af7f0b pools: gate 10 green (round-robin vs source-hash); docs/README/CHANGELOG for 13 gates 2026-11-11 23:45:24 -03:00
lgcosta 422a325a56 docs: drop leftover line in pt-BR patches note 2026-11-11 23:37:38 -03:00
lgcosta 80396e0b36 docs: patches helpers are in tree (verified PATCHES_APPLIED=0 / UNPATCH_DONE); route pool gate 10 written 2026-11-11 23:37:02 -03:00
lgcosta b3e7358868 scrub: gate 09 green (min-ttl/max-mss measured on egress); matrix + guides updated EN+PT-BR 2026-11-11 23:32:32 -03:00
lgcosta 251e5c09e0 demo+docs: 11-step recording (adds route-to), README status for 11/11 gates v0.5.0 2026-11-11 23:18:48 -03:00
lgcosta e26c6d4a36 changelog: v0.5.0 (binat, route-to/dup-to, checksum correctness) 2026-11-11 23:13:07 -03:00
lgcosta e7e941047f docs: route-to/dup-to verified (matrix EN+PT-BR, guides) 2026-11-11 23:11:56 -03:00
lgcosta 1bda8b4fa2 route-to: forced nexthop via rt_gw4/rt_gw_family (Linux resolves neighbours from the gateway only when the route says so); cleanup of debug instrumentation 2026-11-11 23:08:11 -03:00
lgcosta dc73274b19 route-to: fix skb reference imbalance in if_output_tso (stack output consumed the queue's ref -> double kfree_skb, wedging the box); look up the forced route by destination only 2026-11-11 22:35:05 -03:00
lgcosta ecab31d704 docs: caveats EN+PT-BR updated to the verified state (hook order, soak, states, divert/pflog consumers) 2026-11-11 22:21:25 -03:00
lgcosta 2668d99d5a binat: gate 07 green (pfctl's nat-to+rdr-to expansion verified both directions); docs updated 2026-11-11 22:14:44 -03:00
lgcosta bc5b8a4a1e demo: pflog step in the asciinema recording (pflogd: 2 records, DLT_PFLOG pcap); re-recorded cast v0.4.0 2026-11-11 22:07:45 -03:00
lgcosta 06ad1baa3a runner: deploy pdivert/pflogd alongside pf.ko and pfctl 2026-11-11 22:00:15 -03:00
lgcosta f263b63b8e docs: guides reflect verified NAT/redirect/divert/pflog and the fixed state listing; CHANGELOG for v0.4.0 2026-11-11 21:58:57 -03:00
lgcosta 92c015b97b docs: pflog verified (matrix EN+PT-BR, porting notes, README status) 2026-11-11 21:53:55 -03:00
lgcosta 7a81076c6a pflog: /dev/pflog chardev backend (BSD pflog_hdr layout, DLT_PFLOG) + tools/pflogd + gate 12 2026-11-11 21:52:36 -03:00
lgcosta dbb21048e4 tests: 06 gate self-sufficient (bracketed pkill patterns, srv cleanup) - avoids cross-run listener contamination 2026-11-11 21:35:50 -03:00
lgcosta 0eb040fcc8 docs: rdr-to verified; porting notes on checksum strategy across translation paths (GSO regression) 2026-11-11 21:27:32 -03:00
lgcosta 5af7adb08e checksums: GSO-aware recompute after translation (software for plain skbs, CHECKSUM_PARTIAL for GSO); 06 gate asserts 2MB integrity through rdr 2026-11-11 21:24:41 -03:00
lgcosta d9fc3a7e3f checksums: materialize CHECKSUM_PARTIAL before pf so address-rewrite fixups operate on real values; new 06-rdr gate 2026-11-11 20:53:03 -03:00
lgcosta b554c9a6ef demo: ship the asciinema recording in the repo (out/demo-pf-linux.cast) 2026-11-11 20:33:57 -03:00
lgcosta 6cc4e21ae2 docs+demo: asciinema recording of the lab (stateful filtering, tables, drop, NAT, divert); README/matrix reflect the verified state 2026-11-11 20:33:27 -03:00
lgcosta 2cd13eb806 tests: soak helper ignores the CPU SRSO advisory 2026-11-11 20:29:33 -03:00
lgcosta e03df9adc8 docs: data-path semantics and NAT checklist 2026-11-11 20:25:57 -03:00
lgcosta 7d1e4b81e9 cleanup: drop debug instrumentation 2026-11-11 20:24:53 -03:00
lgcosta f3143a54a8 nat: route the reverse-translated packet with an empty source (kernel rejects non-local saddr) 2026-11-11 20:23:44 -03:00
lgcosta 917d3e3fe0 nat: recompute IPv4 header checksum after pf rewrote the addresses (OpenBSD ip_output does this) 2026-11-11 20:20:52 -03:00
lgcosta d520172753 nat: re-route inbound packets whose destination pf rewrote (reverse NAT) instead of delivering them locally 2026-11-11 20:17:27 -03:00
lgcosta 9c3c4b6bbc fix: skb->mark masks were multi-bit (IN mark matched OUT test) - transit traffic never reached the out ruleset; ingress/egress interface groups; pfctl resolves (egress) pool locally 2026-11-11 20:15:35 -03:00
lgcosta c449e01b9a pf: emulate OpenBSD ingress/egress interface groups as any-interface (Linux has no ifgroups) 2026-11-11 20:07:17 -03:00
lgcosta 525fe7ee28 tests: gate assertions corrected (pfctl state format, transit egress device, CPU SRSO advisory excluded); Makefile demo target 2026-11-11 20:04:16 -03:00
lgcosta 18c0a69f74 tests: soak gate via uploaded helper (quoting), 04/05 assertions fixed (pf interface name vs OS device) 2026-11-11 19:59:27 -03:00
lgcosta 3aa0c2d0d0 docs: demo guide (asciinema recording/playback/upload) EN+PT-BR 2026-11-11 19:56:24 -03:00
lgcosta 66939c839a tests: 04 gate rewritten to OpenBSD-correct anchor semantics (reply is evaluated by the in ruleset); 05 gate uses 8.0 nat-to syntax; 14 quoting fixed; demo session + asciinema recorder 2026-11-11 19:54:29 -03:00
lgcosta 444a0555c6 docs: divert verified (matrix + porting notes); FORWARD hook evaluates the out ruleset for transit traffic 2026-11-11 19:50:14 -03:00
lgcosta bdf9db7a21 fix: remove leftover debug fragment in pfk_hooks 2026-11-11 19:45:43 -03:00
lgcosta 57ecc15606 divert: end-to-end working (network-order port, direct IN-verdict handoff, L2-synth reinjection, skip-mark anti-loop, pdivert ICMP checksum fix) 2026-11-11 19:45:29 -03:00
lgcosta 4728648a0c divert: consume PF_TAG_DIVERTED after passthrough (BSD ip_input role); divert layer owns its own skb ref; gate uses pfput path 2026-11-11 19:19:53 -03:00
lgcosta da1f678125 divert: fix NULL-deref on empty listener list (wedged pf0 in gate 11); gate 11 pdivert path; runner accepts gate filter arg 2026-11-11 19:15:48 -03:00