Commit Graph
26 Commits
Author SHA1 Message Date
lgcosta 0a8ba69340 docs: record the stress findings and the 16-gate suite
caveats §13 documents the capacity ceiling the stress test found: the
workqueue backlog grows with the offered rate and OOM-panics a 1 GB guest
above ~1 Gbit/s. Gate count, suite status and the doc indexes move to
16/16 green.
2026-11-12 20:11:12 -03:00
lgcosta 928322edf7 feat(debian13): add host prep script and implementation manual
The manual is the from-zero path on Debian 13 (trixie): packages, the
7.3.4 kernel tree, pf.ko, the userland, the libvirt lab, the gate suite,
the stress test and running PF on the host itself. Every command in it was
executed on the lab's Debian 13.7 host, including the toolchain report
and the kernel.org checksum.

Trixie ships gcc 14 and no gcc-15; the kernel and pf.ko build with gcc
14 unpatched, so the manual documents that instead of the gcc >= 15 the
generic install guide asks for.
2026-11-12 20:11:09 -03:00
lgcosta dff2185b20 Docs: bring the documentation in line with the code
- Demo docs replaced by the screencast docs (single source of truth): the
  legacy demo-session.sh/demo-record.py and the v0.9 renderer were removed
  instead of being kept as a second convention.
- feature-matrix: stale 'M2-in-progress' vocabulary gone; the rows the 15
  gates prove now say VERIFIED-in-VM and name the gate; af-to is now an
  explicit non-goal row (INET6 0 in this build).
- guide: the queues/divert/pflog sections described a state that no longer
  exists (priq-only, divert dropping packets, pflog logging nothing); they
  now describe the shipped behaviour.
- vm-lab: the 15 real gates with what each proves, 15/15 green, and the
  pf0 -> cli/srv/gw2 ssh setup the screencasts rely on.
- install/build: pdivert and pflogd are built and ported (docs said
  otherwise); module object list now includes hfsc.c, fq_codel.c, pfk_pflog.c;
  the screencast pipeline is an optional build stage.
- pf-guide: the 'M2 scope only' framing is gone; honest gaps listed.
- caveats: three new sections about the screencast pipeline (MiniMax video
  generation blocked by the account tier, narration as a separate track, a
  video frame is not a verdict).
- porting-notes: what the recording taught us.
- README/README.pt-BR: status, milestones, layout and video section updated.

All internal doc links verified.
2026-11-12 11:51:25 -03:00
lgcosta cc67531602 Screencasts v2: real shell sessions, MiniMax narration (EN/PT), lab diagram
O problema da primeira serie era a sensacao de 'quadro a quadro': a captura
era a saida de um script despejada em blocos.  Agora cada video e uma sessao
shell de verdade.

- record.py: abre um pty, faz SSH no firewall e digita os comandos tecla a
  tecla (~34 c/s, com jitter).  O tty remoto ecoa, o kernel responde ao vivo;
  a captura e asciicast v2 com timestamps.  'bash --noediting' evita que o
  editor de linha redesenhe o comando e o tty o repita (a linha aparecia
  duplicada).
- record.py --pace: segura a tela enquanto a voz fala - cada batida dura o
  trecho de audio que a acompanha, de modo que uma unica captura serve as
  duas versoes de idioma.
- record.py --ensure-pf: garante pf.ko carregado antes de gravar.  Sem isso
  uma fase que roda 'rmmod' (ativacao) pode derrubar o /dev/pf de outra
  gravacao concorrentes - foi exatamente o que quebrou a fase de roteamento
  na primeira tentativa.
- make_audio.py: geracao da narracao pelo MiniMax TTS, uma voz feminina por
  idioma (English_ConfidentWoman / Portuguese_ConfidentWoman), um arquivo
  por segmento.  A API devolve o audio em hexadecimal, nao base64.
- render_video.py: reproduz a captura por um emulador de terminal e escreve
  os frames direto no ffmpeg como RGB bruto (um PNG por frame era uma ordem
  de grandeza mais lento).  Desenha o diagrama de rede do lab por cima do
  terminal nas fases de roteamento e balanceamento, destacando o caminho em
  discussao, e queima a legenda da narracao.
- assemble.py: mixa os segmentos de audio na linha do tempo.
- join.py: emenda as fases com crossfade sobre um cartao de capitulo e
  sincroniza o audio na mesma linha do tempo.
- build-all.sh: roda o pipeline inteiro.

Arte de fundo e audio vem do MiniMax; a geracao de video do MiniMax nao esta
habilitada nesta conta (erro 2068 em todos os modelos), entao as transicoes
sao renderizadas localmente com ffmpeg.

16 videos: 7 fases + filme completo, em ingles e portugues, com legenda
queimada.
2026-11-12 10:57:29 -03:00
lgcosta fe324259c0 Screencasts: seis videos silenciosos 1080p30 da demonstracao
Novo pipeline de video: scripts/screencast/render.py roda uma fase em um pty,
emula o terminal com pyte, gera frames PNG e mux com ffmpeg (H.264,
yuv420p, +faststart, sem trilha de audio). Fallback de encoder: libx264 ->
libopenh264 -> VP9.

Fases (scripts/screencast/phases/, uma sessao real contra o lab por video):
  1. implementacao no kernel - fontes OpenBSD + compat + build limpo do pf.ko
  2. ativacao - insmod, /dev/pf, pfctl -e, pfctl -s info
  3. regras basicas - tabela, anchor, keep state, block drop
  4. roteamento - route-to e reply-to (provado com tcpdump nas 2 interfaces)
  5. balanceamento - pool round-robin, pool source-hash, dup-to
  6. QOS - filas hfsc, pfctl -s queue -v, transferencia de 2 MB pela fila

Os videos ficam versionados em out/screencast/ (.gitignore ajustado para
nao ignorar *.mp4) e o indice/texto de descricao do YouTube em
docs/{en,pt-BR}/screencast.md.

Correcoes na propria demo (encontradas gravando): a acao de uma regra no pf
e pass OU block (nunca os dois); as regras de um anchor vivem em um ruleset
proprio (pfctl -a <anchor> -f); o nc do QOS precisa rodar na VM do servidor,
nao no firewall; o rotulo 'FASE x/6' vem do numero da fase, nao da posicao
na lista de gravacao.
2026-11-12 08:41:41 -03:00
lgcosta 90c51f43fc M5: queue gate 15 green (hfsc in the data path, pfctl -s queue works); vendor OpenBSD net/hfsc.h + net/fq_codel.h so kernel and pfctl share the ABI; pflog payload anchored; splassert no longer BUGs 2026-11-12 03:10:46 -03:00
lgcosta 4bade69cf9 docs: queue status honest - attach path reached, purge lock assert documented as the blocker (caveats #9) 2026-11-12 01:42:51 -03:00
lgcosta f34c4f92a9 M5 step 1: build OpenBSD hfsc/fq_codel back ends into pf.ko (ml_* API, ifq_q handshake, scheduler types); priq still attached, 14/14 suite unchanged 2026-11-12 00:48:56 -03:00
lgcosta ca75fc49cc docs: guide milestone tables synced with the verified state (EN+PT-BR) 2026-11-11 23:59:03 -03:00
lgcosta 12e9c1a2f3 reply-to: gate 13 green (reply forced onto the gateway link); docs/README/CHANGELOG for 14 gates 2026-11-11 23:56:01 -03:00
lgcosta c2d1fc4c53 docs: fail-to is not an OpenBSD 8.0 keyword (removed upstream) - correct all references; reply-to gate added 2026-11-11 23:52:11 -03:00
lgcosta 3235af7f0b pools: gate 10 green (round-robin vs source-hash); docs/README/CHANGELOG for 13 gates 2026-11-11 23:45:24 -03:00
lgcosta 422a325a56 docs: drop leftover line in pt-BR patches note 2026-11-11 23:37:38 -03:00
lgcosta 80396e0b36 docs: patches helpers are in tree (verified PATCHES_APPLIED=0 / UNPATCH_DONE); route pool gate 10 written 2026-11-11 23:37:02 -03:00
lgcosta b3e7358868 scrub: gate 09 green (min-ttl/max-mss measured on egress); matrix + guides updated EN+PT-BR 2026-11-11 23:32:32 -03:00
lgcosta e7e941047f docs: route-to/dup-to verified (matrix EN+PT-BR, guides) 2026-11-11 23:11:56 -03:00
lgcosta ecab31d704 docs: caveats EN+PT-BR updated to the verified state (hook order, soak, states, divert/pflog consumers) 2026-11-11 22:21:25 -03:00
lgcosta 2668d99d5a binat: gate 07 green (pfctl's nat-to+rdr-to expansion verified both directions); docs updated 2026-11-11 22:14:44 -03:00
lgcosta f263b63b8e docs: guides reflect verified NAT/redirect/divert/pflog and the fixed state listing; CHANGELOG for v0.4.0 2026-11-11 21:58:57 -03:00
lgcosta 92c015b97b docs: pflog verified (matrix EN+PT-BR, porting notes, README status) 2026-11-11 21:53:55 -03:00
lgcosta 0eb040fcc8 docs: rdr-to verified; porting notes on checksum strategy across translation paths (GSO regression) 2026-11-11 21:27:32 -03:00
lgcosta 6cc4e21ae2 docs+demo: asciinema recording of the lab (stateful filtering, tables, drop, NAT, divert); README/matrix reflect the verified state 2026-11-11 20:33:27 -03:00
lgcosta 3aa0c2d0d0 docs: demo guide (asciinema recording/playback/upload) EN+PT-BR 2026-11-11 19:56:24 -03:00
lgcosta 444a0555c6 docs: divert verified (matrix + porting notes); FORWARD hook evaluates the out ruleset for transit traffic 2026-11-11 19:50:14 -03:00
lgcosta 0fe4d83b93 docs: divert status corrected after AF_DIVERT restore (COMPILES untested, M4 e2e gate defined) 2026-11-11 18:50:29 -03:00
lgcosta ac56534c3b AF_DIVERT family restored & compiling: full socket family (PF_DIVERT=32, sock_register), blocking recv/poll, divert_output-compatible sendmsg reinject, mtag type passthrough; pfk_api decls 2026-11-11 18:47:04 -03:00