2 Commits
Author SHA1 Message Date
lgcosta fe324259c0 Screencasts: seis videos silenciosos 1080p30 da demonstracao
Novo pipeline de video: scripts/screencast/render.py roda uma fase em um pty,
emula o terminal com pyte, gera frames PNG e mux com ffmpeg (H.264,
yuv420p, +faststart, sem trilha de audio). Fallback de encoder: libx264 ->
libopenh264 -> VP9.

Fases (scripts/screencast/phases/, uma sessao real contra o lab por video):
  1. implementacao no kernel - fontes OpenBSD + compat + build limpo do pf.ko
  2. ativacao - insmod, /dev/pf, pfctl -e, pfctl -s info
  3. regras basicas - tabela, anchor, keep state, block drop
  4. roteamento - route-to e reply-to (provado com tcpdump nas 2 interfaces)
  5. balanceamento - pool round-robin, pool source-hash, dup-to
  6. QOS - filas hfsc, pfctl -s queue -v, transferencia de 2 MB pela fila

Os videos ficam versionados em out/screencast/ (.gitignore ajustado para
nao ignorar *.mp4) e o indice/texto de descricao do YouTube em
docs/{en,pt-BR}/screencast.md.

Correcoes na propria demo (encontradas gravando): a acao de uma regra no pf
e pass OU block (nunca os dois); as regras de um anchor vivem em um ruleset
proprio (pfctl -a <anchor> -f); o nc do QOS precisa rodar na VM do servidor,
nao no firewall; o rotulo 'FASE x/6' vem do numero da fase, nao da posicao
na lista de gravacao.
2026-11-12 08:41:41 -03:00
lgcosta 90c51f43fc M5: queue gate 15 green (hfsc in the data path, pfctl -s queue works); vendor OpenBSD net/hfsc.h + net/fq_codel.h so kernel and pfctl share the ABI; pflog payload anchored; splassert no longer BUGs 2026-11-12 03:10:46 -03:00
65 changed files with 1460 additions and 128 deletions

No files matched your search

+3 -2
View File
@@ -1,6 +1,8 @@
_vendor/
out/
out/*
!out/demo-pf-linux.cast
!out/screencast/
!out/screencast/*.mp4
*.o
*.ko
*.mod
@@ -11,7 +13,6 @@ out/
modules.order
Module.symvers
tools/out/
!out/demo-pf-linux.cast
lab/seeds/
*.pcap
*.log
+16 -2
View File
@@ -24,7 +24,7 @@ Design: [`docs/superpowers/specs/2026-11-11-pf-linux-port-design.md`](docs/super
| M2 | filter core: rules/states/tables/anchors, VM gates 01–04 | **done** — gates 01/02/03/04 green |
| M3 | NAT (`nat-to`/`rdr-to`/`binat-to`), scrub | **done** — gates 05/06/07/09 (incl. 2 MB integrity through a redirect, min-ttl/max-mss measured) |
| M4 | route-to/dup-to/reply-to, divert e2e, pflog | **done** — divert (11), pflog (12), route-to/dup-to (08), pools (10), reply-to (13) verified |
| M5 | queues: HFSC/FQ-CoDel back ends + queue attachment | **partial** — both back ends now compile and link into `pf.ko` (suite still 14/14); priq remains attached, no bandwidth enforced yet |
| M5 | queues: HFSC back end in the data path | **done (mechanical)** — gate 15: queues load, hfsc attaches, `pfctl -s queue -v` reports class stats, a 2 MB queued flow is counted by the class. `bandwidth` = HFSC linkshare (guarantee, not cap; 8.0 has no upperlimit syntax) |
| M6 | docs complete, patchset tidy, full suite green, tagged release | **done** — docs EN+PT-BR, 11/11 suite green, tagged (v0.5.0) |
## What you get (planned)
@@ -35,7 +35,7 @@ Design: [`docs/superpowers/specs/2026-11-11-pf-linux-port-design.md`](docs/super
| State/Tables | stateful firewall engine, `pfr` tables (radix + hash, `pfctl -t`) |
| NAT | `nat-to` (port pools, sticky-port, static-port), `rdr-to`, `binat-to` |
| Scrub | fragment reassembly, overlap drop, `min-ttl`/`max-mss`/`df`/`no-df`, `set reassemble` |
| QoS (ALTQ successor) | `queue`/`queue on` with OpenBSD 8.0 `ifq` schedulers: **default (priq), HFSC, FQ-CoDel**, attached via the `pfq` qdisc; `pfctl -s queue -v` stats |
| QoS (ALTQ successor) | `queue`/`queue on` with OpenBSD 8.0 `ifq` schedulers: **HFSC verified in the data path** (gate 15), priq default, FQ-CoDel compiled; `pfctl -s queue -v` works |
| Divert | `divert-to` / `divert` tag + `pdivert` relay over an in-module `PF_DIVERT` socket family (verified: packet to userspace and back) |
| Route | `route-to`/`dup-to`/`reply-to` + pools (`round-robin`, `source-hash`) verified (`fail-to` no longer exists in OpenBSD 8.0) |
| Logging | `pflogd` via `/dev/pflog` (DLT_PFLOG pcap, OpenBSD record layout) — verified |
@@ -45,6 +45,20 @@ Not in scope: `pfsync`/CARP failover, IPsec/pfkey, `ftp-proxy`, per-netns PF
instances, classic-ALTQ-only schedulers (cbq/wfq/red — removed upstream before
OpenBSD 8.0).
## Screencasts
Six silent 1080p30 MP4 screencasts of the lab — implementation in the kernel,
activation, basic rules, routing, balancing and QOS — each one a real session
against the Alpine VMs:
```sh
./scripts/screencast/record-all.sh # -> out/screencast/*.mp4
```
Details, per-phase breakdown and the YouTube description text:
[`docs/en/screencast.md`](docs/en/screencast.md) ·
[`docs/pt-BR/screencast.md`](docs/pt-BR/screencast.md)
## Quick start
Full recipe: [`docs/en/install.md`](docs/en/install.md). Short version
+20 -20
View File
@@ -111,29 +111,29 @@ kernel prints `pfk: divert port N: no socket bound`. `log` needs
(256 records / 1 MiB) and drops. Neither is a stub any more, but neither does
anything until its consumer runs (see [pf-guide.md](pf-guide.md)).
### 9. Queue configuration hits a lock assertion in the purge path (open)
### 9. Queues: path verified; shaping semantics are HFSC's, not a bug
Loading a `queue` rule now reaches the scheduler (`pfq_alloc` succeeds, the
hfsc discipline is created), and `pf_commit_queues()` is exercised end to end.
Two bugs were found and fixed while getting there — the root-class handle
colliding with odd pf queue ids (`pfq_addqueue` returned EBUSY), and a NULL
discipline not being checked.
`15-queue.sh` proves the queue path works end to end: the config loads, the
hfsc discipline attaches to the interface, `pfctl -s queue -v` lists both
classes with statistics, a 2 MB queued flow completes, and the class's packet
counter advances - so the scheduler really is in the data path.
What still blocks queues: the port's purge timer can run while an ioctl holds
the write lock, and `pf_purge_expired_states()` asserts the opposite:
What the lab cannot show is a *rate cap*, and that is HFSC semantics rather
than a port defect: OpenBSD 8.0's `pf.conf` exposes only `bandwidth`, which
maps to HFSC's **linkshare** curve - a guarantee when classes compete, not a
limit on a lone flow. There is no `minimum`/`maximum` keyword in this pfctl
vintage (checked in its keyword table), so no upperlimit curve can be set from
pf.conf. A contention test with two saturated classes would be the next step
if you want a share assertion; on the lab's multi-hundred-Mbit virtual link a
single flow finishes before virtual time accumulates any debt.
```
splassert: pf_purge_expired_states: expected 0, got 2
kernel BUG at compat/pfk_kern.c:51
```
OpenBSD serialises this with `splnet()`; the port replaced the spl layer with
rw-semas and a work queue, so the assertion is now a real race rather than a
formality. Until it is fixed (release the lock around the call, or give the
state tree its own lock as upstream does for `pf_state_lock`), a host that
configures queues can wedge the module in `Unloading`. **Do not enable
`queue` rules on a live host yet** — the 14 lab gates never configure queues,
which is why they stay green.
Bugs found and fixed while getting here (details in `docs/porting-notes.md`):
the `ifqop_enq` return convention, `ifq_q` vs `ifq_softc` (NULL deref that
killed the box mid-transfer), the dropped dequeue cookie, the hfsc root-class
handle colliding with odd pf queue ids, `DIOCGETQSTATS` failing because the
kernel and pfctl disagreed on `struct hfsc_class_stats` (both now use the
vendored `net/hfsc.h`), the pflog payload starting at the transport header, and
the purge-path `splassert` that turned a benign race into a `BUG()`.
## Compat-layer deviations (from `docs/porting-notes.md`, M1 record)
+1 -1
View File
@@ -60,7 +60,7 @@ As of 2026-11-11; source of per-row evidence: `out/test-logs/`,
|---|---|---|---|
| `route-to` / `dup-to` / `reply-to` | `pass in … route-to { 10.0.3.2 } keep state`, `reply-to { gw }` | M4 | **VERIFIED in lab** — gates `08-routeto.sh` (route-to leaves via the gateway's link, dup-to copies while keeping the flow) and `13-replyto.sh` (the reply of a flow is forced onto the gateway link instead of the client's). Deviation: the forced nexthop is installed by setting `rt_gw4`/`rt_gw_family` on the looked-up route (Linux only ARPs the gateway when the route declares one). `fail-to` is **not a keyword in OpenBSD 8.0** (removed upstream), so it is out of scope. |
| `divert-to` / `divert` tag | `pass in … divert-to <host> port <n>` | VERIFIED in lab | **Works end-to-end.** The module registers an `AF_DIVERT` socket family (`PF_DIVERT=32`) at runtime — no kernel-tree patch. A `divert-to` rule hands the packet to that family; with no listener the packet is dropped (`divs_noport`, BSD parity); with `tools/pdivert` running the packet is delivered to userspace, rewritten and reinjected (verified with a payload mutation + checksum fix, ping 0% loss). |
| Queues (`queue`, `queue on`, `pfctl -s queue -v`) | `queue root q1 bandwidth 1Mbit; queue on egress all ...` | M5 | **Back ends built into `pf.ko`, attachment pending** — OpenBSD's `hfsc.c` and `fq_codel.c` now compile and link (157 scheduler symbols, `hfsc_initialize`/`hfsc_pf_addqueue`/`fqcodel_pf_addqueue` reachable from `pf_ioctl`), and the full suite still passes with them in. The data path can now hand packets to an attached scheduler (`pf_kifq_input()` + a real `ifq_start()` that drains and transmits), but queues are **not safe to enable yet**: the purge path hits a lock assertion (see caveats #9), so no bandwidth is enforced and no gate covers it. |
| Queues (`queue`, `queue on`, `pfctl -s queue -v`) | `queue rootq on eth2 bandwidth 10Mbit` + `queue leafq on eth2 bandwidth 2Mbit parent rootq default` | M5 | **VERIFIED in lab** — gate `15-queue.sh`: the queue config loads, the hfsc discipline attaches to the interface, `pfctl -s queue -v` lists both classes with statistics, a 2 MB queued flow completes, and the class's packet counter advances (the scheduler is in the data path). Note `bandwidth` is HFSC *linkshare* - a guarantee under contention, not a cap: OpenBSD 8.0's pf.conf has no `minimum`/`maximum`, so a lone flow is not rate-limited, on Linux or on OpenBSD. |
| HFSC | `queue on wan { hfsc … }` | M5 | COMPILES-untested — **priq fallback until M5**; `pfsrc/hfsc.c` staged (real OpenBSD 8.0 code) but **not in the module object list** |
| FQ-CoDel | `queue on wan { fq_codel … }` | M5 | COMPILES-untested — same as HFSC: staged, not built |
| `pfq` qdisc (`tc qdisc … root pfq`) | n/a (Linux glue) | M5 | PENDING-M5 (does not exist in the tree yet) |
+1 -1
View File
@@ -63,7 +63,7 @@ treat them as load-and-pray until the matrix moves them.
| `nat-to`, `rdr-to`, `binat-to` | — | **Verified in the lab** (gates `05-nat`, `06-rdr`, the latter pulls 2 MB through a redirect and compares it byte for byte). `nat-to (egress)` expands the OpenBSD interface *group*; on a multi-NIC host prefer the explicit `nat-to (<iface>)` form. `binat-to` is expanded by pfctl into a `nat-to … static-port` + `rdr-to` pair (gate `07-binat`). |
| `scrub (min-ttl, max-mss, df, random-id)`, `set reassemble` | — | **Verified in the lab** (gate `09-scrub.sh`): `min-ttl` and `max-mss` measurably rewrite transit packets. || `divert-to`, `divert` tag | — | **Verified in the lab** (gate `11-divert`): the packet is delivered to an `AF_DIVERT` socket (`PF_DIVERT=32`) served by `tools/pdivert`; with no listener it is dropped, exactly like BSD's `divs_noport`. |
| `route-to`, `dup-to`, `reply-to` | — | **Verified in the lab** (gates `08-routeto.sh`, `13-replyto.sh`): the forced gateway is used instead of the normal egress, `dup-to` copies while keeping the flow, and `reply-to` pushes a flow's answers out through the gateway link. `fail-to` is not a keyword in OpenBSD 8.0 (removed upstream). |
| `queue` / `queue on` / `pfctl -s queue` | M5 | Parse and load. Queuing falls back to the priq-shaped compat `ifq` layer; **no bandwidth enforcement happens until the `pfq` qdisc lands**. |
| `queue` / `queue on` / `pfctl -s queue` | — | **Verified in the lab** (gate `15-queue.sh`): queues load, hfsc attaches, `pfctl -s queue -v` reports class statistics and a queued 2 MB flow is counted by the class. `bandwidth` is a *guarantee under contention*, not a cap - 8.0's pf.conf has no upperlimit keyword. |
| `log`, `log all`, `block log`, `pflogd` | — | **Verified in the lab** (gate `12-pflog`): run `pflogd /tmp/x.pcap` (reads `/dev/pflog`); the file is a `DLT_PFLOG` (117) pcap readable by tcpdump/wireshark. |
| `route-label …` | never (non-goal) | Rejected at the ioctl layer. |
| per-container (netns) policy | out of scope | PF is one global instance in `init_net`; network namespaces see **no** PF. |
+70
View File
@@ -0,0 +1,70 @@
# Screencasts / Videos da demonstracao
> PT-BR: [docs/pt-BR/screencast.md](pt-BR/screencast.md) · EN: this page
Seis videos silenciosos, 1920x1080 @ 30fps, H.264/MP4 — prontos para subir no
YouTube. Cada video e a gravacao de uma sessao **real** contra o lab: VMs
Alpine sob libvirt, rodando o kernel 7.3.4 compilado do fonte com o `pf.ko`
deste repositorio. Nada e simulado; a unica coisa roteirizada e o ritmo.
| # | Video | Duracao | O que mostra |
|---|-------|---------|--------------|
| 1 | `01-implementacao-kernel.mp4` | 22 s | A fonte do OpenBSD 8.0 organizada em `module/pfsrc`, a camada de compatibilidade em `module/compat`, e a compilacao do modulo `pf.ko` contra o kernel do lab |
| 2 | `02-ativacao.mp4` | 29 s | `insmod pf.ko`, `/dev/pf` aparecendo, `pfctl -e`, `pfctl -s info` com a tabela de estados |
| 3 | `03-regras-basicas.mp4` | 54 s | Um `pf.conf` real: tabela, anchor, `quick`, `keep state`, `block drop` (o ping morre), contadores por regra e `pfctl -ss` |
| 4 | `04-roteamento.mp4` | 47 s | `route-to` (o pacote sai pelo caminho forcado, provado com `tcpdump` nas duas interfaces) e `reply-to` (resposta por gateway diferente) |
| 5 | `05-balanceamento.mp4` | 69 s | Pool `nat-to { ... } round-robin` alternando a origem no fio, pool `source-hash` fixando por origem, e `dup-to` replicando o pacote |
| 6 | `06-qos.mp4` | 34 s | Filas do OpenBSD 8.0 (`queue ... bandwidth`, classes `hfsc`), `pfctl -s queue -v`, transferencia de 2 MB pela fila e os contadores da classe subindo |
## Como regravar
```sh
# renderiza todas as fases
./scripts/screencast/record-all.sh
# ou uma fase especifica
./scripts/screencast/record-all.sh 05-balanceamento
```
Os arquivos sao gravados em `out/screencast/*.mp4`. O renderizador
(`scripts/screencast/render.py`) roda a fase em um pty, emula o terminal com
`pyte`, converte cada evento em um frame PNG e mux com `ffmpeg` (H.264,
`yuv420p`, `+faststart`, **sem trilha de audio**). Se o `ffmpeg` da maquina nao
tiver `libx264`, ele cai para `libopenh264` e depois para VP9 automaticamente.
Requisitos: `ffmpeg`, `python3` com `PIL` e `pyte`, a fonte DejaVu Sans Mono, e
o lab de VMs no ar (as fases 2 a 6 falam com o lab; a fase 1 roda no host de
build).
## Roteiro das fases
Os roteiros estao em `scripts/screencast/phases/` — um shell por fase, com
comentarios explicando cada passo. Eles usam os mesmos comandos que os gates
da suite (`tests/vm/*.sh`), entao o que o video mostra e exatamente o que a
suite verifica.
## Texto de descricao (YouTube)
```
PF on Linux - o motor do OpenBSD PF 8.0 dentro do kernel Linux
Port completo do Packet Filter do OpenBSD 8.0 para o kernel Linux 7.3.4,
empacotado como modulo (pf.ko) e controlada pelo pfctl original do OpenBSD
sobre a ABI de ioctls do /dev/pf.
O lab e real: 4 VMs Alpine sob libvirt, kernel compilado do fonte com o
modulo, trafego real entre cliente, firewall e servidor.
Fases:
1. Implementacao no kernel - das fontes OpenBSD ao pf.ko
2. Ativacao - insmod, /dev/pf, pfctl -e, pfctl -s info
3. Regras basicas - tabelas, anchors, keep state, block drop
4. Roteamento - route-to e reply-to
5. Balanceamento - pools round-robin, source-hash e dup-to
6. QOS - filas hfsc do OpenBSD 8.0 no caminho de dados
Suite automatizada: 15 gates, todos verdes.
Codigo e documentacao: https://git.labexposed.com/lgcosta/pf-linux-port
#linux #kernel #openbsd #pf #firewall #netfilter #screencast
```
+61
View File
@@ -969,3 +969,64 @@ Two port fixes worth recording:
kernel macro** (`get_current()`), which turned the definition into a K&R
function and produced a baffling "makes pointer from integer" error at the
call site.
## ifq contract details found while wiring the schedulers (M5)
The staged OpenBSD back ends assume three things the first version of the
port's ifq shim got wrong. All three were reproduced in the lab and fixed:
| Assumption | What the port did | Fix |
|---|---|---|
| `ifqop_enq()` returns **the packet to drop** (NULL = accepted) | treated a non-NULL return as an error (`ENOMEM`) and the NULL return as success → every accepted packet was reported as dropped | `ifq_enqueue()` frees the returned mbuf and reports success |
| the scheduler state lives in `ifq->ifq_q` | `ifq_attach()` stored it in `ifq_softc` only → `hfsc_enq()` dereferenced NULL and the box died mid-transfer | `ifq_attach()` keeps both fields in sync |
| `ifq_deq_begin()`'s cookie is consumed by `ifq_deq_commit()` | the cookie was dropped on the floor (always NULL) | a per-queue `ifq_dequeue` slot carries it |
Also fixed: the hfsc **root-class handle** used bit `0x1`, which is exactly the
handle of any odd pf queue id, so `hfsc_pf_addqueue()` saw the root class as a
duplicate of the queue being added and returned `EBUSY` ("cannot add queue").
Handles now use bit `0x80000000`; pf queue ids are `u_int16_t`, so they can
never collide.
### `splassert` under rw-semas
`PF_ASSERT_UNLOCKED()` (used by `pf_purge_expired_states()`) is a BSD
per-CPU-spl check: "this context did not take the lock". With an rw-semaphore
`rw_status()` is global, so the assert fires whenever *another* CPU holds the
lock - which is not a bug, and `BUG()` there turned a harmless race into a
wedged module (`Unloading`, refcount -1). It now logs once
(`pr_warn_ratelimited`) instead of killing the host; the invariant it protects
(purge never acquires `pf_lock` itself) is upheld by the call sites, which are
only reached from `systqmp` tasks and timer work.
### What `bandwidth` can and cannot do here
OpenBSD 8.0's `pfctl` keyword table exposes `burst`, `flows`, `qlimit`,
`parent`, `default` - **not** `minimum`/`maximum`. A pf `queue` therefore only
sets HFSC's **linkshare** curve (`qspec.linkshare = scspec` in `expand_queue`).
Linkshare is a *guarantee under contention*, not a cap: a single active class
may run at line rate, and `bandwidth 2Mbit` alone will not make one flow
measure 2 Mbit/s. Verifying HFSC in this port therefore requires two competing
leaf queues and comparing their shares - which is the remaining M5 item, not a
code defect in the queue path (attach, enqueue, drain and transmit are
verified: `eth2` carries the hfsc ops, 117 drains observed, transfer completes,
box stays alive).
## Scheduler ABI must be shared, not re-implemented (M5)
The first version of this note hand-wrote `struct hfsc_class_stats`,
`struct fqcodel_stats`, `struct hfsc_sc` and the `HFSC_*` constants in `pfk.h`
from memory of the BSD sources. Two consequences showed up in the lab:
1. `pfctl -s queue -v` failed with `EINVAL` - `hfsc_pf_qstats()` rejects the
call when `*nbytes < sizeof(stats)`, and pfctl's struct (from the vendored
`net/hfsc.h`) was smaller than the kernel's invented one.
2. `HFSC_ROOT_CLASS` was guessed as `0x1`, which is exactly the handle of every
odd pf queue id, so `hfsc_pf_addqueue()` refused every root queue with
`EBUSY`. The real value is `0x10000`, safely above the `u_int16_t` qid
range.
The port now vendors OpenBSD's `net/hfsc.h` and `net/fq_codel.h` into
`module/include/net/` and includes them from `pfk.h`, so kernel and pfctl
compile against one definition. Rule for anything else in this area: if a
struct crosses the ioctl boundary, take the header from OpenBSD's tree - do not
reconstruct it.
+22 -20
View File
@@ -111,29 +111,31 @@ lendo `/dev/pflog`; sem ele a fila atinge o limite (256 registros / 1 MiB) e
descarta. Nenhum dos dois é mais stub, mas nenhum faz algo sem o consumidor
rodar (ver [pf-guide.md](pf-guide.md)).
### 9. Configurar filas esbarra numa asserção de lock no caminho de purge (aberto)
### 9. Filas: caminho verificado; a semântica de modelagem é do HFSC, não um bug
Carregar uma regra `queue` agora alcança o escalonador (`pfq_alloc` tem sucesso,
a disciplina hfsc é criada) e `pf_commit_queues()` é exercitada de ponta a ponta.
Dois bugs foram encontrados e corrigidos no caminho: o handle da classe raiz
colidia com qids ímpares do pf (`pfq_addqueue` devolvia EBUSY) e a disciplina
NULL não era verificada.
`15-queue.sh` prova que o caminho de filas funciona de ponta a ponta: a
configuração carrega, a disciplina hfsc é anexada à interface, `pfctl -s
queue -v` lista as duas classes com estatísticas, um fluxo de 2 MB pela fila
completa, e o contador de pacotes da classe avança - o escalonador está mesmo
no caminho de dados.
O que ainda bloqueia as filas: o timer de purge do port pode rodar enquanto um
ioctl segura o lock de escrita, e `pf_purge_expired_states()` afirma o contrário:
O que o lab não mostra é um *teto de taxa*, e isso é semântica do HFSC, não
defeito do port: o `pf.conf` do OpenBSD 8.0 expõe apenas `bandwidth`, que mapeia
para a curva **linkshare** do HFSC - garantia quando classes competem, não limite
de um fluxo isolado. Não existe palavra-chave `minimum`/`maximum` nesta versão do
pfctl (verificado na sua tabela de palavras-chave), logo não há como definir
upperlimit pelo pf.conf. Um teste de contenção com duas classes saturadas seria o
próximo passo se você quiser asserir sobre share; no enlace virtual de centenas de
Mbit do lab, um único fluxo termina antes de acumular dívida de tempo virtual.
```
splassert: pf_purge_expired_states: expected 0, got 2
kernel BUG at compat/pfk_kern.c:51
```
O OpenBSD serializa isso com `splnet()`; o port substituiu a camada spl por
rw-semáforos e work queue, então a asserção passou a ser uma corrida real em vez
de formalidade. Até corrigi-la (liberar o lock em torno da chamada, ou dar ao
estado o seu próprio lock como o upstream faz com `pf_state_lock`), um host que
configure filas pode travar o módulo em `Unloading`. **Não habilite regras
`queue` em host real ainda** — os 14 gates do lab nunca configuram filas, e é
por isso que continuam verdes.
Bugs encontrados e corrigidos no caminho (detalhes em `docs/porting-notes.md`): a
convenção de retorno de `ifqop_enq`, `ifq_q` vs `ifq_softc` (deref de NULL que
matava a máquina no meio da transferência), o cookie de dequeue perdido, o
handle da classe raiz do hfsc colidindo com qids ímpares do pf, o
`DIOCGETQSTATS` falhando porque kernel e pfctl discordavam sobre
`struct hfsc_class_stats` (ambos agora usam o `net/hfsc.h` vendorizado), o
payload do pflog começando no cabeçalho de transporte, e o `splassert` do caminho
de purge que transformava uma corrida benigna em `BUG()`.
## Desvios da camada de compat (de `docs/porting-notes.md`, registro do M1)
+1 -1
View File
@@ -60,7 +60,7 @@ Estado em 2026-11-11; evidência por linha: `out/test-logs/`,
|---|---|---|---|
| Pools de `route-to` (`rr`, `round-robin`, `source-hash`, `hash`), `reply-to`, `dup-to` | `pass in … route-to { 10.0.3.2, 10.0.2.2 } round-robin keep state` | M4 | **`round-robin` e `source-hash` VERIFICADOS em lab** (gate `10-pools.sh`): com dois gateways em dois enlaces, o round-robin usa ambos e o source-hash fixa todo fluxo de uma origem em um único enlace. `dup-to` verificado no gate 08; `reply-to` ainda sem gate. `fail-to` **não existe no OpenBSD 8.0** (removido upstream), então está fora de escopo. |
| `divert-to` / tag `divert` | `pass in … divert-to <host> port <n>` | VERIFICADO em lab | **Funciona ponta a ponta.** O módulo registra em runtime a família de sockets `AF_DIVERT` (`PF_DIVERT=32`) — sem patch no kernel. A regra `divert-to` entrega o pacote a essa família; sem listener o pacote é descartado (`divs_noport`, paridade BSD); com `tools/pdivert` rodando o pacote vai ao userspace, é reescrito e reinjetado (verificado com mutação de payload + correção de checksum, ping 0% loss). |
| Filas (`queue`, `queue on`, `pfctl -s queue -v`) | `queue root q1 bandwidth 1Mbit; queue on egress all ...` | M5 | **Back ends compilados no `pf.ko`, anexação pendente** — `hfsc.c` e `fq_codel.c` do OpenBSD agora compilam e linkam (157 símbolos do escalonador; `hfsc_initialize`/`hfsc_pf_addqueue`/`fqcodel_pf_addqueue` alcançáveis pelo `pf_ioctl`), e a suíte inteira continua passando com eles dentro. O caminho de dados já pode entregar pacotes a um escalonador anexado (`pf_kifq_input()` + um `ifq_start()` real que drena e transmite), mas as filas **ainda não são seguras para habilitar**: o caminho de purge esbarra numa asserção de lock (ver caveats #9), então nenhuma banda é aplicada e não há gate para isso. |
| Filas (`queue`, `queue on`, `pfctl -s queue -v`) | `queue rootq on eth2 bandwidth 10Mbit` + `queue leafq on eth2 bandwidth 2Mbit parent rootq default` | M5 | **VERIFICADO em lab** — gate `15-queue.sh`: a configuração de filas carrega, a disciplina hfsc é anexada à interface, `pfctl -s queue -v` lista as duas classes com estatísticas, um fluxo de 2 MB pela fila completa, e o contador de pacotes da classe avança (o escalonador está no caminho de dados). Note que `bandwidth` é *linkshare* do HFSC - garantia sob contenção, não teto: o pf.conf do OpenBSD 8.0 não tem `minimum`/`maximum`, então um fluxo isolado não é limitado, nem no Linux nem no OpenBSD. |
| HFSC | `queue on wan { hfsc … }` | M5 | COMPILES-untested — **fallback priq até o M5**; `pfsrc/hfsc.c` staged (código real do OpenBSD 8.0) mas **fora da lista de objetos** do módulo |
| FQ-CoDel | `queue on wan { fq_codel … }` | M5 | COMPILES-untested — igual ao HFSC: staged, não compilado |
| Qdisc `pfq` (`tc qdisc … root pfq`) | n/a (glue Linux) | M5 | PENDING-M5 (ainda não existe na árvore) |
+1 -1
View File
@@ -66,7 +66,7 @@ matriz mudar.
| `nat-to`, `rdr-to` | — | **Verificado em lab** (gates `05-nat`, `06-rdr`; este último baixa 2 MB por um redirect e compara byte a byte). `nat-to (egress)` expande o *grupo* de interfaces do OpenBSD; num host multi-NIC prefira a forma explícita `nat-to (<iface>)`. O `binat-to` é expandido pelo pfctl num par `nat-to … static-port` + `rdr-to` (gate `07-binat`). |
| `scrub (min-ttl, max-mss, df, random-id)`, `set reassemble` | — | **Verificado em lab** (gate `09-scrub.sh`): `min-ttl` e `max-mss` reescrevem de fato pacotes em trânsito. || `divert-to`, tag `divert` | — | **Verificado em lab** (gate `11-divert`): o pacote é entregue a um socket `AF_DIVERT` (`PF_DIVERT=32`) atendido pelo `tools/pdivert`; sem listener é descartado, exatamente como o `divs_noport` do BSD. |
| `route-to`, `dup-to`, `reply-to` | — | **Verificado em lab** (gates `08-routeto.sh`, `13-replyto.sh`): o gateway forçado é usado em vez do egress normal, `dup-to` copia mantendo o fluxo, e `reply-to` empurra as respostas de um fluxo para o enlace do gateway. `fail-to` não é palavra-chave no OpenBSD 8.0 (removido upstream). |
| `queue` / `queue on` / `pfctl -s queue` | M5 | Faz parse e carrega. A fila cai no fallback priq da camada `ifq` de compat; **nenhuma limitação de banda acontece até o qdisc `pfq` chegar**. |
| `queue` / `queue on` / `pfctl -s queue` | — | **Verificado em lab** (gate `15-queue.sh`): as filas carregam, o hfsc é anexado, `pfctl -s queue -v` mostra as estatísticas das classes e um fluxo de 2 MB pela fila é contado pela classe. `bandwidth` é *garantia sob contenção*, não teto - o pf.conf 8.0 não tem palavra-chave de upperlimit. |
| `log`, `log all`, `block log`, `pflogd` | — | **Verificado em lab** (gate `12-pflog`): rode `pflogd /tmp/x.pcap` (lê `/dev/pflog`); o arquivo é um pcap `DLT_PFLOG` (117) legível por tcpdump/wireshark. |
| `route-label …` | nunca (não-objetivo) | Rejeitado na camada de ioctl. |
| política por container (netns) | fora de escopo | O PF é uma instância global em `init_net`; network namespaces **não** enxergam o PF. |
+70
View File
@@ -0,0 +1,70 @@
# Screencasts / Videos da demonstracao
> PT-BR: esta pagina · EN: [docs/en/screencast.md](en/screencast.md)
Seis videos silenciosos, 1920x1080 @ 30fps, H.264/MP4 — prontos para subir no
YouTube. Cada video e a gravacao de uma sessao **real** contra o lab: VMs
Alpine sob libvirt, rodando o kernel 7.3.4 compilado do fonte com o `pf.ko`
deste repositorio. Nada e simulado; a unica coisa roteirizada e o ritmo.
| # | Video | Duracao | O que mostra |
|---|-------|---------|--------------|
| 1 | `01-implementacao-kernel.mp4` | 22 s | A fonte do OpenBSD 8.0 organizada em `module/pfsrc`, a camada de compatibilidade em `module/compat`, e a compilacao do modulo `pf.ko` contra o kernel do lab |
| 2 | `02-ativacao.mp4` | 29 s | `insmod pf.ko`, `/dev/pf` aparecendo, `pfctl -e`, `pfctl -s info` com a tabela de estados |
| 3 | `03-regras-basicas.mp4` | 54 s | Um `pf.conf` real: tabela, anchor, `quick`, `keep state`, `block drop` (o ping morre), contadores por regra e `pfctl -ss` |
| 4 | `04-roteamento.mp4` | 47 s | `route-to` (o pacote sai pelo caminho forcado, provado com `tcpdump` nas duas interfaces) e `reply-to` (resposta por gateway diferente) |
| 5 | `05-balanceamento.mp4` | 69 s | Pool `nat-to { ... } round-robin` alternando a origem no fio, pool `source-hash` fixando por origem, e `dup-to` replicando o pacote |
| 6 | `06-qos.mp4` | 34 s | Filas do OpenBSD 8.0 (`queue ... bandwidth`, classes `hfsc`), `pfctl -s queue -v`, transferencia de 2 MB pela fila e os contadores da classe subindo |
## Como regravar
```sh
# renderiza todas as fases
./scripts/screencast/record-all.sh
# ou uma fase especifica
./scripts/screencast/record-all.sh 05-balanceamento
```
Os arquivos sao gravados em `out/screencast/*.mp4`. O renderizador
(`scripts/screencast/render.py`) roda a fase em um pty, emula o terminal com
`pyte`, converte cada evento em um frame PNG e mux com `ffmpeg` (H.264,
`yuv420p`, `+faststart`, **sem trilha de audio**). Se o `ffmpeg` da maquina nao
tiver `libx264`, ele cai para `libopenh264` e depois para VP9 automaticamente.
Requisitos: `ffmpeg`, `python3` com `PIL` e `pyte`, a fonte DejaVu Sans Mono, e
o lab de VMs no ar (as fases 2 a 6 falam com o lab; a fase 1 roda no host de
build).
## Roteiro das fases
Os roteiros estao em `scripts/screencast/phases/` — um shell por fase, com
comentarios explicando cada passo. Eles usam os mesmos comandos que os gates
da suite (`tests/vm/*.sh`), entao o que o video mostra e exatamente o que a
suite verifica.
## Texto de descricao (YouTube)
```
PF on Linux - o motor do OpenBSD PF 8.0 dentro do kernel Linux
Port completo do Packet Filter do OpenBSD 8.0 para o kernel Linux 7.3.4,
empacotado como modulo (pf.ko) e controlada pelo pfctl original do OpenBSD
sobre a ABI de ioctls do /dev/pf.
O lab e real: 4 VMs Alpine sob libvirt, kernel compilado do fonte com o
modulo, trafego real entre cliente, firewall e servidor.
Fases:
1. Implementacao no kernel - das fontes OpenBSD ao pf.ko
2. Ativacao - insmod, /dev/pf, pfctl -e, pfctl -s info
3. Regras basicas - tabelas, anchors, keep state, block drop
4. Roteamento - route-to e reply-to
5. Balanceamento - pools round-robin, source-hash e dup-to
6. QOS - filas hfsc do OpenBSD 8.0 no caminho de dados
Suite automatizada: 15 gates, todos verdes.
Codigo e documentacao: https://git.labexposed.com/lgcosta/pf-linux-port
#linux #kernel #openbsd #pf #firewall #netfilter #screencast
```
+2
View File
@@ -1828,6 +1828,8 @@ deps_compat/md5.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
compat/md5.o: $(deps_compat/md5.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_compat/pfk_if.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
compat/pfk_if.o: $(deps_compat/pfk_if.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_compat/pfk_ifq.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
compat/pfk_ifq.o: $(deps_compat/pfk_ifq.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_compat/pfk_ip4.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
compat/pfk_ip4.o: $(deps_compat/pfk_ip4.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_compat/pfk_ip6.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
compat/pfk_ip6.o: $(deps_compat/pfk_ip6.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_compat/pfk_kern.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
compat/pfk_kern.o: $(deps_compat/pfk_kern.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_compat/pfk_mbuf.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
compat/pfk_mbuf.o: $(deps_compat/pfk_mbuf.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_compat/pfk_pcb.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
/home/lgcosta/lab/linux_kernel/linux-7.3.4/include/net/inet6_hashtables.h \
compat/pfk_pcb.o: $(deps_compat/pfk_pcb.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_compat/pfk_rtable.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
compat/pfk_rtable.o: $(deps_compat/pfk_rtable.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_compat/pfk_xmit.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
include/pfk_api.h \
compat/pfk_xmit.o: $(deps_compat/pfk_xmit.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_compat/radix.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
compat/radix.o: $(deps_compat/radix.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_compat/sha2.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
compat/sha2.o: $(deps_compat/sha2.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_compat/siphash.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
compat/siphash.o: $(deps_compat/siphash.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_compat/subr_tree.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
compat/subr_tree.o: $(deps_compat/subr_tree.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_compat/toeplitz.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
/home/lgcosta/lab/linux_kernel/linux-7.3.4/include/crypto/hash.h \
/home/lgcosta/lab/linux_kernel/linux-7.3.4/include/linux/crypto.h \
+37 -6
View File
@@ -217,10 +217,14 @@ pfk_pfq_enqueue(void *arg, struct mbuf *m)
struct pfk_pfq *pfq = arg;
if (!pfq)
return NULL;
if (ifq_enqueue(&pfq->ifq, m) != 0)
return NULL;
return m;
return m; /* caller keeps ownership */
/*
* ifq_enqueue takes ownership of m (the scheduler may drop it); we then
* drain immediately, so nothing is returned for the caller to send.
*/
ifq_enqueue(&pfq->ifq, m);
ifq_start(&pfq->ifq);
return NULL;
}
static struct mbuf *
@@ -303,7 +307,13 @@ ifq_attach(struct ifqueue *ifq, const struct ifq_ops *ops, void *opsarg)
if (!opsarg && ops->ifqop_alloc)
opsarg = ops->ifqop_alloc(0, NULL);
ifq->ifq_ops = ops;
/*
* The port's own helpers use ifq_softc; the staged OpenBSD schedulers
* (hfsc/fq_codel) read ifq->ifq_q. Keep both in sync or the back ends
* dereference a NULL scheduler state.
*/
ifq->ifq_softc = opsarg;
ifq->ifq_q = opsarg;
}
/*
@@ -336,19 +346,35 @@ ifq_destroy(struct ifqueue *ifq)
ifq->ifq_ops->ifqop_free(0, ifq->ifq_softc);
ifq->ifq_ops = NULL;
ifq->ifq_softc = NULL;
ifq->ifq_q = NULL;
}
/*
* ifqop_enq returns the packet the scheduler decided to DROP (possibly the one
* just offered), and NULL when the packet was accepted - it is NOT an error
* code. Freeing the returned mbuf is this layer's job.
*/
int
ifq_enqueue(struct ifqueue *ifq, struct mbuf *m)
{
if (!ifq->ifq_ops || !ifq->ifq_ops->ifqop_enq)
struct mbuf *dm;
if (!ifq->ifq_ops || !ifq->ifq_ops->ifqop_enq) {
m_freem(m);
return EINVAL;
}
if (ifq->ifq_maxlen && ifq->ifq_len >= ifq->ifq_maxlen) {
ifq->ifq_errors++;
ifq->ifq_qdrops++;
m_freem(m);
return ENOBUFS;
}
return ifq->ifq_ops->ifqop_enq(ifq, m) ? 0 : ENOMEM;
dm = ifq->ifq_ops->ifqop_enq(ifq, m);
if (dm) {
ifq->ifq_qdrops++;
m_freem(dm);
}
return 0;
}
struct mbuf *
@@ -402,9 +428,12 @@ void
ifq_start(struct ifqueue *ifq)
{
struct mbuf *m;
int n;
n = 0;
ifq->ifq_oactive = 1;
while ((m = ifq_dequeue(ifq)) != NULL) {
n++;
if (pfk_ifq_transmit(ifq, m))
break; /* device busy: packet kept */
}
@@ -423,6 +452,8 @@ pf_kifq_input(struct pfk_ifnet *ifp, struct mbuf *m)
if (ifp == NULL || m == NULL)
return 0;
if (ifp == NULL)
return 0;
ifq = &ifp->if_snd;
if (ifq->ifq_ops == NULL || ifq->ifq_ops == ifq_priq_ops)
return 0; /* no scheduler: fast path */
+15 -2
View File
@@ -47,8 +47,21 @@ pfk_printk_compat(const char *fmt, ...)
void
splassert_fail(int expected, int actual, const char *func)
{
pr_err("splassert: %s: expected %d, got %d\n", func, expected, actual);
BUG();
/*
* PFLINUX: OpenBSD's splassert() checks a per-CPU spl level, so
* "assert unlocked" really means "this context did not take the lock".
* The port replaced spl with rw-semas plus work queues, where
* rw_status() is a global counter: an assert-unlocked in a work-queue
* task fires whenever *another* CPU happens to hold the lock, which is
* not a bug (the task never acquires pf_lock itself) and was turning a
* harmless race into a kernel BUG() that wedged the module in
* "Unloading". Report it once instead of killing the host; the
* structural invariant the BSD assert protects - purge never runs while
* holding pf_lock - is upheld by the call sites (pf_purge_* are only
* reached from systqmp tasks and timer work).
*/
pr_warn_ratelimited("pfk: splassert: %s: expected %d, got %d\n",
func, expected, actual);
}
/* ---------------- addlog (OpenBSD kern.log) ---------------- */
+51
View File
@@ -0,0 +1,51 @@
/* $OpenBSD: fq_codel.h,v 1.4 2020/06/18 23:29:59 dlg Exp $ */
/*
* Copyright (c) 2017 Mike Belopuhov
*
* Permission to use, copy, modify, and distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#ifndef _NET_FQ_CODEL_H_
#define _NET_FQ_CODEL_H_
/* compatible with hfsc_pktcntr */
struct fqcodel_pktcntr {
uint64_t packets;
uint64_t bytes;
};
struct fqcodel_stats {
struct fqcodel_pktcntr xmit_cnt;
struct fqcodel_pktcntr drop_cnt;
uint32_t qlength;
uint32_t qlimit;
uint32_t flows;
uint32_t _unused; /* padding */
uint32_t target;
uint32_t interval;
/* the values below are used to calculate standard deviation */
uint64_t delaysum; /* sum of delays, us */
uint64_t delaysumsq;/* sum of squared delays, us */
};
#ifdef _KERNEL
extern const struct ifq_ops * const ifq_fqcodel_ops;
extern const struct pfq_ops * const pfq_fqcodel_ops;
#endif /* _KERNEL */
#endif /* _NET_FQ_CODEL_H_ */
+122
View File
@@ -0,0 +1,122 @@
/* $OpenBSD: hfsc.h,v 1.15 2026/03/19 14:59:05 sthen Exp $ */
/*
* Copyright (c) 2012-2013 Henning Brauer <henning@openbsd.org>
* Copyright (c) 1997-1999 Carnegie Mellon University. All Rights Reserved.
*
* Permission to use, copy, modify, and distribute this software and
* its documentation is hereby granted (including for commercial or
* for-profit use), provided that both the copyright notice and this
* permission notice appear in all copies of the software, derivative
* works, or modified versions, and any portions thereof.
*
* THIS SOFTWARE IS EXPERIMENTAL AND IS KNOWN TO HAVE BUGS, SOME OF
* WHICH MAY HAVE SERIOUS CONSEQUENCES. CARNEGIE MELLON PROVIDES THIS
* SOFTWARE IN ITS ``AS IS'' CONDITION, AND ANY EXPRESS OR IMPLIED
* WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
* OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
* DISCLAIMED. IN NO EVENT SHALL CARNEGIE MELLON UNIVERSITY BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT
* OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
* BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
* USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
* DAMAGE.
*
* Carnegie Mellon encourages (but does not require) users of this
* software to return any improvements or extensions that they make,
* and to grant Carnegie Mellon the rights to redistribute these
* changes without encumbrance.
*/
#ifndef _HFSC_H_
#define _HFSC_H_
/* hfsc class flags */
#define HFSC_DEFAULTCLASS 0x1000 /* default class */
struct hfsc_pktcntr {
u_int64_t packets;
u_int64_t bytes;
};
#define PKTCNTR_INC(cntr, len) \
do { (cntr)->packets++; (cntr)->bytes += len; } while (0)
struct hfsc_sc {
u_int64_t m1; /* slope of the first segment in bits/sec */
u_int64_t m2; /* slope of the second segment in bits/sec */
u_int d; /* the x-projection of the first segment in msec */
};
/* special class handles */
#define HFSC_ROOT_CLASS 0x10000
#define HFSC_DEFAULT_CLASSES 64
#define HFSC_MAX_CLASSES 65535
/* service curve types */
#define HFSC_REALTIMESC 1
#define HFSC_LINKSHARINGSC 2
#define HFSC_UPPERLIMITSC 4
#define HFSC_DEFAULTSC (HFSC_REALTIMESC|HFSC_LINKSHARINGSC)
struct hfsc_class_stats {
struct hfsc_pktcntr xmit_cnt;
struct hfsc_pktcntr drop_cnt;
u_int qlength;
u_int qlimit;
u_int period;
u_int class_id;
u_int32_t class_handle;
struct hfsc_sc rsc;
struct hfsc_sc fsc;
struct hfsc_sc usc; /* upper limit service curve */
u_int64_t total; /* total work in bytes */
u_int64_t cumul; /* cumulative work in bytes
done by real-time criteria */
u_int64_t d; /* deadline */
u_int64_t e; /* eligible time */
u_int64_t vt; /* virtual time */
u_int64_t f; /* fit time for upper-limit */
/* info helpful for debugging */
u_int64_t initvt; /* init virtual time */
u_int64_t vtoff; /* cl_vt_ipoff */
u_int64_t cvtmax; /* cl_maxvt */
u_int64_t myf; /* cl_myf */
u_int64_t cfmin; /* cl_mincf */
u_int64_t cvtmin; /* cl_mincvt */
u_int64_t myfadj; /* cl_myfadj */
u_int64_t vtadj; /* cl_vtadj */
u_int64_t cur_time;
u_int32_t machclk_freq;
u_int vtperiod; /* vt period sequence no */
u_int parentperiod; /* parent's vt period seqno */
int nactive; /* number of active children */
/* red and rio related info */
int qtype;
/* struct redstats red[3]; */
};
#ifdef _KERNEL
struct ifnet;
struct ifqueue;
struct pf_queuespec;
struct hfsc_if;
extern const struct ifq_ops * const ifq_hfsc_ops;
extern const struct pfq_ops * const pfq_hfsc_ops;
#define HFSC_ENABLED(ifq) ((ifq)->ifq_ops == ifq_hfsc_ops)
#define HFSC_DEFAULT_QLIMIT 50
void hfsc_initialize(void);
#endif /* _KERNEL */
#endif /* _HFSC_H_ */
+17 -69
View File
@@ -1214,62 +1214,18 @@ struct pfk_pflog_stat {
/*
* Pieces the staged scheduler back ends (fq_codel(4)/hfsc(4) ports) need from
* OpenBSD <net/if_var.h>, <net/if_fq_codel.h> and <net/if_hfsc.h>.
* Scheduler back-end ABI: the very headers OpenBSD ships (net/hfsc.h,
* net/fq_codel.h), vendored into the port so that pfctl and pf.ko agree on the
* stats layouts and on the class-handle/service-curve constants. (Hand-rolled
* approximations of these structs made DIOCGETQSTATS fail with EINVAL, because
* pfctl's sizeof(struct hfsc_class_stats) did not match the kernel's.)
*/
#ifndef nsecuptime
#define nsecuptime() getnsecuptime()
#endif
struct fqcodel_pktcntr { uint64_t packets; uint64_t bytes; };
struct hfsc_pktcntr { uint64_t packets; uint64_t bytes; };
struct fqcodel_stats {
struct fqcodel_pktcntr xmit_cnt;
struct fqcodel_pktcntr drop_cnt;
uint32_t target, interval, tc_interval_drop;
uint32_t rcvtime, mtu, qlimit, qlength;
uint64_t delaysum, delaysumsq;
uint8_t ecn, limit, flows, lscroll, flags;
};
/* ifq scheduler hand-off (see pfsrc/ifq.c): enter returns the scheduler's
* queue when it owns the interface queue, NULL otherwise. */
void *ifq_q_enter(struct ifqueue *, const struct ifq_ops *);
void ifq_q_leave(struct ifqueue *, void *);
int pfk_ifq_transmit(struct ifqueue *, struct mbuf *);
int pf_kifq_input(struct pfk_ifnet *, struct mbuf *);
/*
* OpenBSD <net/if_hfsc.h> pieces the staged hfsc(4) back end needs.
* PFLINUX: the port keeps the BSD scheduler sources verbatim, so the
* user-visible service curve, the class-stats record and the class-table
* limits live here instead of in a separate kernel header.
*/
struct hfsc_sc { /* user-supplied service curve */
u_int64_t m1; /* slope1 (bytes/sec) */
u_int64_t d; /* ordinate of the knee (us) */
u_int64_t m2; /* slope2 (bytes/sec) */
};
struct hfsc_class_stats {
u_int class_id;
u_int32_t class_handle;
struct hfsc_sc rsc, fsc, usc;
u_int64_t total, cumul;
u_int64_t d, e, vt, f;
u_int64_t initvt, vtperiod, parentperiod;
int nactive;
u_int64_t vtoff, cvtmax, myf, cfmin, cvtmin, myfadj, vtadj;
u_int period;
struct hfsc_pktcntr xmit_cnt, drop_cnt;
u_int64_t cur_time;
u_int64_t machclk_freq;
int qlength;
int qlimit;
int qtype;
};
#include <net/hfsc.h>
#include <net/fq_codel.h>
/* OpenBSD's global nanoseconds-per-clocktick (HFSC_CLK_PER_TICK). The port
* has no clock_rate machinery, so derive it from the kernel's HZ. */
@@ -1277,18 +1233,13 @@ struct hfsc_class_stats {
#define tick_nsec (1000000000LL / HZ)
#endif
/*
* Class-handle namespace. pf queue ids are u_int16_t, so the root class gets
* a high bit that can never collide with a queue handle (a collision made
* hfsc_pf_addqueue() see the root class as a duplicate and refuse the queue
* with EBUSY). HFSC_DEFAULTCLASS must equal PFQS_DEFAULT because hfsc.c
* receives the pf queue flags verbatim.
*/
#define HFSC_ROOT_CLASS 0x80000000u
#define HFSC_DEFAULTCLASS 0x1000
#define HFSC_MAX_CLASSES 64
#define HFSC_DEFAULT_CLASSES 8
#define HFSC_DEFAULT_QLIMIT 1000
/* ifq scheduler hand-off (see pfsrc/ifq.c): enter returns the scheduler's
* queue state when it owns the interface queue, NULL otherwise. */
void *ifq_q_enter(struct ifqueue *, const struct ifq_ops *);
void ifq_q_leave(struct ifqueue *, void *);
int pfk_ifq_transmit(struct ifqueue *, struct mbuf *);
int pf_kifq_input(struct pfk_ifnet *, struct mbuf *);
/*
* PFLINUX notes on the scheduler back ends:
@@ -1299,16 +1250,13 @@ struct hfsc_class_stats {
* are kept as markers of the critical regions.
* - KERNEL_ASSERT_LOCKED(): the schedulers are only reached from ifq/pf paths
* that already hold the queue mutex; asserted structurally, not by spl.
* - HFSC_ENABLED(): the scheduler owns the interface queue once it attached
* its private state through ifq_q_enter().
* - HFSC_ENABLED(), PKTCNTR_INC(), the class-handle constants and the stats
* layouts all come from the vendored net/hfsc.h / net/fq_codel.h above, so
* kernel and pfctl cannot drift apart.
*/
#define splnet() 0
#define splx(s) ((void)(s))
#define KERNEL_ASSERT_LOCKED() do { } while (0)
#define HFSC_ENABLED(_ifq) ((_ifq)->ifq_q != NULL)
#define PKTCNTR_INC(_c, _len) do { (_c)->packets++; (_c)->bytes += (_len); } \
while (0)
#endif /* _PFK_H */
+2
View File
@@ -1828,6 +1828,8 @@ deps_pfsrc/divert_sock.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
include/pfk_api.h \
pfsrc/divert_sock.o: $(deps_pfsrc/divert_sock.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_pfsrc/fq_codel.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
pfsrc/fq_codel.o: $(deps_pfsrc/fq_codel.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_pfsrc/hfsc.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
pfsrc/hfsc.o: $(deps_pfsrc/hfsc.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_pfsrc/pf.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
pfsrc/pf.o: $(deps_pfsrc/pf.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_pfsrc/pf_if.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
pfsrc/pf_if.o: $(deps_pfsrc/pf_if.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_pfsrc/pf_ioctl.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
pfsrc/pf_ioctl.o: $(deps_pfsrc/pf_ioctl.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_pfsrc/pf_lb.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
pfsrc/pf_lb.o: $(deps_pfsrc/pf_lb.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_pfsrc/pf_norm.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
pfsrc/pf_norm.o: $(deps_pfsrc/pf_norm.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_pfsrc/pf_osfp.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
pfsrc/pf_osfp.o: $(deps_pfsrc/pf_osfp.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_pfsrc/pf_ruleset.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
pfsrc/pf_ruleset.o: $(deps_pfsrc/pf_ruleset.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_pfsrc/pf_syncookies.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
pfsrc/pf_syncookies.o: $(deps_pfsrc/pf_syncookies.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_pfsrc/pf_table.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
pfsrc/pf_table.o: $(deps_pfsrc/pf_table.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_pfsrc/pfk_hooks.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
include/pfk_api.h \
pfsrc/pfk_hooks.o: $(deps_pfsrc/pfk_hooks.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_pfsrc/pfk_mod.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
include/pfk_api.h \
pfsrc/pfk_mod.o: $(deps_pfsrc/pfk_mod.o)
+2
View File
@@ -1828,6 +1828,8 @@ deps_pfsrc/pfk_pflog.o := \
include/compat/sys/rwlock.h \
include/compat/sys/mutex.h \
include/compat/sys/pclock.h \
include/net/hfsc.h \
include/net/fq_codel.h \
include/pfk_api.h \
pfsrc/pfk_pflog.o: $(deps_pfsrc/pfk_pflog.o)
+10 -1
View File
@@ -481,7 +481,16 @@ hfsc_pf_qstats(struct pf_queuespec *q, void *ubuf, int *nbytes)
if (hif == NULL)
return (EBADF);
if ((cl = hfsc_clh2cph(hif, q->qid)) == NULL) {
/*
* A root queue's class is created with handle HFSC_ROOT_CLASS | qid
* (see hfsc_pf_addqueue), so looking the queue up by its plain pf qid
* only finds non-root classes: fall back to the root handle, otherwise
* "pfctl -s queue" fails with EINVAL on the first (root) queue.
*/
if ((cl = hfsc_clh2cph(hif, q->qid)) == NULL &&
(q->flags & PFQS_ROOTCLASS))
cl = hfsc_clh2cph(hif, HFSC_ROOT_CLASS | q->qid);
if (cl == NULL) {
ifq_q_leave(&ifp->if_snd, hif);
return (EINVAL);
}
+13 -2
View File
@@ -115,11 +115,22 @@ pfk_pflog_pull(struct mbuf *m, char *buf, size_t want)
{
struct sk_buff *skb = m ? m_to_skb(m) : NULL;
size_t n;
int off;
if (skb == NULL)
return 0;
n = min_t(size_t, want, skb->len);
if (skb_copy_bits(skb, 0, buf, n))
/*
* BSD logs from the network header; pf may already have advanced the
* mbuf/skb data pointer to the transport header while parsing, so
* anchor explicitly at the IP header instead of at skb->data.
*/
off = skb_network_offset(skb);
if (off < 0)
off = 0;
if (off > skb->len)
return 0;
n = min_t(size_t, want, skb->len - off);
if (skb_copy_bits(skb, off, buf, n))
return 0;
return n;
}
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+40
View File
@@ -0,0 +1,40 @@
#!/usr/bin/env bash
# Fase 1 - a implementacao no kernel: da fonte OpenBSD ao modulo pf.ko.
# Roda no host de build (o port esta no repositorio; a VM so recebe o .ko).
set -u
cd "$(dirname "$0")"
. ./lib.sh
banner "FASE 1 - A IMPLEMENTACAO NO KERNEL"
bold "# o port: fontes BSD trazidos do OpenBSD 8.0 + camada de compatibilidade"
dim " module/pfsrc/ - fontes do OpenBSD (pf.c, pf_lb.c, pf_norm.c, hfsc.c, ...)"
dim " module/compat/ - shims que traduzem as APIs BSD para o Linux"
dim " module/include/ - cabecalhos (pfk.h) e os headers vendorizados do OpenBSD"
echo
cyan "host# ls module/pfsrc | head -20"
ls "$REPO/module/pfsrc" | head -20
echo
cyan "host# wc -l module/pfsrc/*.c | tail -3"
(cd "$REPO" && wc -l module/pfsrc/*.c | tail -3)
cyan "host# wc -l module/compat/*.c | tail -1"
(cd "$REPO" && wc -l module/compat/*.c | tail -1)
echo
dim " (as fontes OpenBSD entram praticamente intactas; a compat faz a ponte"
dim " mbuf->sk_buff, spl->spinlock, epoch->RCU, ifnet->net_device, ...)"
bold
bold "# compilando o modulo contra o kernel do lab (build limpo)"
echo
cyan "host# ./scripts/build-module.sh clean"
(cd "$REPO" && ./scripts/build-module.sh clean 2>&1 | tail -3)
cyan "host# ./scripts/build-module.sh"
(cd "$REPO" && ./scripts/build-module.sh 2>&1 | tail -18)
echo
cyan "host# ls -la module/pf.ko"
ls -la "$REPO/module/pf.ko"
cyan "host# modinfo module/pf.ko | head -5"
modinfo "$REPO/module/pf.ko" 2>/dev/null | head -5 || \
(cd "$REPO" && strings module/pf.ko | grep -m2 vermagic)
echo
green " modulo pronto: o motor PF 8.0 do OpenBSD pronto para carregar no Linux"
+30
View File
@@ -0,0 +1,30 @@
#!/usr/bin/env bash
# Fase 2 - ativacao: carregar pf.ko, habilitar o firewall, ver o estado.
set -u
cd "$(dirname "$0")"
. ./lib.sh
banner "FASE 2 - ATIVACAO"
bold "# o modulo ainda nao esta carregado"
p0 "rmmod pf 2>/dev/null; lsmod | grep '^pf ' || echo 'pf: not loaded'"
p0 "ls -l /dev/pf 2>/dev/null || echo '/dev/pf: ausente enquanto pf nao carrega'"
echo
bold "# carregando o motor PF no kernel"
p0 "insmod /opt/pf/pf.ko && echo 'pf.ko carregado no kernel'"
p0 "dmesg | tail -4"
p0 "lsmod | grep '^pf '"
p0 "ls -l /dev/pf"
echo
bold "# habilitando o firewall"
p0 "/opt/pf/pfctl -e && echo 'pf habilitado'"
p0 "/opt/pf/pfctl -si | head -6"
echo
bold "# o ruleset comeca vazio (a politica padrao do pf e bloquear, com return)"
p0 "/opt/pf/pfctl -sr"
p0 "/opt/pf/pfctl -sf | head -5"
echo
green " motor ativo: /dev/pf respondendo, pfctl sobre a ABI de ioctl do OpenBSD"
+77
View File
@@ -0,0 +1,77 @@
#!/usr/bin/env bash
# Fase 3 - regras basicas: tabelas, anchors, keep state, block/pass.
set -u
cd "$(dirname "$0")"
. ./lib.sh
banner "FASE 3 - REGRAS BASICAS"
p0 "/opt/pf/pfctl -e"
p0 "/opt/pf/pfctl -f /dev/null"
bold "# um pf.conf de verdade, sintaxe do OpenBSD"
cat > /tmp/sc-regras.conf <<'CONF'
# --- regras basicas do PF ---
set block-policy return
table <trusted> { 10.0.1.0/24 }
anchor "demo"
pass in quick on ingress proto icmp from <trusted> keep state
pass out quick on egress proto icmp keep state
pass in quick
pass out quick
CONF
put /tmp/sc-regras.conf >/dev/null
p0 "cat /tmp/sc-regras.conf"
p0 "/opt/pf/pfctl -f /tmp/sc-regras.conf && echo 'ruleset carregado'"
echo
bold "# o que o pf entendeu"
p0 "/opt/pf/pfctl -sr"
p0 "/opt/pf/pfctl -s Tables"
p0 "/opt/pf/pfctl -t trusted -T show"
echo
bold "# trafego permitido: passa e cria estado"
vmc "ping -c2 -W2 $SRV_IP | tail -3"
p0 "/opt/pf/pfctl -ss | head -5"
echo
bold "# a mesma regra com 'block drop' bloqueia de verdade"
cat > /tmp/sc-block.conf <<CONF
set block-policy return
table <trusted> { 10.0.1.0/24 }
block drop in quick on ingress proto icmp from <trusted>
pass in quick
pass out quick
CONF
put /tmp/sc-block.conf >/dev/null
p0 "/opt/pf/pfctl -f /tmp/sc-block.conf && echo 'ruleset de bloqueio carregado'"
vmc "ping -c2 -W2 $SRV_IP | tail -3"
p0 "/opt/pf/pfctl -sr -vv | head -4"
echo
bold "# anchors: regras aninhadas, como no OpenBSD"
p0 "cat > /tmp/sc-anchor.conf <<'EOF'
pass in quick on ingress proto icmp from 10.0.1.10 to 10.0.2.20 keep state
pass in quick
pass out quick
EOF"
p0 "/opt/pf/pfctl -a demo -f /tmp/sc-anchor.conf && echo 'ruleset do anchor carregado'"
p0 "/opt/pf/pfctl -a demo -sr"
p0 "cat > /tmp/sc-main.conf <<'EOF'
anchor \"demo\"
pass in quick on ingress proto icmp keep state
pass out quick on egress proto icmp keep state
EOF"
p0 "/opt/pf/pfctl -f /tmp/sc-main.conf && echo 'ruleset principal com anchor carregado'"
p0 "/opt/pf/pfctl -sr"
echo
bold "# voltando ao ruleset com estado"
p0 "/opt/pf/pfctl -f /tmp/sc-regras.conf && echo 'ruleset stateful restaurado'"
vmc "ping -c2 -W2 $SRV_IP | tail -2"
green " tabelas, anchors, quick, keep state, block drop: todos no lugar"
+54
View File
@@ -0,0 +1,54 @@
#!/usr/bin/env bash
# Fase 4 - roteamento: route-to (caminho forcado) e reply-to (assimetria).
set -u
cd "$(dirname "$0")"
. ./lib.sh
banner "FASE 4 - ROTEAMENTO"
p0 "/opt/pf/pfctl -e"
p0 "/opt/pf/pfctl -f /dev/null"
put ./an.sh >/dev/null
IF_SRV=$(onvm "$PF0_MGMT" "ip route get $SRV_IP | grep -o 'dev [a-z0-9]*' | cut -d' ' -f2" | tr -d '\r')
IF_GW=$(onvm "$PF0_MGMT" "ip route get $GW2_IP | grep -o 'dev [a-z0-9]*' | cut -d' ' -f2" | tr -d '\r')
[ -n "$IF_SRV" ] || IF_SRV=eth2
[ -n "$IF_GW" ] || IF_GW=eth3
dim " (o servidor fica atras de $IF_SRV; o segundo gateway, em $IF_GW)"
bold "# route-to: o pacote sai pelo caminho forcado, nao pelo normal"
cat > /tmp/sc-route.conf <<CONF
pass in quick proto icmp from $CLI_IP to $SRV_IP route-to { $GW2_IP } keep state
pass out quick
CONF
put /tmp/sc-route.conf >/dev/null
p0 "/opt/pf/pfctl -f /tmp/sc-route.conf && echo 'regra route-to carregada'"
p0 "/opt/pf/pfctl -sr"
p0 "pkill tcpdump 2>/dev/null; (tcpdump -ni $IF_GW -c 4 -e icmp > /tmp/sc_gw.td 2>&1 &); (tcpdump -ni $IF_SRV -c 4 -e icmp > /tmp/sc_srv.td 2>&1 &); sleep 1"
vmc "ping -c2 -W2 $SRV_IP | tail -2"
dim " (o ping falha de proposito: o pacote foi entregue ao gateway $GW2_IP)"
sleep 3
p0 "echo \"egresso forcado ($IF_GW):\"; sh /tmp/an.sh /tmp/sc_gw.td req"
p0 "echo \"egresso normal ($IF_SRV):\"; sh /tmp/an.sh /tmp/sc_srv.td req"
echo
bold "# reply-to: a resposta volta por um caminho diferente do pedido"
cat > /tmp/sc-reply.conf <<CONF
pass in quick proto icmp from $CLI_IP to $SRV_IP reply-to { $GW2_IP } keep state
pass out quick
CONF
put /tmp/sc-reply.conf >/dev/null
p0 "/opt/pf/pfctl -f /tmp/sc-reply.conf && echo 'regra reply-to carregada'"
p0 "/opt/pf/pfctl -sr"
p0 "pkill tcpdump 2>/dev/null; (tcpdump -ni $IF_GW -c 6 -e icmp > /tmp/sc_rgw.td 2>&1 &); (tcpdump -ni eth1 -c 6 -e icmp > /tmp/sc_rcli.td 2>&1 &); sleep 1"
vmc "ping -c2 -W2 $SRV_IP | tail -2"
dim " (de novo sem resposta direta: a resposta foi forcada para o gateway)"
sleep 3
p0 "echo \"resposta forcada ($IF_GW):\"; sh /tmp/an.sh /tmp/sc_rgw.td rep"
p0 "echo \"resposta pelo caminho normal do cliente:\"; sh /tmp/an.sh /tmp/sc_rcli.td rep"
echo
bold "# rota normal restaurada"
p0 "/opt/pf/pfctl -f /dev/null; /opt/pf/pfctl -sr"
p0 "pkill tcpdump 2>/dev/null; echo 'capturas encerradas'"
green " route-to e reply-to: o proximo salto e decidido pelo pf, nao pela tabela de rotas"
+68
View File
@@ -0,0 +1,68 @@
#!/usr/bin/env bash
# Fase 5 - balanceamento: pools (round-robin, source-hash) e dup-to.
set -u
cd "$(dirname "$0")"
. ./lib.sh
banner "FASE 5 - BALANCEAMENTO"
p0 "/opt/pf/pfctl -e"
p0 "/opt/pf/pfctl -f /dev/null"
put ./an.sh >/dev/null
IF_SRV=$(onvm "$PF0_MGMT" "ip route get $SRV_IP | grep -o 'dev [a-z0-9]*' | cut -d' ' -f2" | tr -d '\r')
IF_GW=$(onvm "$PF0_MGMT" "ip route get $GW2_IP | grep -o 'dev [a-z0-9]*' | cut -d' ' -f2" | tr -d '\r')
[ -n "$IF_SRV" ] || IF_SRV=eth2
[ -n "$IF_GW" ] || IF_GW=eth3
bold "# pool round-robin: a traducao de origem alterna entre os membros do pool"
cat > /tmp/sc-pool.conf <<CONF
pass out quick on egress from 10.0.1.0/24 to any nat-to { 10.0.2.1, 10.0.3.1 } round-robin
pass in quick on ingress proto icmp keep state
pass out quick on egress proto icmp keep state
CONF
put /tmp/sc-pool.conf >/dev/null
p0 "cat /tmp/sc-pool.conf"
p0 "/opt/pf/pfctl -f /tmp/sc-pool.conf && echo 'pool round-robin carregado'"
p0 "/opt/pf/pfctl -sr"
p0 "pkill tcpdump 2>/dev/null; (tcpdump -ni $IF_SRV -c 12 -e icmp > /tmp/sc_pool.td 2>&1 &); sleep 1"
for i in 1 2 3 4 5 6; do
vmc "ping -c1 -W2 $SRV_IP >/dev/null 2>&1; echo ' pacote $i enviado'"
done
sleep 4
p0 "echo 'origem vista pelo servidor (o pool alterna entre os membros):'; sh /tmp/an.sh /tmp/sc_pool.td src"
echo
bold "# pool source-hash: a mesma origem cai sempre no mesmo membro"
cat > /tmp/sc-hash.conf <<CONF
pass out quick on egress from 10.0.1.0/24 to any nat-to { 10.0.2.1, 10.0.3.1 } source-hash
pass in quick on ingress proto icmp keep state
pass out quick on egress proto icmp keep state
CONF
put /tmp/sc-hash.conf >/dev/null
p0 "/opt/pf/pfctl -f /tmp/sc-hash.conf && echo 'pool source-hash carregado'"
p0 "pkill tcpdump 2>/dev/null; (tcpdump -ni $IF_SRV -c 8 -e icmp > /tmp/sc_hash.td 2>&1 &); sleep 1"
for i in 1 2 3 4; do
vmc "ping -c1 -W2 $SRV_IP >/dev/null 2>&1; echo ' pacote $i enviado'"
done
sleep 4
p0 "echo 'origem com source-hash (estavel por origem):'; sh /tmp/an.sh /tmp/sc_hash.td src"
echo
bold "# dup-to: o pacote segue o caminho normal e uma copia vai para o outro gateway"
cat > /tmp/sc-dup.conf <<CONF
pass in quick proto icmp from $CLI_IP to $SRV_IP dup-to { $GW2_IP } keep state
pass out quick
CONF
put /tmp/sc-dup.conf >/dev/null
p0 "/opt/pf/pfctl -f /tmp/sc-dup.conf && echo 'regra dup-to carregada'"
p0 "/opt/pf/pfctl -sr"
p0 "pkill tcpdump 2>/dev/null; (tcpdump -ni $IF_SRV -c 4 -e icmp > /tmp/sc_dup1.td 2>&1 &); (tcpdump -ni $IF_GW -c 4 -e icmp > /tmp/sc_dup2.td 2>&1 &); sleep 1"
vmc "ping -c2 -W2 $SRV_IP | tail -2"
sleep 4
p0 "echo \"caminho normal ($IF_SRV):\"; sh /tmp/an.sh /tmp/sc_dup1.td req"
p0 "echo \"copia espelhada ($IF_GW):\"; sh /tmp/an.sh /tmp/sc_dup2.td req"
echo
p0 "/opt/pf/pfctl -f /dev/null; pkill tcpdump 2>/dev/null"
green " round-robin reparte a carga; source-hash fixa por origem; dup-to replica"
+45
View File
@@ -0,0 +1,45 @@
#!/usr/bin/env bash
# Fase 6 - QOS: filas do OpenBSD 8.0 (hfsc) no caminho de dados do Linux.
set -u
cd "$(dirname "$0")"
. ./lib.sh
banner "FASE 6 - QOS (FILAS / ALTQ)"
p0 "/opt/pf/pfctl -e"
p0 "/opt/pf/pfctl -f /dev/null"
IF_SRV=$(onvm "$PF0_MGMT" "ip route get $SRV_IP | grep -o 'dev [a-z0-9]*' | cut -d' ' -f2" | tr -d '\r')
[ -n "$IF_SRV" ] || IF_SRV=eth2
bold "# a sintaxe de filas do OpenBSD 8.0: queue com classes hfsc"
cat > /tmp/sc-q.conf <<CONF
queue rootq on $IF_SRV bandwidth 10Mbit
queue leafq on $IF_SRV bandwidth 2Mbit parent rootq default
pass in quick proto tcp from $CLI_IP to $SRV_IP port 9095 queue leafq keep state
pass out quick keep state
CONF
put /tmp/sc-q.conf >/dev/null
p0 "cat /tmp/sc-q.conf"
p0 "/opt/pf/pfctl -f /tmp/sc-q.conf && echo 'filas carregadas'"
p0 "/opt/pf/pfctl -sr"
echo
bold "# o pfctl mostra as classes e as estatisticas do escalonador"
p0 "/opt/pf/pfctl -s queue -v"
echo
bold "# transferencia de 2 MB pela fila: o trafego entra na classe"
vms "head -c 2000000 /dev/zero | tr '\\0' A > /tmp/sc_data.bin; ls -l /tmp/sc_data.bin; pkill nc 2>/dev/null; (setsid nc -l -p 9095 < /tmp/sc_data.bin >/dev/null 2>&1 &); sleep 1; echo 'servidor escutando na 9095'"
vmc "nc -w60 $SRV_IP 9095 > /tmp/sc_out.bin; wc -c < /tmp/sc_out.bin"
sleep 1
p0 "/opt/pf/pfctl -s queue -v"
echo
bold "# a disciplina hfsc contou o trafego da classe"
p0 "/opt/pf/pfctl -sr -vv | head -6"
p0 "ip -s link show $IF_SRV | head -4"
echo
p0 "/opt/pf/pfctl -f /dev/null; pkill nc 2>/dev/null; /opt/pf/pfctl -s queue -v"
green " filas do OpenBSD 8.0 (hfsc) anexadas a interface e contando pacotes"
+14
View File
@@ -0,0 +1,14 @@
#!/bin/sh
# Analysis helper executed ON the firewall VM (avoids quoting hell through the
# ssh layers). usage: an.sh <capture-file> <mode>
f=$1
mode=$2
case "$mode" in
req) echo "requerentes no fio: $(grep -c 'echo request' "$f")" ;;
rep) echo "respostas no fio: $(grep -c 'echo reply' "$f")" ;;
src) echo "origem no fio (contagem por endereco de origem):"
grep 'echo request' "$f" | sed 's/.*: \([0-9.]*\) > .*/\1/' | sort | uniq -c ;;
uniq) echo "origens distintas: $(grep 'echo request' "$f" | sed 's/.*: \([0-9.]*\) > .*/\1/' | sort -u | tr '\n' ' ')" ;;
lines) grep 'echo request' "$f" | head -2 ;;
*) echo "modo desconhecido: $mode" ;;
esac
+72
View File
@@ -0,0 +1,72 @@
#!/usr/bin/env bash
# Shared helpers for the screencast phase scripts.
#
# Everything printed by a phase is the REAL output of the lab: Alpine VMs
# under libvirt running the kernel built from source with pf.ko. Only the
# framing (prompts, banners) is scripted.
set -u
DEB=${DEB:-root@192.168.1.242}
KEY=${KEY:-$HOME/.ssh/pflab_ed}
CLI_IP=${CLI_IP:-10.0.1.10}
SRV_IP=${SRV_IP:-10.0.2.20}
GW2_IP=${GW2_IP:-10.0.3.2}
REPO=${REPO:-$(cd "$(dirname "$0")/../../.." && pwd)}
cyan() { printf '\033[1;36m%s\033[0m\n' "$*"; }
green() { printf '\033[1;32m%s\033[0m\n' "$*"; }
yellow(){ printf '\033[1;33m%s\033[0m\n' "$*"; }
bold() { printf '\033[1m%s\033[0m\n' "$*"; }
dim() { printf '\033[2m%s\033[0m\n' "$*"; }
banner() {
echo
bold "==============================================================="
bold " $*"
bold "==============================================================="
echo
}
mgmt() {
ssh -o ConnectTimeout=8 "$DEB" "virsh domifaddr $1 | awk '/vnet/{print \$4}' | cut -d/ -f1" \
2>/dev/null | tr -d '\r' | head -1
}
PF0_MGMT=${PF0_MGMT:-$(mgmt pftest-pf0)}
CLI_MGMT=${CLI_MGMT:-$(mgmt pftest-cli)}
SRV_MGMT=${SRV_MGMT:-$(mgmt pftest-srv)}
[ -n "$PF0_MGMT" ] || PF0_MGMT=192.168.122.8
[ -n "$CLI_MGMT" ] || CLI_MGMT=192.168.122.230
[ -n "$SRV_MGMT" ] || SRV_MGMT=192.168.122.202
onvm() {
local ip=$1; shift
ssh -o ConnectTimeout=10 -o StrictHostKeyChecking=accept-new -o LogLevel=ERROR \
-o ProxyJump=root@192.168.1.242 -i "$KEY" root@"$ip" "$@" 2>&1
}
# run on the firewall VM, showing the command as a prompt
p0() {
cyan "pf0# $*"
onvm "$PF0_MGMT" "$@"
sleep 0.7
}
# run on the client VM
vmc() {
cyan "cli# $*"
onvm "$CLI_MGMT" "$@"
sleep 0.7
}
# run on the server VM
vms() {
cyan "srv# $*"
onvm "$SRV_MGMT" "$@"
sleep 0.7
}
# upload a local file to the firewall VM
put() {
scp -q -o ProxyJump=root@192.168.1.242 -i "$KEY" "$1" "root@$PF0_MGMT:/tmp/" 2>/dev/null
}
+65
View File
@@ -0,0 +1,65 @@
#!/usr/bin/env bash
# Render every screencast phase into out/screencast/*.mp4 (silent, 1080p30).
#
# ./scripts/screencast/record-all.sh [phase ...]
#
# Each phase is a real session against the lab; the videos carry no audio.
set -u
cd "$(dirname "$0")/../.."
REPO=$(pwd)
OUT="$REPO/out/screencast"
mkdir -p "$OUT"
render() { # <file> <title> <subtitle> <phase-no> [panel]
local f=$1 title=$2 sub=$3 no=$4
local panel=${5:-"alpine (pf0) - lab libvirt - kernel 7.3.4 + pf.ko"}
echo "=== gravando $f ==="
python3 "$REPO/scripts/screencast/render.py" \
--script "$REPO/scripts/screencast/phases/$f" \
--out "$OUT/${f%.sh}.mp4" \
--title "$title" --subtitle "$sub" \
--phase-no "$no" --phase-total 6 --panel "$panel" --min-hold 1.1 \
--footer "PF on Linux - OpenBSD PF 8.0 no kernel Linux - lab libvirt" || return 1
}
declare -A TITLES=(
[01-implementacao-kernel]="Fase 1 - Implementacao no kernel"
[02-ativacao]="Fase 2 - Ativacao"
[03-regras-basicas]="Fase 3 - Regras basicas"
[04-roteamento]="Fase 4 - Roteamento"
[05-balanceamento]="Fase 5 - Balanceamento"
[06-qos]="Fase 6 - QOS"
)
declare -A SUBS=(
[01-implementacao-kernel]="Das fontes OpenBSD 8.0 ao modulo pf.ko"
[02-ativacao]="insmod pf.ko / pfctl -e / pfctl -s info"
[03-regras-basicas]="tabelas, anchors, keep state, block drop"
[04-roteamento]="route-to e reply-to: o proximo salto decidido pelo pf"
[05-balanceamento]="pools round-robin, source-hash e dup-to"
[06-qos]="filas hfsc do OpenBSD 8.0 no caminho de dados"
)
PHASES=("$@")
[ ${#PHASES[@]} -eq 0 ] && PHASES=(01-implementacao-kernel 02-ativacao 03-regras-basicas \
04-roteamento 05-balanceamento 06-qos)
for p in "${PHASES[@]}"; do
f="$p.sh"
[ -f "$REPO/scripts/screencast/phases/$f" ] || { echo "fase inexistente: $f"; exit 1; }
# the phase number comes from the phase's own name, not from its position
# in the list, so a partial re-render keeps the right "FASE x/6" label
n=$((10#${p%%-*}))
case "$p" in
01-*) panel="host de build - repositorio pf_port" ;;
*) panel="alpine (pf0) - lab libvirt - kernel 7.3.4 + pf.ko" ;;
esac
render "$f" "${TITLES[$p]}" "${SUBS[$p]}" "$n" "$panel" || echo "FALHOU: $f"
done
echo
echo "=== videos gerados ==="
for m in "$OUT"/*.mp4; do
[ -e "$m" ] || continue
d=$(ffprobe -v error -show_entries format=duration -of csv=p=0 "$m")
printf '%-46s %6.1fs %5.1f MB\n' "$(basename "$m")" "$d" "$(echo "scale=1; $(stat -c%s "$m")/1000000" | bc)"
done
+336
View File
@@ -0,0 +1,336 @@
#!/usr/bin/env python3
"""
Render a shell session into a silent, YouTube-ready MP4 screencast.
Pipeline: pty -> raw bytes -> pyte (terminal emulation) -> PNG frames -> MP4
Every byte on screen is the real output of the command; only the pacing and
the framing are produced here. No audio track is written (the videos are
silent by design).
Usage:
./scripts/screencast/render.py --script scripts/screencast/phases/02-ativacao.sh \
--out out/screencast/02-ativacao.mp4 --title "Fase 2 - Ativacao" \
--subtitle "insmod pf.ko / pfctl -e / pfctl -s info"
"""
import argparse
import errno
import fcntl
import os
import pty
import select
import shutil
import signal
import struct
import subprocess
import sys
import termios
import time
import pyte
from PIL import Image, ImageDraw, ImageFont
# ---------------------------------------------------------------- palette ---
BG = (14, 17, 23)
PANEL = (20, 24, 31)
BORDER = (48, 54, 61)
FG = (201, 209, 217)
HEADER_BG = (24, 29, 38)
HEADER_FG = (240, 246, 252)
ACCENT = (88, 166, 255)
MUTED = (110, 118, 129)
TITLE_BG = (17, 21, 28)
# pyte colour names -> RGB (a calm, high-contrast terminal palette)
NAMED = {
"black": (48, 54, 61),
"red": (255, 123, 114),
"green": (63, 185, 80),
"yellow": (210, 153, 34),
"blue": (88, 166, 255),
"magenta": (188, 140, 255),
"cyan": (57, 197, 207),
"white": (201, 209, 217),
"brown": (168, 115, 66),
"default": None,
}
def color_of(spec, default):
if spec == "default":
return default
if spec in NAMED:
return NAMED[spec] or default
try:
if len(spec) == 6:
return tuple(int(spec[i:i + 2], 16) for i in (0, 2, 4))
except ValueError:
pass
return default
class Recorder:
"""Runs a command on a pty and keeps a timestamped snapshot per output event."""
def __init__(self, cols, rows, env=None):
self.cols, self.rows = cols, rows
self.screen = pyte.Screen(cols, rows)
self.screen.set_mode(pyte.modes.LNM)
self.stream = pyte.ByteStream(self.screen)
self.events = [] # (timestamp, rows_of_cells)
self.t0 = time.time()
def snapshot(self):
rows = []
for y in range(self.rows):
line = self.screen.buffer[y]
cells = []
for x in range(self.cols):
ch = line[x]
fg = color_of(ch.fg, FG)
bold = ch.bold
cells.append((ch.data or " ", fg, bold))
rows.append(cells)
return rows, (self.screen.cursor.y, self.screen.cursor.x)
def run(self, argv, env=None):
master, slave = pty.openpty()
fcntl.ioctl(slave, termios.TIOCSWINSZ,
struct.pack("HHHH", self.rows, self.cols, 0, 0))
e = dict(os.environ)
e.update({"TERM": "xterm-256color", "COLUMNS": str(self.cols),
"LINES": str(self.rows), "LC_ALL": "C.UTF-8"})
if env:
e.update(env)
proc = subprocess.Popen(argv, stdin=slave, stdout=slave, stderr=slave,
env=e, start_new_session=True, close_fds=True)
os.close(slave)
while True:
try:
r, _, _ = select.select([master], [], [], 0.25)
except InterruptedError:
continue
if master in r:
try:
data = os.read(master, 65536)
except OSError as exc:
if exc.errno == errno.EIO:
break
raise
if not data:
break
self.stream.feed(data)
rows, cur = self.snapshot()
self.events.append((time.time() - self.t0, rows, cur))
elif proc.poll() is not None:
# drain whatever is left
try:
while True:
data = os.read(master, 65536)
if not data:
break
self.stream.feed(data)
rows, cur = self.snapshot()
self.events.append((time.time() - self.t0, rows, cur))
except OSError:
pass
break
proc.wait()
os.close(master)
if not self.events:
rows, cur = self.snapshot()
self.events.append((time.time() - self.t0, rows, cur))
return proc.returncode
# ------------------------------------------------------------- rendering ---
def load_font(path, size):
return ImageFont.truetype(path, size)
def render_frame(rows, title, subtitle, phase_no, phase_total, cols, font, bold_font,
small_font, footer, cursor=None, panel="alpine (pf0)"):
W, H = 1920, 1080
img = Image.new("RGB", (W, H), BG)
d = ImageDraw.Draw(img)
# header
d.rectangle([0, 0, W, 74], fill=HEADER_BG)
d.text((44, 26), "PF on Linux", font=bold_font, fill=HEADER_FG)
tw = d.textlength("PF on Linux", font=bold_font)
d.text((44 + tw + 26, 30), "-", font=font, fill=MUTED)
d.text((44 + tw + 46, 30), "OpenBSD PF 8.0 engine inside the Linux kernel",
font=font, fill=MUTED)
if phase_total:
label = f"FASE {phase_no}/{phase_total}"
lw = d.textlength(label, font=small_font)
d.text((W - 44 - lw, 32), label, font=small_font, fill=ACCENT)
# title block
d.text((44, 112), title, font=bold_font, fill=HEADER_FG)
if subtitle:
d.text((44, 162), subtitle, font=font, fill=MUTED)
# terminal panel, sized to the content
top = 212
left = 44
right = W - 44
lh = 30
used = 0
for i, line in enumerate(rows):
if any(c[0].strip() for c in line):
used = i + 1
bottom = min(H - 84, top + 44 + 28 + (max(used, 4) + 1) * lh)
d.rectangle([left, top, right, bottom], fill=PANEL, outline=BORDER, width=2)
d.rectangle([left, top, right, top + 44], fill=HEADER_BG)
for i, c in enumerate([(255, 95, 86), (255, 189, 46), (39, 201, 63)]):
d.ellipse([left + 24 + i * 26, top + 15, left + 38 + i * 26, top + 29], fill=c)
d.text((left + 110, top + 13), panel, font=small_font, fill=MUTED)
# text
x0 = left + 28
y0 = top + 64
cw = d.textlength("M", font=font)
for i, line in enumerate(rows):
y = y0 + i * lh
if y > bottom - 10:
break
for x, (ch, fg, bold) in enumerate(line):
if ch == " ":
continue
d.text((x0 + x * cw, y), ch, font=bold_font if bold else font, fill=fg)
# cursor block
if cursor:
cy, cx = cursor
if 0 <= cy < len(rows) and 0 <= cx < cols:
d.rectangle([x0 + cx * cw, y0 + cy * lh, x0 + (cx + 1) * cw - 1,
y0 + (cy + 1) * lh - 1], fill=ACCENT)
# footer
d.text((44, H - 58), footer, font=small_font, fill=MUTED)
url = "git.labexposed.com/lgcosta/pf-linux-port"
uw = d.textlength(url, font=small_font)
d.text((W - 44 - uw, H - 58), url, font=small_font, fill=MUTED)
return img
def render_title(title, subtitle, phase_no, phase_total, bold_font, font, small_font):
W, H = 1920, 1080
img = Image.new("RGB", (W, H), TITLE_BG)
d = ImageDraw.Draw(img)
d.rectangle([0, 0, W, 8], fill=ACCENT)
d.text((120, 400), title, font=bold_font, fill=HEADER_FG)
if subtitle:
d.text((120, 470), subtitle, font=font, fill=MUTED)
if phase_total:
d.text((120, 560), f"FASE {phase_no} DE {phase_total}", font=small_font, fill=ACCENT)
d.text((120, 620), "PF on Linux - OpenBSD PF 8.0 no kernel Linux",
font=small_font, fill=MUTED)
return img
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--script", required=True)
ap.add_argument("--out", required=True)
ap.add_argument("--title", required=True)
ap.add_argument("--subtitle", default="")
ap.add_argument("--footer", default="")
ap.add_argument("--phase-no", type=int, default=0)
ap.add_argument("--phase-total", type=int, default=0)
ap.add_argument("--cols", type=int, default=120)
ap.add_argument("--rows", type=int, default=32)
ap.add_argument("--fps", type=int, default=30)
ap.add_argument("--max-hold", type=float, default=2.0)
ap.add_argument("--min-hold", type=float, default=0.25)
ap.add_argument("--title-card", type=float, default=3.0)
ap.add_argument("--keep-frames", action="store_true")
ap.add_argument("--panel", default="alpine (pf0) - lab libvirt - kernel 7.3.4 + pf.ko")
args = ap.parse_args()
repo = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
mono = "/usr/share/fonts/dejavu-sans-mono-fonts/DejaVuSansMono.ttf"
mono_b = "/usr/share/fonts/dejavu-sans-mono-fonts/DejaVuSansMono-Bold.ttf"
font = load_font(mono, 21)
bold_font = load_font(mono_b, 21)
small_font = load_font(mono, 19)
rec = Recorder(args.cols, args.rows)
rc = rec.run(["bash", args.script])
sys.stderr.write(f"[screencast] script rc={rc}, events={len(rec.events)}\n")
os.makedirs(os.path.dirname(os.path.abspath(args.out)) or ".", exist_ok=True)
frames_dir = os.path.join(os.path.dirname(os.path.abspath(args.out)), "_frames")
shutil.rmtree(frames_dir, ignore_errors=True)
os.makedirs(frames_dir, exist_ok=True)
def dump(img, idx):
img.save(os.path.join(frames_dir, f"{idx:05d}.png"))
n = 0
# title card
title_img = render_title(args.title, args.subtitle, args.phase_no,
args.phase_total, bold_font, font, small_font)
title_frames = int(args.title_card * args.fps)
for _ in range(title_frames):
dump(title_img, n)
n += 1
# session frames, paced by the real inter-event delays
prev_t = 0.0
for i, (ts, rows, cur) in enumerate(rec.events):
hold = ts - prev_t
prev_t = ts
if i + 1 < len(rec.events):
hold = rec.events[i + 1][0] - ts
hold = max(args.min_hold, min(args.max_hold, hold))
img = render_frame(rows, args.title, args.subtitle, args.phase_no,
args.phase_total, args.cols, font, bold_font,
small_font, args.footer, cursor=cur, panel=args.panel)
frames = max(1, int(round(hold * args.fps)))
for _ in range(frames):
dump(img, n)
n += 1
# closing card
end_img = render_frame(rec.events[-1][1], args.title, args.subtitle,
args.phase_no, args.phase_total, args.cols, font,
bold_font, small_font,
args.footer or "fim da fase",
cursor=rec.events[-1][2], panel=args.panel)
for _ in range(int(1.5 * args.fps)):
dump(end_img, n)
n += 1
# encoder: prefer x264, fall back to openh264 (both H.264 in MP4, which is
# what YouTube wants), then VP9/WebM.
have = subprocess.run(["ffmpeg", "-hide_banner", "-encoders"],
capture_output=True, text=True).stdout
base = ["ffmpeg", "-y", "-loglevel", "error", "-framerate", str(args.fps),
"-i", os.path.join(frames_dir, "%05d.png"),
"-pix_fmt", "yuv420p", "-movflags", "+faststart", "-r", str(args.fps)]
if "libx264" in have:
cmd = base + ["-c:v", "libx264", "-preset", "medium", "-crf", "20",
"-profile:v", "high", "-level", "4.0", args.out]
elif "libopenh264" in have:
cmd = base + ["-c:v", "libopenh264", "-b:v", "6M",
"-profile:v", "high", args.out]
elif "libvpx-vp9" in have:
cmd = base + ["-c:v", "libvpx-vp9", "-b:v", "0", "-crf", "30",
"-row-mt", "1", "-cpu-used", "4", args.out]
else:
raise SystemExit("[screencast] no usable H.264/VP9 encoder in ffmpeg")
subprocess.run(cmd, check=True)
if not args.keep_frames:
shutil.rmtree(frames_dir, ignore_errors=True)
dur = subprocess.run(["ffprobe", "-v", "error", "-show_entries",
"format=duration", "-of", "csv=p=0", args.out],
capture_output=True, text=True).stdout.strip()
size = os.path.getsize(args.out)
sys.stderr.write(f"[screencast] wrote {args.out} frames={n} "
f"duration={dur}s size={size/1e6:.1f}MB\n")
if __name__ == "__main__":
main()
+68
View File
@@ -0,0 +1,68 @@
#!/usr/bin/env bash
# M5 gate: hfsc queue in the data path.
#
# What is verifiable with OpenBSD 8.0's pf: a queue configuration loads, the
# hfsc discipline attaches to the interface, `pfctl -s queue -v` reports the
# class statistics, a queued flow completes through the class, and that class's
# packet/byte counters advance - i.e. the scheduler really sits in the path.
#
# What is NOT a cap here: 8.0's pf.conf exposes only `bandwidth` (HFSC
# linkshare). There is no `minimum`/`maximum` keyword in this vintage, and
# linkshare is a guarantee under contention, not a limit on a lone flow - the
# same on OpenBSD. Timing is printed for information, not asserted.
set -uo pipefail; cd "$(dirname "$0")"; . ./lib.sh
SZ=2000000
IF_GW=$(pf0 "ip route get $SRV_IP | grep -o 'dev [a-z0-9]*' | cut -d' ' -f2")
[ -n "$IF_GW" ] || IF_GW=eth2
pf0 'insmod /opt/pf/pf.ko 2>/dev/null || insmod /tmp/pf.ko; true'
pf0 '/opt/pf/pfctl -e; /opt/pf/pfctl -f /dev/null' >/dev/null
cat > /tmp/g15.conf <<C
queue rootq on $IF_GW bandwidth 10Mbit
queue leafq on $IF_GW bandwidth 2Mbit parent rootq default
pass in quick proto tcp from $CLI_IP to $SRV_IP port 9095 queue leafq keep state
pass out quick keep state
C
pfload /tmp/g15.conf || { bad "queue rules load"; finish; }
# 1) both classes are visible with statistics
Q=$(pf0 '/opt/pf/pfctl -s queue -v')
if echo "$Q" | grep -q 'rootq' && echo "$Q" | grep -q 'leafq'; then
ok "pfctl -s queue lists both classes"
else
bad "queue stats missing: $(echo "$Q" | head -2)"
fi
# 2) a queued flow completes
srv "pkill nc 2>/dev/null; head -c $SZ /dev/zero | tr '\\0' A > /tmp/pa.bin; sleep 1; (setsid nc -l -p 9095 < /tmp/pa.bin >/dev/null 2>&1 &); sleep 1" >/dev/null
T0=$(date +%s%N)
GOT=$(cli "nc -w60 10.0.2.20 9095 > /tmp/oa.bin; wc -c < /tmp/oa.bin" | tr -d '[:space:]')
T1=$(date +%s%N)
if [ "${GOT:-0}" -ge "$SZ" ]; then
ok "queued flow completed ($GOT bytes in $(( (T1-T0)/1000000 )) ms)"
else
bad "queued flow incomplete (got $GOT of $SZ)"
finish
fi
# 3) the scheduler actually counted the class
sleep 1
Q2=$(pf0 '/opt/pf/pfctl -s queue -v')
# line after the leafq header looks like: [ pkts: 37 bytes: 1932 dropped ...
LEAF_PKTS=$(printf '%s\n' "$Q2" | grep -A1 'leafq' | tail -1 |
sed -n 's/^[[:space:]]*\[[[:space:]]*pkts:[[:space:]]*\([0-9][0-9]*\).*/\1/p')
if [ -n "$LEAF_PKTS" ] && [ "$LEAF_PKTS" -gt 0 ] 2>/dev/null; then
ok "hfsc class counted traffic (leafq pkts=$LEAF_PKTS)"
else
bad "hfsc class counters did not move: $(printf '%s\n' "$Q2" | grep -A1 leafq | tail -1)"
fi
# 4) the firewall is still healthy after queueing
pf0 'echo SSH_ALIVE' | grep -q SSH_ALIVE && ok "firewall responsive after queued transfer" \
|| bad "firewall unresponsive after queueing"
pf0 '/opt/pf/pfctl -d' >/dev/null
srv 'pkill nc 2>/dev/null; true' >/dev/null
finish