01 Proxy groups changed the problem
Zabbix 7 introduced proxy groups: hosts are assigned to a group rather than to a proxy, and the server moves them between proxies as members come and go. It is a real improvement for polled hosts. For traps it quietly breaks the usual design, because a trap is a datagram that arrives at one place — and if that place is a proxy which no longer owns the host, the trap is dropped with nothing to say so.
Most estates that trap carrier, power, optical or facility gear into Zabbix have not designed for this yet. The question this work set out to answer was what a trap path should look like when the proxies underneath it are allowed to rebalance.
02 A replica honest enough to break
Six virtual machines from Terraform and Ansible on a Proxmox host: two isolated site networks routed through a gateway, inter-site latency injected with netem at roughly 7–8 ms round trip, a Zabbix 7.4 server, and a proxy group with two active proxies. The inter-site link can be severed on demand.
The point of a replica is to be able to kill things. Every milestone below was passed by actually stopping the process in question and measuring what happened, not by reasoning about what should.
03 Fan out, and let the owner keep it
The collector does not try to know which proxy owns which host. It forwards every trap to every proxy; the proxy that currently owns the host matches it, and the others drop it. This is deliberately dumb, and that is what makes the collector immune to proxy-group churn — it has nothing to update when ownership moves.
| Event | Measured |
|---|---|
| Collector master killed | 1.5 s gap; 37 of 40 traps delivered across it at a fixed cadence |
| Route-follow latency | 1 s |
| Proxy stopped — group reassignment | 26 s |
| Proxy stopped — consolidation onto the survivor | 24 s |
| Host values lost during the proxy failover | 0 |
| Collector configuration changes required | 0 |
| SNMPv3 trap with wrong credentials | Rejected before processing |
04 Nothing in the trap path waits on anything
A stated design rule, then proven: zero API calls, lookups or caches between a trap arriving and a value reaching Zabbix. Forty trap definitions from a carrier-class device estate are compiled into a static in-memory classifier that resolves each trap to an event, an active-or-clear state and a severity. Thirty-one of the forty share a trap OID with another and are disambiguated by varbind. Thirteen unit tests pin the resolver.
The rule exists because a trap path that consults an API is a trap path that stalls when the API does — which, during an incident, is exactly when the traps are arriving.
05 What it does under partition
Severing the inter-site link produces behaviour that no design can make pleasant, so it is characterised rather than hidden. Split-brain between the collector pair and the single-gateway chokepoint for the floating address are both written up as architecture decisions with their trade-offs, alongside four others.
The repository carries a 45-page architecture and operations guide written for a reader who has never seen it, and an explicit section on what would have to change before it carried production traps.