Daniel Okafor

Replaced a batch-oriented detection system (15-min cycles) with a streaming architecture. The old system meant attackers had established persistence before alerts fired. The new design uses Kafka for event ingestion, Go consumer services for rule evaluation, and ClickHouse for windowed aggregations.

Key decision: Each rule type gets its own consumer group, so high-volume simple pattern rules don't bottleneck complex multi-step correlation rules. Built a custom windowed aggregation layer in Go with in-memory state and Redis checkpoints to avoid hitting ClickHouse on every event. The state machine engine tracks attack progression across events for multi-stage detection rules that weren't possible in the batch system.

Our first Fortune 500 customer required contractual guarantees of data isolation. The existing system used application-level WHERE clauses for tenant filtering, which had produced two near-miss cross-tenant data exposure bugs in QA. A single app bug could have exposed security telemetry across tenants.

Key decision: Three-layer defense-in-depth instead of relying on any single mechanism. Layer 1: PostgreSQL row-level security policies on all tenant-scoped tables, enforced via SET app.current_tenant. Layer 2: a tenant-aware connection pooler that injects tenant context on every connection checkout. Layer 3: CI integration tests that attempt cross-tenant reads for every API endpoint.

180K-line Django monolith with 31% test coverage, circular imports everywhere, 25-minute deploys. Three frontend teams, two mobile engineers, and six integration partners all consuming the API. A breaking migration would have been a coordination disaster.

Key decision: Strangler fig pattern through a Kong API gateway. Traffic routed to either the monolith or new Go microservices per endpoint. Compatibility tests hit both old and new implementations and diffed responses before each cutover. Migrated 47 endpoints over 4 months with zero consumer-facing breaking changes. First refactored the monolith internally -- broke 34 circular imports to zero, raised test coverage from 31% to 58% -- then extracted services.

High-throughput inference gateway for an ML infrastructure platform. The system needed to serve 50M+ requests per day while maintaining p99 under 200ms and surviving 3x traffic spikes without scaling infrastructure.

Key decision: Request batching plus adaptive load shedding. When traffic spikes hit, the system sheds lower-priority requests gracefully instead of queuing everything until timeouts cascade. Also built the A/B experimentation framework for model deployments with automatic rollback on latency or error-rate regressions.