Can we replay this work safely?
Separate delivery attempts from business effects, then verify the evidence and retention window.
Messaging systems · Background execution · Developer experience
Reliable messaging, background jobs, and the engineering that keeps them running.
I'm a Senior Software Engineer at GitHub, working on a platform that supports roughly 8B background jobs and 150B events each day. I write field notes, source-level studies, and experiments with the evidence—and the limits—made explicit.
José David Baena
Messaging infrastructure · Distributed systems
Start with the work
Inspect a protocol investigation, a structured course, a source-level study, and a runnable failure model.
Open the decision-first Kafka Reliability Field Guide- 01
Protocol debugging
18 min article
Debugging Kafka Batch Offsets Across Appends
Inspect batch-offset invariants, the reported symptom, and the limits of the runnable header example.
Open resource - 02
Guided course
13 parts · complete
Kafka Engineering
Work through protocol internals, platform ownership, and migration boundaries in a deliberate order.
Open resource - 03
Source-guided study
8 parts · complete
Redpanda 26.1 Architecture and Operations
See how a Kafka-compatible system changes durability, storage, and operations under the hood.
Open resource - 04
Interactive lab
Deterministic · runs locally
Async Failure Lab
Compare crashes, timeouts, duplicate dispatch, and durable recovery without uploading data.
Open resource
Start with the failure in front of you
Use a short sequence of labs and checklists to answer one concrete background-job question.
Will the backlog drain by the deadline?
Estimate capacity, test controller lag, and define completion as a measurable objective.
Can old jobs survive a deployment?
Test payload compatibility, worker takeover, rollback, and replay before a mixed-version rollout.
Current threads
Two messaging and platform entries, plus one adjacent systems problem that carries the same reliability discipline.
Kafka EngineeringProduction & PlatformPart 9/13- Published on
- /10 mins read
Kafka Consumer Groups Are Platform Infrastructure
A platform contract should own Kafka client lifecycle, migration, and observability without pretending it can own application side effects.
Redpanda 26.1 Architecture and OperationsProductionPart 8/8- Published on
- /15 mins read
Benchmarking Redpanda: A Reproducible Systems Method
A reproducible Redpanda benchmark controls load, durability, clients, cache state, and failures before interpreting latency or throughput.
Frontier Model EngineeringRuntime SystemsPart 2/4- Published on
- /17 mins read
Agent Loops: Distributed Transactions Without Atomic Commit
Treat tool-using agents as durable workflows: record intent, bind authority, reconcile timeouts, fence workers, compensate effects, and audit every attempt.
Follow future work
Get occasional field notes, follow public posts through RSS, or start a technical conversation.
Follow public posts through RSS
Email delivery is paused here. The public article feed still carries new field notes to any feed reader, without an account or inbox tracking.
Open the RSS feedTalk about the work
For distributed-systems work, collaboration, relevant engineering opportunities, or a technical conversation, email is the simplest path.