a replayable event log and pub/sub fan-out

An append-only log that broadcasts immutable events to multiple downstream systems while allowing consumers to rewind and replay history at their own pace.

This architecture decouples producers from consumers by writing events to a partitioned, append-only disk log. Instead of deleting messages upon delivery, the log retains them for a set duration (e.g., 7 days), enabling any downstream service to rewind its read offset and replay historical data to recover from failures or populate new databases. Simultaneously, the built-in pub/sub engine handles fan-out routing, instantly broadcasting a single incoming event to hundreds of independent microservices (such as payment processing, fraud detection, and email notifications) without performance degradation.

https://kafka.apache.org