Inca — Message Tracing and Loss Detection For Streaming Data @Netflix

by Allen Wang, Real Time Data Infrastructure

At Netflix, our real time data infrastructure have embraced the multi-cluster Kafka architecture and Flink powered stream processing platform, which enable us to deliver trillions of messages per day. This architecture is described in detail in the above blogs and is illustrated in the diagram below.