Bruce - Kafka 生产者守护进程


Apache
Linux
C/C++

软件简介

Bruce 是 Apache Kafka 的生产者守护进程,它简化了客户端发送消息到
Kafka ,无需关注后端的 Kafka 集群。Bruce 主要处理:

  • Routing messages to the proper brokers, and spreading the load evenly across multiple partitions for a given topic

  • Waiting for acknowledgements, and resending messages as necessary due to communication failures or Kafka-reported errors

  • Buffering messages to handle transient load spikes and Kafka-related problems

  • Tracking message discards when serious problems occur; Providing web-based discard reporting and status monitoring interfaces

  • Batching and compressing messages in a configurable manner for improved performance

Bruce runs on each individual host that communicates with Kafka, receiving
messages from local clients over a UNIX domain datagram socket. Clients write
messages to Bruce’s socket in a simple binary format. Once a client has
written a message, no further interaction with Bruce is required. From that
point onward, Bruce takes full responsibility for reliable message delivery.
Bruce serves as a single intake point for a Kafka cluster, receiving messages
from diverse clients regardless of what programming language a client is
written in. Client code is currently available in C, C++, Java, Python, and
PHP. Code contributions for clients in other programming languages are much
appreciated. Technical details on how to send messages to Bruce are provided
here.
Bruce runs on Linux, and has been tested on CentOS versions 7 and 6.5, and
Ubuntu versions 14.04.1 LTS and 13.10. Bruce requires at least version 0.8 of
Kafka.