Apache Kafka: Broker OutOfMemoryError via tagged-field allocation amplification in flexible protocol request parsing
Apache Kafka's code-generated readers for flexible protocol versions read a tagged-field count declared by the sender and retain a RawTaggedField object for each unknown field. That count was bounded only by the outer request frame size (socket.request.max.bytes); it was not checked against the remaining buffer bytes or capped. Because each entry can be encoded in as little as two bytes (unknown tag 0, size 0), a modest request can declare tens of millions of tagged fields and force one object allocation per declared field. Flexible request headers (header version 2, used by flexible APIs such as ApiVersions v3) are parsed this way, so the amplification applies to the request header itself. Impact: The flexible request header is parsed on the network thread before authorization, request dispatch, and request-time quota accounting. A client can send a request of a few tens of megabytes whose header declares about 20 million tagged fields, forcing roughly that many RawTaggedField allocations and exhausting broker heap or inducing process-wide GC stalls, denying broker service. Because the header is parsed for pre-authentication APIs such as ApiVersions (v3), the allocation can occur before authentication; no Kafka resource ACL or quota is required. Mitigation: Upgrade to Apache Kafka 4.2.2, 4.3.2, or 4.4.0. Until then, lowering socket.request.max.bytes and restricting network access to broker listeners reduces, but does not remove, exposure. Resolution: From Apache Kafka 4.2.2, 4.3.2, and 4.4.0: generated readers bounds-check the tagged-field count for flexible-versioned messages, rejecting a negative count, a count exceeding the remaining buffer bytes, or a count exceeding a fixed maximum (10,000), before any per-field allocation.