Search before asking

• I searched in the issues and found nothing similar.

Motivation

As we discuss in #399 , this issue will focus on supportting multiple advertised listeners.

Same as Fluss, both Kafka and Pulsar also used to support listening only one port. However, both supports multiple advertised listeners in the production environment later, thus connection from internal server and external client will use different port and security protocol.

Kafka

As shown in kafka document, Multiple Listeners for Kafka Brokers and KIP-103: Separation of Internal and External traffic, the support multiple type of listeners:

• listeners: Comma-separated list of URIs we will listen on and the listener names. If the listener name is not a security protocol, listener.security.protocol.map must also be set.
• advertised.listeners: Listeners to publish to ZooKeeper for clients to use, if different than the listeners config property. In IaaS environments, this may need to be different from the interface to which the broker binds. If this is not set, the value for listeners will be used. Unlike listeners, it is not valid to advertise the 0.0.0.0 meta-address.
• inter.broker.listener.name: Name of listener used for communication between brokers. If this is unset, the listener name is defined by security.inter.broker.protocolIt is an error to set this and security.inter.broker.protocol properties at the same time.
• control.plane.listener.name: Name of listener used for communication between controller and brokers. A broker will use the control.plane.listener.name to locate the endpoint in listeners list, to listen for connections from the controller.
• controller.listener.names: A comma-separated list of the names of the listeners used by the controller. This is required if running in KRaft mode. When communicating with the controller quorum, the broker will always use the first listener in this list.

process.roles=broker
# the listeners of each port
listeners=CLIENT://localhost:9092,BROKER://localhost:9093,
# the security protocol of each listener
listener.security.protocol.map=CLIENT:SSL,BROKER:PLAINTEXT


inter.broker.listener.name=BROKER

Pulasr

As shown in pulsar document and PIP 61: Advertised multiple addresses , to ensure clients in both internal and external networks can connect to a Pulsar cluster, Pulsar introduces advertisedListeners and internalListenerName configuration options into the broker configuration file to ensure that the broker supports exposing multiple advertised listeners and support the separation of internal and external network traffic.

• The advertisedListeners is used to specify multiple advertised listeners. The broker uses the listener as the broker identifier in the load manager and the bundle owner data. The advertisedListeners is formatted as <listener_name>:pulsar://:, <listener_name>:pulsar+ssl://:. You can set up the advertisedListeners like advertisedListeners=internal:pulsar://192.168.1.11:6660,internal:pulsar+ssl://192.168.1.11:6651.

• The internalListenerName is used to specify the internal service URL that the broker uses. You can specify the internalListenerName by choosing one of the advertisedListeners. The broker uses the listener name of the first advertised listener as the internalListenerName if the internalListenerName is absent.

Solution

No response

Anything else?

No response

Willingness to contribute

• I'm willing to submit a PR!