Apache Pulsar: Namespace encryption requirement not enforced after a failure during topic initialization
Apache Pulsar allows an administrator to require end-to-end message encryption for all topics in a namespace by setting the encryption_required namespace policy. When this policy is set, the broker rejects messages that are published without encryption. When the broker loads a persistent topic, it reads the namespace policies and the topic-level policies as part of topic initialization. If a step of this initialization fails, for example because of an error reading from the metadata store, a failure loading the topic-level policies or a failure in a later initialization step, the broker logs a warning, disables the encryption requirement for that topic, and completes the topic load as if it had succeeded. The topic is then served with encryption_required not enforced, even though the namespace policy requires it. Topic-level policies that could not be loaded are also not applied. While the encryption requirement is not enforced, an authenticated client with permission to produce to the affected topic can publish messages that are not encrypted, and the broker accepts and stores them. This applies both to producers created after the topic was loaded and to producers that were already connected. Messages published with encryption remain encrypted; the issue does not expose the content of encrypted messages or the encryption keys. The condition is not limited to transient failures. If the cause of the initialization failure persists, the encryption requirement is disabled again each time the topic is loaded, for example after a topic unload, a namespace bundle reassignment or a broker restart. An update to the namespace policies restores enforcement on a loaded topic, but only until the topic is loaded again. Enforcement is reliably restored only once the underlying cause of the failure has been resolved and the topic has been loaded again. The broker logs a warning when the encryption requirement is disabled, but the topic load is reported as successful and the state of the encryption requirement is not shown in the topic statistics. In fixed versions, a failure during topic initialization causes the topic load to fail so that it can be retried, instead of loading the topic with the encryption requirement disabled. This issue affects Apache Pulsar versions from 1.22.0-incubating before 4.0.14 and from 4.1.0 before 4.2.5. - 4.0.x version users should upgrade to at least 4.0.14. - 4.1.x and 4.2.x version users should upgrade to at least 4.2.5. Users operating versions prior to those listed above should upgrade to the aforementioned patched versions or newer versions. The currently supported Apache Pulsar release lines are 4.0.x and 5.0.x. The supported versions schedule is available at https://pulsar.apache.org/contribute/release-policy/. Users who rely on encryption_required and cannot upgrade immediately can monitor the broker logs for warnings containing "isEncryptionRequired will be set to false" (the exact wording of the message differs between releases), resolve the underlying cause of the failure, and then reload the affected topic and confirm that the warning does not recur.