CVE-2026-17507

MLS membership checks compare a uint32 leaf_index as signed, admitting an out-of-range sender

Published Oct 2, 2026 CVSS 4.0 · 8.7 High EPSS < 1% KEV · Not listed
Description
In Bouncy Castle for Java before 1.86, the MLS implementation (org.bouncycastle.mls) holds RFC 9420's uint32 leaf_index in a signed int, so a wire value with the top bit set decodes to a negative number. That is a legitimate encoding rather than malformed input, and it must still decode, since the MLS interop test vectors round-trip the full range. GroupKeySet.SecretTree.hasLeaf and Group.validateRemove compared the decoded value directly against the tree's leaf count, and a signed comparison treats any negative int as less than a positive bound, so an out-of-range sender passed the membership check. In the hasLeaf case the SenderData of an unprotected PrivateMessage could then drive LeafIndex.directPath through NodeIndex.parent() arithmetic that never reaches the tree root, growing the resulting node list without bound until the JVM exhausted its heap. A single small message from any current group member could therefore deny service to every other member of the group. Both comparisons now interpret the value as unsigned via Integer.toUnsignedLong, rejecting an out-of-range sender however it was encoded; well-formed leaf indices are unaffected.

Remediation

No solution or workaround provided in the CVE record

Tracking

Sign in to view the affected projects.

Advisories

No advisories available
History
Type Values Removed Values Added
First Time appeared Bouncycastle
Bouncycastle bc-java
CPEs cpe:2.3:a:bouncycastle:bc-java:*:*:*:*:*:*:*:*
Vendors & Products Bouncycastle
Bouncycastle bc-java
Metrics cvssV3_1

{'score': 7.5, 'vector': 'CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H'}

Type Values Removed Values Added
Metrics ssvc

{'options': {'Automatable': 'yes', 'Exploitation': 'none', 'Technical Impact': 'partial'}, 'version': '2.0.3'}

Type Values Removed Values Added
First Time appeared Legion Of The Bouncy Castle Inc.
Legion Of The Bouncy Castle Inc. bc-java
Vendors & Products Legion Of The Bouncy Castle Inc.
Legion Of The Bouncy Castle Inc. bc-java
Type Values Removed Values Added
Description In Bouncy Castle for Java before 1.86, the MLS implementation (org.bouncycastle.mls) holds RFC 9420's uint32 leaf_index in a signed int, so a wire value with the top bit set decodes to a negative number. That is a legitimate encoding rather than malformed input, and it must still decode, since the MLS interop test vectors round-trip the full range. GroupKeySet.SecretTree.hasLeaf and Group.validateRemove compared the decoded value directly against the tree's leaf count, and a signed comparison treats any negative int as less than a positive bound, so an out-of-range sender passed the membership check. In the hasLeaf case the SenderData of an unprotected PrivateMessage could then drive LeafIndex.directPath through NodeIndex.parent() arithmetic that never reaches the tree root, growing the resulting node list without bound until the JVM exhausted its heap. A single small message from any current group member could therefore deny service to every other member of the group. Both comparisons now interpret the value as unsigned via Integer.toUnsignedLong, rejecting an out-of-range sender however it was encoded; well-formed leaf indices are unaffected.
Title MLS membership checks compare a uint32 leaf_index as signed, admitting an out-of-range sender
Weaknesses CWE-195
References
Metrics cvssV4_0

{'score': 8.7, 'vector': 'CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:N/VC:N/VI:N/VA:H/SC:N/SI:N/SA:N/U:Amber'}

Weaknesses

Sources

Status
PUBLISHED
Assigner
bcorg
Published
Updated
Oct 2, 2026, 5:43 PM
Reserved
Jul 26, 2026, 10:40 PM
Status
Analyzed
Published
Oct 2, 2026, 8:17 AM
Modified
Oct 14, 2026, 6:59 PM