CVE-2026-55520

Protego: Exponential backtracking ReDoS in robots.txt URL wildcard matching

Published Aug 28, 2026 CVSS 4.0 · 7.1 High EPSS < 1% KEV · Not listed
Description
Protego is a pure-Python robots.txt parser with support for modern conventions. Prior to 0.6.2, protego._urlpattern._URLPattern._prepare_pattern_for_regex translates every asterisk in an Allow or Disallow directive into a lazy regular-expression wildcard, so a directive containing many asterisks creates exponential backtracking. After protego.Protego.parse processes a crafted robots.txt file, protego.Protego.can_fetch can spend an attacker-controlled period matching a near-miss URL and deny service to the crawler. The vulnerable path is src/protego/_urlpattern.py in the _URLPattern match logic. This issue is fixed in version 0.6.2.

Remediation

No solution or workaround provided in the CVE record

Tracking

Sign in to view the affected projects.

Advisories

Source ID Title
Github GHSA Github GHSA GHSA-wjmf-p669-5m5p Protego has exponential backtracking ReDoS in robots.txt URL wildcard matching
History
Type Values Removed Values Added
References
Metrics threat_severity

None

cvssV3_1

{'score': 6.5, 'vector': 'CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:U/C:N/I:N/A:H'}

threat_severity

Moderate

Type Values Removed Values Added
Metrics ssvc

{'options': {'Automatable': 'yes', 'Exploitation': 'none', 'Technical Impact': 'partial'}, 'version': '2.0.3'}

Type Values Removed Values Added
First Time appeared Scrapy
Scrapy protego
Vendors & Products Scrapy
Scrapy protego
Type Values Removed Values Added
Description Protego is a pure-Python robots.txt parser with support for modern conventions. Prior to 0.6.2, protego._urlpattern._URLPattern._prepare_pattern_for_regex translates every asterisk in an Allow or Disallow directive into a lazy regular-expression wildcard, so a directive containing many asterisks creates exponential backtracking. After protego.Protego.parse processes a crafted robots.txt file, protego.Protego.can_fetch can spend an attacker-controlled period matching a near-miss URL and deny service to the crawler. The vulnerable path is src/protego/_urlpattern.py in the _URLPattern match logic. This issue is fixed in version 0.6.2.
Title Protego: Exponential backtracking ReDoS in robots.txt URL wildcard matching
Weaknesses CWE-1333
CWE-400
References
Metrics cvssV4_0

{'score': 7.1, 'vector': 'CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:P/VC:N/VI:N/VA:H/SC:N/SI:N/SA:N'}

Weaknesses

Sources

Status
PUBLISHED
Assigner
GitHub_M
Published
Updated
Aug 31, 2026, 6:54 PM
Reserved
Jun 16, 2026, 10:44 PM
Status
Deferred
Published
Aug 28, 2026, 8:18 PM
Modified
Sep 9, 2026, 9:09 PM
Severity
Moderate
Public date
Aug 28, 2026, 6:42 PM