Skip to content

Avoid URISyntaxException for common bad input in HttpURLConnection URLs (quick fix) - #12693

Draft
dougqh wants to merge 6 commits into
masterfrom
dougqh/lenient-url-to-uri
Draft

dougqh wants to merge 6 commits into
masterfrom
dougqh/lenient-url-to-uri

Conversation

@dougqh

@dougqh dougqh commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

What Does This Do

Draft / jumping-off point, expect to iterate.

Stops HttpURLConnection requests with common bad-input URLs from throwing and reporting a URISyntaxException on every request.

java.net.URL accepts strings that java.net.URI rejects. HttpUrlConnectionDecorator.url and UrlConnectionDecorator.onURL called URL.toURI(), so each such request threw, was caught in HttpClientDecorator.onRequest ("Error tagging url"), and ended up with no http.url, no peer host/port, and no path-based resource name.

  • Adds URIUtils.toURI(URL), which percent-encodes only what java.net.URI rejects before parsing: a space or control character, any of " < > \ ^ { | }, [or]in the **path** (they stay legal in the query, fragment and an IPv6 host), a%not followed by two hex digits, and a second#`. Non-ASCII letters are kept; non-ASCII spaces/controls are encoded as UTF-8.
  • Well-formed URLs are converted exactly as url.toURI() would, and return the same String with no extra allocation.
  • Anything else URI rejects still throws URISyntaxException, so existing catch blocks behave as before.
  • HttpUrlConnectionDecorator.url and UrlConnectionDecorator.onURL now use it.
  • The repair runs only while bad URLs are arriving. toURI goes through an AdaptiveLatch (ported from Avoid repeated exceptions from malformed Kafka Base64 headers with AdaptiveLatch ("quick" fix) #12672): well-formed URLs take url.toURI() directly and skip the scan, a URISyntaxException switches to the repairing path, and 64 well-formed URLs in a row switch back. A repaired URL keeps it on the repairing path, so a steady stream of bad URLs throws once, not once every 64 calls.
  • AdaptiveLatch gains two things for this: X may be a checked exception (as with ClassLatch), and repaired(result) reports input the cautious path fixed.
  • Adds URIUtilsToURITest (JUnit 5, 17 cases): repaired inputs, unchanged well-formed inputs (compared with url.toURI()), non-ASCII handling, and a case that must still throw.

Motivation

Three Error Tracking issues are this same failure through different entry points: 7eb5eb3c (HttpURLConnection.getInputStream), 80a09d2a (HttpURLConnection.connect, currently IGNORED with no recorded reason) and 7d87bb46 (commons-httpclient, not touched here). Together they are about 200k events a day (all tracer versions; reports aggregate repeats, so the real throw count is higher), and 80a09d2a is still ~28% on 1.65.1 / 1.66.0, so upgrading does not make it go away. A latch that skips work is the wrong tool, because the failure is per URL. The adaptive latch skips nothing: it only chooses between two correct conversions, so a wrong guess costs one exception or one extra scan, never a lost tag.

Additional Notes

  • No benchmark included. The claim is that a repaired URL costs less than a thrown and caught URISyntaxException plus the lost tags; that is reasoned, not measured. The latch's CLOSE_AFTER = 64 is likewise an estimate (one throw under an HTTP client's stack against one scan of a well-formed URL). A JMH comparing the paths (well-formed strict, well-formed scanned, repaired, throwing) would be a good addition before this leaves draft.
  • Behavior change to be aware of: requests that used to get no URL tags now get them, built from the repaired URI. http.url and the SSRF check see the percent-encoded form, and URI.getPath() decodes back to the original characters.
  • Possible iterations:
    • The pathStart / path-end scan is only needed when the string has a [ or ]; computing it lazily would make the repairing path a single pass. Well-formed URLs no longer reach it unless the latch is engaged.
    • Other toURI() callers were not touched: OkHttpClientDecorator (okhttp-2.2, request.url().toURI()), and commons-httpclient (new URI(httpMethod.getURI().toString()), a different source type).
    • The structural alternative is a client-side URIDataAdapter (already used by ~20 server decorators) so that nothing goes through the strict java.net.URI.
  • java.net.URL already rejects malformed authorities at construction, so URI failures after the repair should be rare; the "still throws" test uses the multi-argument URL constructor, which does not validate the host.
  • SpotBugs and Spotless pass on internal-api, and agent-bootstrap compiles. URIUtilsToURITest (21, including 4 latch cases), URIUtilsTest (49) and AdaptiveLatchTest (12) pass.

Contributor Checklist

Jira ticket: N/A

🤖 Generated with Claude Code

java.net.URL accepts strings that java.net.URI rejects (a space, |, {}, [] in
the path, a stray % or a second #). HttpUrlConnectionDecorator and
UrlConnectionDecorator called URL.toURI(), so every such request threw and
reported a URISyntaxException and lost its http.url, peer and path-based
resource tags.

Add URIUtils.toURI(URL), which percent-encodes only the offending characters
before parsing. Well-formed URLs go through unchanged and without extra
allocation.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@dougqh dougqh added type: bug fix Bug fix comp: core Tracer core tag: no release notes Changes to exclude from release notes tag: ai generated Largely based on code generated by an AI or LLM labels Sep 29, 2026
@datadog-prod-us1-3

This comment has been minimized.

@dd-octo-sts

dd-octo-sts Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 13.83 s 13.92 s [-1.2%; -0.0%] (maybe better)
startup:insecure-bank:tracing:Agent 12.81 s 12.88 s [-1.3%; +0.1%] (no difference)
startup:petclinic:appsec:Agent 17.07 s 16.86 s [+0.2%; +2.2%] (maybe worse)
startup:petclinic:iast:Agent 16.79 s 16.98 s [-2.1%; -0.2%] (maybe better)
startup:petclinic:profiling:Agent 16.53 s 16.82 s [-2.9%; -0.5%] (maybe better)
startup:petclinic:sca:Agent 17.05 s 17.01 s [-0.6%; +1.1%] (no difference)
startup:petclinic:tracing:Agent 16.13 s 16.13 s [-0.9%; +0.9%] (no difference)

Commit: df93969e · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@dougqh dougqh changed the title Avoid URISyntaxException for common bad input in HttpURLConnection URLs (draft) Avoid URISyntaxException for common bad input in HttpURLConnection URLs (quick fix) Oct 7, 2026
dougqh and others added 5 commits October 7, 2026 15:55
Brought over unchanged, with its tests, for use in URL-to-URI conversion.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
X may now be a checked exception, as with ClassLatch, so a latch can guard
URI parsing. repaired(result) restarts the engaged count like reject but keeps
the repaired result, so a steady stream of repairable input stays on the
cautious path instead of paying one optimistic failure every closeAfter calls.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
URIUtils.toURI now goes through an AdaptiveLatch: well-formed URLs take
url.toURI() directly and skip the scan, and the repairing conversion runs only
after a URISyntaxException, until 64 well-formed URLs in a row. A repaired URL
keeps the latch engaged. Unrepairable URLs still throw URISyntaxException.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Compares url.toURI() (master), the latched URIUtils.toURI, the latch's repair
path on every call, and reading the URL's fields directly with no URI, each
on well-formed and bad URLs at stack depth 0 and 50.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: core Tracer core tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes type: bug fix Bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant