Skip to content

Skip repeated AbstractMethodError from JDBC getClientInfo - #12702

Open
dougqh wants to merge 18 commits into
masterfrom
dougqh/abstract-method-guard
Open

dougqh wants to merge 18 commits into
masterfrom
dougqh/abstract-method-guard

Conversation

@dougqh

@dougqh dougqh commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

What Does This Do

Adds ClassLatch, a small abstract type for calls that fail the same way every time for a given class, and uses it for Connection.getClientInfo() in JDBCDecorator.parseDBInfoFromConnection. Once a class is known to lack the method, the call is skipped instead of throwing and catching an AbstractMethodError on every call.

A call site holds a static final subclass and calls tryApply:

private static final ClassLatch<Connection, Properties, SQLException> CLIENT_INFO_LATCH =
    new ClassLatch<Connection, Properties, SQLException>() {
      @Override
      protected Properties apply(Connection connection) throws SQLException {
        return handleAbstractMethod(connection, "getClientInfo", Connection::getClientInfo);
      }
    };

clientInfo = CLIENT_INFO_LATCH.tryApply(connection);

The catch around the call is unchanged: it still catches Throwable, logs with EXCLUDE_TELEMETRY, and falls back to a URL-only DBInfo, so the latch is purely an optimization and the behavior is the same as before. (An earlier revision of this PR narrowed the catch to SQLException, which broke existing tests: TestConnection.getClientInfo() throws a bare Throwable, and JDBCInstrumentationV0Test and JDBCWrappedInterfacesTest expect the URL-derived DB info to survive any failure. Narrowing it, so that an unexpected failure reaches the outer handler and shows up in Error Tracking, is a separate decision and would need those tests changed.)

Motivation

Connections from old JDBC drivers and pool proxies lack getClientInfo, so every call threw and caught an AbstractMethodError. #11412 muted the log line (telemetry dropped to zero from 1.63), but the throw and catch still happen on every call. See Error Tracking issue b449bd64, which was ~650k events/day from old tracers.

The JVM does not fast-throw AbstractMethodError (that optimisation covers only a fixed set of implicit exceptions), and it cannot cache the failure on the call site because the error comes from method selection, which depends on the receiver. The latch memoises per call site and receiver class, and skips the call entirely when it can prove the class lacks the method.

Additional Notes

Design

  • tryApply (formerly tryApplyOrNull, renamed once the result was no longer necessarily null) is public final: returns null for a null target, fallback(target) for a latched one, otherwise calls apply. A null result always means "nothing available," whether skipped or genuinely absent, so the caller's fallback behaves the same either way; tryApplyOrDefault is null-coalescing sugar over it. No wrapper result type — a null return needs no allocation. Subclasses write apply as an ordinary try/catch and touch state only via latch/unlatch/latchIfNamed.
  • fallback(target) is an overridable hook: what a latched target yields instead of the operation, and what the handle* helpers return when the call fails, so the failing call and every skipped call after it agree. It defaults to null, which is what JDBC uses. Override it when there is a slower alternative (e.g. an older API, as in Convert JMSDecorator.getDestination to ClassLatch #12720's JMS getQueue/getTopic), instead of re-checking isLatched at the call site to tell a skipped call from an operation that legitimately returned null.
  • handleAbstractMethod(target, methodName, fn) is the common-case helper, taking a new ThrowingFunction<T, R, E> (alongside TriFunction/TriConsumer) so a throwing method reference like getClientInfo can be passed directly. Both AbstractMethodError and UnsupportedOperationException yield fallback(target), but only AbstractMethodError latches, and only when its message names the key class and methodName — a wrapper delegating to a deficient object still gets the fallback, but is never latched. Anything else propagates.
  • handleNoSuchMethod and handleNoSuchOrAbstractMethod are also provided but unused so far (the JDBC site uses handleAbstractMethod). handleNoSuchMethod latches the target's key rather than the whole site, since NoSuchMethodError can originate inside one receiver's implementation.
  • keyOf picks the class the latch is keyed on (default: the target's class) — override it when the target is a wrapper around the actually-deficient object.
  • State is an eager final ClassValue behind a plain (non-volatile) anyLatched flag: one flag read on the common path; a stale read just costs one extra failure, since each thread always sees its own write.
  • Parses both JDK 8 and JDK 11+ AbstractMethodError message formats, verified on real JVMs (8, 11, 17, 21, 25, GraalVM 21). An unparseable message means no latch, never a wrong one.

Observed traffic. Last 3h of "Could not get client info from DB" (~98.8k events, mostly tracer <1.63): 83.5% AbstractMethodError, 14.4% SQLFeatureNotSupportedException, 1.3% other SQLException, 0.8% UnsupportedOperationException. ~85% of the AbstractMethodError events have a non-Datadog top frame — likely a wrapper/pool proxy delegating to a deficient object (inferred from stack shape, not confirmed). This PR's latch isn't keyed on the leaf, so it only covers the direct-call minority; wrapped connections are unaffected.

Not in this PR (tracked in APMLP-1894)

Benchmark. ClassLatchBenchmark (JMH), Zulu 17.0.7 on an M1, single thread, 5 forks. Full table in the benchmark's Javadoc; JDK 8/x86 not measured. Error margins 0.1–3.4%.

Arm depth 0 depth 50 Allocation (0 / 50)
status quo (throw and catch every call) 4,391 ns 6,054 ns 896 B / 2,256 B
wrapper, cannot latch 4,304 ns 5,838 ns 896 B / 2,256 B
latched, handleAbstractMethod 4.87 ns 85.2 ns 0
latched, dedicated subclass 4.86 ns 84.4 ns 0
working path, direct call 4.46 ns 28.5 ns 0
working path, latch 4.76 ns 28.5 ns 0
working path, dedicated subclass 4.72 ns 27.8 ns 0
  • A latched class is ~900x cheaper than the status quo at depth 0, ~70x at depth 50, with zero allocation.
  • A wrapper (can't latch) performs like the status quo, as designed.
  • Method reference vs. dedicated subclass: indistinguishable (within 1%).
  • The latch itself adds ~0.3 ns on the working path at depth 0, nothing measurable at depth 50.
  • Unexplained: at depth 50 a latched skip (~85 ns) is slower than a call that runs (~28.5 ns); all 5 forks agreed, so not noise. Latch (Stop repeated NoSuchFieldError in Jackson 2.16 IAST interner lookup (quick fix) #12670) doesn't show this (~28 ns skip), so it's specific to this per-class path (ClassValue.get/keyOf) or to how this benchmark's recursion inlines it — not investigated further.

Tests. ClassLatchTest/HandleAbstractMethodTest/HandleNoSuchMethodTest cover latching, fallback (on the failing call, once latched, for an unattributed error, never for a real null or a null target), keyOf, wrapper non-latching, both message formats, collisions, null target, and checked/unchecked propagation, including a real AbstractMethodError built with the JDK compiler (JDK 8, 11, 17, 21). ParseDBInfoClientInfoTest confirms URL-based DBInfo survives any getClientInfo failure. Existing JDBC tests (JDBCInstrumentationV0Test, JDBCWrappedInterfacesTest, IastJDBCTest) pass on both regular and latest-dependency tasks; the Docker-backed RemoteJDBCInstrumentationV0Test wasn't run locally.

CODEOWNERS. New files fall under the already-owned /internal-api/src/*/*/datadog/trace/util/ path — no change needed.

Contributor Checklist

Jira ticket: APMLP-1894

🤖 Generated with Claude Code

Connections from old JDBC drivers lack getClientInfo, so every
parseDBInfoFromConnection call threw and caught an AbstractMethodError.
#11412 muted the log line but not the throw.

Add AbstractMethodGuard: a per-call-site object that treats
AbstractMethodError and UnsupportedOperationException as "not supported"
(returns null) and lets everything else, SQLException included, propagate
through a type parameter. A class is latched, so later calls skip the
call, only when the error message names exactly the receiver class, so a
wrapper delegating to a deficient driver is never latched. Both HotSpot
message formats (JDK 8 and 11+) are recognised. Anything unattributable
keeps today's behaviour.

The JDBC call site now narrows its catch from Throwable to SQLException,
so unexpected failures reach the outer handler and stay visible.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@dougqh dougqh added type: bug fix Bug fix tag: no release notes Changes to exclude from release notes inst: jdbc JDBC instrumentation tag: ai generated Largely based on code generated by an AI or LLM labels Sep 30, 2026
Comment thread internal-api/src/main/java/datadog/trace/util/AbstractMethodGuard.java Outdated
@datadog-prod-us1-3

datadog-prod-us1-3 Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

🎯 Code Coverage (details)
• Patch Coverage: 85.71%
• Overall Coverage: 59.26% (+0.01%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 9e5b445 | Docs | Give us feedback!

@dd-octo-sts

dd-octo-sts Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.84 s 14.68 s [+0.0%; +2.1%] (maybe worse)
startup:insecure-bank:tracing:Agent 13.61 s 13.75 s [-1.7%; -0.2%] (maybe better)
startup:petclinic:appsec:Agent 17.01 s 16.79 s [+0.4%; +2.2%] (maybe worse)
startup:petclinic:iast:Agent 16.83 s 17.05 s [-2.0%; -0.5%] (maybe better)
startup:petclinic:profiling:Agent 16.80 s 16.78 s [-0.8%; +0.9%] (no difference)
startup:petclinic:sca:Agent 17.07 s 16.78 s [+0.8%; +2.7%] (maybe worse)
startup:petclinic:tracing:Agent 15.74 s 16.20 s [-7.0%; +1.4%] (no difference)

Commit: 9e5b4459 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

dougqh and others added 4 commits September 30, 2026 12:23
Benchmark the guard against the status quo (throw and catch on every
call), a wrapper that cannot be latched, and the working path, using a
real AbstractMethodError built at setup.

Replace the lazily created ClassValue with an eager final field, and gate
the per-class lookup with a plain anyLatched flag. This removes the
creation race (a lost update could discard every latch so far) and gives
safe publication through the final field. A plain flag measured faster
than a volatile one on the working path.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Make isLatched package-private (only tests and the benchmark use it) and
have invokeOrNull call it so the check has a single definition. Move
CLIENT_INFO_GUARD next to the other private statics in JDBCDecorator.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Replace AbstractMethodGuard with two abstract types meant to be held in
static final fields, one per call site:

- Latch: a one-way, call-site-wide latch, for failures that are the same
  for everyone (e.g. a field missing from the classes on the classpath).
- ClassLatch: a per-class latch keyed through an overridable keyOf, for
  failures that recur for every instance of a class. The per-class state
  is an eager final ClassValue behind a plain anyLatched flag, so it is
  safely published and cannot lose latches to a creation race.

Subclasses implement get in an ordinary try/catch, so checked exceptions
need no generics tricks, and change the state only through protected
helpers (latch, unlatch, latchIfNamed). A protected higher-order
handleAbstractMethod covers the common case: AbstractMethodError is
latched only when its message names the key class, and
UnsupportedOperationException is swallowed without latching.

JDBCDecorator now holds a static final ClassLatch for getClientInfo;
behaviour is unchanged. The benchmark is renamed to ClassLatchBenchmark,
gives each arm its own method, and adds a comparison against a
dedicated-subclass form. Its results are provisional: depth-50 latched
arms showed per-fork JIT modes and a cleaner run with more forks is
planned.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Latch has no user here; it moves to the Jackson NoSuchFieldError fix
(#12670), which is where it is used. ClassLatch's Javadoc no longer
links to it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Comment thread internal-api/src/main/java/datadog/trace/util/ClassLatch.java
dougqh and others added 4 commits September 30, 2026 15:05
Drop the defaultValue hook from ClassLatch. The public methods are now
tryGetOrNull (skip if the target is null or latched, otherwise get; null
means nothing is available) and tryGetOrDefault (null-coalescing sugar
over it), so a call that yields nothing and a skipped call always agree.
handleAbstractMethod returns null on an unsupported call. The protected
hook stays get. A null return needs no allocation and no escape
analysis, unlike a wrapper result type.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Replace the nested ClassLatch.Call with a top-level ThrowingFunction
next to the other functional interfaces (TriFunction, TriConsumer), so
other toolbox types can share it. Behaviour is unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Annotate ClassLatch.handleAbstractMethod with @StrategyConsumer and its
function parameter with @strategy, as ConcurrentHashtable does, and note
in its Javadoc that callers should pass a method reference or a
non-capturing lambda. Documentation and tooling markers only; behaviour
is unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Name the helpers after the JVM errors they handle, as handleNoSuchField
does: handleAbstractMethod (AbstractMethodError), handleNoSuchMethod
(NoSuchMethodError) and handleNoSuchOrAbstractMethod (both), and
cross-link their Javadocs. The two errors are easy to confuse, so the
combined helper is the documented default.

A NoSuchMethodError names the declared type, not the receiver, and can
come from a call made inside one receiver's implementation, so it
latches the target's key and never the whole call site.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
dougqh and others added 5 commits September 30, 2026 17:10
Rename the protected hook get to handle, so it reads the same across the
Latch family and does not suggest a no-arg accessor. The public methods
(tryGetOrNull, tryGetOrDefault) are unchanged. Naming only.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Record a five-fork run on Zulu 17 (M1) in the benchmark's Javadoc: a
latched class against the status quo, a wrapper that cannot latch, and
the working path, at two stack depths, with the method-reference form
compared to a dedicated subclass. The two forms are indistinguishable.
Notes one unexplained result: at depth 50 a latched skip is slower than
a call that runs.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Rename the protected hook handle to apply, and the public methods
tryGetOrNull and tryGetOrDefault to tryApplyOrNull and tryApplyOrDefault.
apply matches ThrowingFunction.apply, which the handleX helpers take, and
no longer overlaps with the handleX helper names. Naming only; benchmark
results are unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Narrowing the catch to SQLException broke existing tests: TestConnection's
getClientInfo throws a bare Throwable, and JDBCInstrumentationV0Test and
JDBCWrappedInterfacesTest expect the URL-derived DB info to survive any
failure of getClientInfo. Restore the old resilience at the call site, so
the latch is purely an optimization, and update ParseDBInfoClientInfoTest
to assert that any failure still yields URL-based DBInfo.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@dougqh
dougqh marked this pull request as ready for review October 1, 2026 14:52
@dougqh
dougqh requested review from a team as code owners October 1, 2026 14:52
@dougqh
dougqh requested review from ygree and removed request for a team October 1, 2026 14:52

@datadog-prod-us1-3 datadog-prod-us1-3 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: FAIL

Two distinct correctness issues exist in ClassLatch.java: a NoSuchMethodError can escape the catch block if a method reference is resolved before the helper is entered, and the latch incorrectly keys on the receiver class alone rather than the specific failing method, causing unrelated valid calls to be permanently suppressed.

Open Bits AI session

🤖 Bits Code Review · Commit d176106 · @DataDog review to ask questions

Comment thread internal-api/src/main/java/datadog/trace/util/ClassLatch.java
Comment thread internal-api/src/main/java/datadog/trace/util/ClassLatch.java Outdated
&& message.length() > end
&& message.charAt(end) == ' ';
}
if (message.startsWith(name) && message.length() > name.length() + 1) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kind of hate this, but it makes the latching mechanism safer.

@ygree ygree left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The provided ClassLatch is a flexible and thoughtful abstraction for handling various errors. However, many of the provided methods are not used for the intended use case.
I will approve it assuming it will have more applications than just the JDBCDecorator use case.

@dougqh

dougqh commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

Yeah, I'm trying to find the right balance for these toolbox PRs.
I don't want to overbuild something that we won't use, but I also don't want someone to skip using it because I didn't handle an obvious use case.

And yes, we do have a few other cases for ClassLatch-es mostly more AbstractMethodError-s, so we'll have to see how this all pans out.

… method-name check

tryApplyOrNull previously only caught NoSuchMethodError, missing the
BootstrapMethodError-wrapped form that some JVM versions raise for the
same linkage failure, which let it escape the latch uncaught.

handleAbstractMethod/latchIfNamed/isNamedIn now also take the method
name the call site expects, since a receiver's own implementation can
raise an AbstractMethodError naming the key class while blaming a
different method it calls internally; without the check that error
would incorrectly latch the class for a method it does implement.

Extracts the duplicated compile() test helper into CompilingClassLoaders.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
dougqh added a commit that referenced this pull request Oct 1, 2026
Replaces the per-call AbstractMethodError catch (JMS <=1.1 producers
lack getDestination) with a ClassLatch so the error is paid once per
class instead of once per call. Distinguishes a producer's legitimate
null destination from a latched/unavailable method via isLatched,
since both can otherwise look identical.

Part of APMLP-1895; stacked on #12702 (ClassLatch).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A latched target, and the call that fails, now yield fallback(target)
instead of a bare null. Call sites with a slower alternative (an older
API) can put it in the latch rather than re-checking isLatched after
tryApplyOrNull, which also removes the ambiguity between a skipped call
and an operation that legitimately returned null. The default is null,
so existing latches are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dougqh added a commit that referenced this pull request Oct 2, 2026
Replaces the per-call AbstractMethodError catch (JMS <=1.1 producers
lack getDestination) with a ClassLatch so the error is paid once per
class instead of once per call. Distinguishes a producer's legitimate
null destination from a latched/unavailable method via isLatched,
since both can otherwise look identical.

Part of APMLP-1895; stacked on #12702 (ClassLatch).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
With an overridable fallback the result is no longer necessarily null
when the operation is skipped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dougqh added a commit that referenced this pull request Oct 2, 2026
Replaces the per-call AbstractMethodError catch (JMS <=1.1 producers
lack getDestination) with a ClassLatch so the error is paid once per
class instead of once per call. Distinguishes a producer's legitimate
null destination from a latched/unavailable method via isLatched,
since both can otherwise look identical.

Part of APMLP-1895; stacked on #12702 (ClassLatch).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@dougqh
dougqh enabled auto-merge October 2, 2026 18:41

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

inst: jdbc JDBC instrumentation tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes type: bug fix Bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants