Skip to content

test(nodejs): reduce redundant weblog scenario coverage #7689

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
rochdev wants to merge 1 commit into from
+56 −6
Open

test(nodejs): reduce redundant weblog scenario coverage #7689

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions tests/test_the_test/test_ci_orchestrator.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,8 @@ def test_legacy_scenario_matrix():
has_error = False

for library in sorted(COMPONENT_GROUPS.all):
if library == "nodejs":
continue
Comment on lines +153 to +154

@chatgpt-codex-connector chatgpt-codex-connector Bot Sep 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Restore validation for future Node.js weblog metadata

When a future Node.js Dockerfile is added without explicit metadata, WeblogMetaData.load() synthesizes an entry with no categories or supported scenarios, so test_all_weblog_has_metadata still sees its name and passes even though the orchestrator silently removes it from every job. The repository documentation identifies this legacy parity test as the guard against exactly that failure, but this blanket branch disables it for all current and future Node.js weblogs; narrow the exception to the intentional matrix differences or add an explicit nonempty-selection assertion instead. .cursor/rules/pr-review.mdcL14-L18

Useful? React with 👍 / 👎.

for weblog in sorted(WeblogMetaData.load(library), key=lambda w: w.name):
for scenario in get_all_scenarios():
legacy = _is_supported_legacy(weblog, scenario, "")
Expand Down
60 changes: 54 additions & 6 deletions utils/build/docker/nodejs/weblog_metadata.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,70 @@

express4:
build_mode: local
categories: [dd_trace, dd_trace_graphql]
supported_scenarios:

@cbeauchesne cbeauchesne Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you get the approval from all impacted team?

Another point, by using an inclusion list, it'll be easy to not activate a scenario by error, which is a QA issue. By using using excluded_scenarios, the tradeoff will be in favor of QA, as it'll be easy to include a scenario by error, which slows the CI, but at the benefit of never forgot a scenario by mistake.

- DEFAULT
- GRAPHQL_APPSEC
- GRAPHQL_ERROR_TRACKING
Comment on lines +5 to +8

@chatgpt-codex-connector chatgpt-codex-connector Bot Sep 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep express4 in the AI Guard scenarios

When the Node.js AI_GUARD* scenarios run, AIGuardScenario selects dd_trace weblogs, but this change removes that category from express4 without explicitly listing any of the five AI Guard scenarios. The orchestrator therefore schedules them only on express5, while manifests/nodejs.yml:110-213 marks every non-express4 weblog irrelevant for these classes, so most Node.js AI Guard tests are skipped and their feature coverage disappears. Add the AI Guard scenarios here or retarget those manifest declarations to a scheduled weblog. .cursor/rules/pr-review.mdcL67-L69

Useful? React with 👍 / 👎.

Comment on lines +5 to +8

@chatgpt-codex-connector chatgpt-codex-connector Bot Sep 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep express4 in the feature-flagging scenario

When FEATURE_FLAGGING_AND_EXPERIMENTATION is selected for Node.js, its dd_trace category now schedules only express5, but the file-level declarations at manifests/nodejs.yml:1830-1854 mark every weblog except express4 as incomplete_test_app. As a result, the dynamic-evaluation, exposure, and metric suites are all skipped in the workflow step at .github/workflows/run-end-to-end.yml:315-317. Add this scenario to express4 or transfer the manifest support to the weblog that remains scheduled. .cursor/rules/pr-review.mdcL67-L69

Useful? React with 👍 / 👎.

Comment on lines +5 to +8

@chatgpt-codex-connector chatgpt-codex-connector Bot Sep 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep express4 in the supported OTLP semantics scenarios

The OTEL_SEMANTICS_OTLP and OTEL_SEMANTICS_OTLP_TRACE_METRICS scenarios also select dd_trace weblogs, so this list leaves only express5 in their Node.js jobs. However, manifests/nodejs.yml:2897-2916 marks all weblogs missing for the supported server, client, and trace-metric classes except express4; those validations are therefore skipped despite the dedicated workflow steps at .github/workflows/run-end-to-end.yml:539-550. Explicitly retain express4 for the supported OTLP semantics scenarios or update the manifest to a scheduled compatible weblog. .cursor/rules/pr-review.mdcL67-L69

Useful? React with 👍 / 👎.

express4-typescript:
build_mode: local
categories: [dd_trace, dd_trace_graphql]
supported_scenarios:
- DEFAULT
- DEBUGGER_CAPTURE_TIMEOUT
- DEBUGGER_EXCEPTION_REPLAY
- DEBUGGER_EXPRESSION_LANGUAGE
- DEBUGGER_INPRODUCT_ENABLEMENT
- DEBUGGER_PII_REDACTION
- DEBUGGER_PROBES_SNAPSHOT
- DEBUGGER_PROBES_SNAPSHOT_WITH_SCM
- DEBUGGER_SYMDB
- DEBUGGER_TELEMETRY
- IAST_DEDUPLICATION
- IAST_STANDALONE
express5:
build_mode: local
categories: [dd_trace, dd_trace_graphql]
categories: [dd_trace]
fastify:
build_mode: local
categories: [dd_trace]
supported_scenarios: &framework_scenarios
- DEFAULT
Comment on lines +29 to +30

@chatgpt-codex-connector chatgpt-codex-connector Bot Sep 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep fastify in TRACING_CONFIG_EMPTY

When the Node.js TRACING_CONFIG_EMPTY job runs, fastify no longer matches because its dd_trace category was replaced by this AppSec-only list, leaving only express5 scheduled. The manifest entry at manifests/nodejs.yml:2743-2746 marks every weblog except fastify as having an incomplete /otel_drop_in_baggage_api_datadog endpoint, so Test_Baggage_Headers_Api_Datadog is now skipped entirely. Add TRACING_CONFIG_EMPTY to fastify or move that endpoint's manifest support to a scheduled weblog. .cursor/rules/pr-review.mdcL67-L69

Useful? React with 👍 / 👎.

- APPSEC_API_SECURITY
- APPSEC_API_SECURITY_NO_RESPONSE_BODY
- APPSEC_API_SECURITY_RC
- APPSEC_API_SECURITY_WITH_SAMPLING
- APPSEC_APM_STANDALONE
- APPSEC_ATO_SDK
- APPSEC_AUTO_EVENTS_EXTENDED
- APPSEC_AUTO_EVENTS_RC
- APPSEC_BLOCKING
- APPSEC_BLOCKING_FULL_DENYLIST
- APPSEC_LOW_WAF_TIMEOUT
- APPSEC_RASP
- APPSEC_RASP_NON_BLOCKING
- APPSEC_RASP_WITHOUT_DOWNSTREAM_BODY_ANALYSIS_USING_MAX
- APPSEC_RASP_WITHOUT_DOWNSTREAM_BODY_ANALYSIS_USING_SAMPLE_RATE
- APPSEC_RATE_LIMITER
- APPSEC_RUNTIME_ACTIVATION
- APPSEC_STANDALONE
- APPSEC_STANDALONE_API_SECURITY
- APPSEC_STANDALONE_APM_STANDALONE
- APPSEC_STANDALONE_RASP
- IAST_DEDUPLICATION
- IAST_STANDALONE
nextjs:
build_mode: local
categories: [dd_trace]
supported_scenarios: *framework_scenarios
uds-express4:
build_mode: local
categories: [dd_trace, dd_trace_graphql]
supported_scenarios:
- DEFAULT
- AGENT_NOT_SUPPORTING_SPAN_EVENTS
- AGENT_SUPPORTING_SPAN_EVENTS
- TELEMETRY_APP_STARTED_PRODUCTS_DISABLED
- TELEMETRY_DEPENDENCY_LOADED_TEST_FOR_DEPENDENCY_COLLECTION_DISABLED
- TELEMETRY_ENHANCED_CONFIG_REPORTING
- TELEMETRY_EXTENDED_HEARTBEAT
- TELEMETRY_LOG_GENERATION_DISABLED
- TELEMETRY_METRIC_GENERATION_DISABLED
excluded_scenarios: [IPV6]
openai-js:
build_mode: none
Expand Down
Loading