Skip to content

test(otel): cover OTEL_TRACES variables parametrically - #7706

Draft
MilanGarnier wants to merge 3 commits into
mainfrom
milan.garnier/test-otel-traces-sampler-arg
Draft

test(otel): cover OTEL_TRACES variables parametrically#7706
MilanGarnier wants to merge 3 commits into
mainfrom
milan.garnier/test-otel-traces-sampler-arg

Conversation

@MilanGarnier

@MilanGarnier MilanGarnier commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Motivation

Add dedicated parametric system-test coverage for the OTEL_TRACES_* variables.

Changes

  • add registry-based coverage for OTEL_TRACES_EXPORTER
  • add registry-based coverage for OTEL_TRACES_SAMPLER
  • add registry-based coverage for OTEL_TRACES_SAMPLER_ARG
  • migrate the corresponding legacy cases into focused test modules
  • declare broad manifest assumptions sourced from the audit data

Tests were intentionally not run during this first phase.

@github-actions

github-actions Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

CODEOWNERS have been resolved as:

tests/parametric/otel_env_vars/test_otel_traces_exporter.py             @DataDog/system-tests-reviewers
tests/parametric/otel_env_vars/test_otel_traces_sampler.py              @DataDog/system-tests-reviewers
tests/parametric/otel_env_vars/test_otel_traces_sampler_arg.py          @DataDog/system-tests-reviewers
manifests/cpp.yml                                                       @DataDog/system-tests-reviewers
manifests/cpp_httpd.yml                                                 @DataDog/system-tests-reviewers
manifests/cpp_nginx.yml                                                 @DataDog/system-tests-reviewers
manifests/dotnet.yml                                                    @DataDog/system-tests-reviewers
manifests/golang.yml                                                    @DataDog/system-tests-reviewers
manifests/java.yml                                                      @DataDog/system-tests-reviewers
manifests/java_otel.yml                                                 @DataDog/system-tests-reviewers
manifests/nodejs.yml                                                    @DataDog/system-tests-reviewers
manifests/nodejs_otel.yml                                               @DataDog/system-tests-reviewers
manifests/php.yml                                                       @DataDog/system-tests-reviewers
manifests/python.yml                                                    @DataDog/system-tests-reviewers
manifests/python_lambda.yml                                             @DataDog/system-tests-reviewers
manifests/python_otel.yml                                               @DataDog/system-tests-reviewers
manifests/ruby.yml                                                      @DataDog/system-tests-reviewers
manifests/rust.yml                                                      @DataDog/system-tests-reviewers
tests/parametric/test_otel_env_vars.py                                  @DataDog/system-tests-reviewers

@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Sep 11, 2026 •
edited by datadog-datadog-us1-prod Bot
Loading

Copy link
Copy Markdown

Pipelines  Tests

⚠️ Warnings

❌ Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 15 Pipeline jobs failed

Testing the test | System Tests (dotnet, dev) / parametric / parametric (2) — ❌ 20 tests failed · 🔧 Needs a code fix, caused by this PR

View more details · View in GitHub Actions

❌ tests.parametric.otel_env_vars.test_otel_traces_exporter.Test_OTEL_TRACES_EXPORTER.test_default_matches_specification[unset, parametric-dotnet] from system_tests_suite 
assert False
 +  where False = any(<generator object _assert_otlp_export.<locals>.<genexpr> at 0x7ff0691736b0>)

self = <tests.parametric.otel_env_vars.test_otel_traces_exporter.Test_OTEL_TRACES_EXPORTER object at 0x7ff06a765b80>
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7ff069011d00>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7ff069408830>

    @pytest.mark.parametrize("library_env", UNSET_VALUE)
    def test_default_matches_specification(self, test_agent: TestAgentAPI, test_library: APMLibrary) -> None:
>       _assert_otlp_export(test_agent, test_library)
...
❌ tests.parametric.otel_env_vars.test_otel_traces_exporter.Test_OTEL_TRACES_EXPORTER.test_deprecated_values[logging, parametric-dotnet] from system_tests_suite 
Failed: DID NOT RAISE <class 'ValueError'>

self = <tests.parametric.otel_env_vars.test_otel_traces_exporter.Test_OTEL_TRACES_EXPORTER object at 0x7ff06a764a70>
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7ff069011d00>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7ff06948b200>

    @pytest.mark.parametrize("library_env", DEPRECATED_VALUES)
    def test_deprecated_values(self, test_agent: TestAgentAPI, test_library: APMLibrary) -> None:
>       _assert_stdout_export(test_agent, test_library)

...
❌ tests.parametric.otel_env_vars.test_otel_traces_exporter.Test_OTEL_TRACES_EXPORTER.test_development_values[otlp/stdout, parametric-dotnet] from system_tests_suite 
Failed: DID NOT RAISE <class 'ValueError'>

self = <tests.parametric.otel_env_vars.test_otel_traces_exporter.Test_OTEL_TRACES_EXPORTER object at 0x7ff06a765820>
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7ff069011d00>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7ff06948b2c0>

    @pytest.mark.parametrize("library_env", DEVELOPMENT_VALUES)
    def test_development_values(self, test_agent: TestAgentAPI, test_library: APMLibrary) -> None:
>       _assert_stdout_export(test_agent, test_library)

...
↳ and 17 more — View all
Testing the test | System Tests (golang, dev) / parametric / parametric (2) — ❌ 18 tests failed · 🔧 Needs a code fix, caused by this PR

View more details · View in GitHub Actions

❌ tests.parametric.otel_env_vars.test_otel_traces_exporter.Test_OTEL_TRACES_EXPORTER.test_default_matches_specification[unset, parametric-golang] from system_tests_suite 
assert False
 +  where False = any(<generator object _assert_otlp_export.<locals>.<genexpr> at 0x7f32266fb5e0>)

self = <tests.parametric.otel_env_vars.test_otel_traces_exporter.Test_OTEL_TRACES_EXPORTER object at 0x7f3226e21c10>
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7f32259275c0>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7f322654efc0>

    @pytest.mark.parametrize("library_env", UNSET_VALUE)
    def test_default_matches_specification(self, test_agent: TestAgentAPI, test_library: APMLibrary) -> None:
>       _assert_otlp_export(test_agent, test_library)
...
❌ tests.parametric.otel_env_vars.test_otel_traces_exporter.Test_OTEL_TRACES_EXPORTER.test_deprecated_values[logging, parametric-golang] from system_tests_suite 
self = <tests.parametric.otel_env_vars.test_otel_traces_exporter.Test_OTEL_TRACES_EXPORTER object at 0x7f3226e50830>
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7f32259275c0>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7f3225be9b20>

    @pytest.mark.parametrize("library_env", DEPRECATED_VALUES)
    def test_deprecated_values(self, test_agent: TestAgentAPI, test_library: APMLibrary) -> None:
>       _assert_stdout_export(test_agent, test_library)

tests/parametric/otel_env_vars/test_otel_traces_exporter.py:186: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
...
❌ tests.parametric.otel_env_vars.test_otel_traces_exporter.Test_OTEL_TRACES_EXPORTER.test_development_values[otlp/stdout, parametric-golang] from system_tests_suite 
self = <tests.parametric.otel_env_vars.test_otel_traces_exporter.Test_OTEL_TRACES_EXPORTER object at 0x7f3226da2c60>
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7f32259275c0>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7f32265f7fb0>

    @pytest.mark.parametrize("library_env", DEVELOPMENT_VALUES)
    def test_development_values(self, test_agent: TestAgentAPI, test_library: APMLibrary) -> None:
>       _assert_stdout_export(test_agent, test_library)

tests/parametric/otel_env_vars/test_otel_traces_exporter.py:190: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
...
↳ and 15 more — View all
Testing the test | System Tests (java, dev) / parametric / parametric (2) — ❌ 20 tests failed · 🔧 Needs a code fix, caused by this PR

View more details · View in GitHub Actions

❌ tests.parametric.otel_env_vars.test_otel_traces_exporter.Test_OTEL_TRACES_EXPORTER.test_default_matches_specification[unset, parametric-java] from system_tests_suite 
assert False
 +  where False = any(<generator object _assert_otlp_export.<locals>.<genexpr> at 0x7f20e0477510>)

self = <tests.parametric.otel_env_vars.test_otel_traces_exporter.Test_OTEL_TRACES_EXPORTER object at 0x7f20e1fed3a0>
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7f20e0623f20>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7f20e1023c20>

    @pytest.mark.parametrize("library_env", UNSET_VALUE)
    def test_default_matches_specification(self, test_agent: TestAgentAPI, test_library: APMLibrary) -> None:
>       _assert_otlp_export(test_agent, test_library)
...
❌ tests.parametric.otel_env_vars.test_otel_traces_exporter.Test_OTEL_TRACES_EXPORTER.test_deprecated_values[logging, parametric-java] from system_tests_suite 
Failed: DID NOT RAISE <class 'ValueError'>

self = <tests.parametric.otel_env_vars.test_otel_traces_exporter.Test_OTEL_TRACES_EXPORTER object at 0x7f6601141fd0>
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7f65d0850530>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7f65d0e091f0>

    @pytest.mark.parametrize("library_env", DEPRECATED_VALUES)
    def test_deprecated_values(self, test_agent: TestAgentAPI, test_library: APMLibrary) -> None:
>       _assert_stdout_export(test_agent, test_library)

...
❌ tests.parametric.otel_env_vars.test_otel_traces_exporter.Test_OTEL_TRACES_EXPORTER.test_development_values[otlp/stdout, parametric-java] from system_tests_suite 
Failed: DID NOT RAISE <class 'ValueError'>

self = <tests.parametric.otel_env_vars.test_otel_traces_exporter.Test_OTEL_TRACES_EXPORTER object at 0x7f66011423f0>
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7f65d0850530>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7f65d14a9910>

    @pytest.mark.parametrize("library_env", DEVELOPMENT_VALUES)
    def test_development_values(self, test_agent: TestAgentAPI, test_library: APMLibrary) -> None:
>       _assert_stdout_export(test_agent, test_library)

...
↳ and 17 more — View all

View all 15 failed jobs.

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 6015ead | Docs | View more details | Give us feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@MilanGarnier