TC8 SOME/IP conformance test suite for someipd#60
TC8 SOME/IP conformance test suite for someipd#60jorgecasal wants to merge 19 commits intoeclipse-score:mainfrom
Conversation
Wire-level pytest tests verifying someipd against the OA TC8 ECU Test Specification v3.0 Chapter 5: service discovery (phases, reboot, timing), message format, event notification, field lifecycle, and TCP transport. - someipd --tc8-standalone mode: request/response, field GET/SET, UDP and TCP events; offer_event uses ET_FIELD so vsomeip delivers the cached field value to new subscribers immediately on subscribe (is_field in JSON config is not honoured in programmatic offer_event calls) - vsomeip config templates for SD timing and service/event/field/TCP tests, with JSON Schema validation at lint time - Architecture doc, requirements, test specification and OA spec traceability - Bazel port isolation via the env attribute: each target receives unique SD and service ports enabling medium targets to run in parallel; timing-sensitive and reboot lifecycle targets retain the exclusive tag for serial execution
License Check Results🚀 The license check job ran with the Bazel command: bazel run //:license-checkStatus: Click to expand output |
jorgecasal
changed the title
jorgecasal
added
documentation
|
The created documentation from the pull request is available at: docu-html |
- Raise check-added-large-files limit from 50 KB to 125 KB to accommodate large but legitimate TC8 test modules and the RST test specification document - Add REUSE.toml annotation for tests/tc8_conformance/config/*.json so the four vsomeip JSON config files pass reuse-lint-file - Prepend Apache-2.0 SPDX comment headers to the two TC8 README.md files that were missing copyright notices
TC8 conformance tests require special setup (multicast route, env vars) and already run in a dedicated step; avoid double-execution via -tc8 filter.
jorgecasal
force-pushed
the
jorgecasal_tc8_conformance_test_infrastructure
branch
from
c507e28 to
63d628d
Compare
jorgecasal
force-pushed
the
jorgecasal_tc8_conformance_test_infrastructure
branch
from
63d628d to
8420326
Compare
|
Documentation overall looks very good, reviewed the current state at https://eclipse-score.github.io/inc_someip_gateway/pr-60/architecture/tc8_conformance_testing.html - not in detail, but general sections and impression. Nice! On my local checkout, running the tests works with the expected failures:
$ time bazel test --test_tag_filters=tc8 --test_env=TC8_HOST_IP=127.0.0.1 //tests/tc8_conformance/...Result: real 6m16.073s |
|
I missed setting up the multicast... now the tests run fine locally as well, can confirm. |
mikehaller
left a comment
mikehaller
left a comment
There was a problem hiding this comment.
As the TC8 Conformity Tests is a complete new part for someip gateway, I'll approve the current state of the PR. It's a very good step forward.
There is one issue regarding the main.cpp where I didn't fully grasp the plans and hence would accept it if it's just temporary.
| bazel test //tests/... --test_tag_filters=tc8 | ||
|
|
||
| # Use a real network interface | ||
| TC8_HOST_IP=<your-host-ip> bazel test //tests/tc8_conformance/... |
There was a problem hiding this comment.
Is this host or device under test ip?
There was a problem hiding this comment.
Got it, it's for choosing a different host interface (e.g. localhost vs. eth0)
There was a problem hiding this comment.
You got it right, nevertheless let me provide a bit more information. In the protocol conformance tests, the DUT (someipd) runs on the same machine as the test framework, so they share the same IP. TC8_HOST_IP is the host's network interface IP, where it's used both to bind the test's raw sockets (for sending/receiving SOME/IP packets) and to configure someipd via the __TC8_HOST_IP__ placeholder in the vsomeip JSON templates.
In short, host IP = DUT IP here, because they're co-located. The variable name reflects the test framework's perspective (the machine under test), not a remote DUT. Happy to rename it to something clearer like TC8_DUT_IP if you think that better matches TC8 terminology.
There was a problem hiding this comment.
I assume the previous state of main.cpp also was just a hardcoded dummy service, and extending it with TC8 specifics as part of the pull request doesn't make a big difference.
In the longer run, would it make sense to have the TC8-enabled version of someipd main.cpp as a separate application?
I ask because someipd is supposed to be an in-vehicle component and mixing production code and test code together like this may not be ideal. If it's just temporary until the other PR's converge, i'll pull back my comment.
There was a problem hiding this comment.
Your assumption is correct, main.cpp was already a hardcoded placeholder with no production logic, so the TC8 additions don't mix test code into real production code. That said, your longer-term concern is valid: once someipd grows its real implementation, the TC8-specific flags should move to a separate binary. We'll have to track that as a follow-up.
jorgecasal
changed the title
There was a problem hiding this comment.
Note: score_docs_as_code version bump required for full traceability
While verifying the TC8 requirements → test specification → code implementation traceability linkage introduced in this PR, a bug was found in score_docs_as_code that silently prevented all testcase need objects from being created (see eclipse-score/docs-as-code#440 (comment) for the detailed root cause).
The fix has been merged in eclipse-score/docs-as-code#440, but is not yet available in a published release. Our MODULE.bazel currently pins:
bazel_dep(name = "score_docs_as_code", version = "2.3.0", dev_dependency = True)- Use local json_schema_validator rule instead of @score_communication - Add integrity hashes to download_archive deps for reproducible builds - Add component descriptions for gatewayd and someipd in architecture docs
jorgecasal
force-pushed
the
jorgecasal_tc8_conformance_test_infrastructure
branch
from
4393f39 to
2dfcff0
Compare
jorgecasal
force-pushed
the
jorgecasal_tc8_conformance_test_infrastructure
branch
from
ead57d7 to
84174d4
Compare
lurtz
left a comment
lurtz
left a comment
There was a problem hiding this comment.
I did not look at the test implementations yet and want to settle high level concerns (test environment, concurrency) first
| sudo ip route add 224.0.0.0/4 dev lo || true | ||
|
|
||
| # Confirm multicast route was added | ||
| ip route show | grep "224\." || echo "WARNING: No multicast route found in routing table!" |
There was a problem hiding this comment.
can this be done from within bazel? I get the impression that I have to setup my environment the same way. This makes it harder to reproduce CI build failures.
There was a problem hiding this comment.
TC8 tests now self configure their network via bazel test --config=tc8 //tests/tc8_conformance/.... The --config=tc8 flag wraps each test in a private network namespace that sets up loopback multicast automatically. Additionally, bazel test //... now excludes TC8 by default via --test_tag_filters=-tc8 in .bazelrc.
Please double check.
| - name: Bazel test targets | ||
| run: | | ||
| bazel test //... --build_tests_only | ||
| bazel test //... --build_tests_only --test_tag_filters=-lint,-tc8 |
There was a problem hiding this comment.
I am not happy that a simple bazel test //... fails now. Can the tests instead be opt-in? Or they are able to setup the environment. Another option would be check the environment and skip tests, which do not fit the current environment.
Right now I get the following test results with the thread sanitizer enabled using the devcontainer:
$ sudo ip route add 224.0.0.0/4 dev lo
$ bazel test //... --nocache_test_results
...
================================================================================
INFO: Found 73 targets and 26 test targets...
INFO: Elapsed time: 127.333s, Critical Path: 59.16s
INFO: 27 processes: 80 action cache hit, 9 disk cache hit, 31 local.
INFO: Build completed, 11 tests FAILED, 27 total actions
//:format.check_Python_with_ruff PASSED in 0.2s
//:format.check_Rust_with_rustfmt PASSED in 0.2s
//:format.check_Starlark_with_buildifier PASSED in 0.2s
//:format.check_YAML_with_yamlfmt PASSED in 0.1s
//:python_requirements_test PASSED in 4.2s
//src/gatewayd:config_file_test PASSED in 0.0s
//src/gatewayd:config_schema.update_test PASSED in 0.1s
//src/gatewayd:mw_com_config_schema_valid PASSED in 0.1s
//src/someipd:mw_com_config_schema_valid PASSED in 0.1s
//tests/benchmarks:benchmark_mw_com_config_schema_valid PASSED in 0.1s
//tests/cpp:cpp_test_main PASSED in 0.1s
//tests/integration:integration PASSED in 0.9s
//tests/tc8_conformance:tc8_someipd_multi_schema_valid PASSED in 0.1s
//tests/tc8_conformance:tc8_someipd_sd_schema_valid PASSED in 0.0s
//tests/tc8_conformance:tc8_someipd_service_schema_valid PASSED in 0.1s
//tests/rust:rust_hello_test FAILED in 0.2s
/var/cache/bazel/7573313f746e4064cdb168fae28a75d9/execroot/_main/bazel-out/k8-fastbuild/testlogs/tests/rust/rust_hello_test/test.log
//tests/tc8_conformance:tc8_event_notification FAILED in 1.0s
/var/cache/bazel/7573313f746e4064cdb168fae28a75d9/execroot/_main/bazel-out/k8-fastbuild/testlogs/tests/tc8_conformance/tc8_event_notification/test.log
//tests/tc8_conformance:tc8_field_conformance FAILED in 6.0s
/var/cache/bazel/7573313f746e4064cdb168fae28a75d9/execroot/_main/bazel-out/k8-fastbuild/testlogs/tests/tc8_conformance/tc8_field_conformance/test.log
//tests/tc8_conformance:tc8_message_format FAILED in 1.3s
/var/cache/bazel/7573313f746e4064cdb168fae28a75d9/execroot/_main/bazel-out/k8-fastbuild/testlogs/tests/tc8_conformance/tc8_message_format/test.log
//tests/tc8_conformance:tc8_multi_service FAILED in 1.1s
/var/cache/bazel/7573313f746e4064cdb168fae28a75d9/execroot/_main/bazel-out/k8-fastbuild/testlogs/tests/tc8_conformance/tc8_multi_service/test.log
//tests/tc8_conformance:tc8_sd_client FAILED in 59.1s
/var/cache/bazel/7573313f746e4064cdb168fae28a75d9/execroot/_main/bazel-out/k8-fastbuild/testlogs/tests/tc8_conformance/tc8_sd_client/test.log
//tests/tc8_conformance:tc8_sd_format FAILED in 1.5s
/var/cache/bazel/7573313f746e4064cdb168fae28a75d9/execroot/_main/bazel-out/k8-fastbuild/testlogs/tests/tc8_conformance/tc8_sd_format/test.log
//tests/tc8_conformance:tc8_sd_phases_timing FAILED in 11.1s
/var/cache/bazel/7573313f746e4064cdb168fae28a75d9/execroot/_main/bazel-out/k8-fastbuild/testlogs/tests/tc8_conformance/tc8_sd_phases_timing/test.log
//tests/tc8_conformance:tc8_sd_reboot FAILED in 50.1s
/var/cache/bazel/7573313f746e4064cdb168fae28a75d9/execroot/_main/bazel-out/k8-fastbuild/testlogs/tests/tc8_conformance/tc8_sd_reboot/test.log
//tests/tc8_conformance:tc8_sd_robustness FAILED in 1.3s
/var/cache/bazel/7573313f746e4064cdb168fae28a75d9/execroot/_main/bazel-out/k8-fastbuild/testlogs/tests/tc8_conformance/tc8_sd_robustness/test.log
//tests/tc8_conformance:tc8_service_discovery FAILED in 1.7s
/var/cache/bazel/7573313f746e4064cdb168fae28a75d9/execroot/_main/bazel-out/k8-fastbuild/testlogs/tests/tc8_conformance/tc8_service_discovery/test.log
There was a problem hiding this comment.
Good point. TC8 tests now skip gracefully instead of failing when the environment isn't ready. The require_tc8_environment autouse fixture in conftest.py does some validations before any TC8 test runs:
TC8_HOST_IPenv var must be set (opt-in via--test_env=TC8_HOST_IP=127.0.0.1)- The value must be a valid IP address
- On lo interface, a multicast route must exist
If any check fails the entire module is skipped — TC( tests only run when explicitly opted into.
| /// Handle SET-field: update field state in-place, return new payload for notification. | ||
| /// Returns a notify payload to broadcast, which is always non-null on SET — TC8-FLD-004. | ||
| // REQ: comp_req__tc8_conformance__fld_get_set | ||
| static std::shared_ptr<vsomeip::payload> HandleTc8SetField( |
There was a problem hiding this comment.
I see Tc8 in many names. What about putting these functions and variables into a namespace or class? Then you do not need to repeat Tc8 in every name. And it is also obvious they belong together.
I see that std::shared_ptr<vsomeip::application> app is passed into many functions. Thus one option could be to encapsulate that within a class called Tc8TestApplication. Then you can skip at least this function parameter and shorten some function names.
However we should also take care not to overcomplicate namespaces.
There was a problem hiding this comment.
That's a valid point, the Tc8 prefix repetition and app threading are some of the signs that this code wants to be in its own namespace or class.
However, someipd as a whole is due for a significant refactoring (class decomposition, dependency injection, configuration-driven service IDs, etc). The current flat main.cpp is a known tech debt item. When that refactoring happens, TC8 standalone mode will either:
- Move into its own class/file naturally as part of the new architecture
- Be eliminated entirely if the new design makes
someipddirectly testable
Adding a Tc8TestApplication class now would create an intermediate abstraction that the refactoring would, most likely, discard. I'd prefer to keep the flat structure for now — it's simple, easy to understand, and easy to delete when the time comes.
If you feel that's still worth, I can wrap the TC8 constants and helpers in a namespace tc8 {} as a lightweight compromise, that's a relatively easy change with zero impact.
| - **Application-Level Tests** — tests the full gateway path | ||
| (mw::com client → ``gatewayd`` → ``someipd`` → network) using C++ apps | ||
| built on ``score::mw::com``. These tests are stack-agnostic. |
There was a problem hiding this comment.
How are these verified if not on the wire level? Maybe the image is missing a second someipd and gatewayd
There was a problem hiding this comment.
You're right. Fixed the diagrams to show both gateway stacks: TC8 Service → gatewayd → someipd ↔ someipd → gatewayd → TC8 Client.
| @startuml | ||
| !theme plain | ||
| scale max 800 width | ||
| skinparam classAttributeIconSize 0 | ||
| skinparam class { | ||
| BackgroundColor<<test>> #E3F2FD | ||
| BorderColor<<test>> #1565C0 | ||
| BackgroundColor<<helper>> #E8F5E9 | ||
| BorderColor<<helper>> #2E7D32 | ||
| } | ||
|
|
||
| title Service Discovery — Test Module Dependencies | ||
|
|
||
| class test_service_discovery <<test>> { | ||
| TC8-SD-001..008, 011, 013, 014 | ||
| SOMEIPSRV_SD_MESSAGE_01–06/14–19 | ||
| SD_BEHAVIOR_03/04 | ||
| ETS_088/091/092/098/099/100/101 | ||
| ETS_107/120/122/128/130/155 | ||
| } | ||
| class test_sd_phases_timing <<test>> { | ||
| TC8-SD-009 / 010 | ||
| } | ||
| class test_sd_reboot <<test>> { | ||
| TC8-SD-012 | ||
| } | ||
| class test_sd_format_compliance <<test>> { | ||
| TC8-SDF (SD Format) | ||
| FORMAT_01/02/04–06/09–13/15/16/18–28 | ||
| OPTIONS_01/02/03/05/06/08–14 | ||
| } | ||
| class test_sd_robustness <<test>> { | ||
| ETS SD Robustness | ||
| Malformed entries, options, | ||
| framing errors, subscribe edges | ||
| } | ||
| class test_sd_client <<test>> { | ||
| ETS SD Client Lifecycle | ||
| ETS_081/082/084 | ||
| } | ||
|
|
||
| class sd_helpers <<helper>> { | ||
| +open_multicast_socket() | ||
| +parse_sd_offers() | ||
| +capture_sd_offers() | ||
| } | ||
| class sd_sender <<helper>> { | ||
| +open_sender_socket() | ||
| +send_find_service() | ||
| +send_subscribe_eventgroup() | ||
| +capture_unicast_sd_entries() | ||
| +capture_some_ip_messages() | ||
| } | ||
| class sd_malformed <<helper>> { | ||
| +build_malformed_entry() | ||
| +build_malformed_option() | ||
| +build_truncated_sd() | ||
| +send_malformed_sd() | ||
| } | ||
| class someip_assertions <<helper>> { | ||
| +assert_sd_offer_entry() | ||
| +assert_offer_has_ipv4_endpoint_option() | ||
| +assert_offer_has_tcp_endpoint_option() | ||
| } | ||
| class timing <<helper>> { | ||
| +collect_sd_offers_from_socket() | ||
| +capture_sd_offers_with_timestamps() | ||
| } | ||
|
|
||
| ' layout: test modules in two rows | ||
| test_service_discovery -right[hidden]- test_sd_phases_timing | ||
| test_sd_phases_timing -right[hidden]- test_sd_reboot | ||
| test_sd_format_compliance -right[hidden]- test_sd_robustness | ||
| test_sd_robustness -right[hidden]- test_sd_client | ||
| test_service_discovery -down[hidden]- test_sd_format_compliance | ||
|
|
||
| ' layout: helpers in a row below tests | ||
| sd_helpers -right[hidden]- sd_sender | ||
| sd_sender -right[hidden]- sd_malformed | ||
| someip_assertions -right[hidden]- timing | ||
| sd_helpers -down[hidden]- someip_assertions | ||
|
|
||
| ' test → helper dependencies | ||
| test_service_discovery -down-> sd_helpers | ||
| test_service_discovery -down-> sd_sender | ||
| test_service_discovery -down-> someip_assertions | ||
| test_service_discovery -down-> timing | ||
| test_sd_phases_timing -down-> timing | ||
| test_sd_phases_timing -down-> sd_helpers | ||
| test_sd_reboot -down-> sd_helpers | ||
| test_sd_format_compliance -down-> sd_helpers | ||
| test_sd_robustness -down-> sd_malformed | ||
| test_sd_robustness -down-> sd_helpers | ||
| test_sd_client -down-> sd_helpers | ||
| test_sd_client -down-> sd_sender | ||
|
|
||
| ' internal helper dependencies | ||
| timing ..> sd_helpers : <<uses>> |
There was a problem hiding this comment.
I suppose this is the inheritance hierarchy present in test source code. Having it here will create future effort to keep it in sync. Can this be automated or skipped?
There was a problem hiding this comment.
Replaced all three detailed class diagrams with compact component diagrams — module names and dependency arrows only, no method signatures or test case IDs.
Much less likely to go stale since it only changes when a whole module is added or removed.
| TO -down-> PO : uses | ||
| @enduml | ||
|
|
||
| The orchestrator starts the ETS application, ``gatewayd``, and ``someipd`` |
There was a problem hiding this comment.
In the rendered document there is no explanation what ETS stands for. It could use some introduction
There was a problem hiding this comment.
Added an introductory sentence expanding ETS, ETC, and the other acronyms.
There was a problem hiding this comment.
For my taste this file covers too much detail. Part of it reads like a test report and requirements tracing. I would have only described the test setups, which are the chapters until including Planned Components
There was a problem hiding this comment.
Trimmed the file down. The spec alignment analysis, coverage tables, and known stack limitations moved to docs/tc8_conformance/traceability.rst. The CI/CD section was already covered by the test README so it was dropped. The port assignment table moved to the README as well.
| package "TC8 Conformance Docs" { | ||
| file "**test_specification.rst**\nDetailed test cases:\npurpose, stimuli,\nexpected results" as TestSpec #f5f5f5 | ||
| file "**traceability.rst**\nOA Spec ID →\nInternal ID →\nRequirement →\nTest Function" as Trace #f5f5f5 | ||
| } |
There was a problem hiding this comment.
to me it looks like an additional level added to the requirements tracing. Or that it is its own requirements tracing. It looks a bit complicated in the picture, but I have no better idea.
There was a problem hiding this comment.
Simplified the diagram by removing the documentation-artifact nodes (test_specification.rst, traceability.rst) — those are supporting files, not requirement levels. The diagram now shows only the standard 3-level S-CORE hierarchy (stkh_req → feat_req → comp_req), the test verification link, and the external OA standard reference.
| "console": "true", | ||
| "file": { | ||
| "enable": "false", | ||
| "path": "/tmp/vsomeip-tc8.log" |
There was a problem hiding this comment.
hardcoded paths will make concurrent test execution impossible if we do not use bazels ´linux-sandbox`
There was a problem hiding this comment.
Good catch. Although file logging is disabled ("enable": "false"), /tmp is shared across network namespaces (unshare --net only isolates the network, not the filesystem). If someone enables logging for debugging, concurrent tests would overwrite each other's log files.
That is now be fixed: replaced the hardcoded path with __TC8_LOG_DIR__/vsomeip-tc8.log in all three config templates. The placeholder is substituted at render time with pytest's tmp_path_factory directory, which is unique per test target. This follows the same pattern already used for __TC8_HOST_IP__, __TC8_SD_PORT__, etc.
| "TC8_SD_PORT": "30490", | ||
| "TC8_SVC_PORT": "30500", |
There was a problem hiding this comment.
is there a way to ensure these ports are actually available? Stuff which might happen:
- test crashes
- Linux keeps port occupied until timeout is reached
- next test run prior reaching the port cleaning timeout will fail
Or some random application might for whatever reason (ephemeral ports) use one of these. Maybe we have to enforce using bazels linux-sandbox, even though I will have then to figure out how to make it work on Ubuntu 24.04 within a devcontainer.
There was a problem hiding this comment.
This is already handled by tc8_net_wrapper.sh. Each test target runs inside a private network namespace (unshare --user --net --map-root-user), configured in .bazelrc via test:tc8 --run_under=//tests/tc8_conformance:tc8_net_wrapper.
Inside a private network namespace the kernel creates a fresh network stack:
- All ports are available — no inherited sockets, no TIME_WAIT from previous runs
- No conflict with host processes or other test namespaces
- On crash, the kernel destroys the namespace and releases all sockets automatically
Bazel's linux-sandbox uses the same kernel mechanism (CLONE_NEWNET) to create network namespaces — see Bazel sandboxing docs ("uses Linux Namespaces (User, Mount, PID, Network and IPC namespaces) to isolate the action from the host") and the linux-sandbox.cc source. Switching to it would not add extra network isolation beyond what our wrapper already provides.
The unique port numbers per target (SD 30490–30499, SVC 30500–30513) are defense-in-depth for the fallback path when unshare is not available (lines 27–29 of the wrapper).
Upgrade json_schema_validator from 2.1.0 to 2.4.0 to resolve 5 compiler warnings (deprecated nlohmann/json 3.11.x APIs, -Wswitch, -Wrange-loop). Adapt BUILD file for 2.4.0 archive layout changes (CMakeLists.txt exclusion, relocated validator binary). Remove the jsonschema Python package (4.23.0) which had zero consumers in the project.
Use `unshare --user --net` to create a private network namespace per test, eliminating the need for `sudo ip route add` before running TC8 conformance tests. Add `--config=tc8` Bazel config that sets the tag filter, host IP, and --run_under wrapper automatically. Also exclude TC8 from `bazel test //...` via default tag filter and standardize terminology from "real NIC" to "non-loopback interface".
| # Try namespace isolation; fall back to direct execution. | ||
| # On locked-down systems, the test's own environment check (conftest.py) | ||
| # handles the skip with an actionable message. | ||
| unshare --user --net --map-root-user -- bash -c ' |
There was a problem hiding this comment.
With --map-root-user I get the following failure using the devcontainer:
lure9428 ➜ /workspaces/score_inc_someip_gateway (main) $ unshare --user --net --map-root-user ip a
unshare: write failed /proc/self/uid_map: Operation not permittedHowever without it works:
lure9428 ➜ /workspaces/score_inc_someip_gateway (main) $ unshare --user --net ip a
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00Is it possible to remove --map-root-user?
There was a problem hiding this comment.
We can't simply remove it, my testing confirmed that w/o --map-root-user, ip link set lo up and ip route add both fail on regular systems. So far, the solution is cascading fallback — i.e., try with --map-root-user first, then without, then direct execution.
Additionally, some useful information can be found here.
| # Fallback: unshare not available — run directly, let conftest.py handle it. | ||
| exec "$@" |
There was a problem hiding this comment.
I already find it a bit confusing that bazel has silent fallbacks as well. E.g. communication has designed tests which require the use of linux-sandbox, which is used by bazel if available. But otherwise it will run these tests with less sandboxing and all in a sudden you wonder why sometimes tests are flaky.
Can you add a warning?
There was a problem hiding this comment.
I added a warning on namespace fallback a suggested.
jorgecasal
force-pushed
the
jorgecasal_tc8_conformance_test_infrastructure
branch
from
0fae264 to
622244e
Compare
- Add shebang and is_executable = True to json_schema_validator.bzl script output. - Simplify TC8 network namespace wrapper documentation across .bazelrc, RST docs, README.md, and the wrapper script comments - Add a warning on namespace fallback in tc8_net_wrapper.sh
jorgecasal
force-pushed
the
jorgecasal_tc8_conformance_test_infrastructure
branch
from
622244e to
cf3a5d8
Compare
jorgecasal
force-pushed
the
jorgecasal_tc8_conformance_test_infrastructure
branch
from
e3cf506 to
70b305e
Compare
…ance_test_infrastructure
Simplify the PlantUML hierarchy diagram by removing documentation- artifact nodes (test_specification.rst, traceability.rst) that were mistaken for additional requirement levels. Replace hardcoded /tmp/vsomeip-tc8.log with __TC8_LOG_DIR__ placeholder in all TC8 config templates, substituted at render time with pytest tmp_path.
3 participants
Summary
Introduces the TC8 SOME/IP conformance test suite for
someipd, covering OA TC8 ECU Test Specification. Tests run wire-level via raw SOME/IP sockets over loopback, withoutgatewayd,using a new--tc8-standaloneflag added tosomeipd.Documentation
docs/architecture/tc8_conformance_testing.rst— new architecture document: standalone mode rationale, port-isolation design, per-target port table, test module structure, and TC8 specification alignment analysis.docs/tc8_conformance/requirements.rst— requirements for SD, message format, event notification, field lifecycle, TCP transport, and robustness, with AUTOSAR PRS traceability IDs.docs/tc8_conformance/test_specification.rst— test-specification document mapping each requirement to one or more test cases.docs/tc8_conformance/traceability.rst— requirement-to-test traceability matrix linking TC8 spec sections to test IDs.docs/architecture/index.rstanddocs/requirements/index.rst— updated to include the new documents.TC8 Conformance Test Directory Structure
New directory
tests/tc8_conformance/with the following layout:Test Framework and Test Cases
All tests use
score_py_pytestBazel targets. Port isolation is achieved via theenvattribute — each target receives uniqueTC8_SD_PORT,TC8_SVC_PORT, and (where needed)TC8_SVC_TCP_PORTvalues, allowing medium-sized targets to run in parallel. Timing-sensitive and reboot lifecycle targets carry theexclusivetag for serial execution.someipd Changes
--tc8-standaloneflag (src/someipd/main.cpp): skips gatewayd IPC and offers the TC8 service/event/field directly, enabling self-contained conformance testing without the full gateway stack.offer_eventusesET_FIELDfor field-type events so vsomeip delivers the cached initial value to new subscribers immediately on subscribe.src/someipd/BUILD.bazel:someipdbinary deps updated for standalone mode.CI Workflow
.github/workflows/build_and_test_host.yml— new stepBazel TC8 Conformance Test Targets:224.0.0.0/4 dev lo) required for vsomeip SOME/IP-SD on the GitHub Actions runner.bazel test --test_tag_filters=tc8 --test_env=TC8_HOST_IP=127.0.0.1 //tests/tc8_conformance/....General Clean-up
MODULE.bazel: dependency version bump.tests/integration/BUILD.bazel: minor dependency alignment.Test Plan
bazel test --test_tag_filters=tc8 --test_env=TC8_HOST_IP=127.0.0.1 //tests/tc8_conformance/...passes locallybazel run //:copyright.checkpasses (all new files have correct headers)build_and_test_host) green on this branchbazel build //docs/...builds without RST errors