diff --git a/scripts/Readme.md b/scripts/Readme.md index df5c2587653c..30441422899b 100644 --- a/scripts/Readme.md +++ b/scripts/Readme.md @@ -6,36 +6,35 @@ This folder contains a lot of useful scripts that can speed up development for r SOF has several build targets depending on whether you are building firmware, tooling, documentation or topologies. This directory has a helper for each. -### Firmware +### Firmware Build (`xtensa-build-zephyr.py`) -Firmware can either be built using west command directly or by the xtensa-build-zephyr.py script. This script wraps up the west commands and can build using either the Zephyr SDK compiler or the Cadence xtensa compiler for xtensa targets. +Firmware can either be built using west commands directly or by the `xtensa-build-zephyr.py` script. This script wraps up the west commands and can build using either the Zephyr SDK compiler or the Cadence xtensa compiler for xtensa targets. -Please run the script with --help to see all options. +Please run the script with `--help` to see all options. E.g to build SOF for Intel Pantherlake: -1) Enable the python virtual environment for west. This should be in your SOF workspace installation direction. Default is ```~/work/sof``` (only needs run once). +1) Enable the python virtual environment for west. This should be in your SOF workspace installation directory. Default is `~/work/sof` (only needs run once). + ```bash source ~/work/sof/.venv/bin/activate ``` -2) Now run the build script. *Note: most build errors are a result of ingredients being out of sync with the west manifest. Please run ```west update``` and rebuild before fixing/reporting build errors.* + +2) Now run the build script. *Note: most build errors are a result of ingredients being out of sync with the west manifest. Please run `west update` and rebuild before fixing/reporting build errors.* + ```bash ./scripts/xtensa-build-zephyr.py -p ptl ``` -### Testbench +### Reproducible Output Builds (`test-repro-build.sh`) -Testbench is a host application that is used to run SOF processing modules on developers PC. This allows for module development using regular host based tooling. +This script can be used to locally reproduce the exact build steps and environment of specific CI validation tests. -Please run ```bash -./rebuild-testbench.sh --help +./scripts/test-repro-build.sh --help ``` -for full options. - -Testbench can be also be built for Cadence simulator targets. -### Tools and Topologies +## Tools and Topologies Tooling and topology can be built together using one script. To build all topologies please run: @@ -43,65 +42,126 @@ Tooling and topology can be built together using one script. To build all topolo ./scripts/build-tools.sh ``` -This script can build: -1) sof-ctl -2) sof-logger -3) probes -4) all topology 1 & 2 and test topologies. -5) Local ALSA git version for alsa-lib and alsa-utils that have features not yet in distro version of ALSA packages. +**Options for `build-tools.sh`:** + +* `-c` : Rebuild `ctl/` tool +* `-l` : Rebuild `logger/` tool +* `-p` : Rebuild `probes/` tool +* `-T` : Rebuild ALL `topology/` targets +* `-X` : Rebuild topology1 only +* `-Y` : Rebuild topology2 only +* `-t` : Rebuild test topologies +* `-A` : Clone and rebuild the local ALSA git version for `alsa-lib` and `alsa-utils` with latest non-distro features. +* `-C` : No build, only CMake re-configuration. Shows CMake targets. + +*Warning: building tools is incremental by default. To build from scratch delete the `tools/build_tools` directory or use `-C`.* + +### ALSA Specific Build (`build-alsa-tools.sh`) + +If you want to pull down and explicitly recompile only the associated ALSA libraries from their public `alsa-lib` GitHub upstream independently of SOF topologies: + +```bash +./scripts/build-alsa-tools.sh +``` + +## Testbench and Emulation + +Testbench is a host application that is used to run SOF processing modules on developers PC. This allows for module development using regular host based tooling. + +### Rebuilding the Testbench (`rebuild-testbench.sh`) + +This script cleans and rebuilds the host test application binary. Ensure you supply the correct target platform wrapper or fuzzing backend. + +**Usage Options:** + +* `-p ` : Build testbench binary for `xt-run` for selected platform (e.g. `-p tgl`). When omitted, performs a `BUILD_TYPE=native`, compile-only check. +* `-f` : Build testbench via a compiler provided by a fuzzer (default path: `.../AFL/afl-gcc`). +* `-j` : Number of parallel make jobs (defaults to `nproc`). + +### Running the Testbench (`host-testbench.sh`) + +Starts the testing sequences. This invokes specific components to ensure basic inputs process without segfaults. + +```bash +./scripts/host-testbench.sh +``` + +### QEMU Check (`qemu-check.sh`) + +Automated verifier for executing firmware builds under QEMU emulation. + +**Usage:** + +```bash +./scripts/qemu-check.sh [platform(s)] +``` + +* Supported platforms are: `imx8`, `imx8x`, `imx8m`. +* Runs all supported platforms by default if none are provided. ## SDK Support There is some SDK support in this directory for speeding up or simplifying tasks with multiple steps. -### New Modules +### New Modules (`sdk-create-module.py`) + +A new module can be created by running the SDK Create Module script. This python helper copies the SOF template audio module and renames all strings, Cmakefiles, and Kconfigs automatically. It also correctly registers a new DSP UUID and TOML entries. -A new module can be created by running the sdk-create-module script. This script will copy the template module and rename all strings, Cmakefiles, Kconfigs to match the new module. It will also create a UUID for the new module and a TOML manifest entry (for targets that need this). +Please run: -Please run ```bash -./sdk-create-module.py new_module_name +./scripts/sdk-create-module.py new_module_name ``` ## Docker -The docker container provided in docker_build sets up a build environment for -building Sound Open Firmware. A working docker installation is needed to run -the docker build container. +The docker container provided in `docker_build` sets up a build environment for building Sound Open Firmware. A working docker installation is needed to run the docker build container. + +*Note: In order to run docker as non sudo/root user please run:* -Note: In order to run docker as non sudo/root user please run. ```bash sudo usermod -aG docker your-user-name ``` + Then logout and login again. -Quick Start: +**Quick Start:** + +First, build the docker container. This step needs to be done initially and when the toolchain or ALSA dependencies are updated. -First, build the docker container. This step needs to be done initially and -when the toolchain or alsa dependencies are updated. ```bash cd scripts/docker_build ./docker-build.sh ``` + After the container is built, it can be used to run the scripts. To build for tigerlake: + ```bash ./scripts/docker-run.sh ./scripts/xtensa-build-all.sh -l tgl ``` -or (may need password test0000 for rimage install) + +or (this command may prompt for a password during rimage installation inside the container) + ```bash ./scripts/docker-run.sh ./scripts/xtensa-build-all.sh tgl ``` + To rebuild the topology and logger: + ```bash ./scripts/docker-run.sh ./scripts/build-tools.sh ``` -An incremental sof.git build: + +An incremental `sof.git` build: + ```bash ./scripts/docker-run.sh make ``` + Or enter a shell: + ```bash ./scripts/docker-run.sh bash ``` diff --git a/src/debug/debug_stream/readme.md b/src/debug/debug_stream/readme.md new file mode 100644 index 000000000000..592edef8a719 --- /dev/null +++ b/src/debug/debug_stream/readme.md @@ -0,0 +1,61 @@ +# SOF Debug Stream + +The `debug_stream` framework is an abstract logging and live-data streaming mechanism allowing the DSP to asynchronously push structured or freeform diagnostic records immediately out to the host. + +## Feature Overview + +Unlike standard tracing (`mtrace`), which requires buffering and complex host parsing logic often tied directly to pipeline topologies or ALSA interfaces, the `debug_stream` bypasses the audio framework entirely. It utilizes the dedicated IPC Memory Windows (specifically the debug slot) to write data. + +The stream is particularly useful for reporting: + +1. **Thread Information:** Real-time data from Zephyr OS threads (like CPU runtime, context switch frequencies, or stack high-water marks). +2. **Text Messages (`ds_msg`):** Lightweight string prints that bypass the standard heavily-formatted logger. + +## How to Enable + +These features are disabled by default to save firmware footprint. You can enable them via Kconfig: + +* `CONFIG_SOF_DEBUG_STREAM_SLOT=y` : Master switch. Reserves exactly one Memory Window 4k block (default Slot 3) mapping to host space. +* `CONFIG_SOF_DEBUG_STREAM_THREAD_INFO=y` : Activates the Zephyr thread statistics compiler integration (`INIT_STACKS`, `THREAD_MONITOR`). +* `CONFIG_SOF_DEBUG_STREAM_TEXT_MSG=y` : Allows calling `ds_msg("...", ...)` scattered throughout DSP C code to emit plain strings. + +## Architecture + +The architecture revolves around a "Slot" abstraction where data is copied sequentially over a ringbuffer into the ADSP debug window slot used for the debug stream (mapped over PCIe/SPI for the Host to read non-destructively). + +```mermaid +graph TD + subgraph SOF Firmware + SysEvent["System Event / OS Timer"] --> |Triggers| DSThread["Thread Info Collector"] + DevCode["Developer Code"] --> |"ds_msg()"| Text["Text Subsystem"] + + DSThread --> Formatter[DS Formatter] + Text --> Formatter + + Formatter --> Slot[Memory Window Slot 3] + end + + subgraph Host System + PyTool[tools/debug_stream/debug_stream.py] + Slot -.->|PCIe DMA / IPC Memory| PyTool + PyTool --> |Stdout| User[Developer Terminal] + end +``` + +## Usage Example + +If you enable `CONFIG_SOF_DEBUG_STREAM_TEXT_MSG=y`, developers can insert rapid debug markers without setting up topology traces: + +```c +#include + +void my_function() { + ds_msg("Reached tricky initialization state! Value: %d", some_val); +} +``` + +On the host machine, you extract this continuous output stream by running the provided SOF tooling: + +```bash +python3 tools/debug_stream/debug_stream.py +``` diff --git a/src/debug/gdb/readme.md b/src/debug/gdb/readme.md new file mode 100644 index 000000000000..33b6a31915c1 --- /dev/null +++ b/src/debug/gdb/readme.md @@ -0,0 +1,65 @@ +# GDB Remote Debugging Stub + +The Sound Open Firmware (SOF) project carries a GNU Debugger (GDB) stub directly integrated with the framework's exception handlers. This translates commands sent by a GDB client (running on the host Linux machine) into architecture-specific logic. + +## Feature Overview + +Instead of completely relying on complex JTAG setups, developers can use this stub to dynamically introspect panic states, stack traces, and variable states during firmware execution, particularly inside isolated SoC DSP cores. + +When the firmware faults or hits a defined breakpoint, the exception vector routes control into this stub. It then waits for GDB Remote Protocol packet streams (ASCII formatted over the SOF mailbox/shared memory window). The Host reads these mailbox slots and pushes/pulls responses to its active GNU Debugger session. + +## Architecture + +Data moves between the Host GDB environment, the physical mailboxes bounding the DSP domain, the DSP firmware's built-in stub, and the active exception state. + +```mermaid +sequenceDiagram + participant GdbSession as Host GDB Client + participant IPC as SOF Driver / ALSA + participant Stub as Firmware GDB Stub (gdb_parser) + participant HW as DSP Context Registers + + HW-->>Stub: Hard Fault / Breakpoint Hit + activate Stub + + Note over Stub: Stores fault context (sregs/aregs) + + Stub-->>IPC: Write Mailbox: Breakpoint Notification + + GdbSession->>IPC: Send Packet (e.g., $g#67 to Read Regs) + IPC->>Stub: Passes 'g' string + + Stub->>HW: Reads requested register values + HW-->>Stub: Values + + Stub->>Stub: mem_to_hex() formatting + Stub-->>GdbSession: Returns hex payload via Mailbox + + GdbSession->>IPC: Send Packet ($c#63 to Continue) + IPC->>Stub: Process 'c' + Stub->>HW: Restores Context, Retains Execution + deactivate Stub +``` + +## How to Enable + +A basic GDB debugging configuration is exposed via Kconfig and must be explicitly bound: + +* `CONFIG_GDB_DEBUG`: Needs to be toggled `=y` to compile `src/debug/gdb/gdb.c` into the main application. + +Additionally, the overarching architecture requires the corresponding Exception vectors to be rewritten. In Zephyr OS based builds (which currently drive native architectures), fatal exception handling must be configured to pass register dumps recursively to `gdb_handle_exception()`. + +## Usage and Protocols + +The protocol adheres precisely to the standard GDB remote serial specification. Each string packet expects the format: + +`$#` + +Supported Command Handlers inside the Stub: + +* `g` (Read all registers) / `G` (Write all registers) +* `m` (Read memory) / `M` (Write memory) +* `p` (Read specific register) / `P` (Write specific register) +* `v` (Query architecture/support details like `vCont`) +* `c` / `s` (Continue execution / Single-step) +* `z` / `Z` (Insert/Remove breakpoints) diff --git a/src/debug/telemetry/readme.md b/src/debug/telemetry/readme.md new file mode 100644 index 000000000000..19664ad2be8c --- /dev/null +++ b/src/debug/telemetry/readme.md @@ -0,0 +1,48 @@ +# Telemetry and Performance Measurements + +The SOF Telemetry subsystem is a suite of built-in diagnostics measuring code execution efficiencies, cycle overheads, and hardware I/O throughput. + +## Feature Overview + +Latency and real-time execution bounds are critical in DSP firmware. The telemetry feature provides mechanisms to monitor these bounds accurately without intrusive breakpoints or slowing down the pipeline too aggressively. + +Capabilities include: + +1. **Component Performance Tracking**: For every instantiated component in the graph, it measures the pure execution time bounds (min/max/average) of that component's `comp_copy()` routines. +2. **I/O Throughput Tracking**: Measures hardware bus speeds or message handling by counting bytes, state changes, or tokens across distinct interfaces: IPC, DMIC, I2S, HD/A, I2C, SPI, etc. +3. **Zephyr Systick Measurement**: Specifically tracks the overall scheduler overhead bounding RTOS ticks. + +Measurements are batched into a ringbuffer locally, then synced across mapped ADSP memory windows into user space, limiting the impact on the active instruction cache. + +## Architecture + +The architecture bridges the component layer (like pure IPC or Audio Component wrappers) directly into independent statistics accumulators. + +```mermaid +graph TD + subgraph DSP Environment + Comp[Audio Component X] --> |"comp_copy() Execution"| Telemetry[perf_measure_execute] + HW[I2S / DMIC HW Driver] --> |"State Change Count"| Telemetry + + Telemetry --> RingBuffer[Statistics Ringbuffer] + RingBuffer --> Sync["Memory Window 3 (perf_data_sync)"] + end + + subgraph Host Userspace + Dev[sof-logger / IPC Tooling] --> |Reads/Queries| Sync + end +``` + +## How to Enable + +Telemetry depends strictly on NOT being built inside a host-userspace environment simulator (`depends on !SOF_USERSPACE_LL`). Ensure your target is a physical or emulated DSP target. + +Settings to configure in `Kconfig`: + +* `CONFIG_SOF_TELEMETRY=y` : Enable the overarching telemetry interfaces, giving you systick and basic task metrics over Memory Window 2 interfaces. +* `CONFIG_SOF_TELEMETRY_PERFORMANCE_MEASUREMENTS=y` : Adds granular tracking to audio components (creating the explicit `telemetry.c` ringbuffer maps via Memory Window 3 slots). Be aware that only a specific configured amount (`PERFORMANCE_DATA_ENTRIES_COUNT`) can be actively tracked due to RAM constraints. +* `CONFIG_SOF_TELEMETRY_IO_PERFORMANCE_MEASUREMENTS=y` : Instructs hardware and communication buses to start pumping data into the metrics collector. + +## Extracting Data + +You can fetch these metrics via `sof-logger` or standard IPC interrogation tools that support polling the corresponding debug window slots mapped for your particular platform's `ADSP_MW`. diff --git a/src/debug/tester/readme.md b/src/debug/tester/readme.md new file mode 100644 index 000000000000..bd558fd80a4d --- /dev/null +++ b/src/debug/tester/readme.md @@ -0,0 +1,54 @@ +# Tester Component + +The `tester` directory implements a pseudo-component inside the SOF architecture. Instead of processing audio like an EQ or Mixer, it acts as a test execution harness designed specifically for Continuous Integration (CI) and automated system verifications. + +## Feature Overview + +Continuous integration often requires validating low-level system interactions that cannot be reached or easily triggered via standard ALSA driver behaviors. To overcome this, the `tester` component can be instantiated via Topologies or explicit IPC commands. Once bound, the host can send configuration blobs triggering embedded testing subroutines compiled directly into the firmware. + +Currently, it supports functionalities like: + +* **Dummy Operations**: Echoing parameters to verify full-stack serialization roundtrips. +* **DRAM Verification**: Stress-testing memory allocation arrays within the `LLB`/`L2` domains to detect caching bugs or out-of-bounds pointer crashes (`tester_simple_dram_test`). + +## Architecture + +The architecture functions identical to an IPC Version 4 standard module, containing `comp_driver`, `comp_dev`, `comp_buffer` wrappers and implementing `tester_params()` handlers. + +```mermaid +graph TD + subgraph Host Automation + Testbench[CI Testbench / ALSA Tool] + Testbench -->> |Topology Payload: CMD_TEST| IPC + end + + subgraph SOF Firmware + IPC[IPC4 Handler] -->> |ipc4_set_module_params| Tester[Tester Component] + + Tester -->> |Switch on Test Type| Dummy[tester_dummy_test.c] + Tester -->> |Switch on Test Type| DRAM[tester_simple_dram_test.c] + + Dummy -->> IPC + DRAM -->> |Runs Data Verifications| ExtMemory[SoC DRAM / Caches] + DRAM -->> IPC + end +``` + +## How to Enable + +Because the tester artificially exposes internal memory bounds limits and allows arbitrary firmware injection tests, it is locked out of standard release payloads. + +To compile it into your build, you must append: + +* `CONFIG_COMP_TESTER=y` + +**Constraints:** +It strictly `depends on IPC_MAJOR_4`. You cannot instantiate this tester under the legacy IPC3 streaming mechanisms. + +## Creating New Tests + +Developers diagnosing tricky hardware bugs can extend the tester by: + +1. Adding a new case to the test selection switch in `tester.c:tester_init()` (and, if needed, extending runtime handling in `tester_set_configuration()`). +2. Defining a new `test_runner()` C file implementing their data patterns. +3. Adding the target configurations in `tester.toml` to automatically sync with the module offset calculators (`llext_offset_calc`).