Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions AGENTS.md
19 changes: 11 additions & 8 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co

## Project Overview

QAS CLI (`qas-cli`) is a Node.js CLI tool for uploading test automation results (JUnit XML / Playwright JSON) to [QA Sphere](https://qasphere.com/). It matches test case markers (e.g., `PRJ-123`) in report files to QA Sphere test cases, creates or reuses test runs, and uploads results with optional attachments.
QAS CLI (`qas-cli`) is a Node.js CLI tool for uploading test automation results (JUnit XML / Playwright JSON / Allure results directories) to [QA Sphere](https://qasphere.com/). It matches test case references (e.g., `PRJ-123` markers or Allure TMS links) to QA Sphere test cases, creates or reuses test runs, and uploads results with optional attachments.

## Commands

Expand All @@ -29,20 +29,21 @@ Node.js compatibility tests: `cd mnode-test && ./docker-test.sh` (requires Docke
### Entry Point & CLI Framework

- `src/bin/qasphere.ts` — Entry point (`#!/usr/bin/env node`). Validates Node version, delegates to `run()`.
- `src/commands/main.ts` — Yargs setup. Registers two commands (`junit-upload`, `playwright-json-upload`) as instances of the same `ResultUploadCommandModule` class.
- `src/commands/main.ts` — Yargs setup. Registers three commands (`junit-upload`, `playwright-json-upload`, `allure-upload`) as instances of the same `ResultUploadCommandModule` class.
- `src/commands/resultUpload.ts` — `ResultUploadCommandModule` defines CLI options shared by both commands. Loads env vars, then delegates to `ResultUploadCommandHandler`.

### Core Upload Pipeline (src/utils/result-upload/)

The upload flow has two stages handled by two classes:

1. **`ResultUploadCommandHandler`** — Orchestrates the overall flow:
- Parses report files using the appropriate parser (JUnit XML or Playwright JSON)
- Detects project code from test case names (or from `--run-url`)
- Creates a new test run (or reuses an existing one if title conflicts)
- Delegates actual result uploading to `ResultUploader`

2. **`ResultUploader`** — Handles the upload-to-run mechanics:
- Parses report inputs using the appropriate parser (JUnit XML file, Playwright JSON file, or Allure results directory)
- Detects project code from test case names (or from `--run-url`)
- Creates a new test run (or reuses an existing one if title conflicts)
- Delegates actual result uploading to `ResultUploader`

1. **`ResultUploader`** — Handles the upload-to-run mechanics:
- Fetches test cases from the run, maps parsed results to them via marker matching
- Validates unmatched/missing test cases (respects `--force`, `--ignore-unmatched`)
- Uploads file attachments concurrently (max 10 parallel), then creates results in batches (max 50 per request)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Numbered list regression: The second item in this list changed from 2. to 1. (and the sub-items lost their nested indentation). This appears to be an accidental formatting regression.

Suggested change
- Uploads file attachments concurrently (max 10 parallel), then creates results in batches (max 50 per request)
1. **`ResultUploader`** — Handles the upload-to-run mechanics:

Expand All @@ -51,6 +52,7 @@ The upload flow has two stages handled by two classes:

- `junitXmlParser.ts` — Parses JUnit XML via `xml2js` + Zod validation. Extracts attachments from `[[ATTACHMENT|path]]` markers in system-out/failure/error/skipped elements.
- `playwrightJsonParser.ts` — Parses Playwright JSON report. Supports two test case linking methods: (1) test annotations with `type: "test case"` and URL description, (2) marker in test name. Handles nested suites recursively.
- `allureParser.ts` — Parses Allure 2 JSON results directories (`*-result.json` files only). Supports test case linking via TMS links (`type: "tms"`) or marker in test name, maps Allure statuses to QA Sphere result statuses, and resolves attachments via `attachments[].source`.
- `types.ts` — Shared `TestCaseResult` and `Attachment` interfaces used by both parsers.

### API Layer (src/api/)
Expand All @@ -72,9 +74,10 @@ Composable fetch wrappers using higher-order functions:

Tests use **Vitest** with **MSW** (Mock Service Worker) for API mocking. Test files are in `src/tests/`:

- `result-upload.spec.ts` — Integration tests for the full upload flow (both JUnit and Playwright), with MSW intercepting all API calls
- `result-upload.spec.ts` — Integration tests for the full upload flow (JUnit, Playwright, and Allure), with MSW intercepting all API calls
- `junit-xml-parsing.spec.ts` — Unit tests for JUnit XML parser
- `playwright-json-parsing.spec.ts` — Unit tests for Playwright JSON parser
- `allure-parsing.spec.ts` — Unit tests for Allure parser
- `template-string-processing.spec.ts` — Unit tests for run name template processing

Test fixtures live in `src/tests/fixtures/` (XML files, JSON files, and mock test case data).
Expand Down
37 changes: 32 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

The QAS CLI is a command-line tool for submitting your test automation results to [QA Sphere](https://qasphere.com/). It provides the most efficient way to collect and report test results from your test automation workflow, CI/CD pipeline, and build servers.

The tool can upload test case results from JUnit XML and Playwright JSON files to QA Sphere test runs by matching test case names (mentions of special markers) to QA Sphere's test cases.
The tool can upload test case results from JUnit XML files, Playwright JSON files, and Allure result directories to QA Sphere test runs by matching test case references to QA Sphere test cases.

## Installation

Expand Down Expand Up @@ -39,7 +39,7 @@ Verify installation: `qasphere --version`
The CLI requires the following variables to be defined:

- `QAS_TOKEN` - QA Sphere API token (see [docs](https://docs.qasphere.com/api/authentication) if you need help generating one)
- `QAS_URL` - Base URL of your QA Sphere instance (e.g., https://qas.eu2.qasphere.com)
- `QAS_URL` - Base URL of your QA Sphere instance (e.g., `https://qas.eu2.qasphere.com`)

These variables could be defined:

Expand All @@ -59,9 +59,9 @@ QAS_URL=https://qas.eu1.qasphere.com
# QAS_URL=https://qas.eu1.qasphere.com
```

## Commands: `junit-upload`, `playwright-json-upload`
## Commands: `junit-upload`, `playwright-json-upload`, `allure-upload`

The `junit-upload` and `playwright-json-upload` commands upload test results from JUnit XML and Playwright JSON reports to QA Sphere respectively.
The `junit-upload`, `playwright-json-upload`, and `allure-upload` commands upload test results to QA Sphere.

There are two modes for uploading results using the commands:

Expand All @@ -70,6 +70,7 @@ There are two modes for uploading results using the commands:

### Options

- `<files..>` / `<directories..>` - Input paths. Use report files for `junit-upload` and `playwright-json-upload`, and Allure results directories for `allure-upload`
- `-r`/`--run-url` - Upload results to an existing test run
- `--project-code`, `--run-name`, `--create-tcases` - Create a new test run and upload results to it
- `--project-code` - Project code for creating new test run. It can also be auto detected from test case markers in the results, but this is not fully reliable, so it is recommended to specify the project code explicitly
Expand Down Expand Up @@ -170,14 +171,28 @@ Ensure the required environment variables are defined before running these comma
This will exclude stdout from passed tests while still including it for failed, blocked, or skipped tests.

10. Skip both stdout and stderr for passed tests:

```bash
qasphere junit-upload --skip-report-stdout on-success --skip-report-stderr on-success ./test-results.xml
```

This is useful when you have verbose logging in tests but only want to see output for failures.

11. Upload Allure results from a directory:

```bash
qasphere allure-upload -r https://qas.eu1.qasphere.com/project/P1/run/23 ./allure-results
```

12. Continue Allure upload when some `*-result.json` files are malformed (skip invalid files):

```bash
qasphere allure-upload --force -r https://qas.eu1.qasphere.com/project/P1/run/23 ./allure-results
```

## Test Report Requirements

The QAS CLI maps test results from your reports (JUnit XML or Playwright JSON) to corresponding test cases in QA Sphere using test case markers. If a test result lacks a valid marker, the CLI will display an error unless you use `--create-tcases` to automatically create test cases, or `--ignore-unmatched`/`--force` to skip unmatched results.
The QAS CLI maps test results from your reports (JUnit XML, Playwright JSON, or Allure) to corresponding test cases in QA Sphere. If a test result lacks a valid marker/reference, the CLI will display an error unless you use `--create-tcases` to automatically create test cases, or `--ignore-unmatched`/`--force` to skip unmatched results.

### JUnit XML

Expand Down Expand Up @@ -218,6 +233,18 @@ Playwright JSON reports support two methods for referencing test cases (checked

2. **Test Case Marker in Name** - Include the `PROJECT-SEQUENCE` marker in the test name (same format as JUnit XML)

### Allure

Allure results use one `*-result.json` file per test in a results directory. `allure-upload` matches test cases using:

1. **TMS links (Recommended)** - `links[]` entries with:
- `type`: `"tms"`
- `url`: QA Sphere test case URL, e.g. `https://qas.eu1.qasphere.com/project/PRJ/tcase/123`
2. **TMS link name fallback** - If `url` is not a QA Sphere URL, a marker in `links[].name` is used (for example `PRJ-123`)
3. **Test case marker in name** - Marker in `name` field (same `PROJECT-SEQUENCE` format as JUnit XML)

Only Allure 2 JSON (`*-result.json`) is supported. Legacy Allure 1 XML files are ignored.

## Development (for those who want to contribute to the tool)

1. Install and build: `npm install && npm run build && npm link`
Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions src/commands/main.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ Required variables: ${qasEnvs.join(', ')}
)
.command(new ResultUploadCommandModule('junit-upload'))
.command(new ResultUploadCommandModule('playwright-json-upload'))
.command(new ResultUploadCommandModule('allure-upload'))
.demandCommand(1, '')
.help('h')
.alias('h', 'help')
Expand Down
43 changes: 30 additions & 13 deletions src/commands/resultUpload.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,19 @@ import {
const commandTypeDisplayStrings: Record<UploadCommandType, string> = {
'junit-upload': 'JUnit XML',
'playwright-json-upload': 'Playwright JSON',
'allure-upload': 'Allure',
}

const commandTypeFileExtensions: Record<UploadCommandType, string> = {
'junit-upload': 'xml',
'playwright-json-upload': 'json',
const commandTypeInputKinds: Record<UploadCommandType, string> = {
'junit-upload': 'files',
'playwright-json-upload': 'files',
'allure-upload': 'directories',
}

const commandTypeExampleInputs: Record<UploadCommandType, string> = {
'junit-upload': './test-results.xml',
'playwright-json-upload': './test-results.json',
'allure-upload': './allure-results',
}

export class ResultUploadCommandModule implements CommandModule<unknown, ResultUploadCommandArgs> {
Expand All @@ -25,10 +33,19 @@ export class ResultUploadCommandModule implements CommandModule<unknown, ResultU
}

get describe() {
return `Upload ${commandTypeDisplayStrings[this.type]} files to a new or existing test run`
return `Upload ${commandTypeDisplayStrings[this.type]} ${commandTypeInputKinds[this.type]} to a new or existing test run`
}

builder = (argv: Argv) => {
argv.positional('files', {
describe:
this.type === 'allure-upload'
? 'One or more Allure results directories'
: 'One or more test report files',
type: 'string',
array: true,
})

argv.options({
'run-url': {
alias: 'r',
Expand Down Expand Up @@ -83,38 +100,38 @@ export class ResultUploadCommandModule implements CommandModule<unknown, ResultU
})

argv.example(
`$0 ${this.type} -r https://qas.eu1.qasphere.com/project/P1/run/23 ./test-results.${
commandTypeFileExtensions[this.type]
`$0 ${this.type} -r https://qas.eu1.qasphere.com/project/P1/run/23 ${
commandTypeExampleInputs[this.type]
}`,
'Upload results to existing run ID 23 of project P1'
)

argv.example(
`$0 ${this.type} ./test-results.${commandTypeFileExtensions[this.type]}`,
`$0 ${this.type} ${commandTypeExampleInputs[this.type]}`,
'Create a new test run with default name template and upload results. Project code is detected from test case markers in the results'
)

argv.example(
`$0 ${this.type} --project-code P1 --run-name "v1.4.4-rc5" ./test-results.${
commandTypeFileExtensions[this.type]
`$0 ${this.type} --project-code P1 --run-name "v1.4.4-rc5" ${
commandTypeExampleInputs[this.type]
}`,
'Create a new test run with name template without any placeholders and upload results'
)

argv.example(
`$0 ${
this.type
} --project-code P1 --run-name "CI Build {env:BUILD_NUMBER} - {YYYY}-{MM}-{DD}" ./test-results.${
commandTypeFileExtensions[this.type]
} --project-code P1 --run-name "CI Build {env:BUILD_NUMBER} - {YYYY}-{MM}-{DD}" ${
commandTypeExampleInputs[this.type]
}`,
'Create a new test run with name template using environment variable and date placeholders and upload results'
)

argv.example(
`$0 ${
this.type
} --project-code P1 --run-name "Nightly Tests {YYYY}/{MM}/{DD} {HH}:{mm}" --create-tcases ./test-results.${
commandTypeFileExtensions[this.type]
} --project-code P1 --run-name "Nightly Tests {YYYY}/{MM}/{DD} {HH}:{mm}" --create-tcases ${
commandTypeExampleInputs[this.type]
}`,
'Create a new test run with name template using date and time placeholders and create test cases for results without valid markers and upload results'
)
Expand Down
Loading