Testing Tools and Techniques¶
CDS takes a principled approach to tooling: we maintain preferred defaults across all engagements, but these are starting points rather than mandates. The right tool for the engagement always wins over the most familiar one.
This page documents the tools we use across each area of testing, with the rationale behind each choice.
Unit testing¶
Unit tests are a developer responsibility — the test team's role is awareness and alignment to ensure unit testing is in place as part of the Definition of Done. Framework selection is based on the language in use on the engagement. Consistency within an engagement matters more than which framework is chosen — mixing frameworks on a single project adds unnecessary complexity.
| Language | Primary tool | Alternative |
|---|---|---|
| .NET | xUnit (new projects) or NUnit (existing) | MSTest |
| JavaScript / TypeScript | Jest | Vitest (for Vite-based projects) |
| Java | JUnit 5 | TestNG |
| Python | pytest | unittest |
API and contract testing¶
API testing¶
Postman and Newman are the CDS default for API testing. Postman is used for collection authoring and management; Newman runs collections in CI/CD pipelines. Collections are versioned in source control alongside application code.
Contract testing¶
Contract testing validates that the interface between services remains compatible as each evolves independently. Pact is the CDS standard for consumer-driven contract testing, supporting JavaScript/TypeScript, .NET, and Python.
Where a full contract testing framework is not required, Postman supports lightweight contract validation via OpenAPI/Swagger spec validation — asserting that API responses conform to a defined schema.
On Java and Spring engagements, Spring Cloud Contract is an approved alternative to Pact.
End-to-end and browser automation¶
Playwright is the CDS standard for browser-based end-to-end testing. It is cross-browser, actively maintained, and designed for reliability — with auto-waiting, network interception, and strong TypeScript support.
End-to-end tests are expensive to maintain. We focus them on critical user journeys rather than exhaustive permutation coverage. The instinct to reach for a UI test should always prompt the question: can the underlying logic be covered at unit or API level instead, with the E2E test reserved for the journey validation only?
Cypress is an approved alternative to Playwright where it is already established on an engagement, though its cross-browser support is more limited. Selenium is not an appropriate default and requires a business case — it is only considered where a specific legacy or client constraint prevents use of Playwright.
For manual cross-browser and device testing, we use BrowserStack Live — providing access to real browsers and devices without local installation.
Performance testing¶
k6 is the CDS default for performance and load testing. It uses JavaScript-based scripting, integrates cleanly with Azure DevOps pipelines, and produces clear, actionable output.
Performance testing is established early in each engagement — not reserved for pre-go-live. Initial baselines are set against the staging environment once it is stable, and load tests run post-merge throughout delivery so that degradation is caught incrementally rather than discovered in production.
| Tool | When to use |
|---|---|
| k6 | Default for all new engagements |
| JMeter | Where the team has existing JMeter expertise or a GUI-based authoring approach is required |
| Locust | On Python-heavy engagements or where existing Locust expertise exists |
Accessibility testing¶
Accessibility testing combines automated pipeline checks with manual auditing. Automated tools catch a defined subset of WCAG violations — manual testing is required to cover what automation cannot: keyboard navigation, screen reader behaviour, and contextual judgement.
WCAG 2.2 Level AA is the default minimum standard on every CDS engagement with a user interface.
Automated¶
axe-core via Playwright is the CDS default. Accessibility checks run as part of the existing end-to-end suite — no additional pipeline stage required. Critical and serious violations fail the build.
Where Playwright is not in use, Lighthouse CLI is an approved alternative. It integrates directly into CI/CD pipelines as a standalone tool, produces structured reports, and covers WCAG 2.1 and 2.2 A and AA criteria.
Manual tooling¶
| Purpose | Tool |
|---|---|
| Screen reader (Windows) | NVDA |
| Screen reader (Windows — alternative) | Narrator (built-in, no installation required) |
| Screen reader (macOS / iOS) | VoiceOver |
| Screen reader (Android) | TalkBack |
| Colour contrast | WebAIM Contrast Checker |
| Browser inspection | axe DevTools extension, Chrome DevTools Accessibility Panel |
| Heading structure | HeadingsMap browser extension |
Screen reader testing covers at minimum NVDA on Windows and VoiceOver on macOS. Mobile screen reader testing is conducted where the engagement has a mobile audience.
Security testing¶
Security testing covers three layers: static analysis, dynamic analysis, and dependency scanning. Tooling is confirmed per engagement in the engagement testing strategy.
| Type | Tool | Notes |
|---|---|---|
| SAST (static analysis) | SonarCloud | Runs on every PR and build |
| SAST (alternative) | Semgrep | Open source, free tier available |
| SAST (alternative) | SonarQube | Self-hosted alternative to SonarCloud — appropriate where data residency or client constraints prevent use of a cloud-hosted tool |
| DAST (dynamic analysis) | OWASP ZAP | Open source, runs nightly against staging |
| DAST (alternative) | Burp Suite | Industry standard for DAST and manual penetration testing — requires a business case and cost approval due to licence cost |
| Dependency scanning | Dependabot (GitHub) or Snyk (ADO) | Automated, continuous |
The OWASP Top 10 is the baseline for vulnerability classification across all engagements.
Visual regression testing¶
Visual regression testing detects unintended visual changes between builds. Playwright's built-in visual comparison capability is the CDS default — baseline screenshots are stored in version control and updated intentionally via a reviewed pull request when UI changes are approved.
Visual regression carries a real maintenance cost: pixel-level comparisons are sensitive to minor rendering differences and can generate noise if not carefully managed. It is applied selectively to the highest-risk UI components rather than used as blanket coverage of every page.
AI-assisted testing¶
AI tooling is used to accelerate testing workflows — scaffolding test frameworks, generating test cases, writing and reviewing automation code, and documentation support. All AI-generated output is reviewed by a practitioner before use.
- Claude Code — used for automation code authoring, test case generation, and test documentation
- Microsoft Copilot — useful for documentation support and test planning within the Microsoft 365 ecosystem
See Agentic Engineering for more on how CDS uses AI tools across delivery.