mirror of
https://github.com/n8n-io/n8n.git
synced 2026-10-17 05:29:44 +02:00
chore: DEVP-1038 · c2: Add --test-files forwarding and CLI Stryker scope config (#37679)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
7ac335d94b
commit
e4e78da0d6
|
|
@ -57,6 +57,7 @@ That divergence is exactly why this project exists.
|
|||
| `mutate.mjs` | The whole engine. Runs Stryker over a package and emits an actionable summary. Exposed as `pnpm mutate`. |
|
||||
| `mutate.test.mjs` | Unit tests for its pure helpers (`node --test 'scripts/mutation-health/*.test.mjs'`). |
|
||||
| `stryker.default.mjs` | Shared Stryker config for any vitest package. A package that needs special handling ships its own `stryker.config.mjs`, which `mutate.mjs` prefers. |
|
||||
| `stryker.cli.mjs` | The default plus `vitest.related: false`, used for `packages/cli` targets. See [Scoping the tests](#scoping-the-tests-with---test-files). |
|
||||
|
||||
Outputs land in `<package>/reports/mutation/` (gitignored):
|
||||
|
||||
|
|
@ -81,6 +82,10 @@ pnpm mutate packages/@n8n/crdt/src/utils.ts:40-75
|
|||
|
||||
# Package-relative target.
|
||||
pnpm mutate src/cron.ts --package-dir packages/workflow
|
||||
|
||||
# One file, scoped to the tests that must kill its mutants.
|
||||
pnpm mutate packages/cli/src/credentials/external-secrets.utils.ts:32-68 \
|
||||
--test-files packages/cli/src/credentials/__tests__/external-secrets.utils.test.ts
|
||||
```
|
||||
|
||||
Exit codes: `0` gate passed · `1` gate failed (summary.json still written — this is the
|
||||
|
|
@ -103,6 +108,44 @@ cost tracks the related suite rather than package size. Measured end-to-end, who
|
|||
`@n8n/decorators` 1s · `@n8n/scheduler` 3s · `packages/workflow` 13s · `nodes-base` 26s ·
|
||||
`packages/cli` 88s. Line-scoping cuts these further.
|
||||
|
||||
### Scoping the tests with `--test-files`
|
||||
|
||||
By default Stryker's vitest runner uses [related mode](https://vitest.dev/guide/cli.html#vitest-related): it
|
||||
walks the import graph of the mutated file and runs every test file that reaches it. That is the
|
||||
right default for most packages, and it is what keeps cost tracking the related suite rather than
|
||||
package size.
|
||||
|
||||
`--test-files` replaces that discovery with an explicit list. The value goes to Stryker's
|
||||
[`testFiles`](https://stryker-mutator.io/docs/stryker-js/configuration/#testfiles-string) config
|
||||
field, so only those files run — which also answers a sharper question: can this module's *own*
|
||||
unit tests kill its mutants, without help from the rest of the suite?
|
||||
|
||||
The flag repeats and it also takes a comma-separated list. Paths may be repo-relative (what you
|
||||
type) or package-relative (what Stryker matches); `mutate.mjs` converts them. Globs work.
|
||||
|
||||
```bash
|
||||
pnpm mutate packages/@n8n/crdt/src/utils.ts --test-files packages/@n8n/crdt/src/__tests__/utils.test.ts
|
||||
pnpm mutate src/utils.ts --package-dir packages/@n8n/crdt --test-files 'src/__tests__/*.test.ts'
|
||||
```
|
||||
|
||||
`--test-files` needs a single target, so it does not combine with `--diff`.
|
||||
|
||||
#### `packages/cli` requires it
|
||||
|
||||
`packages/cli` runs vitest with `pool: 'forks'` and a global setup, so each test file costs a
|
||||
process. Related discovery finds hundreds of them for a typical source file, and the dry run alone
|
||||
outlives any usable timeout. A `packages/cli` target therefore **fails with exit `2`** until you
|
||||
name the test files:
|
||||
|
||||
```bash
|
||||
$ pnpm mutate packages/cli/src/credentials/external-secrets.utils.ts:32-68
|
||||
Mutating packages/cli needs --test-files.
|
||||
```
|
||||
|
||||
Those runs also get `stryker.cli.mjs` instead of the shared default — same settings, with
|
||||
`vitest.related` turned off, because the explicit list already decides the scope. With it, the
|
||||
example above runs its 36-test file and finishes in seconds.
|
||||
|
||||
### In-place mutation
|
||||
|
||||
Runs use Stryker's `--inPlace`. Its default sandbox copy breaks on any package whose vitest
|
||||
|
|
@ -141,6 +184,9 @@ Not scored:
|
|||
- `.vue` single-file components — every SFC package crashed Stryker's mutate step in the 2026-06 sweep, and the component layer is low-value to mutate.
|
||||
- Tests, declarations, stories, configs, migrations and build output.
|
||||
|
||||
`packages/cli` is scored, but only with an explicit
|
||||
[`--test-files`](#packagescli-requires-it) list.
|
||||
|
||||
## Gate semantics
|
||||
|
||||
A run passes only when **both**:
|
||||
|
|
|
|||
|
|
@ -4,6 +4,7 @@
|
|||
* script works for any package. Run it as `pnpm mutate` from the repo root.
|
||||
*
|
||||
* pnpm mutate <file>[:<start>-<end>] [--package-dir <dir>] [--config <path>]
|
||||
* [--test-files <path>[,<path>…]]
|
||||
* pnpm mutate --diff [--base <ref>] [--config <path>]
|
||||
*
|
||||
* Use `--diff` before you merge. It reads the changed line ranges from
|
||||
|
|
@ -12,10 +13,18 @@
|
|||
* per package. This makes the gate apply to the patch: it scores the lines you
|
||||
* changed, not the debt you inherited.
|
||||
*
|
||||
* Use `--test-files` to name the test files that must kill the mutants. The
|
||||
* value goes to Stryker's `testFiles` config field, thus Stryker runs those
|
||||
* files instead of the whole related-test graph. The flag repeats and it also
|
||||
* takes a comma-separated list. `packages/cli` requires it — see
|
||||
* `cliScopeError` below.
|
||||
*
|
||||
* Stryker config resolution (first match wins):
|
||||
* 1. --config <path> explicit override
|
||||
* 2. <package-dir>/stryker.config.mjs package-local (e.g. workflow's vm carve-out)
|
||||
* 3. scripts/mutation-health/stryker.default.mjs shared default (points at the
|
||||
* 3. scripts/mutation-health/stryker.cli.mjs packages/cli only (related-test
|
||||
* discovery off — see stryker.cli.mjs)
|
||||
* 4. scripts/mutation-health/stryker.default.mjs shared default (points at the
|
||||
* package's own vitest.config.* — no
|
||||
* bespoke vitest config required)
|
||||
*
|
||||
|
|
@ -337,6 +346,87 @@ export function splitRange(target) {
|
|||
return m ? { file: m[1], range: `${m[2]}-${m[3]}` } : { file: target, range: null };
|
||||
}
|
||||
|
||||
// --- explicit test-file scoping (pure helpers exported for the unit tests) ---
|
||||
|
||||
// `packages/cli` forks a process per test file and runs a global setup, thus
|
||||
// Stryker's related-test discovery pulls in hundreds of files and no run
|
||||
// finishes inside the timeout. An explicit `--test-files` list is required
|
||||
// there, and the shared default config is swapped for the cli one.
|
||||
export const CLI_PACKAGE_DIR = 'packages/cli';
|
||||
|
||||
// Path comparisons in this file are posix-shaped. `path.relative` gives
|
||||
// backslashes on Windows, so normalise before matching a package dir.
|
||||
function toPosix(p) {
|
||||
return p.split(path.sep).join('/');
|
||||
}
|
||||
|
||||
/**
|
||||
* Flatten the raw `--test-files` values into one list. The flag repeats and
|
||||
* each value also takes a comma-separated list, thus `--test-files a,b` and
|
||||
* `--test-files a --test-files b` mean the same thing. Blanks are dropped and
|
||||
* duplicates collapse, so a repeated path never runs its file twice.
|
||||
*/
|
||||
export function parseTestFiles(values) {
|
||||
const out = [];
|
||||
for (const value of values) {
|
||||
for (const part of String(value).split(',')) {
|
||||
const file = part.trim();
|
||||
if (file && !out.includes(file)) out.push(file);
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Stryker matches `testFiles` patterns against the files under its run cwd,
|
||||
* which is the package dir. A repo-relative path (what the user types, and
|
||||
* what `--diff` prints) therefore has to lose its package prefix. A path that
|
||||
* is already package-relative, and a glob, both pass through untouched.
|
||||
*/
|
||||
export function toPackageRelative(file, packageDir) {
|
||||
const normalised = toPosix(file).replace(/^\.\//, '');
|
||||
const prefix = `${toPosix(packageDir)}/`;
|
||||
return normalised.startsWith(prefix) ? normalised.slice(prefix.length) : normalised;
|
||||
}
|
||||
|
||||
/**
|
||||
* The Stryker config fields this run overrides on top of the resolved config
|
||||
* file. Stryker merges its CLI options into the config object it runs with,
|
||||
* thus every field here lands in that object.
|
||||
*/
|
||||
export function buildStrykerConfig({ targets, testFiles = [] }) {
|
||||
const config = { mutate: [...targets] };
|
||||
if (testFiles.length > 0) config.testFiles = [...testFiles];
|
||||
return config;
|
||||
}
|
||||
|
||||
// Turn the config overrides into Stryker CLI flags. Stryker keeps only the last
|
||||
// occurrence of a repeated flag, thus every list field goes over comma-joined.
|
||||
export function strykerCliArgs(config) {
|
||||
return Object.entries(config).flatMap(([field, value]) => [
|
||||
`--${field}`,
|
||||
Array.isArray(value) ? value.join(',') : String(value),
|
||||
]);
|
||||
}
|
||||
|
||||
// Why a cli target may not run, or null when it may. Named so the message says
|
||||
// what to do, not only what went wrong.
|
||||
export function cliScopeError(packageDir, testFiles) {
|
||||
if (toPosix(packageDir) !== CLI_PACKAGE_DIR) return null;
|
||||
if (testFiles.length > 0) return null;
|
||||
return (
|
||||
`Mutating ${CLI_PACKAGE_DIR} needs --test-files.\n` +
|
||||
'Without it Stryker discovers every related test file in the package, forks a ' +
|
||||
'process for each one and never finishes. Name the test files that cover the target:\n' +
|
||||
' pnpm mutate packages/cli/src/foo.ts:10-40 --test-files packages/cli/src/__tests__/foo.test.ts'
|
||||
);
|
||||
}
|
||||
|
||||
// Which shared config a package gets when it ships none of its own.
|
||||
export function defaultConfigNameFor(packageDir) {
|
||||
return toPosix(packageDir) === CLI_PACKAGE_DIR ? 'stryker.cli.mjs' : 'stryker.default.mjs';
|
||||
}
|
||||
|
||||
function git(args) {
|
||||
const res = spawnSync('git', args, {
|
||||
cwd: repoRoot,
|
||||
|
|
@ -430,7 +520,8 @@ function planFromDiff(base) {
|
|||
function resolveConfig(pkgRoot, configArg) {
|
||||
if (configArg) return path.resolve(repoRoot, configArg);
|
||||
const local = path.join(pkgRoot, 'stryker.config.mjs');
|
||||
return existsSync(local) ? local : path.join(__dirname, 'stryker.default.mjs');
|
||||
if (existsSync(local)) return local;
|
||||
return path.join(__dirname, defaultConfigNameFor(path.relative(repoRoot, pkgRoot)));
|
||||
}
|
||||
|
||||
// Try the package's own copy first, then the root devDep. A package that pins
|
||||
|
|
@ -674,10 +765,14 @@ function restoreWorkingTree({ bytes, dirtyBefore }) {
|
|||
return [...restored];
|
||||
}
|
||||
|
||||
async function runJob({ pkgRoot, packageDir, targets }, { configArg }) {
|
||||
async function runJob({ pkgRoot, packageDir, targets }, { configArg, testFiles = [] }) {
|
||||
const configPath = resolveConfig(pkgRoot, configArg);
|
||||
const strykerBin = resolveStrykerBin(pkgRoot, packageDir);
|
||||
const mutateArg = formatMutateArg(targets);
|
||||
const strykerConfig = buildStrykerConfig({
|
||||
targets,
|
||||
testFiles: testFiles.map((f) => toPackageRelative(f, packageDir)),
|
||||
});
|
||||
|
||||
const reportDir = path.join(pkgRoot, 'reports/mutation');
|
||||
const rawJsonPath = path.join(reportDir, 'raw.json');
|
||||
|
|
@ -694,6 +789,9 @@ async function runJob({ pkgRoot, packageDir, targets }, { configArg }) {
|
|||
`\nRunning Stryker on ${packageDir} — ${targets.length} target(s) ` +
|
||||
`(config: ${path.relative(repoRoot, configPath)}, threshold: ${THRESHOLD}%)\n`,
|
||||
);
|
||||
if (strykerConfig.testFiles) {
|
||||
process.stderr.write(` testFiles: ${strykerConfig.testFiles.join(', ')}\n`);
|
||||
}
|
||||
|
||||
const snapshot = snapshotWorkingTree([
|
||||
...new Set(targets.map((t) => path.join(pkgRoot, splitRange(t).file))),
|
||||
|
|
@ -736,7 +834,7 @@ async function runJob({ pkgRoot, packageDir, targets }, { configArg }) {
|
|||
strykerExitCode = await new Promise((resolve) => {
|
||||
child = spawn(
|
||||
process.execPath,
|
||||
[strykerBin, 'run', configPath, '--inPlace', '--mutate', mutateArg],
|
||||
[strykerBin, 'run', configPath, '--inPlace', ...strykerCliArgs(strykerConfig)],
|
||||
{ cwd: pkgRoot, stdio: ['inherit', 'pipe', 'pipe'] },
|
||||
);
|
||||
child.stdout.on('data', (chunk) => {
|
||||
|
|
@ -840,14 +938,28 @@ function reportJob({ packageDir, summaryJsonPath, summary, noTests, failed }) {
|
|||
|
||||
const USAGE = `Usage:
|
||||
node scripts/mutation-health/mutate.mjs <file>[:<start>-<end>] [--package-dir <dir>] [--config <path>]
|
||||
[--test-files <path>[,<path>...]]
|
||||
node scripts/mutation-health/mutate.mjs --diff [--base <ref>] [--config <path>]
|
||||
|
||||
Options:
|
||||
--package-dir <dir> Package the target belongs to. Inferred when omitted.
|
||||
--config <path> Stryker config file. Overrides the resolution order.
|
||||
--base <ref> Branch point --diff measures against. Default origin/master.
|
||||
--diff Mutate every line this branch changed, one run per package.
|
||||
--test-files <paths> Test files Stryker runs, instead of the whole related-test
|
||||
graph. Goes to Stryker's testFiles config field. Comma-
|
||||
separated, and the flag repeats. Paths may be repo-relative
|
||||
or package-relative. Required for ${CLI_PACKAGE_DIR} targets.
|
||||
|
||||
# one file, whole
|
||||
node scripts/mutation-health/mutate.mjs packages/@n8n/crdt/src/utils.ts
|
||||
# one file, only lines 40-75
|
||||
node scripts/mutation-health/mutate.mjs packages/@n8n/crdt/src/utils.ts:40-75
|
||||
# package-relative target
|
||||
node scripts/mutation-health/mutate.mjs src/cron.ts --package-dir packages/workflow
|
||||
# a cli target, scoped to the tests that must kill its mutants
|
||||
node scripts/mutation-health/mutate.mjs packages/cli/src/credentials/external-secrets.utils.ts:32-68 \\
|
||||
--test-files packages/cli/src/credentials/__tests__/external-secrets.utils.test.ts
|
||||
# every line this branch changed, batched one Stryker run per package
|
||||
node scripts/mutation-health/mutate.mjs --diff --base origin/master`;
|
||||
|
||||
|
|
@ -889,24 +1001,51 @@ function planFromTarget(targetArg, packageDirArg) {
|
|||
};
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const argv = process.argv.slice(2);
|
||||
let packageDirArg;
|
||||
let configArg;
|
||||
let targetArg;
|
||||
let baseArg = 'origin/master';
|
||||
let diffMode = false;
|
||||
/**
|
||||
* Read the command line. Pure, so the unit tests can assert what each flag
|
||||
* parses to. Validation stays in `main`, which owns the exit codes.
|
||||
*/
|
||||
export function parseArgs(argv) {
|
||||
const parsed = {
|
||||
packageDirArg: undefined,
|
||||
configArg: undefined,
|
||||
targetArg: undefined,
|
||||
baseArg: 'origin/master',
|
||||
diffMode: false,
|
||||
helpMode: false,
|
||||
testFiles: [],
|
||||
};
|
||||
const rawTestFiles = [];
|
||||
for (let i = 0; i < argv.length; i++) {
|
||||
const a = argv[i];
|
||||
if (a === '--package-dir') packageDirArg = argv[++i];
|
||||
else if (a === '--config') configArg = argv[++i];
|
||||
else if (a === '--base') baseArg = argv[++i];
|
||||
else if (a === '--diff') diffMode = true;
|
||||
else if (!a.startsWith('--') && targetArg === undefined) targetArg = a;
|
||||
if (a === '--package-dir') parsed.packageDirArg = argv[++i];
|
||||
else if (a === '--config') parsed.configArg = argv[++i];
|
||||
else if (a === '--base') parsed.baseArg = argv[++i];
|
||||
else if (a === '--test-files') rawTestFiles.push(argv[++i] ?? '');
|
||||
else if (a === '--diff') parsed.diffMode = true;
|
||||
else if (a === '--help' || a === '-h') parsed.helpMode = true;
|
||||
else if (!a.startsWith('--') && parsed.targetArg === undefined) parsed.targetArg = a;
|
||||
}
|
||||
parsed.testFiles = parseTestFiles(rawTestFiles);
|
||||
return parsed;
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const { packageDirArg, configArg, targetArg, baseArg, diffMode, helpMode, testFiles } = parseArgs(
|
||||
process.argv.slice(2),
|
||||
);
|
||||
|
||||
if (helpMode) {
|
||||
process.stdout.write(`${USAGE}\n`);
|
||||
process.exit(0);
|
||||
}
|
||||
if (diffMode && targetArg) die(2, `--diff takes no positional target.\n${USAGE}`);
|
||||
if (!diffMode && !targetArg) die(2, `Missing mutate target.\n${USAGE}`);
|
||||
// --diff plans a run per package, so one test-file list cannot say which
|
||||
// package it belongs to. Refuse rather than apply it to all of them.
|
||||
if (diffMode && testFiles.length > 0) {
|
||||
die(2, `--test-files needs a single target; it does not work with --diff.\n${USAGE}`);
|
||||
}
|
||||
|
||||
let jobs;
|
||||
let skipped = [];
|
||||
|
|
@ -923,11 +1062,13 @@ async function main() {
|
|||
);
|
||||
} else {
|
||||
jobs = [planFromTarget(targetArg, packageDirArg)];
|
||||
const scopeError = cliScopeError(jobs[0].packageDir, testFiles);
|
||||
if (scopeError) die(2, scopeError);
|
||||
}
|
||||
|
||||
const results = [];
|
||||
for (const job of jobs) {
|
||||
results.push(await runJob(job, { configArg }));
|
||||
results.push(await runJob(job, { configArg, testFiles }));
|
||||
}
|
||||
|
||||
process.stderr.write('\n=== Mutation summary ===\n');
|
||||
|
|
|
|||
|
|
@ -8,22 +8,29 @@ import path from 'node:path';
|
|||
|
||||
import {
|
||||
buildNoTestsSummary,
|
||||
buildStrykerConfig,
|
||||
buildSummary,
|
||||
classifyRun,
|
||||
cliScopeError,
|
||||
coverageFromCounts,
|
||||
createCleanup,
|
||||
defaultConfigNameFor,
|
||||
filesToCheckout,
|
||||
findStrykerSetupFiles,
|
||||
formatMutateArg,
|
||||
isMutableSource,
|
||||
mergeRanges,
|
||||
parseArgs,
|
||||
parseHunkRanges,
|
||||
parseTestFiles,
|
||||
registerCleanupHandlers,
|
||||
removeStrykerSetupFiles,
|
||||
restoreFiles,
|
||||
scoreFromCounts,
|
||||
snapshotFiles,
|
||||
splitRange,
|
||||
strykerCliArgs,
|
||||
toPackageRelative,
|
||||
} from './mutate.mjs';
|
||||
|
||||
// A minimal Stryker Mutation Testing Elements report for one source file. Mix
|
||||
|
|
@ -361,6 +368,233 @@ describe('splitRange', () => {
|
|||
});
|
||||
});
|
||||
|
||||
describe('parseTestFiles', () => {
|
||||
it('splits a comma-separated value', () => {
|
||||
assert.deepEqual(parseTestFiles(['a.test.ts,b.test.ts']), ['a.test.ts', 'b.test.ts']);
|
||||
});
|
||||
|
||||
it('collects a repeated flag', () => {
|
||||
assert.deepEqual(parseTestFiles(['a.test.ts', 'b.test.ts']), ['a.test.ts', 'b.test.ts']);
|
||||
});
|
||||
|
||||
it('accepts both forms at once, and trims the spaces around a comma', () => {
|
||||
assert.deepEqual(parseTestFiles(['a.test.ts, b.test.ts', 'c.test.ts']), [
|
||||
'a.test.ts',
|
||||
'b.test.ts',
|
||||
'c.test.ts',
|
||||
]);
|
||||
});
|
||||
|
||||
// A duplicate would make Stryker run the same file twice for every mutant.
|
||||
it('drops blanks and duplicates', () => {
|
||||
assert.deepEqual(parseTestFiles(['a.test.ts,,a.test.ts', ' ', 'b.test.ts']), [
|
||||
'a.test.ts',
|
||||
'b.test.ts',
|
||||
]);
|
||||
});
|
||||
|
||||
it('returns nothing when the flag was not given', () => {
|
||||
assert.deepEqual(parseTestFiles([]), []);
|
||||
});
|
||||
});
|
||||
|
||||
describe('parseArgs', () => {
|
||||
it('reads --test-files as a comma-separated list', () => {
|
||||
const parsed = parseArgs([
|
||||
'packages/cli/src/foo.ts:10-40',
|
||||
'--test-files',
|
||||
'packages/cli/src/__tests__/foo.test.ts,packages/cli/src/__tests__/bar.test.ts',
|
||||
]);
|
||||
assert.equal(parsed.targetArg, 'packages/cli/src/foo.ts:10-40');
|
||||
assert.deepEqual(parsed.testFiles, [
|
||||
'packages/cli/src/__tests__/foo.test.ts',
|
||||
'packages/cli/src/__tests__/bar.test.ts',
|
||||
]);
|
||||
});
|
||||
|
||||
it('reads a repeated --test-files flag', () => {
|
||||
const parsed = parseArgs([
|
||||
'src/foo.ts',
|
||||
'--test-files',
|
||||
'src/__tests__/foo.test.ts',
|
||||
'--test-files',
|
||||
'src/__tests__/bar.test.ts',
|
||||
]);
|
||||
assert.deepEqual(parsed.testFiles, ['src/__tests__/foo.test.ts', 'src/__tests__/bar.test.ts']);
|
||||
});
|
||||
|
||||
it('leaves testFiles empty when the flag is absent', () => {
|
||||
assert.deepEqual(parseArgs(['src/foo.ts']).testFiles, []);
|
||||
});
|
||||
|
||||
it('keeps reading the other flags', () => {
|
||||
const parsed = parseArgs([
|
||||
'src/cron.ts',
|
||||
'--package-dir',
|
||||
'packages/workflow',
|
||||
'--config',
|
||||
'custom.mjs',
|
||||
'--base',
|
||||
'upstream/master',
|
||||
]);
|
||||
assert.equal(parsed.packageDirArg, 'packages/workflow');
|
||||
assert.equal(parsed.configArg, 'custom.mjs');
|
||||
assert.equal(parsed.baseArg, 'upstream/master');
|
||||
assert.equal(parsed.diffMode, false);
|
||||
});
|
||||
|
||||
it('defaults the base ref and reads --diff', () => {
|
||||
const parsed = parseArgs(['--diff']);
|
||||
assert.equal(parsed.diffMode, true);
|
||||
assert.equal(parsed.baseArg, 'origin/master');
|
||||
assert.equal(parsed.targetArg, undefined);
|
||||
});
|
||||
|
||||
it('reads --help', () => {
|
||||
assert.equal(parseArgs(['--help']).helpMode, true);
|
||||
assert.equal(parseArgs(['-h']).helpMode, true);
|
||||
assert.equal(parseArgs(['src/foo.ts']).helpMode, false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('toPackageRelative', () => {
|
||||
// Stryker matches testFiles against the files under its run cwd, which is
|
||||
// the package dir.
|
||||
it('strips the package prefix off a repo-relative path', () => {
|
||||
assert.equal(
|
||||
toPackageRelative('packages/cli/src/__tests__/foo.test.ts', 'packages/cli'),
|
||||
'src/__tests__/foo.test.ts',
|
||||
);
|
||||
});
|
||||
|
||||
it('leaves an already package-relative path alone', () => {
|
||||
assert.equal(
|
||||
toPackageRelative('src/__tests__/foo.test.ts', 'packages/cli'),
|
||||
'src/__tests__/foo.test.ts',
|
||||
);
|
||||
});
|
||||
|
||||
it('leaves a glob alone', () => {
|
||||
assert.equal(toPackageRelative('src/**/*.test.ts', 'packages/cli'), 'src/**/*.test.ts');
|
||||
});
|
||||
|
||||
it('drops a leading ./', () => {
|
||||
assert.equal(toPackageRelative('./src/foo.test.ts', 'packages/cli'), 'src/foo.test.ts');
|
||||
});
|
||||
|
||||
// `packages/cli-utils` must not lose its prefix to `packages/cli`.
|
||||
it('only strips a whole path segment', () => {
|
||||
assert.equal(
|
||||
toPackageRelative('packages/cli-utils/src/foo.test.ts', 'packages/cli'),
|
||||
'packages/cli-utils/src/foo.test.ts',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('buildStrykerConfig', () => {
|
||||
it('puts the supplied test files in the testFiles config field', () => {
|
||||
const config = buildStrykerConfig({
|
||||
targets: ['src/credentials/external-secrets.utils.ts:32-68'],
|
||||
testFiles: ['src/credentials/__tests__/external-secrets.utils.test.ts'],
|
||||
});
|
||||
assert.deepEqual(config.testFiles, [
|
||||
'src/credentials/__tests__/external-secrets.utils.test.ts',
|
||||
]);
|
||||
assert.deepEqual(config.mutate, ['src/credentials/external-secrets.utils.ts:32-68']);
|
||||
});
|
||||
|
||||
// An empty `testFiles` is not the same as an absent one: Stryker reads a
|
||||
// non-empty list as "run only these", so an empty one must not be sent.
|
||||
it('leaves testFiles out when no test file was named', () => {
|
||||
const config = buildStrykerConfig({ targets: ['src/cron.ts'] });
|
||||
assert.equal('testFiles' in config, false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('strykerCliArgs', () => {
|
||||
it('forwards testFiles as one comma-joined flag', () => {
|
||||
const args = strykerCliArgs(
|
||||
buildStrykerConfig({
|
||||
targets: ['src/a.ts:1-4', 'src/b.ts'],
|
||||
testFiles: ['src/__tests__/a.test.ts', 'src/__tests__/b.test.ts'],
|
||||
}),
|
||||
);
|
||||
assert.deepEqual(args, [
|
||||
'--mutate',
|
||||
'src/a.ts:1-4,src/b.ts',
|
||||
'--testFiles',
|
||||
'src/__tests__/a.test.ts,src/__tests__/b.test.ts',
|
||||
]);
|
||||
});
|
||||
|
||||
it('sends no --testFiles flag when no test file was named', () => {
|
||||
const args = strykerCliArgs(buildStrykerConfig({ targets: ['src/cron.ts'] }));
|
||||
assert.deepEqual(args, ['--mutate', 'src/cron.ts']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('cliScopeError', () => {
|
||||
it('refuses a packages/cli target with no --test-files', () => {
|
||||
const error = cliScopeError('packages/cli', []);
|
||||
assert.match(error, /--test-files/);
|
||||
});
|
||||
|
||||
it('allows a packages/cli target once test files are named', () => {
|
||||
assert.equal(cliScopeError('packages/cli', ['src/__tests__/foo.test.ts']), null);
|
||||
});
|
||||
|
||||
it('leaves every other package alone', () => {
|
||||
assert.equal(cliScopeError('packages/workflow', []), null);
|
||||
// A prefix match would sweep in an unrelated package.
|
||||
assert.equal(cliScopeError('packages/cli-utils', []), null);
|
||||
});
|
||||
});
|
||||
|
||||
describe('defaultConfigNameFor', () => {
|
||||
it('gives packages/cli its own config, so related-test discovery stays off', () => {
|
||||
assert.equal(defaultConfigNameFor('packages/cli'), 'stryker.cli.mjs');
|
||||
});
|
||||
|
||||
it('gives every other package the shared default', () => {
|
||||
assert.equal(defaultConfigNameFor('packages/workflow'), 'stryker.default.mjs');
|
||||
assert.equal(defaultConfigNameFor('packages/@n8n/decorators'), 'stryker.default.mjs');
|
||||
});
|
||||
});
|
||||
|
||||
describe('the cli scope guard end to end', () => {
|
||||
const wrapper = path.join(import.meta.dirname, 'mutate.mjs');
|
||||
const target = 'packages/cli/src/credentials/external-secrets.utils.ts:32-68';
|
||||
|
||||
function run(args) {
|
||||
return spawnSync(process.execPath, [wrapper, ...args], {
|
||||
cwd: path.resolve(import.meta.dirname, '../..'),
|
||||
encoding: 'utf8',
|
||||
});
|
||||
}
|
||||
|
||||
// The run must stop before Stryker starts, so a mistyped command costs
|
||||
// nothing instead of forking a process for each of the package's test files.
|
||||
it('exits non-zero and names the flag when a cli target has no --test-files', () => {
|
||||
const res = run([target]);
|
||||
assert.equal(res.status, 2);
|
||||
assert.match(res.stderr, /--test-files/);
|
||||
// Stryker never started.
|
||||
assert.doesNotMatch(res.stderr, /Running Stryker/);
|
||||
});
|
||||
|
||||
it('refuses --test-files together with --diff', () => {
|
||||
const res = run(['--diff', '--test-files', 'packages/cli/src/__tests__/foo.test.ts']);
|
||||
assert.equal(res.status, 2);
|
||||
assert.match(res.stderr, /--diff/);
|
||||
});
|
||||
|
||||
it('documents the flag in --help', () => {
|
||||
const res = run(['--help']);
|
||||
assert.equal(res.status, 0);
|
||||
assert.match(res.stdout, /--test-files/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('filesToCheckout', () => {
|
||||
it('picks the files the run made dirty', () => {
|
||||
assert.deepEqual(filesToCheckout(['a.ts'], ['a.ts', 'b.ts']), ['b.ts']);
|
||||
|
|
|
|||
22
scripts/mutation-health/stryker.cli.mjs
Normal file
22
scripts/mutation-health/stryker.cli.mjs
Normal file
|
|
@ -0,0 +1,22 @@
|
|||
/**
|
||||
* Stryker config for `packages/cli`. It is the shared default plus one change:
|
||||
* related-test discovery is off.
|
||||
*
|
||||
* `packages/cli` runs vitest with `pool: 'forks'` and a global setup, thus each
|
||||
* test file costs a process. Stryker's vitest runner defaults to
|
||||
* [related mode](https://vitest.dev/guide/cli.html#vitest-related), which walks
|
||||
* the import graph of the mutated file and finds hundreds of test files in this
|
||||
* package — the dry run alone then outlives any usable timeout.
|
||||
*
|
||||
* With `related: false` the run tests exactly the files `--test-files` names, so
|
||||
* `mutate.mjs` requires that flag for cli targets (see `cliScopeError`). Stryker
|
||||
* also uses the resolved `testFiles` as its global test filter, thus a mutant
|
||||
* with no per-test coverage stays inside the same set.
|
||||
*/
|
||||
import defaultConfig from './stryker.default.mjs';
|
||||
|
||||
/** @type {import('@stryker-mutator/api/core').PartialStrykerOptions} */
|
||||
export default {
|
||||
...defaultConfig,
|
||||
vitest: { related: false },
|
||||
};
|
||||
Loading…
Reference in New Issue
Block a user