mattermost-mobile-upstream/detox/utils/build-tsio-job-config.test.js
yasser khan 16dd7dbdba
ci(e2e): make E2E/Override waive checks instead of cancelling runs (#9975)
* ci(e2e): make E2E/Override waive checks instead of cancelling runs

E2E/Override previously cancelled in-flight E2E runs and posted success on
their commit statuses. Cancelling destroys the very signal the maintainer
might still want, and reporting success for a run that never finished
attributes a conclusion to work that never happened.

Override now means one thing only: the required e2e-test/* contexts go green
so the PR can merge. Runs keep going and keep reporting to TSIO.

- Add .github/actions/e2e-override-status to post success (or pending) on all
  five required e2e-test/* contexts for a SHA.
- Label manager posts success on `labeled` and pending on `unlabeled`, so
  removing the escape hatch re-blocks the PR instead of leaving a stale waiver
  on the head SHA. It comments the runs it left running rather than killing
  them, and drops to `actions: read` now that it never cancels.
- tsio-report-status re-reads the PR labels before posting a failure. Because
  runs are no longer cancelled, a result landing after the override was applied
  would otherwise re-red a check the maintainer deliberately bypassed. The
  lookup fails closed so an unreadable label list cannot turn a real failure
  green, and the step itself no longer fails under override.
- Guard e2e-cancel-on-label-removal: stripping E2E/Run* is exactly what the
  Honor E2E/Override step does, so without this the indirect unlabeled event
  would cancel the runs the redesign is meant to preserve.
- e2e-detox-pr posts the success itself when the policy skips E2E, since every
  downstream job is skipped and a skipped job leaves the required contexts
  absent, blocking the merge forever.
- cancel-e2e-runs drops status_state: a cancelled run has no conclusion to
  report, so it always resets to pending.
- Thread `issues: read` through the reusable workflow callers so finalize can
  read the labels; a reusable workflow cannot escalate past its caller.

Co-authored-by: Cursor <cursoragent@cursor.com>

* ci(e2e): derive required E2E contexts from one definition

The override and cancel actions each carried their own copy of the five
required e2e-test/* contexts, kept aligned with build-tsio-job-config.js by
comment alone. A job renamed in PR_MAIN_JOBS would silently leave the actions
waiving or resetting a context that no longer exists, so a PR either cannot
merge or merges while a real context still blocks it.

Export E2E_STATUS_CONTEXTS from detox/utils/build-tsio-job-config.js, derived
from PR_MAIN_JOBS, and require it from both composite actions. github-script
resolves relative requires against the workspace, and every call site already
checks out the repo before using the action.

Also set persist-credentials: false on the e2e-label-manager checkout, matching
the other checkout steps.

Co-authored-by: Cursor <cursoragent@cursor.com>

* ci(e2e): drop the cancel action for native concurrency, keep GitHub calls in Actions

Review feedback from @saturninoabril.

Superseding an in-flight E2E run is what GitHub's `concurrency` already does, so
`e2e-detox-pr.yml` now declares `concurrency: e2e-<ref>` with `cancel-in-progress`
instead of cancelling from a step inside the run that supersedes it. The older run
is cancelled the moment the new dispatch is created rather than once a runner picks
up the first job.

That removes the only two remaining callers of `.github/actions/cancel-e2e-runs`,
along with the commit statuses it forced to `pending` on cancel — a cancelled run
has no conclusion to report, so the state is left to GitHub. Forcing a commit status
is now reserved for real test results and the override waiver. The action is deleted.

`e2e-cancel-on-label-removal.yml` goes with it: with `E2E/Run` on its way out and
override no longer cancelling, removing the label to stop a run has no remaining
practical case, and dropping the workflow also removes the back-door cancel guard
it needed.

The `E2E/Override` lookup moves out of `tsio-report-status.js` into a new
`.github/actions/e2e-override-label` composite action, which the finalize steps
call and pass in via `--e2e-override`. Talking to GitHub stays in the Actions
layer and the reporter is left deciding on its inputs.

Co-authored-by: Cursor <cursoragent@cursor.com>

* ci(e2e): paginate PR label lookups for E2E/Override

Co-authored-by: Cursor <cursoragent@cursor.com>

* ci(e2e): re-post E2E/Override success on new commits

Override success is SHA-scoped: it is posted only on the head SHA at
label-apply time (e2e-label-manager 'labeled' path) and when Matterwick
dispatches e2e-detox-pr.yml (post-e2e-override-status). On a push after
E2E/Override is applied, refresh-e2e-run-label strips the run labels and
returns — Matterwick is not dispatched, so the new head SHA gets no
e2e-test/* statuses. With those checks Required, the new commit is
blocked until the label is re-toggled.

Carry the override onto the new commit: refresh-e2e-run-label now emits
an e2e_override output, and when it is set a gated checkout +
e2e-override-status step re-posts success on the new head SHA. Reuses
the existing e2e-override-status action (same 5 required e2e-test/*
contexts, same 'E2E bypassed' description), so it is idempotent with the
labeled path. Adds statuses: write / contents: read to the job.

Note: refresh-e2e-run-label is gated on base.ref == 'main', so stacked
PRs (base != main) are not covered; non-main-base already posts success
at dispatch time.

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-31 14:22:19 +02:00

149 lines
6.4 KiB
JavaScript

// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
// See LICENSE.txt for license information.
const assert = require('node:assert/strict');
const {describe, it} = require('node:test');
const {
E2E_STATUS_CONTEXTS,
buildTsioJobConfig,
buildTsioJobConfigMap,
jobKeysForPlatform,
cmtJobKeys,
webhookBucketForReportName,
} = require('./build-tsio-job-config');
// The override/cancel actions waive and reset exactly these contexts; if this
// list drifts from the required checks, a PR either cannot merge or merges
// while a context still blocks it.
describe('E2E_STATUS_CONTEXTS', () => {
it('should cover every required PR context exactly once', () => {
assert.deepEqual(E2E_STATUS_CONTEXTS, [
'e2e-test/detox-ios',
'e2e-test/detox-android',
'e2e-test/detox-ipad',
'e2e-test/maestro-ios',
'e2e-test/maestro-android',
]);
});
});
describe('buildTsioJobConfig', () => {
const base = {
repository: 'mattermost/mattermost-mobile',
commit_sha: 'abc1234',
gh_run_id: '99',
gh_run_attempt: '1',
branch: 'feat/x',
name: 'mobile-pr',
run_group: 'mobile-pr',
gh_pr_number: '9893',
framework: 'detox',
};
it('should give each job its own report group and status context', () => {
const cfg = buildTsioJobConfig(base, 'detox-ios');
assert.equal(cfg.total_reports_expected, 1);
assert.equal(cfg.status_context, 'e2e-test/detox-ios');
assert.equal(cfg.composite_identity.name, 'mobile-pr-detox-ios');
assert.equal(cfg.composite_identity.run_group, 'mobile-pr-detox-ios');
assert.equal(cfg.composite_identity.framework, 'detox');
assert.equal(cfg.composite_identity.commit_sha, 'abc1234');
});
it('should label maestro jobs with maestro framework and context', () => {
const cfg = buildTsioJobConfig(base, 'maestro-android-e2e');
assert.equal(cfg.status_context, 'e2e-test/maestro-android');
assert.equal(cfg.composite_identity.framework, 'maestro');
assert.equal(cfg.composite_identity.name, 'mobile-pr-maestro-android-e2e');
});
it('should support CMT shard names with <shard> context', () => {
const cmtBase = {...base, name: 'mobile-release', run_group: 'mobile-release'};
const cfg = buildTsioJobConfig(cmtBase, 'detox-ios-Server_11.9.0');
assert.equal(cfg.total_reports_expected, 1);
assert.equal(cfg.composite_identity.name, 'mobile-release-detox-ios-Server_11.9.0');
assert.equal(cfg.status_context, 'e2e-test/detox-ios-Server_11.9.0');
});
// The CMT base identity hardcodes framework=detox, but the upload action posts
// framework=maestro for maestro shards — both must describe the same group.
it('should infer maestro framework for CMT maestro shards', () => {
const cmtBase = {...base, name: 'mobile-release', run_group: 'mobile-release'};
const cfg = buildTsioJobConfig(cmtBase, 'maestro-android-Server_11.9.0');
assert.equal(cfg.composite_identity.framework, 'maestro');
assert.equal(cfg.status_context, 'e2e-test/maestro-android-Server_11.9.0');
});
it('should let an explicit override win over the inferred framework', () => {
const cfg = buildTsioJobConfig(base, 'maestro-ios-Server_11.9.0', {framework: 'detox'});
assert.equal(cfg.composite_identity.framework, 'detox');
});
});
describe('buildTsioJobConfigMap / jobKeysForPlatform', () => {
it('should only include ios jobs for PLATFORM=ios', () => {
assert.deepEqual(jobKeysForPlatform('ios'), ['detox-ios', 'detox-ipad', 'maestro-ios-e2e']);
});
it('should build a map for the selected keys', () => {
const base = {name: 'mobile-main', repository: 'mattermost/mattermost-mobile', commit_sha: 'deadbee'};
const map = buildTsioJobConfigMap(base, jobKeysForPlatform('android'));
assert.deepEqual(Object.keys(map).sort(), ['detox-android', 'maestro-android-e2e']);
assert.equal(map['detox-android'].status_context, 'e2e-test/detox-android');
});
});
describe('cmtJobKeys', () => {
const cmtMatrix = {
server: [
{version: '11.9.0', latest: true},
{version: '10.5.14', latest: false},
],
};
// Maestro has no smoke-on-older variant, so its shards must not be expanded
// across every server version — otherwise the rollup polls groups that never exist.
it('should expand detox shards per server version and maestro only for the latest', () => {
const maestroMatrix = {server: [{version: '11.9.0', latest: true}]};
assert.deepEqual(cmtJobKeys(cmtMatrix, maestroMatrix), [
'detox-ios-Server_11.9.0',
'detox-ipad-Server_11.9.0',
'detox-android-Server_11.9.0',
'detox-ios-Server_10.5.14',
'detox-ipad-Server_10.5.14',
'detox-android-Server_10.5.14',
'maestro-ios-Server_11.9.0',
'maestro-android-Server_11.9.0',
]);
});
it('should produce keys that resolve to the mobile-release webhook bucket', () => {
const base = {name: 'mobile-release', run_group: 'mobile-release', commit_sha: 'abc1234'};
const map = buildTsioJobConfigMap(base, cmtJobKeys(cmtMatrix, {server: [{version: '11.9.0'}]}));
assert.equal(Object.keys(map).length, 8);
assert.equal(map['maestro-ios-Server_11.9.0'].composite_identity.framework, 'maestro');
assert.equal(
webhookBucketForReportName(map['detox-ios-Server_11.9.0'].composite_identity.name),
'mobile-release',
);
});
it('should omit maestro shards when no maestro matrix is provided', () => {
assert.deepEqual(cmtJobKeys({server: [{version: '11.9.0'}]}), [
'detox-ios-Server_11.9.0',
'detox-ipad-Server_11.9.0',
'detox-android-Server_11.9.0',
]);
});
});
describe('webhookBucketForReportName', () => {
it('should map per-job report names back to PR/main/release buckets', () => {
assert.equal(webhookBucketForReportName('mobile-pr-detox-ios'), 'mobile-pr');
assert.equal(webhookBucketForReportName('mobile-main-maestro-ios-e2e'), 'mobile-main');
assert.equal(webhookBucketForReportName('mobile-release-detox-android-Server_10.5.0'), 'mobile-release');
assert.equal(webhookBucketForReportName('mobile-pr'), 'mobile-pr');
});
});