mirror of
https://github.com/mattermost/mattermost-mobile.git
synced 2026-11-04 18:08:54 +00:00
* ci(e2e): make E2E/Override waive checks instead of cancelling runs E2E/Override previously cancelled in-flight E2E runs and posted success on their commit statuses. Cancelling destroys the very signal the maintainer might still want, and reporting success for a run that never finished attributes a conclusion to work that never happened. Override now means one thing only: the required e2e-test/* contexts go green so the PR can merge. Runs keep going and keep reporting to TSIO. - Add .github/actions/e2e-override-status to post success (or pending) on all five required e2e-test/* contexts for a SHA. - Label manager posts success on `labeled` and pending on `unlabeled`, so removing the escape hatch re-blocks the PR instead of leaving a stale waiver on the head SHA. It comments the runs it left running rather than killing them, and drops to `actions: read` now that it never cancels. - tsio-report-status re-reads the PR labels before posting a failure. Because runs are no longer cancelled, a result landing after the override was applied would otherwise re-red a check the maintainer deliberately bypassed. The lookup fails closed so an unreadable label list cannot turn a real failure green, and the step itself no longer fails under override. - Guard e2e-cancel-on-label-removal: stripping E2E/Run* is exactly what the Honor E2E/Override step does, so without this the indirect unlabeled event would cancel the runs the redesign is meant to preserve. - e2e-detox-pr posts the success itself when the policy skips E2E, since every downstream job is skipped and a skipped job leaves the required contexts absent, blocking the merge forever. - cancel-e2e-runs drops status_state: a cancelled run has no conclusion to report, so it always resets to pending. - Thread `issues: read` through the reusable workflow callers so finalize can read the labels; a reusable workflow cannot escalate past its caller. Co-authored-by: Cursor <cursoragent@cursor.com> * ci(e2e): derive required E2E contexts from one definition The override and cancel actions each carried their own copy of the five required e2e-test/* contexts, kept aligned with build-tsio-job-config.js by comment alone. A job renamed in PR_MAIN_JOBS would silently leave the actions waiving or resetting a context that no longer exists, so a PR either cannot merge or merges while a real context still blocks it. Export E2E_STATUS_CONTEXTS from detox/utils/build-tsio-job-config.js, derived from PR_MAIN_JOBS, and require it from both composite actions. github-script resolves relative requires against the workspace, and every call site already checks out the repo before using the action. Also set persist-credentials: false on the e2e-label-manager checkout, matching the other checkout steps. Co-authored-by: Cursor <cursoragent@cursor.com> * ci(e2e): drop the cancel action for native concurrency, keep GitHub calls in Actions Review feedback from @saturninoabril. Superseding an in-flight E2E run is what GitHub's `concurrency` already does, so `e2e-detox-pr.yml` now declares `concurrency: e2e-<ref>` with `cancel-in-progress` instead of cancelling from a step inside the run that supersedes it. The older run is cancelled the moment the new dispatch is created rather than once a runner picks up the first job. That removes the only two remaining callers of `.github/actions/cancel-e2e-runs`, along with the commit statuses it forced to `pending` on cancel — a cancelled run has no conclusion to report, so the state is left to GitHub. Forcing a commit status is now reserved for real test results and the override waiver. The action is deleted. `e2e-cancel-on-label-removal.yml` goes with it: with `E2E/Run` on its way out and override no longer cancelling, removing the label to stop a run has no remaining practical case, and dropping the workflow also removes the back-door cancel guard it needed. The `E2E/Override` lookup moves out of `tsio-report-status.js` into a new `.github/actions/e2e-override-label` composite action, which the finalize steps call and pass in via `--e2e-override`. Talking to GitHub stays in the Actions layer and the reporter is left deciding on its inputs. Co-authored-by: Cursor <cursoragent@cursor.com> * ci(e2e): paginate PR label lookups for E2E/Override Co-authored-by: Cursor <cursoragent@cursor.com> * ci(e2e): re-post E2E/Override success on new commits Override success is SHA-scoped: it is posted only on the head SHA at label-apply time (e2e-label-manager 'labeled' path) and when Matterwick dispatches e2e-detox-pr.yml (post-e2e-override-status). On a push after E2E/Override is applied, refresh-e2e-run-label strips the run labels and returns — Matterwick is not dispatched, so the new head SHA gets no e2e-test/* statuses. With those checks Required, the new commit is blocked until the label is re-toggled. Carry the override onto the new commit: refresh-e2e-run-label now emits an e2e_override output, and when it is set a gated checkout + e2e-override-status step re-posts success on the new head SHA. Reuses the existing e2e-override-status action (same 5 required e2e-test/* contexts, same 'E2E bypassed' description), so it is idempotent with the labeled path. Adds statuses: write / contents: read to the job. Note: refresh-e2e-run-label is gated on base.ref == 'main', so stacked PRs (base != main) are not covered; non-main-base already posts success at dispatch time. --------- Co-authored-by: Cursor <cursoragent@cursor.com>
229 lines
8.0 KiB
JavaScript
229 lines
8.0 KiB
JavaScript
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
|
// See LICENSE.txt for license information.
|
|
|
|
// CI util unit tests: run with `node --test detox/utils/tsio-report-status.test.js`.
|
|
|
|
const assert = require('node:assert/strict');
|
|
const {describe, it} = require('node:test');
|
|
|
|
const {
|
|
buildDisplayReportUrl,
|
|
decideStatus,
|
|
decideTargetUrl,
|
|
overrideCommitStatus,
|
|
} = require('./tsio-report-status');
|
|
|
|
describe('tsio-report-status', () => {
|
|
describe('overrideCommitStatus', () => {
|
|
it('reports a failure as success so E2E/Override does not block the merge', () => {
|
|
assert.deepEqual(
|
|
overrideCommitStatus('failure', '8 passed, 2 failed, 0 skipped'),
|
|
{
|
|
state: 'success',
|
|
description: 'E2E/Override — 8 passed, 2 failed, 0 skipped',
|
|
},
|
|
);
|
|
});
|
|
|
|
it('leaves non-failure states untouched', () => {
|
|
assert.deepEqual(
|
|
overrideCommitStatus('success', '10 passed, 0 skipped'),
|
|
{state: 'success', description: '10 passed, 0 skipped'},
|
|
);
|
|
});
|
|
|
|
it('truncates the labelled description to the commit status limit', () => {
|
|
const {description} = overrideCommitStatus('failure', 'x'.repeat(200));
|
|
assert.equal(description.length, 140);
|
|
});
|
|
});
|
|
|
|
describe('decideStatus', () => {
|
|
it('returns success for completed report with no failures when upstream succeeded', () => {
|
|
assert.deepEqual(
|
|
decideStatus(
|
|
{status: 'completed', test_stats: {passed: 10, failed: 0, skipped: 2}},
|
|
true,
|
|
),
|
|
{
|
|
state: 'success',
|
|
description: '10 passed, 2 skipped',
|
|
both_terminal: true,
|
|
timed_out: false,
|
|
},
|
|
);
|
|
});
|
|
|
|
it('returns failure for completed report with failures', () => {
|
|
assert.deepEqual(
|
|
decideStatus(
|
|
{status: 'completed', test_stats: {passed: 8, failed: 2, skipped: 1}},
|
|
true,
|
|
),
|
|
{
|
|
state: 'failure',
|
|
description: '8 passed, 2 failed, 1 skipped',
|
|
both_terminal: true,
|
|
timed_out: false,
|
|
},
|
|
);
|
|
});
|
|
|
|
it('returns failure for incomplete terminal status even with zero failures', () => {
|
|
assert.deepEqual(
|
|
decideStatus(
|
|
{status: 'incomplete', test_stats: {passed: 5, failed: 0, skipped: 0}},
|
|
true,
|
|
),
|
|
{
|
|
state: 'failure',
|
|
description: 'Report incomplete — 5 passed, 0 skipped',
|
|
both_terminal: true,
|
|
timed_out: false,
|
|
},
|
|
);
|
|
});
|
|
|
|
it('uses timeout path for non-terminal status', () => {
|
|
assert.deepEqual(
|
|
decideStatus({status: 'processing', test_stats: {passed: 1, failed: 0}}, true),
|
|
{
|
|
state: 'success',
|
|
description: 'TSIO poll timed out (status=processing) — using CI job status',
|
|
both_terminal: false,
|
|
timed_out: true,
|
|
},
|
|
);
|
|
});
|
|
|
|
it('overrides pass when upstreamSucceeded is false on completed clean report', () => {
|
|
assert.deepEqual(
|
|
decideStatus(
|
|
{status: 'completed', test_stats: {passed: 10, failed: 0, skipped: 0}},
|
|
false,
|
|
),
|
|
{
|
|
state: 'failure',
|
|
description: '10 passed, 0 skipped',
|
|
both_terminal: true,
|
|
timed_out: false,
|
|
},
|
|
);
|
|
});
|
|
|
|
it('returns failure on timeout when upstreamSucceeded is false', () => {
|
|
assert.equal(
|
|
decideStatus({status: 'pending'}, false).state,
|
|
'failure',
|
|
);
|
|
assert.equal(decideStatus({status: 'pending'}, false).timed_out, true);
|
|
});
|
|
});
|
|
|
|
describe('decideTargetUrl', () => {
|
|
const display = 'https://test-io.test.mattermost.com/reports/mattermost-mobile/main/abc1234/mobile-pr';
|
|
const runUrl = 'https://github.com/mattermost/mattermost-mobile/actions/runs/99';
|
|
|
|
it('returns TSIO URL with gid when both terminal and success', () => {
|
|
assert.equal(
|
|
decideTargetUrl('success', true, display, 'gid-1', runUrl, {
|
|
failed: 0,
|
|
reportStatus: 'completed',
|
|
}),
|
|
`${display}?gid=gid-1`,
|
|
);
|
|
});
|
|
|
|
it('returns TSIO URL when tests failed but report uploaded', () => {
|
|
assert.equal(
|
|
decideTargetUrl('failure', true, display, 'gid-1', runUrl, {
|
|
failed: 187,
|
|
reportStatus: 'completed',
|
|
upstreamSucceeded: false,
|
|
}),
|
|
`${display}?gid=gid-1`,
|
|
);
|
|
});
|
|
|
|
it('returns TSIO URL for incomplete terminal report', () => {
|
|
assert.equal(
|
|
decideTargetUrl('failure', true, display, 'gid-1', runUrl, {
|
|
failed: 0,
|
|
reportStatus: 'incomplete',
|
|
upstreamSucceeded: true,
|
|
}),
|
|
`${display}?gid=gid-1`,
|
|
);
|
|
});
|
|
|
|
it('returns run URL when CI failed outside tests with a clean completed report', () => {
|
|
assert.equal(
|
|
decideTargetUrl('failure', true, display, 'gid-1', runUrl, {
|
|
failed: 0,
|
|
reportStatus: 'completed',
|
|
upstreamSucceeded: false,
|
|
}),
|
|
runUrl,
|
|
);
|
|
});
|
|
|
|
it('returns run URL when not both terminal', () => {
|
|
assert.equal(
|
|
decideTargetUrl('success', false, display, 'gid-1', runUrl),
|
|
runUrl,
|
|
);
|
|
});
|
|
|
|
it('returns run URL when report id is missing', () => {
|
|
assert.equal(
|
|
decideTargetUrl('failure', true, display, null, runUrl, {
|
|
failed: 2,
|
|
reportStatus: 'completed',
|
|
}),
|
|
runUrl,
|
|
);
|
|
});
|
|
});
|
|
|
|
describe('buildDisplayReportUrl', () => {
|
|
const base = 'https://test-io.test.mattermost.com';
|
|
|
|
it('strips refs/heads/ and encodes path slashes as ~', () => {
|
|
assert.equal(
|
|
buildDisplayReportUrl(base, {
|
|
repository: 'mattermost/mattermost-mobile',
|
|
branch: 'refs/heads/feat/tsio-mobile',
|
|
commit_sha: 'abcdef1234567890',
|
|
name: 'mobile-pr',
|
|
}),
|
|
`${base}/reports/mattermost-mobile/feat~tsio-mobile/abcdef1/mobile-pr`,
|
|
);
|
|
});
|
|
|
|
it('prefers run_group over name', () => {
|
|
assert.equal(
|
|
buildDisplayReportUrl(base, {
|
|
repository: 'mattermost/mattermost-mobile',
|
|
branch: 'main',
|
|
commit_sha: 'deadbeef',
|
|
name: 'detox-ios',
|
|
run_group: 'mobile-pr',
|
|
}),
|
|
`${base}/reports/mattermost-mobile/main/deadbee/mobile-pr`,
|
|
);
|
|
});
|
|
|
|
it('falls back to name when run_group is absent', () => {
|
|
assert.equal(
|
|
buildDisplayReportUrl(base, {
|
|
repository: 'mattermost/mattermost-mobile',
|
|
branch: 'main',
|
|
commit_sha: 'deadbeef',
|
|
name: 'mobile-release',
|
|
}),
|
|
`${base}/reports/mattermost-mobile/main/deadbee/mobile-release`,
|
|
);
|
|
});
|
|
});
|
|
});
|