mirror of
https://github.com/mattermost/mattermost-mobile.git
synced 2026-11-05 10:28:54 +00:00
65b8a408ed
* ci(e2e): triage CMT failures per test and add run breakdown tool
CMT had no per-test judging: its Detox template summary jobs fail on
FAILURES > 0, so 17/642 iOS and 12/650 Android failures (plus one iOS
shard whose simulator hit NSPOSIXErrorDomain in simctl privacy) made
the latest CMT run red, as in all 9 CMT runs in the last 30 days.
Wire the existing e2e-triage composite into compatibility-matrix-
testing.yml (report-only unless E2E_TRIAGE_MODE=enforce, same gate as
PR/main), with a new triage-lanes input so CMT summarises its per-shard
contexts instead of the PR lanes.
Also add detox/utils/e2e-run-breakdown.js: groups a gh run's jobs by
framework (Detox/Maestro) and platform (iOS/iPad/Android) so failure
rates per lane no longer need TSIO or signed-in log access.
* ci(e2e): serialize CMT triage status writes and count timed-out jobs as failures
The toolkit's status publisher has no compare-and-swap, and its
status.ts requires every writer of the shared e2e-test/triage context
to hold the same non-cancelling, SHA-scoped lock. The CMT tsio-triage
matrix wrote that context in parallel with no lock. Add max-parallel: 1
so matrix entries run one at a time without discarding queued entries,
plus a job concurrency group keyed on the CMT commit SHA with
cancel-in-progress: false to serialize against triage writers from
other runs of the same commit.
Also count GitHub job conclusion timed_out as a failure in
e2e-run-breakdown; it previously fell through to other, so a timed-out
job showed as zero failures and was omitted from failingJobs.
* ci(e2e): judge CMT channel-message failures as flaky, infra, known, or suspected app bug
The CMT channel rollup only posted totals, so every red run needed a
human to open TSIO and decide whether the app broke. The rollup already
fetches each leg's TSIO consolidated payload for per-leg counts; it now
also extracts this commit's failing tests with their error text and
earlier history, and the channel message groups them: infra/environment
(server-disabled feature, simctl, network, hook timeout, setup crash, or
a sibling cascade in the same suite), flaky (passed on retry or mixed
pass/fail history), known failing (failed before this run), and
suspected app bug (first seen, no infra cause — labelled as needing a
human confirm). When TSIO has detail for fewer tests than the failing
total, the message says how many were judged.
* ci(e2e): require a valid job and suite key for CMT cascade judgements
A failure with no job and a title that reduces to an empty suite key
produced the cascade key '|', which could match an infra failure with
the same empty key and reclassify a suspected app bug as a cascade.
Build the cascade key only when both the job and the suite are
non-empty, exclude invalid keys from the infra set, and require a
valid key before reclassifying.
* ci(e2e): require corroboration before the CMT message says suspected app bug
Suspected app bug was the fallback for any first-seen failure with no
infra marker, so ordinary visibility/existence timeouts (MM-T3199,
MM-T3206, MM-70693_2 on the 833 run) were labelled app bugs on a single
observation. Now that label needs a direct content/state assertion —
never a wait expiring — plus corroboration: the same test failing on
another leg of this run, or earlier passes in TSIO history now failing
that assertion. Everything weaker lands in a new Needs a closer look
bucket that names why the evidence is insufficient.
* ci(e2e): tighten CMT judgement corroboration (legs, run attempts, matcher text)
Three review fixes in cmt-channel-notify:
- alsoFailedElsewhere counted an empty job name as a second leg, so one
real leg plus a missing job name could promote a failure to suspected
app bug. Only non-empty job names are tracked now.
- The consolidated endpoint ignores gh_run_attempt (verified live:
gh_run_attempt=99 returns full data, run_attempt=99 returns none), so
fetchConsolidated now sends run_attempt. failuresFromConsolidated also
excludes every attempt of this GitHub run from previousStatuses, so a
pass in attempt 1 cannot corroborate an app-bug call in attempt 2.
- Removed 'doesn't match the selected view' from the timing patterns so
a matcher mismatch that carries a direct assertion (toHaveText and
friends) can reach the hard-assertion gate instead of being dismissed
as a timing wait.
---------
(cherry picked from commit 3927243232)
Co-authored-by: yasser khan <attitude3cena.yf@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
701 lines
30 KiB
YAML
701 lines
30 KiB
YAML
name: Compatibility Matrix Testing
|
||
|
||
on:
|
||
workflow_dispatch:
|
||
inputs:
|
||
CMT_MATRIX:
|
||
description: "A JSON object representing the testing matrix"
|
||
required: true
|
||
type: string
|
||
MOBILE_VERSION:
|
||
description: "The mattermost mobile version to test"
|
||
required: true
|
||
|
||
# Cancel overlapping CMT runs for the same ref; matterwick cleans up cancelled servers via workflow_run completed.
|
||
concurrency:
|
||
group: ${{ github.workflow }}-${{ github.ref }}
|
||
cancel-in-progress: true
|
||
|
||
# CMT runs against a freshly-cut build-release-NNN branch, so the app content
|
||
# never matches main's fingerprint — the S3 build cache would always miss.
|
||
# CMT's optimization is structural instead: each app (iOS, Android Detox,
|
||
# Android Maestro) is built ONCE below and reused across every server-version
|
||
# leg of the matrix via github.run_id artifacts.
|
||
jobs:
|
||
calculate-commit-hash:
|
||
runs-on: ubuntu-24.04
|
||
outputs:
|
||
MOBILE_SHA: ${{ steps.repo.outputs.MOBILE_SHA }}
|
||
steps:
|
||
- name: cmt/checkout-mobile
|
||
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
|
||
with:
|
||
persist-credentials: false
|
||
ref: ${{ inputs.MOBILE_VERSION }}
|
||
|
||
- name: cmt/calculate-mattermost-sha
|
||
id: repo
|
||
run: echo "MOBILE_SHA=$(git rev-parse HEAD)" >> "${GITHUB_OUTPUT}"
|
||
|
||
# Phase 5 — base TSIO identity for this CMT run. Each matrix leg builds its
|
||
# own per-job report group (total_reports_expected=1) from this base + tsio-shard-name.
|
||
cmt-identity:
|
||
name: Calculate CMT TSIO base identity
|
||
runs-on: ubuntu-24.04
|
||
permissions:
|
||
contents: read
|
||
needs:
|
||
- calculate-commit-hash
|
||
- prepare-maestro-matrix
|
||
outputs:
|
||
tsio-base-config: ${{ steps.id.outputs.tsio-base-config }}
|
||
tsio-job-configs: ${{ steps.id.outputs.tsio-job-configs }}
|
||
tsio-job-keys: ${{ steps.id.outputs.tsio-job-keys }}
|
||
steps:
|
||
- name: Sparse checkout TSIO job-config helper
|
||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||
with:
|
||
ref: ${{ inputs.MOBILE_VERSION }}
|
||
persist-credentials: false
|
||
sparse-checkout: |
|
||
detox/utils/build-tsio-job-config.js
|
||
|
||
- id: id
|
||
env:
|
||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||
MM_SHA: ${{ needs.calculate-commit-hash.outputs.MOBILE_SHA }}
|
||
MOBILE_VERSION: ${{ inputs.MOBILE_VERSION }}
|
||
CMT_MATRIX: ${{ inputs.CMT_MATRIX }}
|
||
MAESTRO_MATRIX: ${{ needs.prepare-maestro-matrix.outputs.matrix }}
|
||
run: |
|
||
BRANCH=$(echo "$MOBILE_VERSION" | sed 's#^refs/heads/##; s#^refs/tags/##')
|
||
BASE=$(jq -nc \
|
||
--arg repo "$GITHUB_REPOSITORY" --arg sha "$MM_SHA" --arg run_id "$GITHUB_RUN_ID" \
|
||
--arg attempt "$GITHUB_RUN_ATTEMPT" --arg branch "$BRANCH" \
|
||
'{repository:$repo, commit_sha:$sha, gh_run_id:$run_id, gh_run_attempt:$attempt, branch:$branch, name:"mobile-release", run_group:"mobile-release", framework:"detox"}')
|
||
# No status_context — templates expand per tsio-shard-name via buildTsioJobConfig.
|
||
TSIO_BASE=$(jq -nc --argjson identity "$BASE" \
|
||
'{composite_identity: $identity, total_reports_expected: 1}')
|
||
echo "tsio-base-config=$TSIO_BASE" >> "$GITHUB_OUTPUT"
|
||
|
||
# Per-shard configs so the final rollup can poll every leg's TSIO group.
|
||
export BASE CMT_MATRIX MAESTRO_MATRIX
|
||
JOB_CONFIGS=$(node -e '
|
||
const {buildTsioJobConfigMap, cmtJobKeys} = require("./detox/utils/build-tsio-job-config.js");
|
||
const base = JSON.parse(process.env.BASE);
|
||
const keys = cmtJobKeys(JSON.parse(process.env.CMT_MATRIX), JSON.parse(process.env.MAESTRO_MATRIX));
|
||
process.stdout.write(JSON.stringify(buildTsioJobConfigMap(base, keys)));
|
||
')
|
||
echo "tsio-job-configs=$JOB_CONFIGS" >> "$GITHUB_OUTPUT"
|
||
JOB_KEYS=$(node -e '
|
||
const {cmtJobKeys} = require("./detox/utils/build-tsio-job-config.js");
|
||
process.stdout.write(JSON.stringify(cmtJobKeys(JSON.parse(process.env.CMT_MATRIX), JSON.parse(process.env.MAESTRO_MATRIX))));
|
||
')
|
||
echo "tsio-job-keys=$JOB_KEYS" >> "$GITHUB_OUTPUT"
|
||
|
||
update-initial-status:
|
||
runs-on: ubuntu-24.04
|
||
permissions:
|
||
contents: read
|
||
statuses: write
|
||
needs:
|
||
- calculate-commit-hash
|
||
- cmt-identity
|
||
steps:
|
||
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||
with:
|
||
ref: ${{ inputs.MOBILE_VERSION }}
|
||
persist-credentials: false
|
||
|
||
# Pending statuses are posted per matrix leg when each job starts; keep a
|
||
# single umbrella pending only when MOBILE_SHA is known so the PR/commit
|
||
# UI shows CMT is running before the first shard uploads.
|
||
- name: Build TSIO report URL (release group prefix)
|
||
id: tsio-url
|
||
env:
|
||
TSIO_BASE: ${{ needs.cmt-identity.outputs.tsio-base-config }}
|
||
run: |
|
||
URL=$(jq -nr --argjson cfg "$TSIO_BASE" --arg base "https://test-io.test.mattermost.com" '
|
||
($cfg.composite_identity) as $id
|
||
| ($id.repository | split("/") | .[-1]) as $repo
|
||
| ($id.branch // "main"
|
||
| sub("^refs/heads/"; "")
|
||
| sub("^refs/tags/"; "")
|
||
| gsub("/"; "~")) as $branch
|
||
| ($id.commit_sha[0:7]) as $sha
|
||
| "\($base)/reports/\($repo|@uri)/\($branch|@uri)/\($sha)/mobile-release?gh_run_id=\($id.gh_run_id)"
|
||
')
|
||
echo "target_url=$URL" >> "$GITHUB_OUTPUT"
|
||
|
||
- uses: ./.github/actions/update-commit-status
|
||
if: ${{ needs.calculate-commit-hash.outputs.MOBILE_SHA != '' }}
|
||
env:
|
||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||
with:
|
||
repository_full_name: ${{ github.repository }}
|
||
commit_sha: ${{ needs.calculate-commit-hash.outputs.MOBILE_SHA }}
|
||
context: e2e-test/compatibility-matrix-testing
|
||
description: "Compatibility Matrix Testing for ${{ inputs.MOBILE_VERSION }} version — per-job checks also posted"
|
||
status: pending
|
||
target_url: ${{ steps.tsio-url.outputs.target_url }}
|
||
|
||
# Maestro only runs against the latest server version; matrix filtering happens here since job-level if: can't access matrix.
|
||
prepare-maestro-matrix:
|
||
runs-on: ubuntu-24.04
|
||
outputs:
|
||
matrix: ${{ steps.filter.outputs.matrix }}
|
||
steps:
|
||
- name: Filter CMT matrix to latest server only
|
||
id: filter
|
||
env:
|
||
CMT_MATRIX: ${{ inputs.CMT_MATRIX }}
|
||
run: |
|
||
matrix=$(echo "$CMT_MATRIX" | jq -c '{server: [.server[] | select(.latest == true)]}')
|
||
if [ "$(echo "$matrix" | jq '.server | length')" -eq 0 ]; then
|
||
matrix=$(echo "$CMT_MATRIX" | jq -c '{server: [.server[-1]]}')
|
||
fi
|
||
echo "matrix=$matrix" >> "$GITHUB_OUTPUT"
|
||
|
||
# PR E2E runs detox provision (demo plugin, license, config) before tests.
|
||
# CMT previously skipped that and only used Matterwick-booted servers, so
|
||
# interactive_dialog_plugin failed 24× with "Demo plugin is not active".
|
||
provision-cmt-servers:
|
||
name: Provision CMT test servers
|
||
runs-on: ubuntu-24.04
|
||
needs:
|
||
- calculate-commit-hash
|
||
steps:
|
||
- name: Checkout Repository
|
||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||
with:
|
||
ref: ${{ inputs.MOBILE_VERSION }}
|
||
persist-credentials: false
|
||
|
||
- name: Collect unique matrix server URLs
|
||
id: urls
|
||
env:
|
||
CMT_MATRIX: ${{ inputs.CMT_MATRIX }}
|
||
run: |
|
||
URLS=$(echo "$CMT_MATRIX" | jq -r '
|
||
.server[]
|
||
| .android_site_1_url, .android_site_2_url, .ios_site_1_url, .ios_site_2_url, .site_3_url
|
||
| select(. != null and . != "")
|
||
' | awk 'NF && !seen[$0]++')
|
||
if [ -z "$URLS" ]; then
|
||
echo "::error::CMT_MATRIX contained no server URLs to provision"
|
||
exit 1
|
||
fi
|
||
echo "Provisioning $(echo "$URLS" | wc -l | tr -d ' ') unique server(s)"
|
||
{
|
||
echo "list<<EOF"
|
||
printf '%s\n' "$URLS"
|
||
echo "EOF"
|
||
} >> "$GITHUB_OUTPUT"
|
||
|
||
- name: Provision test servers
|
||
uses: ./.github/actions/provision-e2e-servers
|
||
env:
|
||
ADMIN_USERNAME: ${{ secrets.MM_MOBILE_E2E_ADMIN_USERNAME }}
|
||
ADMIN_PASSWORD: ${{ secrets.MM_MOBILE_E2E_ADMIN_PASSWORD }}
|
||
with:
|
||
server-urls: ${{ steps.urls.outputs.list }}
|
||
|
||
# Input follows the below schema
|
||
# {
|
||
# "server": [
|
||
# {
|
||
# "version": "9.6.1",
|
||
# "android_site_1_url": "https://android-1.example.test",
|
||
# "android_site_2_url": "https://android-2.example.test",
|
||
# "ios_site_1_url": "https://ios-1.example.test",
|
||
# "ios_site_2_url": "https://ios-2.example.test",
|
||
# "site_3_url": "https://shared-3.example.test"
|
||
# },
|
||
# {
|
||
# "version": "9.5.2",
|
||
# "android_site_1_url": "https://android-1-older.example.test",
|
||
# "android_site_2_url": "https://android-2-older.example.test",
|
||
# "ios_site_1_url": "https://ios-1-older.example.test",
|
||
# "ios_site_2_url": "https://ios-2-older.example.test",
|
||
# "site_3_url": "https://shared-3-older.example.test"
|
||
# }
|
||
# ]
|
||
# }
|
||
build-ios-simulator:
|
||
# macos-14 predates Xcode 26 and has none pre-installed, so prepare-ios-build's
|
||
# Xcode 26 requirement always fails here. e2e-detox-pr.yml and
|
||
# e2e-maestro-template.yml already build on macos-26 successfully — match that.
|
||
runs-on: macos-26
|
||
needs:
|
||
- update-initial-status
|
||
permissions:
|
||
contents: read
|
||
actions: write
|
||
steps:
|
||
- name: Checkout Repository
|
||
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
|
||
with:
|
||
persist-credentials: false
|
||
ref: ${{ inputs.MOBILE_VERSION }}
|
||
|
||
- name: Prepare iOS Build
|
||
uses: ./.github/actions/prepare-ios-build
|
||
|
||
- name: Build iOS Simulator
|
||
env:
|
||
TAG: "${{ inputs.MOBILE_VERSION }}"
|
||
GITHUB_TOKEN: "${{ secrets.MM_MOBILE_GITHUB_TOKEN }}"
|
||
run: bundle exec fastlane ios simulator --env ios.simulator
|
||
working-directory: ./fastlane
|
||
|
||
- name: Upload iOS Simulator Build
|
||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
||
with:
|
||
name: ios-build-simulator-${{ github.run_id }}
|
||
path: Mattermost-simulator-*.app.zip
|
||
|
||
build-android-apk:
|
||
runs-on: ubuntu-latest-8-cores
|
||
needs:
|
||
- update-initial-status
|
||
permissions:
|
||
contents: read
|
||
actions: write
|
||
env:
|
||
ORG_GRADLE_PROJECT_jvmargs: -Xmx8g
|
||
steps:
|
||
- name: Prune Docker to free up space
|
||
run: docker system prune -af
|
||
|
||
- name: Checkout Repository
|
||
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
|
||
with:
|
||
persist-credentials: false
|
||
ref: ${{ inputs.MOBILE_VERSION }}
|
||
|
||
- name: Prepare Android Build
|
||
uses: ./.github/actions/prepare-android-build
|
||
env:
|
||
STORE_FILE: ${{ secrets.MM_MOBILE_STORE_FILE }}
|
||
STORE_ALIAS: ${{ secrets.MM_MOBILE_STORE_ALIAS }}
|
||
STORE_PASSWORD: ${{ secrets.MM_MOBILE_STORE_PASSWORD }}
|
||
MATTERMOST_BUILD_GH_TOKEN: ${{ secrets.MATTERMOST_BUILD_GH_TOKEN }}
|
||
|
||
- name: Detox build
|
||
run: |
|
||
cd detox
|
||
npm ci --prefer-offline --no-audit --no-fund
|
||
npm run e2e:android-build
|
||
|
||
- name: Upload Android Build
|
||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
||
with:
|
||
name: android-build-files-${{ github.run_id }}
|
||
# Match e2e-detox-pr.yml's glob shape exactly: the wildcard must start
|
||
# right after "build/" so upload-artifact only strips "android/app/build/"
|
||
# and preserves the outputs/apk/... subtree. e2e-android-template.yml's
|
||
# Download APK artifact step (shared by PR + CMT) expects that shape.
|
||
path: "android/app/build/**/*"
|
||
|
||
build-android-apk-maestro:
|
||
runs-on: ubuntu-latest-8-cores
|
||
needs:
|
||
- update-initial-status
|
||
permissions:
|
||
contents: read
|
||
actions: write
|
||
env:
|
||
ORG_GRADLE_PROJECT_jvmargs: -Xmx8g
|
||
steps:
|
||
- name: Prune Docker to free up space
|
||
run: docker system prune -af
|
||
|
||
- name: Checkout Repository
|
||
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
|
||
with:
|
||
persist-credentials: false
|
||
ref: ${{ inputs.MOBILE_VERSION }}
|
||
|
||
- name: Prepare Android Build
|
||
uses: ./.github/actions/prepare-android-build
|
||
env:
|
||
STORE_FILE: ${{ secrets.MM_MOBILE_STORE_FILE }}
|
||
STORE_ALIAS: ${{ secrets.MM_MOBILE_STORE_ALIAS }}
|
||
STORE_PASSWORD: ${{ secrets.MM_MOBILE_STORE_PASSWORD }}
|
||
MATTERMOST_BUILD_GH_TOKEN: ${{ secrets.MATTERMOST_BUILD_GH_TOKEN }}
|
||
|
||
- name: Build Maestro Android APK
|
||
uses: ./.github/actions/build-android-maestro-apk
|
||
|
||
- name: Upload Maestro Android Build
|
||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
||
with:
|
||
name: android-maestro-build-files-${{ github.run_id }}
|
||
path: android/app/build/outputs/apk/release/
|
||
|
||
# latest entry runs full suite, older entries run smoke only. Job-level if: can't conditionally skip matrix legs, so inputs vary instead.
|
||
detox-e2e:
|
||
name: mobile-cmt-ios${{ matrix.server.latest && '-full' || '-smoke' }}-${{ matrix.server.version }}
|
||
permissions:
|
||
contents: read
|
||
# nested template mints an OIDC token for the TSIO upload.
|
||
id-token: write
|
||
actions: read
|
||
statuses: write
|
||
# Nested template finalize reads PR labels to honour E2E/Override; a
|
||
# reusable workflow cannot escalate past the caller, so grant it here too.
|
||
issues: read
|
||
uses: ./.github/workflows/e2e-ios-template.yml
|
||
needs:
|
||
- build-ios-simulator
|
||
- cmt-identity
|
||
- provision-cmt-servers
|
||
strategy:
|
||
fail-fast: false
|
||
matrix: ${{ fromJson(inputs.CMT_MATRIX) }}
|
||
secrets:
|
||
MM_MOBILE_E2E_ADMIN_EMAIL: ${{ secrets.MM_MOBILE_E2E_ADMIN_EMAIL }}
|
||
MM_MOBILE_E2E_ADMIN_USERNAME: ${{ secrets.MM_MOBILE_E2E_ADMIN_USERNAME }}
|
||
MM_MOBILE_E2E_ADMIN_PASSWORD: ${{ secrets.MM_MOBILE_E2E_ADMIN_PASSWORD }}
|
||
MM_MOBILE_E2E_ZEPHYR_API_KEY: ${{ secrets.MM_MOBILE_E2E_ZEPHYR_API_KEY }}
|
||
MM_MOBILE_E2E_TEST_CYCLE_LINK_PREFIX: ${{ secrets.MM_MOBILE_E2E_TEST_CYCLE_LINK_PREFIX }}
|
||
MM_MOBILE_E2E_WEBHOOK_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_URL }}
|
||
MM_E2E_MASTER_HEALTH_WEBHOOK_URL: ${{ secrets.MM_E2E_MASTER_HEALTH_WEBHOOK_URL }}
|
||
MM_E2E_RELEASE_WEBHOOK_URL: ${{ secrets.MM_E2E_RELEASE_WEBHOOK_URL }}
|
||
MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID: ${{ secrets.MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID }}
|
||
MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY: ${{ secrets.MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY }}
|
||
MM_MOBILE_E2E_WEBHOOK_PUBLIC_BASE_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_PUBLIC_BASE_URL }}
|
||
MM_MOBILE_E2E_CLOUDFLARED_TUNNEL_TOKEN: ${{ secrets.MM_MOBILE_E2E_CLOUDFLARED_TUNNEL_TOKEN }}
|
||
with:
|
||
tsio-config: ${{ needs.cmt-identity.outputs.tsio-base-config }}
|
||
tsio-shard-name: detox-ios-Server_${{ matrix.server.version }}
|
||
run-type: "RELEASE"
|
||
MM_TEST_SERVER_URL: ${{ matrix.server.ios_site_1_url }}
|
||
MM_TEST_SERVER_URL_2: ${{ matrix.server.ios_site_2_url }}
|
||
MM_TEST_SERVER_URL_3: ${{ matrix.server.site_3_url }}
|
||
MOBILE_VERSION: ${{ inputs.MOBILE_VERSION }}
|
||
search_path: ${{ matrix.server.latest && 'detox/e2e/test' || 'detox/e2e/test/products/channels/smoke_test' }}
|
||
# Full suite matches the PR/main shard count (template default 28); smoke stays single-shard.
|
||
parallelism: ${{ matrix.server.latest && '28' || '1' }}
|
||
artifact_run_id: ${{ github.run_id }}
|
||
|
||
detox-ipad-e2e:
|
||
name: mobile-cmt-ipad-${{ matrix.server.version }}
|
||
permissions:
|
||
contents: read
|
||
id-token: write
|
||
actions: read
|
||
statuses: write
|
||
# Nested template finalize reads PR labels to honour E2E/Override; a
|
||
# reusable workflow cannot escalate past the caller, so grant it here too.
|
||
issues: read
|
||
uses: ./.github/workflows/e2e-ios-template.yml
|
||
needs:
|
||
- build-ios-simulator
|
||
- cmt-identity
|
||
- provision-cmt-servers
|
||
strategy:
|
||
fail-fast: false
|
||
matrix: ${{ fromJson(inputs.CMT_MATRIX) }}
|
||
secrets:
|
||
MM_MOBILE_E2E_ADMIN_EMAIL: ${{ secrets.MM_MOBILE_E2E_ADMIN_EMAIL }}
|
||
MM_MOBILE_E2E_ADMIN_USERNAME: ${{ secrets.MM_MOBILE_E2E_ADMIN_USERNAME }}
|
||
MM_MOBILE_E2E_ADMIN_PASSWORD: ${{ secrets.MM_MOBILE_E2E_ADMIN_PASSWORD }}
|
||
MM_MOBILE_E2E_ZEPHYR_API_KEY: ${{ secrets.MM_MOBILE_E2E_ZEPHYR_API_KEY }}
|
||
MM_MOBILE_E2E_TEST_CYCLE_LINK_PREFIX: ${{ secrets.MM_MOBILE_E2E_TEST_CYCLE_LINK_PREFIX }}
|
||
MM_MOBILE_E2E_WEBHOOK_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_URL }}
|
||
MM_E2E_MASTER_HEALTH_WEBHOOK_URL: ${{ secrets.MM_E2E_MASTER_HEALTH_WEBHOOK_URL }}
|
||
MM_E2E_RELEASE_WEBHOOK_URL: ${{ secrets.MM_E2E_RELEASE_WEBHOOK_URL }}
|
||
MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID: ${{ secrets.MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID }}
|
||
MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY: ${{ secrets.MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY }}
|
||
MM_MOBILE_E2E_WEBHOOK_PUBLIC_BASE_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_PUBLIC_BASE_URL }}
|
||
MM_MOBILE_E2E_CLOUDFLARED_TUNNEL_TOKEN: ${{ secrets.MM_MOBILE_E2E_CLOUDFLARED_TUNNEL_TOKEN }}
|
||
with:
|
||
tsio-config: ${{ needs.cmt-identity.outputs.tsio-base-config }}
|
||
tsio-shard-name: detox-ipad-Server_${{ matrix.server.version }}
|
||
run-type: "RELEASE"
|
||
MM_TEST_SERVER_URL: ${{ matrix.server.ios_site_1_url }}
|
||
MM_TEST_SERVER_URL_2: ${{ matrix.server.ios_site_2_url }}
|
||
MM_TEST_SERVER_URL_3: ${{ matrix.server.site_3_url }}
|
||
MOBILE_VERSION: ${{ inputs.MOBILE_VERSION }}
|
||
ios_device_name: "iPad Pro 13-inch (M5)"
|
||
ios_device_os_name: "iOS 26.2"
|
||
search_path: "detox/e2e/test/products/channels/ipad"
|
||
parallelism: "1"
|
||
detox_config: "ios.ipad.debug"
|
||
artifact_run_id: ${{ github.run_id }}
|
||
|
||
# Maestro runs only against the latest server version (no smoke-on-older variant).
|
||
maestro-ios-e2e:
|
||
name: mobile-cmt-maestro-ios-${{ matrix.server.version }}
|
||
permissions:
|
||
contents: read
|
||
id-token: write
|
||
actions: read
|
||
statuses: write
|
||
# Nested template finalize reads PR labels to honour E2E/Override; a
|
||
# reusable workflow cannot escalate past the caller, so grant it here too.
|
||
issues: read
|
||
uses: ./.github/workflows/e2e-maestro-template.yml
|
||
needs:
|
||
- build-ios-simulator
|
||
- prepare-maestro-matrix
|
||
- cmt-identity
|
||
- provision-cmt-servers
|
||
strategy:
|
||
fail-fast: false
|
||
matrix: ${{ fromJson(needs.prepare-maestro-matrix.outputs.matrix) }}
|
||
secrets:
|
||
MM_MOBILE_E2E_ADMIN_EMAIL: ${{ secrets.MM_MOBILE_E2E_ADMIN_EMAIL }}
|
||
MM_MOBILE_E2E_ADMIN_USERNAME: ${{ secrets.MM_MOBILE_E2E_ADMIN_USERNAME }}
|
||
MM_MOBILE_E2E_ADMIN_PASSWORD: ${{ secrets.MM_MOBILE_E2E_ADMIN_PASSWORD }}
|
||
MM_MOBILE_E2E_ZEPHYR_API_KEY: ${{ secrets.MM_MOBILE_E2E_ZEPHYR_API_KEY }}
|
||
MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID: ${{ secrets.MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID }}
|
||
MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY: ${{ secrets.MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY }}
|
||
MM_MOBILE_E2E_WEBHOOK_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_URL }}
|
||
MM_E2E_MASTER_HEALTH_WEBHOOK_URL: ${{ secrets.MM_E2E_MASTER_HEALTH_WEBHOOK_URL }}
|
||
MM_E2E_RELEASE_WEBHOOK_URL: ${{ secrets.MM_E2E_RELEASE_WEBHOOK_URL }}
|
||
with:
|
||
tsio-config: ${{ needs.cmt-identity.outputs.tsio-base-config }}
|
||
tsio-shard-name: maestro-ios-Server_${{ matrix.server.version }}
|
||
platform: ios
|
||
run-type: "RELEASE"
|
||
MM_TEST_SERVER_URL: ${{ matrix.server.ios_site_1_url }}
|
||
MOBILE_VERSION: ${{ inputs.MOBILE_VERSION }}
|
||
record_tests_in_zephyr: true
|
||
artifact_run_id: ${{ github.run_id }}
|
||
|
||
detox-android-e2e:
|
||
name: mobile-cmt-android${{ matrix.server.latest && '-full' || '-smoke' }}-${{ matrix.server.version }}
|
||
permissions:
|
||
contents: read
|
||
id-token: write
|
||
actions: read
|
||
statuses: write
|
||
# Nested template finalize reads PR labels to honour E2E/Override; a
|
||
# reusable workflow cannot escalate past the caller, so grant it here too.
|
||
issues: read
|
||
uses: ./.github/workflows/e2e-android-template.yml
|
||
needs:
|
||
- build-android-apk
|
||
- cmt-identity
|
||
- provision-cmt-servers
|
||
strategy:
|
||
fail-fast: false
|
||
matrix: ${{ fromJson(inputs.CMT_MATRIX) }}
|
||
secrets:
|
||
MM_MOBILE_E2E_ADMIN_EMAIL: ${{ secrets.MM_MOBILE_E2E_ADMIN_EMAIL }}
|
||
MM_MOBILE_E2E_ADMIN_USERNAME: ${{ secrets.MM_MOBILE_E2E_ADMIN_USERNAME }}
|
||
MM_MOBILE_E2E_ADMIN_PASSWORD: ${{ secrets.MM_MOBILE_E2E_ADMIN_PASSWORD }}
|
||
MM_MOBILE_E2E_ZEPHYR_API_KEY: ${{ secrets.MM_MOBILE_E2E_ZEPHYR_API_KEY }}
|
||
MM_MOBILE_E2E_TEST_CYCLE_LINK_PREFIX: ${{ secrets.MM_MOBILE_E2E_TEST_CYCLE_LINK_PREFIX }}
|
||
MM_MOBILE_E2E_WEBHOOK_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_URL }}
|
||
MM_E2E_MASTER_HEALTH_WEBHOOK_URL: ${{ secrets.MM_E2E_MASTER_HEALTH_WEBHOOK_URL }}
|
||
MM_E2E_RELEASE_WEBHOOK_URL: ${{ secrets.MM_E2E_RELEASE_WEBHOOK_URL }}
|
||
MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID: ${{ secrets.MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID }}
|
||
MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY: ${{ secrets.MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY }}
|
||
MM_MOBILE_E2E_WEBHOOK_PUBLIC_BASE_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_PUBLIC_BASE_URL }}
|
||
MM_MOBILE_E2E_CLOUDFLARED_TUNNEL_TOKEN: ${{ secrets.MM_MOBILE_E2E_CLOUDFLARED_TUNNEL_TOKEN }}
|
||
with:
|
||
tsio-config: ${{ needs.cmt-identity.outputs.tsio-base-config }}
|
||
tsio-shard-name: detox-android-Server_${{ matrix.server.version }}
|
||
run-android-tests: true
|
||
run-type: "RELEASE"
|
||
MM_TEST_SERVER_URL: ${{ matrix.server.android_site_1_url }}
|
||
MM_TEST_SERVER_URL_2: ${{ matrix.server.android_site_2_url }}
|
||
MM_TEST_SERVER_URL_3: ${{ matrix.server.site_3_url }}
|
||
MOBILE_VERSION: ${{ inputs.MOBILE_VERSION }}
|
||
search_path: ${{ matrix.server.latest && 'detox/e2e/test' || 'detox/e2e/test/products/channels/smoke_test' }}
|
||
# Full suite matches the PR/main shard count (template default 28); smoke stays single-shard.
|
||
parallelism: ${{ matrix.server.latest && '28' || '1' }}
|
||
artifact_run_id: ${{ github.run_id }}
|
||
|
||
# Maestro runs only against the latest server version (no smoke-on-older variant).
|
||
maestro-android-e2e:
|
||
name: mobile-cmt-maestro-android-${{ matrix.server.version }}
|
||
permissions:
|
||
contents: read
|
||
id-token: write
|
||
actions: read
|
||
statuses: write
|
||
# Nested template finalize reads PR labels to honour E2E/Override; a
|
||
# reusable workflow cannot escalate past the caller, so grant it here too.
|
||
issues: read
|
||
uses: ./.github/workflows/e2e-maestro-template.yml
|
||
needs:
|
||
- build-android-apk-maestro
|
||
- prepare-maestro-matrix
|
||
- cmt-identity
|
||
- provision-cmt-servers
|
||
strategy:
|
||
fail-fast: false
|
||
matrix: ${{ fromJson(needs.prepare-maestro-matrix.outputs.matrix) }}
|
||
secrets:
|
||
MM_MOBILE_E2E_ADMIN_EMAIL: ${{ secrets.MM_MOBILE_E2E_ADMIN_EMAIL }}
|
||
MM_MOBILE_E2E_ADMIN_USERNAME: ${{ secrets.MM_MOBILE_E2E_ADMIN_USERNAME }}
|
||
MM_MOBILE_E2E_ADMIN_PASSWORD: ${{ secrets.MM_MOBILE_E2E_ADMIN_PASSWORD }}
|
||
MM_MOBILE_E2E_ZEPHYR_API_KEY: ${{ secrets.MM_MOBILE_E2E_ZEPHYR_API_KEY }}
|
||
MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID: ${{ secrets.MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID }}
|
||
MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY: ${{ secrets.MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY }}
|
||
MM_MOBILE_E2E_WEBHOOK_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_URL }}
|
||
MM_E2E_MASTER_HEALTH_WEBHOOK_URL: ${{ secrets.MM_E2E_MASTER_HEALTH_WEBHOOK_URL }}
|
||
MM_E2E_RELEASE_WEBHOOK_URL: ${{ secrets.MM_E2E_RELEASE_WEBHOOK_URL }}
|
||
with:
|
||
tsio-config: ${{ needs.cmt-identity.outputs.tsio-base-config }}
|
||
tsio-shard-name: maestro-android-Server_${{ matrix.server.version }}
|
||
platform: android
|
||
run-type: "RELEASE"
|
||
MM_TEST_SERVER_URL: ${{ matrix.server.android_site_1_url }}
|
||
MOBILE_VERSION: ${{ inputs.MOBILE_VERSION }}
|
||
artifact_run_id: ${{ github.run_id }}
|
||
record_tests_in_zephyr: true
|
||
|
||
update-final-status:
|
||
runs-on: ubuntu-24.04
|
||
if: always()
|
||
permissions:
|
||
contents: read
|
||
statuses: write
|
||
# the channel rollup mints an OIDC token to read each shard's TSIO group.
|
||
id-token: write
|
||
needs:
|
||
- cmt-identity
|
||
- calculate-commit-hash
|
||
- provision-cmt-servers
|
||
- detox-e2e
|
||
- detox-android-e2e
|
||
- detox-ipad-e2e
|
||
- maestro-ios-e2e
|
||
- maestro-android-e2e
|
||
steps:
|
||
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||
with:
|
||
ref: ${{ inputs.MOBILE_VERSION }}
|
||
persist-credentials: false
|
||
|
||
- name: Resolve upstream matrix outcome
|
||
id: upstream
|
||
run: |
|
||
UPSTREAM="true"
|
||
for r in "${{ needs.provision-cmt-servers.result }}" "${{ needs.detox-e2e.result }}" "${{ needs.detox-android-e2e.result }}" "${{ needs.detox-ipad-e2e.result }}" "${{ needs.maestro-ios-e2e.result }}" "${{ needs.maestro-android-e2e.result }}"; do
|
||
case "$r" in
|
||
success|skipped) ;;
|
||
*) UPSTREAM="false" ;;
|
||
esac
|
||
done
|
||
echo "succeeded=$UPSTREAM" >> "$GITHUB_OUTPUT"
|
||
|
||
# Umbrella context only — each matrix leg already posted <shard> from TSIO.
|
||
- name: Finalize umbrella CMT commit status from CI job results
|
||
env:
|
||
GITHUB_TOKEN: ${{ github.token }}
|
||
COMMIT_SHA: ${{ needs.calculate-commit-hash.outputs.MOBILE_SHA }}
|
||
REPO: ${{ github.repository }}
|
||
TARGET_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
|
||
UPSTREAM: ${{ steps.upstream.outputs.succeeded }}
|
||
run: |
|
||
if [ "$UPSTREAM" = "true" ]; then
|
||
STATE=success
|
||
DESC="Compatibility Matrix Testing completed — see per-job e2e-test/* checks"
|
||
else
|
||
STATE=failure
|
||
DESC="Compatibility Matrix Testing failed — see per-job e2e-test/* checks"
|
||
fi
|
||
gh api --method POST "repos/${REPO}/statuses/${COMMIT_SHA}" \
|
||
-f state="$STATE" \
|
||
-f context="e2e-test/compatibility-matrix-testing" \
|
||
-f description="$DESC" \
|
||
-f target_url="$TARGET_URL"
|
||
|
||
# Per-shard TSIO groups are finalized in each matrix leg, which leaves channel
|
||
# notify off so N legs do not spam. This umbrella polls every shard group and
|
||
# posts one aggregated summary (no second commit status) — same path as PR/main.
|
||
- name: Post Mattermost CMT channel rollup
|
||
if: always()
|
||
env:
|
||
JOB_CONFIGS: ${{ needs.cmt-identity.outputs.tsio-job-configs }}
|
||
MATTERMOST_CMT_WEBHOOK_URL: ${{ secrets.MM_E2E_RELEASE_WEBHOOK_URL }}
|
||
UPSTREAM_SUCCEEDED: ${{ steps.upstream.outputs.succeeded }}
|
||
run: |
|
||
node detox/utils/tsio-channel-notify-rollup.js \
|
||
--job-configs="$JOB_CONFIGS" \
|
||
--upstream-succeeded="$UPSTREAM_SUCCEEDED" \
|
||
--poll-attempts=6 || true
|
||
|
||
# Instance cleanup is handled by matterwick on workflow_run completed.
|
||
|
||
# Same per-test judging the PR/main lanes get in e2e-detox-pr.yml: without it
|
||
# every CMT shard is raw any-failure-red (FAILURES > 0 in the template summary
|
||
# jobs), so a handful of known-flaky failures out of ~650 tests per leg turned
|
||
# every CMT run red. Report-only (the default) only posts the verdict;
|
||
# E2E_TRIAGE_MODE=enforce makes the verdict the shard's commit status, exactly
|
||
# as on PR/main. The workflow run itself stays red on real failures either way.
|
||
tsio-triage:
|
||
name: Triage CMT failures (${{ matrix.job-key }})
|
||
runs-on: ubuntu-24.04
|
||
if: >-
|
||
!cancelled() &&
|
||
(needs.detox-e2e.result == 'failure' ||
|
||
needs.detox-android-e2e.result == 'failure' ||
|
||
needs.detox-ipad-e2e.result == 'failure' ||
|
||
needs.maestro-ios-e2e.result == 'failure' ||
|
||
needs.maestro-android-e2e.result == 'failure')
|
||
needs:
|
||
- calculate-commit-hash
|
||
- cmt-identity
|
||
- detox-e2e
|
||
- detox-android-e2e
|
||
- detox-ipad-e2e
|
||
- maestro-ios-e2e
|
||
- maestro-android-e2e
|
||
permissions:
|
||
contents: read
|
||
statuses: write
|
||
pull-requests: read
|
||
# Every lane writes the shared e2e-test/triage context, and the toolkit's
|
||
# status publisher has no compare-and-swap: its status.ts requires every
|
||
# caller to hold the same non-cancelling, SHA-scoped job concurrency lock
|
||
# across its reads and writes. max-parallel serializes the matrix entries
|
||
# within this run without discarding queued entries; the concurrency group
|
||
# serializes against triage writers from any other run of the same commit.
|
||
# The workflow-level cancel-in-progress group above does neither.
|
||
concurrency:
|
||
group: e2e-test-triage-${{ needs.calculate-commit-hash.outputs.MOBILE_SHA }}
|
||
cancel-in-progress: false
|
||
strategy:
|
||
fail-fast: false
|
||
max-parallel: 1
|
||
matrix:
|
||
job-key: ${{ fromJSON(needs.cmt-identity.outputs.tsio-job-keys) }}
|
||
steps:
|
||
# Checked out from the workflow ref (the build-release-* branch under
|
||
# test runs no code with the API key beyond this composite's own steps,
|
||
# which are pinned actions; same pattern as e2e-detox-pr.yml).
|
||
- name: Checkout triage action
|
||
continue-on-error: true
|
||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||
with:
|
||
persist-credentials: false
|
||
sparse-checkout: .github/actions/e2e-triage/
|
||
sparse-checkout-cone-mode: false
|
||
|
||
- name: Resolve CMT triage lanes
|
||
id: lanes
|
||
env:
|
||
JOB_CONFIGS: ${{ needs.cmt-identity.outputs.tsio-job-configs }}
|
||
run: |
|
||
echo "contexts=$(echo "$JOB_CONFIGS" | jq -r '[.[].status_context] | join(",")')" >> "$GITHUB_OUTPUT"
|
||
|
||
- name: e2e/triage
|
||
continue-on-error: true
|
||
uses: ./.github/actions/e2e-triage
|
||
with:
|
||
job-key: ${{ matrix.job-key }}
|
||
composite-identity: ${{ toJSON(fromJSON(needs.cmt-identity.outputs.tsio-job-configs)[matrix.job-key].composite_identity) }}
|
||
status-context: ${{ fromJSON(needs.cmt-identity.outputs.tsio-job-configs)[matrix.job-key].status_context }}
|
||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||
anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
|
||
triage-lanes: ${{ steps.lanes.outputs.contexts }}
|
||
# A triage failure must never change the outcome of the CMT run itself.
|
||
mode: ${{ vars.E2E_TRIAGE_MODE || 'report-only' }}
|