mattermost-mobile-upstream/.github/workflows/compatibility-matrix-testing.yml
yasser khan 3e1331248c
Some checks are pending
ci / lint-typecheck (push) Waiting to run
ci / detox-lint-typecheck (push) Waiting to run
ci / test (push) Waiting to run
ci(e2e): say whether each main-run failure is new, still failing, flaky or a setup problem (#10284)
* ci(e2e): say whether each main-run failure is new, still failing, flaky or a setup problem, from the triage verdict

The channel message judged failures from their error text, beside the E2E triage
jobs that judge the same failures from TSIO history, and the two disagreed: on
main run 40764686027 it listed tests that passed on retry as needing a look. The
message now waits for triage and reads each run's findings. On main, a test that
passed every earlier main run and fails now is new, with the changes merged since
the last main run named (TSIO's previous run, GitHub's compare); a run where most
tests failed at once is called out first. Runs with no triage verdict keep the
error-text judgement.

* ci(e2e): judge from the error text each failed test triage named no verdict for, and say how many were not checked

* ci(e2e): count each test once per run, so a test that failed and then passed its retry is flaky, not a failure

* ci(e2e): a main-run failure triage calls flaky or a test problem is not blamed on new changes; a same-commit re-run says nothing was merged; a long PR list says it is partial

* Fix TSIO current report selection

Co-authored-by: yasserfaraazkhan <15979783+yasserfaraazkhan@users.noreply.github.com>

* Fix lint comment spacing

Co-authored-by: yasserfaraazkhan <15979783+yasserfaraazkhan@users.noreply.github.com>

* ci(e2e): update the TSIO upload action, so a failed screenshot upload no longer drops a shard's results

* ci(e2e): an earlier attempt of the same main run is not the previous run, so a re-run still lists the changes merged since the last run

* ci(e2e): drop the note from the TSIO upload action pin comment

* ci(e2e): a re-run replaces the first attempt's triage findings, so the channel message never posts stale verdicts

* ci(e2e): the main-run message says in a few lines which failures may be product bugs and which are not, and posts nothing when none may be

* ci(e2e): main and CMT call a failure a product bug only when it fails again on a fresh device, and clear what the error text proves is not the app

* ci(e2e): a re-run hands Android Detox and Maestro results to the channel message too

* ci(e2e): CMT re-runs start after a failed lane; a spec that differs from main, or a feature the server has off, is not a product bug

* ci(e2e): the CMT message plans the re-run as the planner job does, leaving out tests that differ from main

* ci(e2e): one channel post per run, and a test's name can never mention the channel

* ci(e2e): a re-run keeps the run's own report and runs only the failed flows

* ci(e2e): a PR run posts to the channel only when something failed

* ci(e2e): an E2E/Override PR keeps its statuses on a new push; a few simpler lines in the channel notify

* ci(e2e): exclude isolated report-problem flows from reruns and prevent label/action concurrency collisions

<!-- coderabbit-ai-generated:v1 -->

* ci(e2e): pass support paths to E2E triage

* ci(e2e): pin TSIO report-upload to release-0.16.0 (main@3212731)

Brings the TSIO action pin up to the latest release, which includes
mattermost-test-system-io#129, #130 and #124. The report-upload action itself
is unchanged between the two pins.

* ci(e2e): re-run E2E jobs once when a lane is red only on infrastructure

Triage (rerun-attempts: 1, with actions: read to read the failed jobs) asks for
one re-run when a lane is red only because jobs died or reports never arrived;
a build or test-server failure holds it. e2e-tests-rerun.yml carries the re-run
out once the run finishes. CMT keeps its own re-run of failed specs.

---------
2026-11-03 07:45:59 +05:30

1109 lines
50 KiB
YAML
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

name: Compatibility Matrix Testing
on:
workflow_dispatch:
inputs:
CMT_MATRIX:
description: "A JSON object representing the testing matrix"
required: true
type: string
MOBILE_VERSION:
description: "The mattermost mobile version to test"
required: true
# Cancel overlapping CMT runs for the same ref; matterwick cleans up cancelled servers via workflow_run completed.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
# CMT runs against a freshly-cut build-release-NNN branch, so the app content
# never matches main's fingerprint — the S3 build cache would always miss.
# CMT's optimization is structural instead: each app (iOS, Android Detox,
# Android Maestro) is built ONCE below and reused across every server-version
# leg of the matrix via github.run_id artifacts.
jobs:
calculate-commit-hash:
runs-on: ubuntu-24.04
outputs:
MOBILE_SHA: ${{ steps.repo.outputs.MOBILE_SHA }}
steps:
- name: cmt/checkout-mobile
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
ref: ${{ inputs.MOBILE_VERSION }}
- name: cmt/calculate-mattermost-sha
id: repo
run: echo "MOBILE_SHA=$(git rev-parse HEAD)" >> "${GITHUB_OUTPUT}"
# Phase 5 — base TSIO identity for this CMT run. Each matrix leg builds its
# own per-job report group (total_reports_expected=1) from this base + tsio-shard-name.
cmt-identity:
name: Calculate CMT TSIO base identity
runs-on: ubuntu-24.04
permissions:
contents: read
needs:
- calculate-commit-hash
- prepare-maestro-matrix
outputs:
tsio-base-config: ${{ steps.id.outputs.tsio-base-config }}
tsio-job-configs: ${{ steps.id.outputs.tsio-job-configs }}
tsio-job-keys: ${{ steps.id.outputs.tsio-job-keys }}
steps:
- name: Sparse checkout TSIO job-config helper
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
ref: ${{ inputs.MOBILE_VERSION }}
persist-credentials: false
sparse-checkout: |
detox/utils/build-tsio-job-config.js
- id: id
env:
GITHUB_REPOSITORY: ${{ github.repository }}
MM_SHA: ${{ needs.calculate-commit-hash.outputs.MOBILE_SHA }}
MOBILE_VERSION: ${{ inputs.MOBILE_VERSION }}
CMT_MATRIX: ${{ inputs.CMT_MATRIX }}
MAESTRO_MATRIX: ${{ needs.prepare-maestro-matrix.outputs.matrix }}
run: |
BRANCH=$(echo "$MOBILE_VERSION" | sed 's#^refs/heads/##; s#^refs/tags/##')
BASE=$(jq -nc \
--arg repo "$GITHUB_REPOSITORY" --arg sha "$MM_SHA" --arg run_id "$GITHUB_RUN_ID" \
--arg attempt "$GITHUB_RUN_ATTEMPT" --arg branch "$BRANCH" \
'{repository:$repo, commit_sha:$sha, gh_run_id:$run_id, gh_run_attempt:$attempt, branch:$branch, name:"mobile-release", run_group:"mobile-release", framework:"detox"}')
# No status_context — templates expand per tsio-shard-name via buildTsioJobConfig.
TSIO_BASE=$(jq -nc --argjson identity "$BASE" \
'{composite_identity: $identity, total_reports_expected: 1}')
echo "tsio-base-config=$TSIO_BASE" >> "$GITHUB_OUTPUT"
# Per-shard configs so the final rollup can poll every leg's TSIO group.
export BASE CMT_MATRIX MAESTRO_MATRIX
JOB_CONFIGS=$(node -e '
const {buildTsioJobConfigMap, cmtJobKeys} = require("./detox/utils/build-tsio-job-config.js");
const base = JSON.parse(process.env.BASE);
const keys = cmtJobKeys(JSON.parse(process.env.CMT_MATRIX), JSON.parse(process.env.MAESTRO_MATRIX));
process.stdout.write(JSON.stringify(buildTsioJobConfigMap(base, keys)));
')
echo "tsio-job-configs=$JOB_CONFIGS" >> "$GITHUB_OUTPUT"
JOB_KEYS=$(node -e '
const {cmtJobKeys} = require("./detox/utils/build-tsio-job-config.js");
process.stdout.write(JSON.stringify(cmtJobKeys(JSON.parse(process.env.CMT_MATRIX), JSON.parse(process.env.MAESTRO_MATRIX))));
')
echo "tsio-job-keys=$JOB_KEYS" >> "$GITHUB_OUTPUT"
update-initial-status:
runs-on: ubuntu-24.04
permissions:
contents: read
statuses: write
needs:
- calculate-commit-hash
- cmt-identity
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
ref: ${{ inputs.MOBILE_VERSION }}
persist-credentials: false
# Pending statuses are posted per matrix leg when each job starts; keep a
# single umbrella pending only when MOBILE_SHA is known so the PR/commit
# UI shows CMT is running before the first shard uploads.
- name: Build TSIO report URL (release group prefix)
id: tsio-url
env:
TSIO_BASE: ${{ needs.cmt-identity.outputs.tsio-base-config }}
run: |
URL=$(jq -nr --argjson cfg "$TSIO_BASE" --arg base "https://test-io.test.mattermost.com" '
($cfg.composite_identity) as $id
| ($id.repository | split("/") | .[-1]) as $repo
| ($id.branch // "main"
| sub("^refs/heads/"; "")
| sub("^refs/tags/"; "")
| gsub("/"; "~")) as $branch
| ($id.commit_sha[0:7]) as $sha
| "\($base)/reports/\($repo|@uri)/\($branch|@uri)/\($sha)/mobile-release?gh_run_id=\($id.gh_run_id)"
')
echo "target_url=$URL" >> "$GITHUB_OUTPUT"
- uses: ./.github/actions/update-commit-status
if: ${{ needs.calculate-commit-hash.outputs.MOBILE_SHA != '' }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
repository_full_name: ${{ github.repository }}
commit_sha: ${{ needs.calculate-commit-hash.outputs.MOBILE_SHA }}
context: e2e-test/compatibility-matrix-testing
description: "Compatibility Matrix Testing for ${{ inputs.MOBILE_VERSION }} version — per-job checks also posted"
status: pending
target_url: ${{ steps.tsio-url.outputs.target_url }}
# Maestro only runs against the latest server version; matrix filtering happens here since job-level if: can't access matrix.
prepare-maestro-matrix:
runs-on: ubuntu-24.04
outputs:
matrix: ${{ steps.filter.outputs.matrix }}
steps:
- name: Filter CMT matrix to latest server only
id: filter
env:
CMT_MATRIX: ${{ inputs.CMT_MATRIX }}
run: |
matrix=$(echo "$CMT_MATRIX" | jq -c '{server: [.server[] | select(.latest == true)]}')
if [ "$(echo "$matrix" | jq '.server | length')" -eq 0 ]; then
matrix=$(echo "$CMT_MATRIX" | jq -c '{server: [.server[-1]]}')
fi
echo "matrix=$matrix" >> "$GITHUB_OUTPUT"
# Provisions license, config and plugins (demo plugin on full-suite site 1 and site 2) before tests.
# CMT previously skipped that and only used Matterwick-booted servers, so
# interactive_dialog_plugin failed 24× with "Demo plugin is not active".
provision-cmt-servers:
name: Provision CMT test servers
runs-on: ubuntu-24.04
needs:
- calculate-commit-hash
steps:
- name: Checkout Repository
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
ref: ${{ inputs.MOBILE_VERSION }}
persist-credentials: false
- name: Collect unique matrix server URLs
id: urls
env:
CMT_MATRIX: ${{ inputs.CMT_MATRIX }}
run: |
# Full-suite rows run the server-mutation specs, which need their own site 2; fail here, not after the builds.
BAD_ROWS=$(echo "$CMT_MATRIX" | jq -r '
.server[]
| select(.latest == true)
| . as $row
| ["android", "ios"][]
| select(($row[. + "_site_2_url"] // "") == "" or $row[. + "_site_2_url"] == $row[. + "_site_1_url"])
| "\($row.version // "unknown"): \(.)_site_2_url is missing or equals \(.)_site_1_url"
')
if [ -n "$BAD_ROWS" ]; then
echo "::error::Full-suite CMT rows need a distinct site 2 URL per platform:"
printf '%s\n' "$BAD_ROWS"
exit 1
fi
# The demo plugin goes only to full-suite site 1 and site 2 URLs; a shared SITE_3 or a smoke server must not be one of them.
DEMO_CONFLICTS=$(echo "$CMT_MATRIX" | jq -r '
. as $m
| ([ $m.server[] | select(.latest != true) | .android_site_1_url, .android_site_2_url, .ios_site_1_url, .ios_site_2_url, .site_3_url ]
+ [ $m.server[] | select(.latest == true) | .site_3_url ]
| map(select(. != null and . != ""))) as $other
| ([ $m.server[] | select(.latest == true) | .android_site_1_url, .ios_site_1_url ] | map(select(. != null and . != ""))) as $pool
| $m.server[] | select(.latest == true) | . as $row
| ["android", "ios"][] as $p
| ["_site_1_url", "_site_2_url"][] as $k
| $row[$p + $k] as $url
| select($url != null and $url != "" and (($other | any(. == $url)) or ($k == "_site_2_url" and ($pool | any(. == $url)))))
| "\($row.version // "unknown"): \($p)\($k) \($url) is also a pool, shared or smoke server"
')
if [ -n "$DEMO_CONFLICTS" ]; then
echo "::error::These full-suite URLs are also pool, shared or smoke servers (a site 2 must not be any pool site 1, and the demo plugin must not land on a shared or smoke server):"
printf '%s\n' "$DEMO_CONFLICTS"
exit 1
fi
URLS=$(echo "$CMT_MATRIX" | jq -r '
.server[]
| .android_site_1_url, .android_site_2_url, .ios_site_1_url, .ios_site_2_url, .site_3_url
| select(. != null and . != "")
' | awk 'NF && !seen[$0]++')
if [ -z "$URLS" ]; then
echo "::error::CMT_MATRIX contained no server URLs to provision"
exit 1
fi
# Only full-suite rows run the dialog specs, on pool site 1 and mutation site 2 (smoke rows use smoke_test, which has none).
DEMO_URLS=$(echo "$CMT_MATRIX" | jq -r '
.server[]
| select(.latest == true)
| .android_site_1_url, .android_site_2_url, .ios_site_1_url, .ios_site_2_url
| select(. != null and . != "")
' | awk 'NF && !seen[$0]++')
echo "Provisioning $(echo "$URLS" | wc -l | tr -d ' ') unique server(s)"
{
echo "list<<EOF"
printf '%s\n' "$URLS"
echo "EOF"
echo "demo<<EOF"
if [ -n "$DEMO_URLS" ]; then printf '%s\n' "$DEMO_URLS"; fi
echo "EOF"
} >> "$GITHUB_OUTPUT"
- name: Provision test servers
uses: ./.github/actions/provision-e2e-servers
env:
ADMIN_USERNAME: ${{ secrets.MM_MOBILE_E2E_ADMIN_USERNAME }}
ADMIN_PASSWORD: ${{ secrets.MM_MOBILE_E2E_ADMIN_PASSWORD }}
with:
server-urls: ${{ steps.urls.outputs.list }}
demo-plugin-server-urls: ${{ steps.urls.outputs.demo }}
# Input follows the below schema
# {
# "server": [
# {
# "version": "9.6.1",
# "android_site_1_url": "https://android-1.example.test",
# "android_site_2_url": "https://android-2.example.test",
# "ios_site_1_url": "https://ios-1.example.test",
# "ios_site_2_url": "https://ios-2.example.test",
# "site_3_url": "https://shared-3.example.test"
# },
# {
# "version": "9.5.2",
# "android_site_1_url": "https://android-1-older.example.test",
# "android_site_2_url": "https://android-2-older.example.test",
# "ios_site_1_url": "https://ios-1-older.example.test",
# "ios_site_2_url": "https://ios-2-older.example.test",
# "site_3_url": "https://shared-3-older.example.test"
# }
# ]
# }
build-ios-simulator:
# macos-14 predates Xcode 26 and has none pre-installed, so prepare-ios-build's
# Xcode 26 requirement always fails here. e2e-detox-pr.yml and
# e2e-maestro-template.yml already build on macos-26 successfully — match that.
runs-on: macos-26
needs:
- update-initial-status
permissions:
contents: read
actions: write
steps:
- name: Checkout Repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
ref: ${{ inputs.MOBILE_VERSION }}
- name: Prepare iOS Build
uses: ./.github/actions/prepare-ios-build
- name: Build iOS Simulator
env:
TAG: "${{ inputs.MOBILE_VERSION }}"
GITHUB_TOKEN: "${{ secrets.MM_MOBILE_GITHUB_TOKEN }}"
run: bundle exec fastlane ios simulator --env ios.simulator
working-directory: ./fastlane
- name: Upload iOS Simulator Build
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: ios-build-simulator-${{ github.run_id }}
path: Mattermost-simulator-*.app.zip
build-android-apk:
runs-on: ubuntu-latest-8-cores
needs:
- update-initial-status
permissions:
contents: read
actions: write
env:
ORG_GRADLE_PROJECT_jvmargs: -Xmx8g
steps:
- name: Prune Docker to free up space
run: docker system prune -af
- name: Checkout Repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
ref: ${{ inputs.MOBILE_VERSION }}
- name: Prepare Android Build
uses: ./.github/actions/prepare-android-build
env:
STORE_FILE: ${{ secrets.MM_MOBILE_STORE_FILE }}
STORE_ALIAS: ${{ secrets.MM_MOBILE_STORE_ALIAS }}
STORE_PASSWORD: ${{ secrets.MM_MOBILE_STORE_PASSWORD }}
MATTERMOST_BUILD_GH_TOKEN: ${{ secrets.MATTERMOST_BUILD_GH_TOKEN }}
- name: Detox build
run: |
cd detox
npm ci --prefer-offline --no-audit --no-fund
npm run e2e:android-build
- name: Upload Android Build
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: android-build-files-${{ github.run_id }}
# Match e2e-detox-pr.yml's glob shape exactly: the wildcard must start
# right after "build/" so upload-artifact only strips "android/app/build/"
# and preserves the outputs/apk/... subtree. e2e-android-template.yml's
# Download APK artifact step (shared by PR + CMT) expects that shape.
path: "android/app/build/**/*"
build-android-apk-maestro:
runs-on: ubuntu-latest-8-cores
needs:
- update-initial-status
permissions:
contents: read
actions: write
env:
ORG_GRADLE_PROJECT_jvmargs: -Xmx8g
steps:
- name: Prune Docker to free up space
run: docker system prune -af
- name: Checkout Repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
ref: ${{ inputs.MOBILE_VERSION }}
- name: Prepare Android Build
uses: ./.github/actions/prepare-android-build
env:
STORE_FILE: ${{ secrets.MM_MOBILE_STORE_FILE }}
STORE_ALIAS: ${{ secrets.MM_MOBILE_STORE_ALIAS }}
STORE_PASSWORD: ${{ secrets.MM_MOBILE_STORE_PASSWORD }}
MATTERMOST_BUILD_GH_TOKEN: ${{ secrets.MATTERMOST_BUILD_GH_TOKEN }}
- name: Build Maestro Android APK
uses: ./.github/actions/build-android-maestro-apk
- name: Upload Maestro Android Build
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: android-maestro-build-files-${{ github.run_id }}
path: android/app/build/outputs/apk/release/
# latest entry runs full suite, older entries run smoke only. Job-level if: can't conditionally skip matrix legs, so inputs vary instead.
detox-e2e:
name: mobile-cmt-ios${{ matrix.server.latest && '-full' || '-smoke' }}-${{ matrix.server.version }}
permissions:
contents: read
# nested template mints an OIDC token for the TSIO upload.
id-token: write
actions: read
statuses: write
# Nested template finalize reads PR labels to honour E2E/Override; a
# reusable workflow cannot escalate past the caller, so grant it here too.
issues: read
uses: ./.github/workflows/e2e-ios-template.yml
needs:
- build-ios-simulator
- cmt-identity
- provision-cmt-servers
strategy:
fail-fast: false
matrix: ${{ fromJson(inputs.CMT_MATRIX) }}
secrets:
MM_MOBILE_E2E_ADMIN_EMAIL: ${{ secrets.MM_MOBILE_E2E_ADMIN_EMAIL }}
MM_MOBILE_E2E_ADMIN_USERNAME: ${{ secrets.MM_MOBILE_E2E_ADMIN_USERNAME }}
MM_MOBILE_E2E_ADMIN_PASSWORD: ${{ secrets.MM_MOBILE_E2E_ADMIN_PASSWORD }}
MM_MOBILE_E2E_ZEPHYR_API_KEY: ${{ secrets.MM_MOBILE_E2E_ZEPHYR_API_KEY }}
MM_MOBILE_E2E_TEST_CYCLE_LINK_PREFIX: ${{ secrets.MM_MOBILE_E2E_TEST_CYCLE_LINK_PREFIX }}
MM_MOBILE_E2E_WEBHOOK_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_URL }}
MM_E2E_MASTER_HEALTH_WEBHOOK_URL: ${{ secrets.MM_E2E_MASTER_HEALTH_WEBHOOK_URL }}
MM_E2E_RELEASE_WEBHOOK_URL: ${{ secrets.MM_E2E_RELEASE_WEBHOOK_URL }}
MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID: ${{ secrets.MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID }}
MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY: ${{ secrets.MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY }}
MM_MOBILE_E2E_WEBHOOK_PUBLIC_BASE_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_PUBLIC_BASE_URL }}
MM_MOBILE_E2E_CLOUDFLARED_TUNNEL_TOKEN: ${{ secrets.MM_MOBILE_E2E_CLOUDFLARED_TUNNEL_TOKEN }}
with:
tsio-config: ${{ needs.cmt-identity.outputs.tsio-base-config }}
tsio-shard-name: detox-ios-Server_${{ matrix.server.version }}
run-type: "RELEASE"
MM_OTHER_PLATFORM_POOL_URL: ${{ matrix.server.android_site_1_url }}
MM_TEST_SERVER_URL: ${{ matrix.server.ios_site_1_url }}
MM_TEST_SERVER_URL_2: ${{ matrix.server.ios_site_2_url }}
MM_TEST_SERVER_URL_3: ${{ matrix.server.site_3_url }}
MOBILE_VERSION: ${{ inputs.MOBILE_VERSION }}
search_path: ${{ matrix.server.latest && 'detox/e2e/test' || 'detox/e2e/test/products/channels/smoke_test' }}
# Full suite matches the PR/main shard count (template default 32); smoke stays single-shard.
parallelism: ${{ matrix.server.latest && '32' || '1' }}
artifact_run_id: ${{ github.run_id }}
detox-ipad-e2e:
name: mobile-cmt-ipad-${{ matrix.server.version }}
permissions:
contents: read
id-token: write
actions: read
statuses: write
# Nested template finalize reads PR labels to honour E2E/Override; a
# reusable workflow cannot escalate past the caller, so grant it here too.
issues: read
uses: ./.github/workflows/e2e-ios-template.yml
needs:
- build-ios-simulator
- cmt-identity
- provision-cmt-servers
strategy:
fail-fast: false
matrix: ${{ fromJson(inputs.CMT_MATRIX) }}
secrets:
MM_MOBILE_E2E_ADMIN_EMAIL: ${{ secrets.MM_MOBILE_E2E_ADMIN_EMAIL }}
MM_MOBILE_E2E_ADMIN_USERNAME: ${{ secrets.MM_MOBILE_E2E_ADMIN_USERNAME }}
MM_MOBILE_E2E_ADMIN_PASSWORD: ${{ secrets.MM_MOBILE_E2E_ADMIN_PASSWORD }}
MM_MOBILE_E2E_ZEPHYR_API_KEY: ${{ secrets.MM_MOBILE_E2E_ZEPHYR_API_KEY }}
MM_MOBILE_E2E_TEST_CYCLE_LINK_PREFIX: ${{ secrets.MM_MOBILE_E2E_TEST_CYCLE_LINK_PREFIX }}
MM_MOBILE_E2E_WEBHOOK_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_URL }}
MM_E2E_MASTER_HEALTH_WEBHOOK_URL: ${{ secrets.MM_E2E_MASTER_HEALTH_WEBHOOK_URL }}
MM_E2E_RELEASE_WEBHOOK_URL: ${{ secrets.MM_E2E_RELEASE_WEBHOOK_URL }}
MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID: ${{ secrets.MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID }}
MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY: ${{ secrets.MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY }}
MM_MOBILE_E2E_WEBHOOK_PUBLIC_BASE_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_PUBLIC_BASE_URL }}
MM_MOBILE_E2E_CLOUDFLARED_TUNNEL_TOKEN: ${{ secrets.MM_MOBILE_E2E_CLOUDFLARED_TUNNEL_TOKEN }}
with:
tsio-config: ${{ needs.cmt-identity.outputs.tsio-base-config }}
tsio-shard-name: detox-ipad-Server_${{ matrix.server.version }}
run-type: "RELEASE"
MM_OTHER_PLATFORM_POOL_URL: ${{ matrix.server.android_site_1_url }}
MM_TEST_SERVER_URL: ${{ matrix.server.ios_site_1_url }}
MM_TEST_SERVER_URL_2: ${{ matrix.server.ios_site_2_url }}
MM_TEST_SERVER_URL_3: ${{ matrix.server.site_3_url }}
MOBILE_VERSION: ${{ inputs.MOBILE_VERSION }}
ios_device_name: "iPad Pro 13-inch (M5)"
ios_device_os_name: "iOS 26.2"
search_path: "detox/e2e/test/products/channels/ipad"
parallelism: "1"
detox_config: "ios.ipad.debug"
artifact_run_id: ${{ github.run_id }}
# Maestro runs only against the latest server version (no smoke-on-older variant).
maestro-ios-e2e:
name: mobile-cmt-maestro-ios-${{ matrix.server.version }}
permissions:
contents: read
id-token: write
actions: read
statuses: write
# Nested template finalize reads PR labels to honour E2E/Override; a
# reusable workflow cannot escalate past the caller, so grant it here too.
issues: read
uses: ./.github/workflows/e2e-maestro-template.yml
needs:
- build-ios-simulator
- prepare-maestro-matrix
- cmt-identity
- provision-cmt-servers
strategy:
fail-fast: false
matrix: ${{ fromJson(needs.prepare-maestro-matrix.outputs.matrix) }}
secrets:
MM_MOBILE_E2E_ADMIN_EMAIL: ${{ secrets.MM_MOBILE_E2E_ADMIN_EMAIL }}
MM_MOBILE_E2E_ADMIN_USERNAME: ${{ secrets.MM_MOBILE_E2E_ADMIN_USERNAME }}
MM_MOBILE_E2E_ADMIN_PASSWORD: ${{ secrets.MM_MOBILE_E2E_ADMIN_PASSWORD }}
MM_MOBILE_E2E_ZEPHYR_API_KEY: ${{ secrets.MM_MOBILE_E2E_ZEPHYR_API_KEY }}
MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID: ${{ secrets.MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID }}
MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY: ${{ secrets.MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY }}
MM_MOBILE_E2E_WEBHOOK_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_URL }}
MM_E2E_MASTER_HEALTH_WEBHOOK_URL: ${{ secrets.MM_E2E_MASTER_HEALTH_WEBHOOK_URL }}
MM_E2E_RELEASE_WEBHOOK_URL: ${{ secrets.MM_E2E_RELEASE_WEBHOOK_URL }}
with:
tsio-config: ${{ needs.cmt-identity.outputs.tsio-base-config }}
tsio-shard-name: maestro-ios-Server_${{ matrix.server.version }}
platform: ios
run-type: "RELEASE"
MM_TEST_SERVER_URL: ${{ matrix.server.ios_site_1_url }}
MOBILE_VERSION: ${{ inputs.MOBILE_VERSION }}
record_tests_in_zephyr: true
artifact_run_id: ${{ github.run_id }}
detox-android-e2e:
name: mobile-cmt-android${{ matrix.server.latest && '-full' || '-smoke' }}-${{ matrix.server.version }}
permissions:
contents: read
id-token: write
actions: read
statuses: write
# Nested template finalize reads PR labels to honour E2E/Override; a
# reusable workflow cannot escalate past the caller, so grant it here too.
issues: read
uses: ./.github/workflows/e2e-android-template.yml
needs:
- build-android-apk
- cmt-identity
- provision-cmt-servers
strategy:
fail-fast: false
matrix: ${{ fromJson(inputs.CMT_MATRIX) }}
secrets:
MM_MOBILE_E2E_ADMIN_EMAIL: ${{ secrets.MM_MOBILE_E2E_ADMIN_EMAIL }}
MM_MOBILE_E2E_ADMIN_USERNAME: ${{ secrets.MM_MOBILE_E2E_ADMIN_USERNAME }}
MM_MOBILE_E2E_ADMIN_PASSWORD: ${{ secrets.MM_MOBILE_E2E_ADMIN_PASSWORD }}
MM_MOBILE_E2E_ZEPHYR_API_KEY: ${{ secrets.MM_MOBILE_E2E_ZEPHYR_API_KEY }}
MM_MOBILE_E2E_TEST_CYCLE_LINK_PREFIX: ${{ secrets.MM_MOBILE_E2E_TEST_CYCLE_LINK_PREFIX }}
MM_MOBILE_E2E_WEBHOOK_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_URL }}
MM_E2E_MASTER_HEALTH_WEBHOOK_URL: ${{ secrets.MM_E2E_MASTER_HEALTH_WEBHOOK_URL }}
MM_E2E_RELEASE_WEBHOOK_URL: ${{ secrets.MM_E2E_RELEASE_WEBHOOK_URL }}
MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID: ${{ secrets.MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID }}
MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY: ${{ secrets.MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY }}
MM_MOBILE_E2E_WEBHOOK_PUBLIC_BASE_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_PUBLIC_BASE_URL }}
MM_MOBILE_E2E_CLOUDFLARED_TUNNEL_TOKEN: ${{ secrets.MM_MOBILE_E2E_CLOUDFLARED_TUNNEL_TOKEN }}
with:
tsio-config: ${{ needs.cmt-identity.outputs.tsio-base-config }}
tsio-shard-name: detox-android-Server_${{ matrix.server.version }}
run-android-tests: true
run-type: "RELEASE"
MM_OTHER_PLATFORM_POOL_URL: ${{ matrix.server.ios_site_1_url }}
MM_TEST_SERVER_URL: ${{ matrix.server.android_site_1_url }}
MM_TEST_SERVER_URL_2: ${{ matrix.server.android_site_2_url }}
MM_TEST_SERVER_URL_3: ${{ matrix.server.site_3_url }}
MOBILE_VERSION: ${{ inputs.MOBILE_VERSION }}
search_path: ${{ matrix.server.latest && 'detox/e2e/test' || 'detox/e2e/test/products/channels/smoke_test' }}
# Full suite matches the PR/main shard count (template default 32); smoke stays single-shard.
parallelism: ${{ matrix.server.latest && '32' || '1' }}
artifact_run_id: ${{ github.run_id }}
# Maestro runs only against the latest server version (no smoke-on-older variant).
maestro-android-e2e:
name: mobile-cmt-maestro-android-${{ matrix.server.version }}
permissions:
contents: read
id-token: write
actions: read
statuses: write
# Nested template finalize reads PR labels to honour E2E/Override; a
# reusable workflow cannot escalate past the caller, so grant it here too.
issues: read
uses: ./.github/workflows/e2e-maestro-template.yml
needs:
- build-android-apk-maestro
- prepare-maestro-matrix
- cmt-identity
- provision-cmt-servers
strategy:
fail-fast: false
matrix: ${{ fromJson(needs.prepare-maestro-matrix.outputs.matrix) }}
secrets:
MM_MOBILE_E2E_ADMIN_EMAIL: ${{ secrets.MM_MOBILE_E2E_ADMIN_EMAIL }}
MM_MOBILE_E2E_ADMIN_USERNAME: ${{ secrets.MM_MOBILE_E2E_ADMIN_USERNAME }}
MM_MOBILE_E2E_ADMIN_PASSWORD: ${{ secrets.MM_MOBILE_E2E_ADMIN_PASSWORD }}
MM_MOBILE_E2E_ZEPHYR_API_KEY: ${{ secrets.MM_MOBILE_E2E_ZEPHYR_API_KEY }}
MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID: ${{ secrets.MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID }}
MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY: ${{ secrets.MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY }}
MM_MOBILE_E2E_WEBHOOK_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_URL }}
MM_E2E_MASTER_HEALTH_WEBHOOK_URL: ${{ secrets.MM_E2E_MASTER_HEALTH_WEBHOOK_URL }}
MM_E2E_RELEASE_WEBHOOK_URL: ${{ secrets.MM_E2E_RELEASE_WEBHOOK_URL }}
with:
tsio-config: ${{ needs.cmt-identity.outputs.tsio-base-config }}
tsio-shard-name: maestro-android-Server_${{ matrix.server.version }}
platform: android
run-type: "RELEASE"
MM_TEST_SERVER_URL: ${{ matrix.server.android_site_1_url }}
MOBILE_VERSION: ${{ inputs.MOBILE_VERSION }}
artifact_run_id: ${{ github.run_id }}
record_tests_in_zephyr: true
update-final-status:
runs-on: ubuntu-24.04
if: always()
permissions:
contents: read
statuses: write
outputs:
upstream-succeeded: ${{ steps.upstream.outputs.succeeded }}
needs:
- cmt-identity
- calculate-commit-hash
- provision-cmt-servers
- detox-e2e
- detox-android-e2e
- detox-ipad-e2e
- maestro-ios-e2e
- maestro-android-e2e
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
ref: ${{ inputs.MOBILE_VERSION }}
persist-credentials: false
- name: Resolve upstream matrix outcome
id: upstream
run: |
UPSTREAM="true"
for r in "${{ needs.provision-cmt-servers.result }}" "${{ needs.detox-e2e.result }}" "${{ needs.detox-android-e2e.result }}" "${{ needs.detox-ipad-e2e.result }}" "${{ needs.maestro-ios-e2e.result }}" "${{ needs.maestro-android-e2e.result }}"; do
case "$r" in
success|skipped) ;;
*) UPSTREAM="false" ;;
esac
done
echo "succeeded=$UPSTREAM" >> "$GITHUB_OUTPUT"
# Umbrella context only — each matrix leg already posted <shard> from TSIO.
- name: Finalize umbrella CMT commit status from CI job results
env:
GITHUB_TOKEN: ${{ github.token }}
COMMIT_SHA: ${{ needs.calculate-commit-hash.outputs.MOBILE_SHA }}
REPO: ${{ github.repository }}
TARGET_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
UPSTREAM: ${{ steps.upstream.outputs.succeeded }}
run: |
if [ "$UPSTREAM" = "true" ]; then
STATE=success
DESC="Compatibility Matrix Testing completed — see per-job e2e-test/* checks"
else
STATE=failure
DESC="Compatibility Matrix Testing failed — see per-job e2e-test/* checks"
fi
gh api --method POST "repos/${REPO}/statuses/${COMMIT_SHA}" \
-f state="$STATE" \
-f context="e2e-test/compatibility-matrix-testing" \
-f description="$DESC" \
-f target_url="$TARGET_URL"
# Per-shard TSIO groups are finalized in each matrix leg, which leaves channel
# notify off so N legs do not spam. This job polls every shard group and posts
# one summary — same path as PR/main. After triage, so the message carries its
# verdict on each failed test.
# Re-run, on fresh devices, the failed specs triage could not clear, against the same server version, so the
# channel message can say for certain whether each is a product bug (see e2e-detox-pr.yml plan-rerun).
plan-rerun-cmt:
name: Plan re-run of failed specs
runs-on: ubuntu-24.04
if: '!cancelled() && needs.tsio-triage.result != ''skipped'''
needs:
- tsio-triage
permissions:
contents: read
outputs:
detox-ios-matrix: ${{ steps.plan.outputs.detox-ios-matrix }}
detox-android-matrix: ${{ steps.plan.outputs.detox-android-matrix }}
detox-ipad-matrix: ${{ steps.plan.outputs.detox-ipad-matrix }}
maestro-ios-matrix: ${{ steps.plan.outputs.maestro-ios-matrix }}
maestro-android-matrix: ${{ steps.plan.outputs.maestro-android-matrix }}
stale: ${{ steps.plan.outputs.stale }}
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
ref: ${{ inputs.MOBILE_VERSION }}
persist-credentials: false
sparse-checkout: |
detox/utils/rerun-failed.js
detox/utils/tsio-channel-notify-rollup.js
detox/utils/tsio-report-status.js
detox/utils/cmt-channel-notify.js
detox/utils/build-tsio-job-config.js
detox/utils/cli-args.js
- name: Download triage findings
continue-on-error: true
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: e2e-triage-findings-*-a${{ github.run_attempt }}
path: triage-findings
# A spec that differs from main's is out of date on this release branch: it is not re-run on the old build.
- name: Fetch main to compare specs
continue-on-error: true
run: git fetch --no-tags --depth=1 origin main
- name: Plan the re-run
id: plan
env:
CMT_MATRIX: ${{ inputs.CMT_MATRIX }}
run: node detox/utils/rerun-failed.js --findings-dir triage-findings --cmt-matrix "$CMT_MATRIX" --compare-ref FETCH_HEAD
rerun-detox-e2e:
name: rerun-mobile-cmt-ios${{ matrix.server.latest && '-full' || '-smoke' }}-${{ matrix.server.version }}
# !cancelled(): a lane failed upstream (that is why there is a re-run), which would skip this job otherwise.
if: >-
!cancelled() &&
needs.plan-rerun-cmt.result == 'success' &&
needs.build-ios-simulator.result == 'success' &&
needs.provision-cmt-servers.result == 'success' &&
fromJson(needs.plan-rerun-cmt.outputs.detox-ios-matrix).server[0] != null
permissions:
contents: read
# nested template mints an OIDC token for the TSIO upload.
id-token: write
actions: read
statuses: write
# Nested template finalize reads PR labels to honour E2E/Override; a
# reusable workflow cannot escalate past the caller, so grant it here too.
issues: read
uses: ./.github/workflows/e2e-ios-template.yml
needs:
- plan-rerun-cmt
- build-ios-simulator
- cmt-identity
- provision-cmt-servers
strategy:
fail-fast: false
matrix: ${{ fromJson(needs.plan-rerun-cmt.outputs.detox-ios-matrix) }}
secrets:
MM_MOBILE_E2E_ADMIN_EMAIL: ${{ secrets.MM_MOBILE_E2E_ADMIN_EMAIL }}
MM_MOBILE_E2E_ADMIN_USERNAME: ${{ secrets.MM_MOBILE_E2E_ADMIN_USERNAME }}
MM_MOBILE_E2E_ADMIN_PASSWORD: ${{ secrets.MM_MOBILE_E2E_ADMIN_PASSWORD }}
MM_MOBILE_E2E_ZEPHYR_API_KEY: ${{ secrets.MM_MOBILE_E2E_ZEPHYR_API_KEY }}
MM_MOBILE_E2E_TEST_CYCLE_LINK_PREFIX: ${{ secrets.MM_MOBILE_E2E_TEST_CYCLE_LINK_PREFIX }}
MM_MOBILE_E2E_WEBHOOK_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_URL }}
MM_E2E_MASTER_HEALTH_WEBHOOK_URL: ${{ secrets.MM_E2E_MASTER_HEALTH_WEBHOOK_URL }}
MM_E2E_RELEASE_WEBHOOK_URL: ${{ secrets.MM_E2E_RELEASE_WEBHOOK_URL }}
MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID: ${{ secrets.MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID }}
MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY: ${{ secrets.MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY }}
MM_MOBILE_E2E_WEBHOOK_PUBLIC_BASE_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_PUBLIC_BASE_URL }}
MM_MOBILE_E2E_CLOUDFLARED_TUNNEL_TOKEN: ${{ secrets.MM_MOBILE_E2E_CLOUDFLARED_TUNNEL_TOKEN }}
with:
tsio-config: ''
tsio-shard-name: ''
run-type: "NONE"
MM_OTHER_PLATFORM_POOL_URL: ${{ matrix.server.android_site_1_url }}
MM_TEST_SERVER_URL: ${{ matrix.server.ios_site_1_url }}
MM_TEST_SERVER_URL_2: ${{ matrix.server.ios_site_2_url }}
MM_TEST_SERVER_URL_3: ${{ matrix.server.site_3_url }}
MOBILE_VERSION: ${{ inputs.MOBILE_VERSION }}
search_path: ${{ matrix.server.latest && 'detox/e2e/test' || 'detox/e2e/test/products/channels/smoke_test' }}
# Full suite matches the PR/main shard count (template default 32); smoke stays single-shard.
parallelism: '1'
artifact_run_id: ${{ github.run_id }}
spec_files: ${{ matrix.server.specs }}
results-artifact: ${{ format('e2e-rerun-results-detox-ios-Server_{0}-a{1}', matrix.server.version, github.run_attempt) }}
rerun-detox-ipad-e2e:
name: rerun-mobile-cmt-ipad-${{ matrix.server.version }}
# !cancelled(): a lane failed upstream (that is why there is a re-run), which would skip this job otherwise.
if: >-
!cancelled() &&
needs.plan-rerun-cmt.result == 'success' &&
needs.build-ios-simulator.result == 'success' &&
needs.provision-cmt-servers.result == 'success' &&
fromJson(needs.plan-rerun-cmt.outputs.detox-ipad-matrix).server[0] != null
permissions:
contents: read
id-token: write
actions: read
statuses: write
# Nested template finalize reads PR labels to honour E2E/Override; a
# reusable workflow cannot escalate past the caller, so grant it here too.
issues: read
uses: ./.github/workflows/e2e-ios-template.yml
needs:
- plan-rerun-cmt
- build-ios-simulator
- cmt-identity
- provision-cmt-servers
strategy:
fail-fast: false
matrix: ${{ fromJson(needs.plan-rerun-cmt.outputs.detox-ipad-matrix) }}
secrets:
MM_MOBILE_E2E_ADMIN_EMAIL: ${{ secrets.MM_MOBILE_E2E_ADMIN_EMAIL }}
MM_MOBILE_E2E_ADMIN_USERNAME: ${{ secrets.MM_MOBILE_E2E_ADMIN_USERNAME }}
MM_MOBILE_E2E_ADMIN_PASSWORD: ${{ secrets.MM_MOBILE_E2E_ADMIN_PASSWORD }}
MM_MOBILE_E2E_ZEPHYR_API_KEY: ${{ secrets.MM_MOBILE_E2E_ZEPHYR_API_KEY }}
MM_MOBILE_E2E_TEST_CYCLE_LINK_PREFIX: ${{ secrets.MM_MOBILE_E2E_TEST_CYCLE_LINK_PREFIX }}
MM_MOBILE_E2E_WEBHOOK_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_URL }}
MM_E2E_MASTER_HEALTH_WEBHOOK_URL: ${{ secrets.MM_E2E_MASTER_HEALTH_WEBHOOK_URL }}
MM_E2E_RELEASE_WEBHOOK_URL: ${{ secrets.MM_E2E_RELEASE_WEBHOOK_URL }}
MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID: ${{ secrets.MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID }}
MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY: ${{ secrets.MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY }}
MM_MOBILE_E2E_WEBHOOK_PUBLIC_BASE_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_PUBLIC_BASE_URL }}
MM_MOBILE_E2E_CLOUDFLARED_TUNNEL_TOKEN: ${{ secrets.MM_MOBILE_E2E_CLOUDFLARED_TUNNEL_TOKEN }}
with:
tsio-config: ''
tsio-shard-name: ''
run-type: "NONE"
MM_OTHER_PLATFORM_POOL_URL: ${{ matrix.server.android_site_1_url }}
MM_TEST_SERVER_URL: ${{ matrix.server.ios_site_1_url }}
MM_TEST_SERVER_URL_2: ${{ matrix.server.ios_site_2_url }}
MM_TEST_SERVER_URL_3: ${{ matrix.server.site_3_url }}
MOBILE_VERSION: ${{ inputs.MOBILE_VERSION }}
ios_device_name: "iPad Pro 13-inch (M5)"
ios_device_os_name: "iOS 26.2"
search_path: "detox/e2e/test/products/channels/ipad"
parallelism: '1'
detox_config: "ios.ipad.debug"
artifact_run_id: ${{ github.run_id }}
spec_files: ${{ matrix.server.specs }}
results-artifact: ${{ format('e2e-rerun-results-detox-ipad-Server_{0}-a{1}', matrix.server.version, github.run_attempt) }}
# Maestro runs only against the latest server version (no smoke-on-older variant).
rerun-detox-android-e2e:
name: rerun-mobile-cmt-android${{ matrix.server.latest && '-full' || '-smoke' }}-${{ matrix.server.version }}
# !cancelled(): a lane failed upstream (that is why there is a re-run), which would skip this job otherwise.
if: >-
!cancelled() &&
needs.plan-rerun-cmt.result == 'success' &&
needs.build-android-apk.result == 'success' &&
needs.provision-cmt-servers.result == 'success' &&
fromJson(needs.plan-rerun-cmt.outputs.detox-android-matrix).server[0] != null
permissions:
contents: read
id-token: write
actions: read
statuses: write
# Nested template finalize reads PR labels to honour E2E/Override; a
# reusable workflow cannot escalate past the caller, so grant it here too.
issues: read
uses: ./.github/workflows/e2e-android-template.yml
needs:
- plan-rerun-cmt
- build-android-apk
- cmt-identity
- provision-cmt-servers
strategy:
fail-fast: false
matrix: ${{ fromJson(needs.plan-rerun-cmt.outputs.detox-android-matrix) }}
secrets:
MM_MOBILE_E2E_ADMIN_EMAIL: ${{ secrets.MM_MOBILE_E2E_ADMIN_EMAIL }}
MM_MOBILE_E2E_ADMIN_USERNAME: ${{ secrets.MM_MOBILE_E2E_ADMIN_USERNAME }}
MM_MOBILE_E2E_ADMIN_PASSWORD: ${{ secrets.MM_MOBILE_E2E_ADMIN_PASSWORD }}
MM_MOBILE_E2E_ZEPHYR_API_KEY: ${{ secrets.MM_MOBILE_E2E_ZEPHYR_API_KEY }}
MM_MOBILE_E2E_TEST_CYCLE_LINK_PREFIX: ${{ secrets.MM_MOBILE_E2E_TEST_CYCLE_LINK_PREFIX }}
MM_MOBILE_E2E_WEBHOOK_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_URL }}
MM_E2E_MASTER_HEALTH_WEBHOOK_URL: ${{ secrets.MM_E2E_MASTER_HEALTH_WEBHOOK_URL }}
MM_E2E_RELEASE_WEBHOOK_URL: ${{ secrets.MM_E2E_RELEASE_WEBHOOK_URL }}
MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID: ${{ secrets.MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID }}
MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY: ${{ secrets.MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY }}
MM_MOBILE_E2E_WEBHOOK_PUBLIC_BASE_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_PUBLIC_BASE_URL }}
MM_MOBILE_E2E_CLOUDFLARED_TUNNEL_TOKEN: ${{ secrets.MM_MOBILE_E2E_CLOUDFLARED_TUNNEL_TOKEN }}
with:
tsio-config: ''
tsio-shard-name: ''
run-android-tests: true
run-type: "NONE"
MM_OTHER_PLATFORM_POOL_URL: ${{ matrix.server.ios_site_1_url }}
MM_TEST_SERVER_URL: ${{ matrix.server.android_site_1_url }}
MM_TEST_SERVER_URL_2: ${{ matrix.server.android_site_2_url }}
MM_TEST_SERVER_URL_3: ${{ matrix.server.site_3_url }}
MOBILE_VERSION: ${{ inputs.MOBILE_VERSION }}
search_path: ${{ matrix.server.latest && 'detox/e2e/test' || 'detox/e2e/test/products/channels/smoke_test' }}
# Full suite matches the PR/main shard count (template default 32); smoke stays single-shard.
parallelism: '1'
artifact_run_id: ${{ github.run_id }}
spec_files: ${{ matrix.server.specs }}
results-artifact: ${{ format('e2e-rerun-results-detox-android-Server_{0}-a{1}', matrix.server.version, github.run_attempt) }}
# Maestro runs only against the latest server version (no smoke-on-older variant).
rerun-maestro-ios-e2e:
name: rerun-mobile-cmt-maestro-ios-${{ matrix.server.version }}
# !cancelled(): a lane failed upstream (that is why there is a re-run), which would skip this job otherwise.
if: >-
!cancelled() &&
needs.plan-rerun-cmt.result == 'success' &&
needs.build-ios-simulator.result == 'success' &&
needs.provision-cmt-servers.result == 'success' &&
fromJson(needs.plan-rerun-cmt.outputs.maestro-ios-matrix).server[0] != null
permissions:
contents: read
id-token: write
actions: read
statuses: write
# Nested template finalize reads PR labels to honour E2E/Override; a
# reusable workflow cannot escalate past the caller, so grant it here too.
issues: read
uses: ./.github/workflows/e2e-maestro-template.yml
needs:
- plan-rerun-cmt
- build-ios-simulator
- prepare-maestro-matrix
- cmt-identity
- provision-cmt-servers
strategy:
fail-fast: false
matrix: ${{ fromJson(needs.plan-rerun-cmt.outputs.maestro-ios-matrix) }}
secrets:
MM_MOBILE_E2E_ADMIN_EMAIL: ${{ secrets.MM_MOBILE_E2E_ADMIN_EMAIL }}
MM_MOBILE_E2E_ADMIN_USERNAME: ${{ secrets.MM_MOBILE_E2E_ADMIN_USERNAME }}
MM_MOBILE_E2E_ADMIN_PASSWORD: ${{ secrets.MM_MOBILE_E2E_ADMIN_PASSWORD }}
MM_MOBILE_E2E_ZEPHYR_API_KEY: ${{ secrets.MM_MOBILE_E2E_ZEPHYR_API_KEY }}
MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID: ${{ secrets.MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID }}
MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY: ${{ secrets.MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY }}
MM_MOBILE_E2E_WEBHOOK_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_URL }}
MM_E2E_MASTER_HEALTH_WEBHOOK_URL: ${{ secrets.MM_E2E_MASTER_HEALTH_WEBHOOK_URL }}
MM_E2E_RELEASE_WEBHOOK_URL: ${{ secrets.MM_E2E_RELEASE_WEBHOOK_URL }}
with:
tsio-config: ''
tsio-shard-name: ''
platform: ios
run-type: "NONE"
MM_TEST_SERVER_URL: ${{ matrix.server.ios_site_1_url }}
MOBILE_VERSION: ${{ inputs.MOBILE_VERSION }}
record_tests_in_zephyr: false
artifact_run_id: ${{ github.run_id }}
flow-path: ${{ matrix.server.specs }}
results-artifact: ${{ format('e2e-rerun-results-maestro-ios-Server_{0}-a{1}', matrix.server.version, github.run_attempt) }}
rerun-maestro-android-e2e:
name: rerun-mobile-cmt-maestro-android-${{ matrix.server.version }}
# !cancelled(): a lane failed upstream (that is why there is a re-run), which would skip this job otherwise.
if: >-
!cancelled() &&
needs.plan-rerun-cmt.result == 'success' &&
needs.build-android-apk-maestro.result == 'success' &&
needs.provision-cmt-servers.result == 'success' &&
fromJson(needs.plan-rerun-cmt.outputs.maestro-android-matrix).server[0] != null
permissions:
contents: read
id-token: write
actions: read
statuses: write
# Nested template finalize reads PR labels to honour E2E/Override; a
# reusable workflow cannot escalate past the caller, so grant it here too.
issues: read
uses: ./.github/workflows/e2e-maestro-template.yml
needs:
- plan-rerun-cmt
- build-android-apk-maestro
- prepare-maestro-matrix
- cmt-identity
- provision-cmt-servers
strategy:
fail-fast: false
matrix: ${{ fromJson(needs.plan-rerun-cmt.outputs.maestro-android-matrix) }}
secrets:
MM_MOBILE_E2E_ADMIN_EMAIL: ${{ secrets.MM_MOBILE_E2E_ADMIN_EMAIL }}
MM_MOBILE_E2E_ADMIN_USERNAME: ${{ secrets.MM_MOBILE_E2E_ADMIN_USERNAME }}
MM_MOBILE_E2E_ADMIN_PASSWORD: ${{ secrets.MM_MOBILE_E2E_ADMIN_PASSWORD }}
MM_MOBILE_E2E_ZEPHYR_API_KEY: ${{ secrets.MM_MOBILE_E2E_ZEPHYR_API_KEY }}
MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID: ${{ secrets.MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID }}
MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY: ${{ secrets.MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY }}
MM_MOBILE_E2E_WEBHOOK_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_URL }}
MM_E2E_MASTER_HEALTH_WEBHOOK_URL: ${{ secrets.MM_E2E_MASTER_HEALTH_WEBHOOK_URL }}
MM_E2E_RELEASE_WEBHOOK_URL: ${{ secrets.MM_E2E_RELEASE_WEBHOOK_URL }}
with:
tsio-config: ''
tsio-shard-name: ''
platform: android
run-type: "NONE"
MM_TEST_SERVER_URL: ${{ matrix.server.android_site_1_url }}
MOBILE_VERSION: ${{ inputs.MOBILE_VERSION }}
artifact_run_id: ${{ github.run_id }}
flow-path: ${{ matrix.server.specs }}
results-artifact: ${{ format('e2e-rerun-results-maestro-android-Server_{0}-a{1}', matrix.server.version, github.run_attempt) }}
record_tests_in_zephyr: false
tsio-channel-notify:
name: Post Mattermost CMT channel rollup
runs-on: ubuntu-24.04
if: always() && needs.cmt-identity.result == 'success'
needs:
- cmt-identity
- update-final-status
- tsio-triage
- rerun-detox-e2e
- rerun-detox-ipad-e2e
- rerun-detox-android-e2e
- rerun-maestro-ios-e2e
- rerun-maestro-android-e2e
- plan-rerun-cmt
permissions:
contents: read
# the channel rollup mints an OIDC token to read each shard's TSIO group.
id-token: write
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
ref: ${{ inputs.MOBILE_VERSION }}
persist-credentials: false
# This attempt's triage verdicts (none when no shard failed, or triage did not finish).
- name: Download triage findings
continue-on-error: true
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: e2e-triage-findings-*-a${{ github.run_attempt }}
path: triage-findings
# The re-run's result per test (none when nothing was re-run).
- name: Download re-run results
continue-on-error: true
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: e2e-rerun-results-*-a${{ github.run_attempt }}
path: rerun-results
- name: Post Mattermost CMT channel rollup
env:
JOB_CONFIGS: ${{ needs.cmt-identity.outputs.tsio-job-configs }}
TRIAGE_FINDINGS_DIR: triage-findings
RERUN_RESULTS_DIR: rerun-results
RERUN_STALE: ${{ needs.plan-rerun-cmt.outputs.stale || '{}' }}
MATTERMOST_CMT_WEBHOOK_URL: ${{ secrets.MM_E2E_RELEASE_WEBHOOK_URL }}
UPSTREAM_SUCCEEDED: ${{ needs.update-final-status.outputs.upstream-succeeded || 'false' }}
run: |
node detox/utils/tsio-channel-notify-rollup.js \
--job-configs="$JOB_CONFIGS" \
--upstream-succeeded="$UPSTREAM_SUCCEEDED" \
--poll-attempts=6 || true
# Instance cleanup is handled by matterwick on workflow_run completed.
# Same per-test judging the PR/main lanes get in e2e-detox-pr.yml: without it
# every CMT shard is raw any-failure-red (FAILURES > 0 in the template summary
# jobs), so a handful of known-flaky failures out of ~650 tests per leg turned
# every CMT run red. Report-only (the default) only posts the verdict;
# E2E_TRIAGE_MODE=enforce makes the verdict the shard's commit status, exactly
# as on PR/main. The workflow run itself stays red on real failures either way.
tsio-triage:
name: Triage CMT failures (${{ matrix.job-key }})
runs-on: ubuntu-24.04
# The channel message waits for this job: a hang must not hold it for hours (recent runs take under a minute).
timeout-minutes: 15
if: >-
!cancelled() &&
(needs.detox-e2e.result == 'failure' ||
needs.detox-android-e2e.result == 'failure' ||
needs.detox-ipad-e2e.result == 'failure' ||
needs.maestro-ios-e2e.result == 'failure' ||
needs.maestro-android-e2e.result == 'failure')
needs:
- calculate-commit-hash
- cmt-identity
- detox-e2e
- detox-android-e2e
- detox-ipad-e2e
- maestro-ios-e2e
- maestro-android-e2e
permissions:
contents: read
statuses: write
pull-requests: read
# Every lane writes the shared e2e-test/triage context, and the toolkit's
# status publisher has no compare-and-swap: its status.ts requires every
# caller to hold the same non-cancelling, SHA-scoped job concurrency lock
# across its reads and writes. max-parallel serializes the matrix entries
# within this run without discarding queued entries; the concurrency group
# serializes against triage writers from any other run of the same commit.
# The workflow-level cancel-in-progress group above does neither.
concurrency:
group: e2e-test-triage-${{ needs.calculate-commit-hash.outputs.MOBILE_SHA }}
cancel-in-progress: false
strategy:
fail-fast: false
max-parallel: 1
matrix:
job-key: ${{ fromJSON(needs.cmt-identity.outputs.tsio-job-keys) }}
steps:
# Checked out from the workflow ref (the build-release-* branch under
# test runs no code with the API key beyond this composite's own steps,
# which are pinned actions; same pattern as e2e-detox-pr.yml).
- name: Checkout triage action
continue-on-error: true
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
sparse-checkout: .github/actions/e2e-triage/
sparse-checkout-cone-mode: false
- name: Resolve CMT triage lanes
id: lanes
env:
JOB_CONFIGS: ${{ needs.cmt-identity.outputs.tsio-job-configs }}
run: |
echo "contexts=$(echo "$JOB_CONFIGS" | jq -r '[.[].status_context] | join(",")')" >> "$GITHUB_OUTPUT"
- name: e2e/triage
continue-on-error: true
uses: ./.github/actions/e2e-triage
with:
job-key: ${{ matrix.job-key }}
composite-identity: ${{ toJSON(fromJSON(needs.cmt-identity.outputs.tsio-job-configs)[matrix.job-key].composite_identity) }}
status-context: ${{ fromJSON(needs.cmt-identity.outputs.tsio-job-configs)[matrix.job-key].status_context }}
github-token: ${{ secrets.GITHUB_TOKEN }}
anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
triage-lanes: ${{ steps.lanes.outputs.contexts }}
# A triage failure must never change the outcome of the CMT run itself.
mode: ${{ vars.E2E_TRIAGE_MODE || 'report-only' }}