mattermost-mobile-upstream/.github/workflows/e2e-android-template.yml
yasser khan 3e1331248c
Some checks are pending
ci / lint-typecheck (push) Waiting to run
ci / detox-lint-typecheck (push) Waiting to run
ci / test (push) Waiting to run
ci(e2e): say whether each main-run failure is new, still failing, flaky or a setup problem (#10284)
* ci(e2e): say whether each main-run failure is new, still failing, flaky or a setup problem, from the triage verdict

The channel message judged failures from their error text, beside the E2E triage
jobs that judge the same failures from TSIO history, and the two disagreed: on
main run 40764686027 it listed tests that passed on retry as needing a look. The
message now waits for triage and reads each run's findings. On main, a test that
passed every earlier main run and fails now is new, with the changes merged since
the last main run named (TSIO's previous run, GitHub's compare); a run where most
tests failed at once is called out first. Runs with no triage verdict keep the
error-text judgement.

* ci(e2e): judge from the error text each failed test triage named no verdict for, and say how many were not checked

* ci(e2e): count each test once per run, so a test that failed and then passed its retry is flaky, not a failure

* ci(e2e): a main-run failure triage calls flaky or a test problem is not blamed on new changes; a same-commit re-run says nothing was merged; a long PR list says it is partial

* Fix TSIO current report selection

Co-authored-by: yasserfaraazkhan <15979783+yasserfaraazkhan@users.noreply.github.com>

* Fix lint comment spacing

Co-authored-by: yasserfaraazkhan <15979783+yasserfaraazkhan@users.noreply.github.com>

* ci(e2e): update the TSIO upload action, so a failed screenshot upload no longer drops a shard's results

* ci(e2e): an earlier attempt of the same main run is not the previous run, so a re-run still lists the changes merged since the last run

* ci(e2e): drop the note from the TSIO upload action pin comment

* ci(e2e): a re-run replaces the first attempt's triage findings, so the channel message never posts stale verdicts

* ci(e2e): the main-run message says in a few lines which failures may be product bugs and which are not, and posts nothing when none may be

* ci(e2e): main and CMT call a failure a product bug only when it fails again on a fresh device, and clear what the error text proves is not the app

* ci(e2e): a re-run hands Android Detox and Maestro results to the channel message too

* ci(e2e): CMT re-runs start after a failed lane; a spec that differs from main, or a feature the server has off, is not a product bug

* ci(e2e): the CMT message plans the re-run as the planner job does, leaving out tests that differ from main

* ci(e2e): one channel post per run, and a test's name can never mention the channel

* ci(e2e): a re-run keeps the run's own report and runs only the failed flows

* ci(e2e): a PR run posts to the channel only when something failed

* ci(e2e): an E2E/Override PR keeps its statuses on a new push; a few simpler lines in the channel notify

* ci(e2e): exclude isolated report-problem flows from reruns and prevent label/action concurrency collisions

<!-- coderabbit-ai-generated:v1 -->

* ci(e2e): pass support paths to E2E triage

* ci(e2e): pin TSIO report-upload to release-0.16.0 (main@3212731)

Brings the TSIO action pin up to the latest release, which includes
mattermost-test-system-io#129, #130 and #124. The report-upload action itself
is unchanged between the two pins.

* ci(e2e): re-run E2E jobs once when a lane is red only on infrastructure

Triage (rerun-attempts: 1, with actions: read to read the failed jobs) asks for
one re-run when a lane is red only because jobs died or reports never arrived;
a build or test-server failure holds it. e2e-tests-rerun.yml carries the re-run
out once the run finishes. CMT keeps its own re-run of failed specs.

---------
2026-11-03 07:45:59 +05:30

929 lines
41 KiB
YAML

name: Detox Android E2E Tests Template
on:
workflow_call:
secrets:
MM_MOBILE_E2E_ADMIN_EMAIL:
required: false
MM_MOBILE_E2E_ADMIN_USERNAME:
required: false
MM_MOBILE_E2E_ADMIN_PASSWORD:
required: false
MM_MOBILE_E2E_ZEPHYR_API_KEY:
required: false
MM_MOBILE_E2E_TEST_CYCLE_LINK_PREFIX:
required: false
MM_MOBILE_E2E_WEBHOOK_URL:
required: false
# Channel-notify routing for the per-job TSIO finalize step. Callers that use
# an explicit `secrets:` block must forward these or notify silently no-ops.
MM_E2E_MASTER_HEALTH_WEBHOOK_URL:
required: false
MM_E2E_RELEASE_WEBHOOK_URL:
required: false
MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID:
required: false
MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY:
required: false
# Webhook sidecar. Unset means mm_blocks specs fall back to trycloudflare.
MM_MOBILE_E2E_WEBHOOK_PUBLIC_BASE_URL:
required: false
MM_MOBILE_E2E_CLOUDFLARED_TUNNEL_TOKEN:
required: false
inputs:
MM_TEST_SERVER_URL:
description: "The test server URL (SITE_1)"
required: false
type: string
MM_TEST_SERVER_URL_2:
description: "The second test server URL (SITE_2); optional, no automatic fallback"
required: false
type: string
MM_TEST_SERVER_URL_3:
description: "The third test server URL (SITE_3); optional, no automatic fallback"
required: false
type: string
MM_OTHER_PLATFORM_POOL_URL:
description: "The other platform's SITE_1 server, used as this platform's pool SITE_2; empty falls back to SITE_1"
required: false
type: string
default: ""
MM_TEST_USER_NAME:
description: "The admin username of the test instance"
required: false
type: string
MM_TEST_PASSWORD:
description: "The admin password of the test instance"
required: false
type: string
MOBILE_VERSION:
description: "The mobile version to test"
required: false
default: ""
type: string
run-android-tests:
description: "Run Android tests"
required: true
type: boolean
run-type:
type: string
required: false
default: "PR"
record_tests_in_zephyr:
description: "Record test results in Zephyr, typically for release and main runs"
required: false
type: boolean
default: false
low_bandwidth_mode:
description: "Enable low bandwidth mode"
required: false
type: boolean
default: false
search_path:
description: "Path to search for test specs (use for smoke test subset)"
required: false
type: string
default: "detox/e2e/test"
parallelism:
description: "Number of parallel test shards"
required: false
type: string
default: "32"
android_avd_name:
description: "Android Emulator name"
required: false
type: string
default: "detox_pixel_8"
android_api_level:
description: "Android API level"
required: false
type: string
default: "35"
artifact_run_id:
description: "Run ID of the workflow that uploaded android-build-files-* (defaults to caller run)"
required: false
type: string
default: ""
tsio-config:
description: >
TSIO reporting config: {"composite_identity":{...},"total_reports_expected":N}.
Empty disables TSIO upload for this platform.
required: false
type: string
default: ""
tsio-shard-name:
description: "Display name for this shard in TSIO (e.g. detox-android-Server_11.9.0-rc3)"
required: false
type: string
default: ""
spec_files:
description: >
Space-separated spec files to run instead of every spec under search_path:
a re-run of failed specs. Pair it with an empty tsio-config, run-type NONE and
results-artifact, so the re-run reports only to the channel message.
required: false
type: string
default: ""
results-artifact:
description: "Artifact name for this run's merged per-test results (a re-run of failed specs); empty uploads none"
required: false
type: string
default: ""
durations_key:
description: >
Spec-duration manifest key (e.g. detox-android). Enables duration-aware
shard packing. Empty keeps the even alphabetical split.
required: false
type: string
default: ""
outputs:
STATUS:
value: ${{ jobs.generate-report.outputs.STATUS }}
TARGET_URL:
value: ${{ jobs.generate-report.outputs.TARGET_URL }}
FAILURES:
value: ${{ jobs.generate-report.outputs.FAILURES }}
PASSES:
value: ${{ jobs.generate-report.outputs.PASSES }}
TOTAL:
value: ${{ jobs.generate-report.outputs.TOTAL }}
SKIPPED:
value: ${{ jobs.generate-report.outputs.SKIPPED }}
ERRORS:
value: ${{ jobs.generate-report.outputs.ERRORS }}
PERCENTAGE:
value: ${{ jobs.generate-report.outputs.PERCENTAGE }}
env:
AWS_REGION: "us-east-1"
ADMIN_EMAIL: ${{ secrets.MM_MOBILE_E2E_ADMIN_EMAIL }}
ADMIN_USERNAME: ${{ inputs.MM_TEST_USER_NAME || secrets.MM_MOBILE_E2E_ADMIN_USERNAME }}
ADMIN_PASSWORD: ${{ inputs.MM_TEST_PASSWORD || secrets.MM_MOBILE_E2E_ADMIN_PASSWORD }}
BRANCH: ${{ github.event_name == 'pull_request' && github.head_ref || github.ref_name }}
COMMIT_HASH: ${{ github.sha }}
DEVICE_NAME: ${{ inputs.android_avd_name }} # This is needed to split tests as same code is used in iOS job
DEVICE_OS_VERSION: ${{ inputs.android_api_level }}
DETOX_AWS_S3_BUCKET: "mattermost-detox-report"
HEADLESS: "true"
TYPE: ${{ inputs.run-type }}
PULL_REQUEST: "https://github.com/mattermost/mattermost-mobile/pull/${{ github.event.number }}"
SITE_1_URL: ${{ inputs.MM_TEST_SERVER_URL }}
SITE_2_URL: ${{ inputs.MM_TEST_SERVER_URL_2 }}
SITE_3_URL: ${{ inputs.MM_TEST_SERVER_URL_3 }}
ZEPHYR_ENABLE: ${{ inputs.record_tests_in_zephyr }}
JIRA_PROJECT_KEY: "MM"
ZEPHYR_API_KEY: ${{ secrets.MM_MOBILE_E2E_ZEPHYR_API_KEY }}
ZEPHYR_FOLDER_ID: "3233873"
TEST_CYCLE_LINK_PREFIX: ${{ secrets.MM_MOBILE_E2E_TEST_CYCLE_LINK_PREFIX }}
# save_report's own channel post, only for a run that does not report to TSIO: a TSIO run is posted once, for the
# whole run, by the tsio-channel-notify rollup.
WEBHOOK_URL: ${{ (inputs.tsio-config == '' || vars.TSIO_UPLOAD_ENABLED == 'false') && secrets.MM_MOBILE_E2E_WEBHOOK_URL || '' }}
FAILURE_MESSAGE: "Something has failed"
IOS: "false"
RUNNING_E2E: "true"
AVD_NAME: ${{ inputs.android_avd_name }}
SDK_VERSION: ${{ inputs.android_api_level }}
ARTIFACT_RUN_ID: ${{ inputs.artifact_run_id }}
jobs:
generate-specs:
runs-on: ubuntu-24.04
outputs:
specs: ${{ steps.generate-specs.outputs.specs }}
shard_count: ${{ steps.generate-specs.outputs.shard_count }}
build_id: ${{ steps.resolve-device.outputs.BUILD_ID }}
mobile_sha: ${{ steps.resolve-device.outputs.MOBILE_SHA }}
mobile_ref: ${{ steps.resolve-device.outputs.MOBILE_REF }}
workflow_hash: ${{ steps.resolve-device.outputs.WORKFLOW_HASH }}
steps:
- name: Checkout Repository
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
ref: ${{ inputs.MOBILE_VERSION }}
- name: Set Build ID
id: resolve-device
run: |
BUILD_ID="${{ github.run_id }}-${{ env.AVD_NAME }}-${{ env.SDK_VERSION}}"
WORKFLOW_HASH=$(tr -dc a-z0-9 </dev/urandom | head -c 10)
## We need that hash to separate the artifacts
echo "WORKFLOW_HASH=${WORKFLOW_HASH}" >> ${GITHUB_OUTPUT}
echo "BUILD_ID=$(echo ${BUILD_ID} | sed 's/ /_/g')" >> ${GITHUB_OUTPUT}
echo "MOBILE_SHA=$(git rev-parse HEAD)" >> ${GITHUB_OUTPUT}
echo "MOBILE_REF=$(git rev-parse --abbrev-ref HEAD)" >> ${GITHUB_OUTPUT}
# Duration history from the last main-push run; absent → even split.
# continue-on-error: this job gates the entire test matrix, and shard packing
# is an optimization — a missing manifest or an AWS hiccup must not stop the
# run, it must fall back to the even split.
- name: Restore spec-duration manifest
if: inputs.durations_key != ''
continue-on-error: true
uses: ./.github/actions/s3-spec-durations
env:
DETOX_AWS_ACCESS_KEY_ID: ${{ secrets.MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID }}
DETOX_AWS_SECRET_ACCESS_KEY: ${{ secrets.MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY }}
with:
key: ${{ inputs.durations_key }}
bucket: ${{ env.DETOX_AWS_S3_BUCKET }}
mode: restore
# Server B gets a mutation shard when it is distinct from A. Android has no
# site3 mutation target; its specs whose iosTarget is site3 run on B.
- name: Resolve mutation server targets
id: mutation-targets
env:
SERVER_A: ${{ inputs.MM_TEST_SERVER_URL }}
SERVER_B: ${{ inputs.MM_TEST_SERVER_URL_2 }}
run: |
# No targets without a distinct second server; split-tests then fails only if a mutation spec is selected.
TARGETS=""
if [ -n "$SERVER_B" ] && [ "$SERVER_B" != "$SERVER_A" ]; then
TARGETS="B"
fi
echo "targets=$TARGETS" >> "$GITHUB_OUTPUT"
echo "Mutation targets: ${TARGETS:-none}"
- name: Generate Test Specs
id: generate-specs
uses: ./.github/actions/generate-specs
with:
parallelism: ${{ inputs.parallelism }}
search_path: ${{ inputs.search_path }}
spec_files: ${{ inputs.spec_files }}
device_name: ${{ env.AVD_NAME }}
device_os_version: ${{ env.SDK_VERSION }}
durations_file: ${{ inputs.durations_key != '' && 'detox/artifacts/spec-durations.json' || '' }}
platform: android
mutation_targets: ${{ steps.mutation-targets.outputs.targets }}
e2e-android:
name: ${{ matrix.role == 'mutation' && 'mutation-' || '' }}machine-${{ matrix.runId }}-api-${{ matrix.deviceOsVersion }}
runs-on: ubuntu-latest-8-cores
continue-on-error: true
timeout-minutes: 150
env:
ANDROID_HOME: /usr/local/lib/android/sdk
ANDROID_SDK_ROOT: /usr/local/lib/android/sdk
needs:
- generate-specs
strategy:
fail-fast: false
matrix: ${{ fromJSON(needs.generate-specs.outputs.specs) }}
steps:
- name: Checkout Repository
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
ref: ${{ inputs.MOBILE_VERSION }}
# Pool shards pair server A with the other platform's A; mutation shards own server B alone.
# SITE_3 is also every pool shard's read-only third server, so site3 mutations must not touch login.
- name: Assign test sites by shard role
env:
ROLE: ${{ matrix.role }}
TARGET: ${{ matrix.target }}
SERVER_A: ${{ inputs.MM_TEST_SERVER_URL }}
SERVER_B: ${{ inputs.MM_TEST_SERVER_URL_2 }}
SERVER_SHARED: ${{ inputs.MM_TEST_SERVER_URL_3 }}
OTHER_PLATFORM_POOL: ${{ inputs.MM_OTHER_PLATFORM_POOL_URL }}
run: |
case "$ROLE" in
pool)
SITE_1_URL="$SERVER_A"; SITE_1_LABEL="server-a"
if [ -n "$OTHER_PLATFORM_POOL" ]; then
SITE_2_URL="$OTHER_PLATFORM_POOL"; SITE_2_LABEL="other-platform-a"
else
# Never SITE_3: on iOS it is also a mutation target.
echo "::warning::MM_OTHER_PLATFORM_POOL_URL is empty; pool SITE_2 falls back to this shard's own server A"
SITE_2_URL="$SERVER_A"; SITE_2_LABEL="server-a"
fi
SITE_3_URL="$SERVER_SHARED"; SITE_3_LABEL="site3"
GUARD="enforce"
;;
mutation)
case "$TARGET" in
B) SITE_1_URL="$SERVER_B"; SITE_1_LABEL="server-b" ;;
site3) SITE_1_URL="$SERVER_SHARED"; SITE_1_LABEL="site3" ;;
A) echo "::error::No isolated server for mutation specs (MM_TEST_SERVER_URL_2 missing or equal to MM_TEST_SERVER_URL); they would change state on the pool server"; exit 1 ;;
*) echo "::error::Unknown mutation target '$TARGET'"; exit 1 ;;
esac
SITE_2_URL="$SERVER_A"; SITE_2_LABEL="server-a"
SITE_3_URL=""; SITE_3_LABEL="unset"
GUARD="off"
;;
*)
echo "::error::Unknown shard role '$ROLE'"; exit 1 ;;
esac
if [ -z "$SITE_1_URL" ]; then
echo "::error::SITE_1 is empty for role=$ROLE target=$TARGET"; exit 1
fi
{
echo "SITE_1_URL=$SITE_1_URL"
echo "SITE_2_URL=$SITE_2_URL"
echo "SITE_3_URL=$SITE_3_URL"
echo "E2E_SERVER_ROLE=$ROLE"
echo "E2E_POOL_GUARD=$GUARD"
} >> "$GITHUB_ENV"
echo "role=$ROLE target=$TARGET: SITE_1=$SITE_1_LABEL SITE_2=$SITE_2_LABEL SITE_3=$SITE_3_LABEL pool_guard=$GUARD"
# Only libpulse0 is a real dependency here — the Android emulator needs it for
# audio. xvfb is already in the runner image (apt reports "already the newest
# version"), and it is listed only so a future image change fails loudly.
- name: Install Dependencies
timeout-minutes: 6
run: |
APT_OPTS=(-o Acquire::Retries=3 -o Acquire::http::Timeout=15 -o DPkg::Lock::Timeout=120)
PACKAGES=(libpulse0 xvfb)
sudo apt-get "${APT_OPTS[@]}" install -y "${PACKAGES[@]}" || {
sudo apt-get "${APT_OPTS[@]}" update
sudo apt-get "${APT_OPTS[@]}" install -y "${PACKAGES[@]}"
}
- name: Enable KVM
run: |
echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules
sudo udevadm control --reload-rules
sudo udevadm trigger --name-match=kvm
- name: ci/prepare-node-deps
uses: ./.github/actions/prepare-node-deps
- name: Cache detox node_modules
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: detox/node_modules
key: ${{ runner.os }}-detox-deps-${{ hashFiles('detox/package-lock.json') }}
restore-keys: ${{ runner.os }}-detox-deps-
- name: Install Detox Dependencies
run: |
cd detox
npm install
- name: Set .env with RUNNING_E2E=true
run: |
cat > .env <<EOF
RUNNING_E2E=true
EOF
# Must precede the download: download-artifact consumes ARTIFACT_RUN_ID and
# fails with an opaque API error when it is empty.
- name: Verify artifact_run_id was passed by the parent workflow
env:
ARTIFACT_RUN_ID: ${{ env.ARTIFACT_RUN_ID }}
run: |
if [ -z "${ARTIFACT_RUN_ID}" ]; then
echo "❌ ARTIFACT_RUN_ID is empty — parent workflow must pass artifact_run_id from build-android-apk"
exit 1
fi
- name: Download Detox Android build from parent workflow
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: android-build-files-${{ env.ARTIFACT_RUN_ID }}
repository: ${{ github.repository }}
run-id: ${{ env.ARTIFACT_RUN_ID }}
github-token: ${{ github.token }}
path: android/app/build
- name: Verify Detox Android app from parent build
run: |
APP_APK="android/app/build/outputs/apk/debug/app-debug.apk"
TEST_APK="android/app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk"
if [ ! -f "$APP_APK" ]; then
echo "❌ Missing ${APP_APK} from android-build-files-${ARTIFACT_RUN_ID}"
find android/app/build -name '*.apk' 2>/dev/null || true
exit 1
fi
if [ ! -f "$TEST_APK" ]; then
echo "❌ Missing ${TEST_APK} from android-build-files-${ARTIFACT_RUN_ID}"
find android/app/build -name '*.apk' 2>/dev/null || true
exit 1
fi
echo "✅ Detox will use Android APKs from parent workflow run ${ARTIFACT_RUN_ID}:"
ls -lh "$APP_APK" "$TEST_APK"
env:
ARTIFACT_RUN_ID: ${{ env.ARTIFACT_RUN_ID }}
- name: Set up Android SDK
run: |
export ANDROID_HOME=/usr/local/lib/android/sdk
export PATH=$ANDROID_HOME/cmdline-tools/latest/bin:$ANDROID_HOME/emulator:$ANDROID_HOME/tools/bin:$ANDROID_HOME/platform-tools:$PATH
echo "ANDROID_HOME=$ANDROID_HOME" >> $GITHUB_ENV
echo "PATH=$PATH" >> $GITHUB_ENV
- name: Start Xvfb
run: |
Xvfb :99 -screen 0 1920x1080x24 &
export DISPLAY=:99
echo "DISPLAY=:99" >> $GITHUB_ENV
- name: Accept Android licenses
run: |
mkdir -p ~/.android
yes | sdkmanager --licenses > sdk_licenses_output.txt || true
# Check if "All SDK package licenses accepted" appears in output
if grep -q "All SDK package licenses accepted" sdk_licenses_output.txt; then
echo "✅ All licenses accepted successfully."
else
echo "❌ Licenses not fully accepted."
cat sdk_licenses_output.txt
exit 1
fi
# Cache SDK under the workspace (not /usr/local) — tar restore into system
# SDK dirs often fails with exit code 2 when paths already exist on the image.
- name: Cache Android SDK components
id: android-sdk-cache
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: .android-sdk-cache
key: android-sdk-${{ inputs.android_api_level }}-google-apis-x86_64-v2
- name: Apply cached Android SDK components
if: steps.android-sdk-cache.outputs.cache-hit == 'true'
env:
API_LEVEL: ${{ inputs.android_api_level }}
run: |
for component in platform-tools emulator; do
if [ -d ".android-sdk-cache/${component}" ]; then
rm -rf "${ANDROID_HOME:?}/${component}"
cp -a ".android-sdk-cache/${component}" "${ANDROID_HOME}/"
fi
done
if [ -d ".android-sdk-cache/platforms/android-${API_LEVEL}" ]; then
rm -rf "${ANDROID_HOME}/platforms/android-${API_LEVEL}"
mkdir -p "${ANDROID_HOME}/platforms"
cp -a ".android-sdk-cache/platforms/android-${API_LEVEL}" "${ANDROID_HOME}/platforms/"
fi
if [ -d ".android-sdk-cache/system-images/android-${API_LEVEL}" ]; then
rm -rf "${ANDROID_HOME}/system-images/android-${API_LEVEL}"
mkdir -p "${ANDROID_HOME}/system-images"
cp -a ".android-sdk-cache/system-images/android-${API_LEVEL}" "${ANDROID_HOME}/system-images/"
fi
- name: Install Android SDK components
if: steps.android-sdk-cache.outputs.cache-hit != 'true'
env:
API_LEVEL: ${{ inputs.android_api_level }}
JAVA_HOME: ${{ env.JAVA_HOME_17_X64 }}
run: |
if ! [[ "$API_LEVEL" =~ ^[0-9]+$ ]]; then
echo "Invalid android_api_level: $API_LEVEL"
exit 1
fi
for attempt in 1 2 3; do
echo "SDK install attempt $attempt..."
if yes | sdkmanager --install "platform-tools" "emulator" "platforms;android-${API_LEVEL}" "system-images;android-${API_LEVEL};google_apis;x86_64"; then
echo "SDK install succeeded on attempt $attempt"
break
fi
echo "Attempt $attempt failed, retrying..."
if [ "$attempt" = "3" ]; then
echo "SDK install failed after 3 attempts"
exit 1
fi
sleep 10
done
mkdir -p .android-sdk-cache/platforms .android-sdk-cache/system-images
cp -a "${ANDROID_HOME}/platform-tools" .android-sdk-cache/
cp -a "${ANDROID_HOME}/emulator" .android-sdk-cache/
cp -a "${ANDROID_HOME}/platforms/android-${API_LEVEL}" .android-sdk-cache/platforms/
cp -a "${ANDROID_HOME}/system-images/android-${API_LEVEL}" .android-sdk-cache/system-images/
# CI AVDs live under detox/.android/avd (see create_android_emulator.sh), not ~/.android/avd.
- name: Cache Android Emulator & AVD
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: |
detox/.android
detox/${{ env.AVD_NAME }}_api_${{ inputs.android_api_level }}
key: android-emulator-${{ runner.os }}-${{ runner.arch }}-${{ env.AVD_NAME }}-api-${{ inputs.android_api_level }}-v3
restore-keys: |
android-emulator-${{ runner.os }}-${{ runner.arch }}-${{ env.AVD_NAME }}-api-${{ inputs.android_api_level }}-
android-emulator-${{ runner.os }}-${{ runner.arch }}-
- name: Start mm_blocks webhook sidecar
if: contains(matrix.specs, 'mm_blocks_')
continue-on-error: true
env:
SITE_1_URL: ${{ env.SITE_1_URL }}
WEBHOOK_PUBLIC_BASE_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_PUBLIC_BASE_URL }}
CLOUDFLARED_TUNNEL_TOKEN: ${{ secrets.MM_MOBILE_E2E_CLOUDFLARED_TUNNEL_TOKEN }}
run: bash detox/scripts/start_webhook_sidecar.sh
- name: Create and run Android Emulator
env:
MATRIX_SPECS: ${{ matrix.specs }}
SITE_1_URL: ${{ env.SITE_1_URL }}
SITE_2_URL: ${{ env.SITE_2_URL }}
SITE_3_URL: ${{ env.SITE_3_URL }}
MAX_ATTEMPTS: "2"
ATTEMPT_TIMEOUT_MIN: "60"
shell: bash
# Attempt 2 retries failed/unrun files only. Cold-boots only if the emulator is dead.
run: |
chmod +x detox/scripts/run_android_ci_shard.sh detox/create_android_emulator.sh
# Split on whitespace into an array, then expand quoted: an unquoted
# $MATRIX_SPECS is also subject to pathname expansion, so a spec path
# containing a glob character would silently become a different argument
# list (or vanish). Matches the iOS template's read -ra handling.
read -ra specs <<< "$MATRIX_SPECS"
# The output is also kept for the triage evidence step below. Not a pipe
# into tee: the shard script leaves the emulator, adb and Metro running,
# they inherit the pipe, and tee would wait on them until the job times
# out. Write to the file and stream it until the script exits.
log="$RUNNER_TEMP/detox-run.log"
: > "$log"
detox/scripts/run_android_ci_shard.sh "$SDK_VERSION" "$AVD_NAME" "${specs[@]}" > "$log" 2>&1 &
shard=$!
tail -n +1 -f --pid="$shard" "$log"
wait "$shard"
continue-on-error: true # We want to run all the tests
# For the E2E triage judge: for each failed test, Detox's failure screenshot,
# the requests Detox reported the app was still waiting on, and the test's
# own server calls that failed.
- name: Collect triage evidence for failed tests
if: always()
continue-on-error: true
working-directory: detox
run: node utils/triage-evidence.js --results artifacts/jest-results.json --log "$RUNNER_TEMP/detox-run.log" --dir artifacts/triage-evidence --artifacts artifacts --out "artifacts/triage-evidence/evidence-${{ matrix.runId }}.json"
- name: Upload triage evidence
if: always() && inputs.tsio-shard-name != '' && hashFiles('detox/artifacts/triage-evidence/*.json') != ''
continue-on-error: true
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: triage-evidence-${{ inputs.tsio-shard-name }}-${{ needs.generate-specs.outputs.workflow_hash }}-${{ matrix.runId }}
path: detox/artifacts/triage-evidence/
# Second attempt for the same reason as the iOS template: losing this upload to a
# transient ETIMEDOUT reports the shard as having produced no results at all.
- name: Upload Android Test Report
id: upload-android-report
if: always() && hashFiles('detox/artifacts/**') != ''
continue-on-error: true
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: android-results-${{ needs.generate-specs.outputs.workflow_hash }}-${{ matrix.runId }}
path: detox/artifacts/
# Same two-retry sequence as the iOS template (see there).
- name: Await results service
if: |
!cancelled() &&
steps.upload-android-report.outcome == 'failure'
uses: ./.github/actions/await-results-service
- name: Upload Android Test Report (retry 1)
id: upload-android-report-retry
if: |
!cancelled() &&
steps.upload-android-report.outcome == 'failure'
continue-on-error: true
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: android-results-${{ needs.generate-specs.outputs.workflow_hash }}-${{ matrix.runId }}
path: detox/artifacts/
- name: Await results service (retry 2)
if: |
!cancelled() &&
steps.upload-android-report.outcome == 'failure' &&
steps.upload-android-report-retry.outcome == 'failure'
uses: ./.github/actions/await-results-service
- name: Upload Android Test Report (retry 2)
if: |
!cancelled() &&
steps.upload-android-report.outcome == 'failure' &&
steps.upload-android-report-retry.outcome == 'failure'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: android-results-${{ needs.generate-specs.outputs.workflow_hash }}-${{ matrix.runId }}
path: detox/artifacts/
overwrite: true
generate-report:
# TSIO displays gh-job-name as the shard label — keep it aligned with tsio-shard-name.
name: ${{ inputs.tsio-shard-name != '' && inputs.tsio-shard-name || 'generate-report' }}
if: always()
runs-on: ubuntu-24.04
needs:
- generate-specs
- e2e-android
permissions:
contents: read
# test-system-io-report-upload authenticates via the workflow's OIDC
# token and resolves gh_job_id via the GitHub API.
id-token: write
actions: read
# Per-job TSIO finalize posts detox-android (or CMT <shard>).
statuses: write
# Finalize re-reads the PR labels so a result landing after E2E/Override was
# applied reports success instead of re-blocking the merge.
issues: read
outputs:
TARGET_URL: ${{ steps.set-url.outputs.TARGET_URL }}
STATUS: ${{ steps.determine-status.outputs.STATUS }}
FAILURES: ${{ steps.summary.outputs.FAILURES }}
PASSES: ${{ steps.summary.outputs.PASSES }}
TOTAL: ${{ steps.summary.outputs.TOTAL }}
SKIPPED: ${{ steps.summary.outputs.SKIPPED }}
ERRORS: ${{ steps.summary.outputs.ERRORS }}
PERCENTAGE: ${{ steps.summary.outputs.PERCENTAGE }}
steps:
- name: Checkout Repository
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
ref: ${{ inputs.MOBILE_VERSION }}
- name: ci/prepare-node-deps
uses: ./.github/actions/prepare-node-deps
- name: Download Android Artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
path: detox/artifacts/
pattern: android-results-${{ needs.generate-specs.outputs.workflow_hash }}-*
continue-on-error: true
- name: Generate Report Path
id: s3
env:
BUILD_ID: ${{ needs.generate-specs.outputs.build_id }}
MOBILE_SHA: ${{ needs.generate-specs.outputs.mobile_sha }}
MOBILE_REF: ${{ needs.generate-specs.outputs.mobile_ref }}
# A re-run of failed specs runs in the same workflow run: its own report, not over the run's.
RERUN: ${{ inputs.results-artifact != '' && '-rerun' || '' }}
run: |
path="${BUILD_ID}-${MOBILE_SHA}-${MOBILE_REF}${RERUN}"
echo "path=$(echo "${path}" | sed 's/\./-/g')" >> "$GITHUB_OUTPUT"
# ── Spec durations for the next run's shard packing ────────────────────
# Must run before e2e:save-report, which deletes the android-results-* dirs
# the per-shard jest-results.json live in. Only main-push publishes (a PR
# must not steer how other branches shard); PR runs still compute it so the
# step is exercised on every PR that touches this workflow.
# Re-fetch first: this is a different runner from generate-specs, so without
# it the EWMA would have no history and collapse to "last run only".
- name: Restore previous spec-duration manifest
if: always() && inputs.durations_key != ''
continue-on-error: true
uses: ./.github/actions/s3-spec-durations
env:
DETOX_AWS_ACCESS_KEY_ID: ${{ secrets.MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID }}
DETOX_AWS_SECRET_ACCESS_KEY: ${{ secrets.MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY }}
with:
key: ${{ inputs.durations_key }}
bucket: ${{ env.DETOX_AWS_S3_BUCKET }}
mode: restore
- name: Build spec-duration manifest
if: always() && inputs.durations_key != ''
continue-on-error: true
env:
DURATIONS_KEY: ${{ inputs.durations_key }}
run: |
node detox/utils/spec-durations.js \
--dir detox/artifacts \
--previous detox/artifacts/spec-durations.json \
--output detox/artifacts/spec-durations.next.json \
--key "$DURATIONS_KEY" \
--repo-root "$GITHUB_WORKSPACE"
- name: Publish spec-duration manifest
if: always() && inputs.durations_key != '' && (inputs.run-type == 'MAIN' || inputs.run-type == 'MASTER') && hashFiles('detox/artifacts/spec-durations.next.json') != ''
continue-on-error: true
uses: ./.github/actions/s3-spec-durations
env:
DETOX_AWS_ACCESS_KEY_ID: ${{ secrets.MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID }}
DETOX_AWS_SECRET_ACCESS_KEY: ${{ secrets.MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY }}
with:
key: ${{ inputs.durations_key }}
bucket: ${{ env.DETOX_AWS_S3_BUCKET }}
manifest-path: detox/artifacts/spec-durations.next.json
mode: save
# For TSIO, or for a re-run of failed specs, which hands its results to the channel message.
- name: Merge per-shard Jest JSON for TSIO
if: always() && ((inputs.tsio-config != '' && vars.TSIO_UPLOAD_ENABLED != 'false') || inputs.results-artifact != '')
id: tsio-merge
env:
# Built from repo filenames, so it must reach the shell as data, never spliced
# into the script text: a spec named x'$(...)'.e2e.ts would otherwise execute.
EXPECTED_SPECS: ${{ needs.generate-specs.outputs.specs }}
run: |
set +e
# TSIO extractDetox expects testFilePath + nested testResults (not Jest CLI
# name + assertionResults). Convert while merging shard JSON files.
if ! node detox/utils/merge-jest-results-for-tsio.js \
--dir detox/artifacts \
--repo-root "${GITHUB_WORKSPACE}" \
--expected-specs "$EXPECTED_SPECS" \
--output detox/artifacts/jest-results-merged.json; then
echo "skip=true" >> "$GITHUB_OUTPUT"
echo "No jest-results.json found or merge failed — skipping TSIO upload."
exit 0
fi
echo "skip=false" >> "$GITHUB_OUTPUT"
# A re-run of failed specs hands its per-test results to the channel message.
- name: Upload re-run results
if: always() && inputs.results-artifact != '' && steps.tsio-merge.outputs.skip == 'false'
continue-on-error: true
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: ${{ inputs.results-artifact }}
path: detox/artifacts/jest-results-merged.json
retention-days: 1
- name: Write TSIO failure stub when Detox JSON is missing
if: always() && inputs.tsio-config != '' && vars.TSIO_UPLOAD_ENABLED != 'false' && steps.tsio-merge.outputs.skip == 'true'
env:
TSIO_SHARD_NAME: ${{ inputs.tsio-shard-name != '' && inputs.tsio-shard-name || 'generate-report' }}
run: |
node detox/utils/write-tsio-failure-stub.mjs \
--format jest \
--output detox/artifacts/jest-results-merged.json \
--job-name "$TSIO_SHARD_NAME" \
--reason "Detox job failed or was cancelled before jest-results.json was written"
- name: Resolve per-job TSIO config
if: always() && inputs.tsio-config != '' && vars.TSIO_UPLOAD_ENABLED != 'false'
id: tsio-cfg
env:
TSIO_CONFIG: ${{ inputs.tsio-config }}
TSIO_SHARD_NAME: ${{ inputs.tsio-shard-name }}
run: |
CFG=$(bash detox/utils/resolve-tsio-job-config.sh)
{
echo "config<<EOF"
echo "$CFG"
echo "EOF"
} >> "$GITHUB_OUTPUT"
echo "Resolved TSIO job: $(echo "$CFG" | jq -r '.composite_identity.name') context=$(echo "$CFG" | jq -r '.status_context')"
- name: Upload report to Test System IO
if: always() && inputs.tsio-config != '' && vars.TSIO_UPLOAD_ENABLED != 'false' && hashFiles('detox/artifacts/jest-results-merged.json') != '' && steps.tsio-cfg.outputs.config != ''
uses: mattermost/mattermost-test-system-io/.github/actions/test-system-io-report-upload@3212731263d782a3993dbdf4c8c54f143e69a7b2 # release-0.16.0 / main@3212731
with:
composite-identity: ${{ toJSON(fromJSON(steps.tsio-cfg.outputs.config).composite_identity) }}
total-reports-expected: ${{ fromJSON(steps.tsio-cfg.outputs.config).total_reports_expected }}
framework: detox
github-token: ${{ github.token }}
gh-job-name: ${{ inputs.tsio-shard-name != '' && inputs.tsio-shard-name || 'generate-report' }}
json-path: detox/artifacts/jest-results-merged.json
screenshots-dir: detox/artifacts/
# Resolved here rather than inside tsio-report-status.js so the GitHub API
# call stays in the Actions layer and the reporter acts on its inputs.
- name: Check E2E/Override on the PR
id: e2e-override
if: always() && inputs.tsio-config != '' && vars.TSIO_UPLOAD_ENABLED != 'false' && steps.tsio-cfg.outputs.config != ''
uses: ./.github/actions/e2e-override-label
with:
pr_number: ${{ fromJSON(steps.tsio-cfg.outputs.config).composite_identity.gh_pr_number }}
- name: Post per-job TSIO commit status
if: always() && inputs.tsio-config != '' && vars.TSIO_UPLOAD_ENABLED != 'false' && steps.tsio-cfg.outputs.config != ''
env:
GITHUB_TOKEN: ${{ github.token }}
TSIO_CONFIG: ${{ steps.tsio-cfg.outputs.config }}
E2E_OVERRIDE: ${{ steps.e2e-override.outputs.override }}
# The runs e2e-detox-pr.yml triages (its tsio-triage job), so a red lane
# says triage is coming instead of looking final.
ANNOUNCE_TRIAGE: ${{ (inputs.run-type == 'PR' || inputs.run-type == 'MAIN' || inputs.run-type == 'MASTER') && vars.E2E_TRIAGE_MODE == 'enforce' }}
SPECS_RESULT: ${{ needs.generate-specs.result }}
E2E_RESULT: ${{ needs.e2e-android.result }}
JOB_STATUS: ${{ job.status }}
MATTERMOST_E2E_WEBHOOK_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_URL }}
MATTERMOST_MASTER_HEALTH_WEBHOOK_URL: ${{ secrets.MM_E2E_MASTER_HEALTH_WEBHOOK_URL }}
MATTERMOST_CMT_WEBHOOK_URL: ${{ secrets.MM_E2E_RELEASE_WEBHOOK_URL }}
run: |
set +e
UPSTREAM="true"
CANCELLED="false"
# This job runs with always(), so job.status stays success when an upstream
# job is cancelled — read the needs results to detect that.
if [ "$SPECS_RESULT" = "cancelled" ] ||
[ "$E2E_RESULT" = "cancelled" ] ||
[ "$JOB_STATUS" = "cancelled" ]; then
UPSTREAM="false"
CANCELLED="true"
fi
IDENTITY=$(echo "$TSIO_CONFIG" | jq -c '.composite_identity')
CONTEXT=$(echo "$TSIO_CONFIG" | jq -r '.status_context // "e2e-test/detox-android"')
TOTAL=$(echo "$TSIO_CONFIG" | jq -r '.total_reports_expected // 1')
node detox/utils/tsio-report-status.js \
--composite-identity="$IDENTITY" \
--total-reports-expected="$TOTAL" \
--github-token="$GITHUB_TOKEN" \
--context="$CONTEXT" \
--upstream-succeeded="$UPSTREAM" \
--cancelled="$CANCELLED" \
--e2e-override="$E2E_OVERRIDE" \
--announce-triage="$ANNOUNCE_TRIAGE" \
--poll-attempts=24
RC=$?
echo "tsio-report-status exit=$RC upstream_succeeded=$UPSTREAM context=$CONTEXT"
if [ "$RC" -ne 0 ] && [ "$UPSTREAM" = "true" ]; then
echo "::error::TSIO reported test failures for $CONTEXT"
exit 1
fi
exit 0
- name: Save report Detox Dependencies
id: report-link
if: always()
continue-on-error: true
run: |
cd detox
npm ci
npm run e2e:save-report
env:
DETOX_AWS_ACCESS_KEY_ID: ${{ secrets.MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID }}
DETOX_AWS_SECRET_ACCESS_KEY: ${{ secrets.MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY }}
BUILD_ID: ${{ needs.generate-specs.outputs.build_id }}
REPORT_PATH: ${{ steps.s3.outputs.path }}
## These are needed for the MM Webhook report
COMMIT_HASH: ${{ needs.generate-specs.outputs.mobile_sha }}
BRANCH: ${{ needs.generate-specs.outputs.mobile_ref }}
- name: Calculate failures
id: summary
if: always()
run: |
if [ -f detox/artifacts/summary.json ]; then
echo "FAILURES=$(cat detox/artifacts/summary.json | jq .stats.failures)" >> ${GITHUB_OUTPUT}
echo "PASSES=$(cat detox/artifacts/summary.json | jq .stats.passes)" >> ${GITHUB_OUTPUT}
echo "SKIPPED=$(cat detox/artifacts/summary.json | jq .stats.skipped)" >> ${GITHUB_OUTPUT}
echo "TOTAL=$(cat detox/artifacts/summary.json | jq .stats.tests)" >> ${GITHUB_OUTPUT}
echo "ERRORS=$(cat detox/artifacts/summary.json | jq .stats.errors)" >> ${GITHUB_OUTPUT}
echo "PERCENTAGE=$(cat detox/artifacts/summary.json | jq .stats.passPercent)" >> ${GITHUB_OUTPUT}
else
echo "⚠️ No summary.json found — all test machines likely failed before producing results"
echo "FAILURES=-1" >> ${GITHUB_OUTPUT}
echo "PASSES=0" >> ${GITHUB_OUTPUT}
echo "SKIPPED=0" >> ${GITHUB_OUTPUT}
echo "TOTAL=0" >> ${GITHUB_OUTPUT}
echo "ERRORS=0" >> ${GITHUB_OUTPUT}
echo "PERCENTAGE=0" >> ${GITHUB_OUTPUT}
fi
- name: Set Target URL
id: set-url
if: always()
env:
S3_BUCKET: ${{ env.DETOX_AWS_S3_BUCKET }}
REPORT_PATH: ${{ steps.s3.outputs.path }}
run: |
echo "TARGET_URL=https://${S3_BUCKET}.s3.amazonaws.com/${REPORT_PATH}/jest-stare/android-report.html" >> "$GITHUB_OUTPUT"
- name: Determine Status
id: determine-status
if: always()
env:
FAILURES: ${{ steps.summary.outputs.FAILURES }}
ERRORS: ${{ steps.summary.outputs.ERRORS }}
TOTAL: ${{ steps.summary.outputs.TOTAL }}
run: |
# Fail on missing/empty reports (infra died before tests) and on real failures.
if [[ "$FAILURES" == "-1" || "$FAILURES" -gt 0 || "${ERRORS:-0}" -gt 0 || "${TOTAL:-0}" -eq 0 ]]; then
echo "STATUS=failure" >> ${GITHUB_OUTPUT}
else
echo "STATUS=success" >> ${GITHUB_OUTPUT}
fi
- name: Generate Summary
if: always()
env:
TOTAL: ${{ steps.summary.outputs.TOTAL }}
PASSES: ${{ steps.summary.outputs.PASSES }}
FAILURES: ${{ steps.summary.outputs.FAILURES }}
SKIPPED: ${{ steps.summary.outputs.SKIPPED }}
ERRORS: ${{ steps.summary.outputs.ERRORS }}
PERCENTAGE: ${{ steps.summary.outputs.PERCENTAGE }}
TARGET_URL: ${{ steps.set-url.outputs.TARGET_URL }}
run: |
echo "| Tests | Passed :white_check_mark: | Failed :x: | Skipped :fast_forward: | Errors :warning: | " >> ${GITHUB_STEP_SUMMARY}
echo "|:---:|:---:|:---:|:---:|:---:|" >> ${GITHUB_STEP_SUMMARY}
echo "| ${TOTAL} | ${PASSES} | ${FAILURES} | ${SKIPPED} | ${ERRORS} |" >> ${GITHUB_STEP_SUMMARY}
echo "" >> ${GITHUB_STEP_SUMMARY}
echo "You can check the full report [here](${TARGET_URL})" >> ${GITHUB_STEP_SUMMARY}
echo "There was **${PERCENTAGE}%** success rate." >> ${GITHUB_STEP_SUMMARY}
- name: Fail job on test failures
if: always() && steps.determine-status.outputs.STATUS == 'failure'
env:
FAILURES: ${{ steps.summary.outputs.FAILURES }}
TOTAL: ${{ steps.summary.outputs.TOTAL }}
ERRORS: ${{ steps.summary.outputs.ERRORS }}
run: |
echo "::error::Detox reported test failures (FAILURES=${FAILURES}, TOTAL=${TOTAL}, ERRORS=${ERRORS})"
exit 1