mirror of
https://github.com/mattermost/mattermost-mobile.git
synced 2026-11-15 07:30:30 +00:00
* ci(e2e): per-job TSIO report groups and commit statuses E2E jobs previously shared a single `e2e/mobile` commit status backed by one TSIO report group, so concurrent platform legs raced to finalize the group and clobbered each other's results. Give each Detox/Maestro job its own report group (`mobile-<flow>-<job>`, total_reports_expected=1) and its own commit status context (`detox-ios`, `detox-android`, `detox-ipad`, `maestro-ios`, `maestro-android`); CMT reports per shard plus an umbrella context. Channel notifications move to a single rollup step so N platform legs no longer spam the webhook, and the five-server topology is dispatched with explicit per-platform inputs that still resolve to the existing SITE_*_URL contract consumed by the test config. Also adds the webhook sidecar launcher, Android emulator CI prep, and TSIO screenshot path linking that keeps folder basenames matching Jest full names. The new detox type-check job surfaced pre-existing broken import paths in maestro/fixtures/calls_seed.ts, which pointed at detox/detox/e2e; corrected so the guard passes on its own first run. Note: the CMT matrix now reads per-platform server URLs (`ios_site_1_url`, `android_site_1_url`, `site_3_url`) instead of a single `url`, which needs the matching Matterwick dispatch change to land alongside this. Co-authored-by: Cursor <cursoragent@cursor.com> * ci(e2e): aggregate CMT channel rollup from TSIO and wire notify secrets The CMT umbrella posted a placeholder summary built from CI job outcomes (passed:0, failed:0|1, reports:[]), so the release channel message carried no per-leg detail. It now runs the same tsio-channel-notify-rollup.js path as PR/main: cmt-identity emits a per-shard TSIO config map and the rollup polls every shard group, so counts and per-leg rows come from TSIO. The rollup job also needs id-token: write to mint its OIDC token. Webhook secrets used by the per-job TSIO finalize steps were referenced but never declared in the templates' workflow_call.secrets, so GitHub resolved them to empty strings and channel notify silently no-opped. Declare them in the iOS/Android/Maestro templates and forward them from e2e-detox.yml and the CMT callers. The Maestro PR path worked only because its callers use secrets: inherit. Also from review: - split-tests.js emitted two matrix jobs at parallelism=1 when mm_blocks specs were present; merge into one shard instead of exceeding the requested count. - Move the Android ARTIFACT_RUN_ID emptiness check ahead of download-artifact, which consumes it and otherwise fails with an opaque API error. - resolve-tsio-job-config.sh now reuses webhookBucketForReportName instead of re-deriving the bucket prefix with a looser startsWith check. Co-authored-by: Cursor <cursoragent@cursor.com> * ci(e2e): add local preflight for CI PRs The webhook-secret and OIDC-permission gaps in this branch were invisible locally and in a green CI run — the affected steps exit 0 and just skip the notify or commit status. Add a script engineers can run before opening a PR that touches .github/ or detox/utils: cd detox && npm run preflight:ci # :ci-fast skips lint + tsc It runs actionlint (which flags secrets not declared in a callee's workflow_call.secrets), scans for exit-code capture broken by a trailing backslash, asserts every job running a TSIO OIDC script declares id-token: write, then runs the detox/utils tests and npm run check. actionlint and the CodeRabbit CLI are optional and skipped with an install hint when absent. Co-authored-by: Cursor <cursoragent@cursor.com> * ci(e2e): use mktemp for preflight logs Fixed /tmp log paths can be pre-created as symlinks, so the preflight script would truncate whatever they point at (CWE-377). Create both logs with mktemp and drop them on exit. Co-authored-by: Cursor <cursoragent@cursor.com> * ci(e2e): address review feedback on TSIO statuses and sidecar A cancelled E2E run reached no conclusion, so posting `success` on its commit statuses let a PR merge on a check that never ran. Cancelled runs now post `pending`; `success` is reserved for the one deliberate bypass, the E2E/Override label. Commit-status contexts move into the `e2e-test/*` namespace to match the mattermost monorepo, with the prefix owned by build-tsio-job-config.js and mirrored in the cancel action's context list. Legacy unprefixed and `e2e/*` contexts are dropped rather than carried forward. The `preflight:ci` script and its OIDC-permission checker duplicated what actionlint, the unit tests and `npm run check` already do, so they are removed and detox/CLAUDE.md documents the underlying failure modes and the commands to catch them instead. cloudflared and the TSIO CLI are pinned to known-good releases rather than `latest`, and detox/README.md now documents the webhook sidecar: when a tunnel is needed, the WEBHOOK_PUBLIC_BASE_URL / CLOUDFLARED_TUNNEL_TOKEN precedence, and why quick tunnels report callbacks as unreachable. Co-authored-by: Cursor <cursoragent@cursor.com> * ci(e2e): detect upstream cancellation and surface status reset failures generate-report runs with always(), so job.status stays "success" when an upstream job is cancelled and TSIO was published as a real failure. Read the generate-specs/e2e-* needs results so a cancelled run is reported as such. In cancel-e2e-runs, only reset the E2E commit statuses for runs that were actually stopped — a run we failed to cancel keeps publishing its own results. Aggregate createCommitStatus failures across every context and fail the action afterwards instead of silently warning, so a stale check cannot go unnoticed. --------- Co-authored-by: Cursor <cursoragent@cursor.com>
716 lines
30 KiB
YAML
716 lines
30 KiB
YAML
name: Detox Android E2E Tests Template
|
|
|
|
on:
|
|
workflow_call:
|
|
secrets:
|
|
MM_MOBILE_E2E_ADMIN_EMAIL:
|
|
required: false
|
|
MM_MOBILE_E2E_ADMIN_USERNAME:
|
|
required: false
|
|
MM_MOBILE_E2E_ADMIN_PASSWORD:
|
|
required: false
|
|
MM_MOBILE_E2E_ZEPHYR_API_KEY:
|
|
required: false
|
|
MM_MOBILE_E2E_TEST_CYCLE_LINK_PREFIX:
|
|
required: false
|
|
MM_MOBILE_E2E_WEBHOOK_URL:
|
|
required: false
|
|
# Channel-notify routing for the per-job TSIO finalize step. Callers that use
|
|
# an explicit `secrets:` block must forward these or notify silently no-ops.
|
|
MM_E2E_MASTER_HEALTH_WEBHOOK_URL:
|
|
required: false
|
|
MM_E2E_RELEASE_WEBHOOK_URL:
|
|
required: false
|
|
MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID:
|
|
required: false
|
|
MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY:
|
|
required: false
|
|
# Webhook sidecar. Unset means mm_blocks specs fall back to trycloudflare.
|
|
MM_MOBILE_E2E_WEBHOOK_PUBLIC_BASE_URL:
|
|
required: false
|
|
MM_MOBILE_E2E_CLOUDFLARED_TUNNEL_TOKEN:
|
|
required: false
|
|
inputs:
|
|
MM_TEST_SERVER_URL:
|
|
description: "The test server URL (SITE_1)"
|
|
required: false
|
|
type: string
|
|
MM_TEST_SERVER_URL_2:
|
|
description: "The second test server URL (SITE_2); optional, no automatic fallback"
|
|
required: false
|
|
type: string
|
|
MM_TEST_SERVER_URL_3:
|
|
description: "The third test server URL (SITE_3); optional, no automatic fallback"
|
|
required: false
|
|
type: string
|
|
MM_TEST_USER_NAME:
|
|
description: "The admin username of the test instance"
|
|
required: false
|
|
type: string
|
|
MM_TEST_PASSWORD:
|
|
description: "The admin password of the test instance"
|
|
required: false
|
|
type: string
|
|
MOBILE_VERSION:
|
|
description: "The mobile version to test"
|
|
required: false
|
|
default: ""
|
|
type: string
|
|
run-android-tests:
|
|
description: "Run Android tests"
|
|
required: true
|
|
type: boolean
|
|
run-type:
|
|
type: string
|
|
required: false
|
|
default: "PR"
|
|
record_tests_in_zephyr:
|
|
description: "Record test results in Zephyr, typically for release and main runs"
|
|
required: false
|
|
type: boolean
|
|
default: false
|
|
low_bandwidth_mode:
|
|
description: "Enable low bandwidth mode"
|
|
required: false
|
|
type: boolean
|
|
default: false
|
|
search_path:
|
|
description: "Path to search for test specs (use for smoke test subset)"
|
|
required: false
|
|
type: string
|
|
default: "detox/e2e/test"
|
|
parallelism:
|
|
description: "Number of parallel test shards"
|
|
required: false
|
|
type: string
|
|
default: "20"
|
|
android_avd_name:
|
|
description: "Android Emulator name"
|
|
required: false
|
|
type: string
|
|
default: "detox_pixel_8"
|
|
android_api_level:
|
|
description: "Android API level"
|
|
required: false
|
|
type: string
|
|
default: "35"
|
|
artifact_run_id:
|
|
description: "Run ID of the workflow that uploaded android-build-files-* (defaults to caller run)"
|
|
required: false
|
|
type: string
|
|
default: ""
|
|
tsio-config:
|
|
description: >
|
|
TSIO reporting config: {"composite_identity":{...},"total_reports_expected":N}.
|
|
Empty disables TSIO upload for this platform.
|
|
required: false
|
|
type: string
|
|
default: ""
|
|
tsio-shard-name:
|
|
description: "Display name for this shard in TSIO (e.g. detox-android-Server_11.9.0-rc3)"
|
|
required: false
|
|
type: string
|
|
default: ""
|
|
outputs:
|
|
STATUS:
|
|
value: ${{ jobs.generate-report.outputs.STATUS }}
|
|
TARGET_URL:
|
|
value: ${{ jobs.generate-report.outputs.TARGET_URL }}
|
|
FAILURES:
|
|
value: ${{ jobs.generate-report.outputs.FAILURES }}
|
|
PASSES:
|
|
value: ${{ jobs.generate-report.outputs.PASSES }}
|
|
TOTAL:
|
|
value: ${{ jobs.generate-report.outputs.TOTAL }}
|
|
SKIPPED:
|
|
value: ${{ jobs.generate-report.outputs.SKIPPED }}
|
|
ERRORS:
|
|
value: ${{ jobs.generate-report.outputs.ERRORS }}
|
|
PERCENTAGE:
|
|
value: ${{ jobs.generate-report.outputs.PERCENTAGE }}
|
|
|
|
env:
|
|
AWS_REGION: "us-east-1"
|
|
ADMIN_EMAIL: ${{ secrets.MM_MOBILE_E2E_ADMIN_EMAIL }}
|
|
ADMIN_USERNAME: ${{ inputs.MM_TEST_USER_NAME || secrets.MM_MOBILE_E2E_ADMIN_USERNAME }}
|
|
ADMIN_PASSWORD: ${{ inputs.MM_TEST_PASSWORD || secrets.MM_MOBILE_E2E_ADMIN_PASSWORD }}
|
|
BRANCH: ${{ github.event_name == 'pull_request' && github.head_ref || github.ref_name }}
|
|
COMMIT_HASH: ${{ github.sha }}
|
|
DEVICE_NAME: ${{ inputs.android_avd_name }} # This is needed to split tests as same code is used in iOS job
|
|
DEVICE_OS_VERSION: ${{ inputs.android_api_level }}
|
|
DETOX_AWS_S3_BUCKET: "mattermost-detox-report"
|
|
HEADLESS: "true"
|
|
TYPE: ${{ inputs.run-type }}
|
|
PULL_REQUEST: "https://github.com/mattermost/mattermost-mobile/pull/${{ github.event.number }}"
|
|
SITE_1_URL: ${{ inputs.MM_TEST_SERVER_URL }}
|
|
SITE_2_URL: ${{ inputs.MM_TEST_SERVER_URL_2 }}
|
|
SITE_3_URL: ${{ inputs.MM_TEST_SERVER_URL_3 }}
|
|
ZEPHYR_ENABLE: ${{ inputs.record_tests_in_zephyr }}
|
|
JIRA_PROJECT_KEY: "MM"
|
|
ZEPHYR_API_KEY: ${{ secrets.MM_MOBILE_E2E_ZEPHYR_API_KEY }}
|
|
ZEPHYR_FOLDER_ID: "3233873"
|
|
TEST_CYCLE_LINK_PREFIX: ${{ secrets.MM_MOBILE_E2E_TEST_CYCLE_LINK_PREFIX }}
|
|
WEBHOOK_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_URL }}
|
|
FAILURE_MESSAGE: "Something has failed"
|
|
IOS: "false"
|
|
RUNNING_E2E: "true"
|
|
AVD_NAME: ${{ inputs.android_avd_name }}
|
|
SDK_VERSION: ${{ inputs.android_api_level }}
|
|
ARTIFACT_RUN_ID: ${{ inputs.artifact_run_id }}
|
|
|
|
jobs:
|
|
generate-specs:
|
|
runs-on: ubuntu-24.04
|
|
outputs:
|
|
specs: ${{ steps.generate-specs.outputs.specs }}
|
|
build_id: ${{ steps.resolve-device.outputs.BUILD_ID }}
|
|
mobile_sha: ${{ steps.resolve-device.outputs.MOBILE_SHA }}
|
|
mobile_ref: ${{ steps.resolve-device.outputs.MOBILE_REF }}
|
|
workflow_hash: ${{ steps.resolve-device.outputs.WORKFLOW_HASH }}
|
|
steps:
|
|
- name: Checkout Repository
|
|
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
|
with:
|
|
ref: ${{ inputs.MOBILE_VERSION }}
|
|
|
|
- name: Set Build ID
|
|
id: resolve-device
|
|
run: |
|
|
BUILD_ID="${{ github.run_id }}-${{ env.AVD_NAME }}-${{ env.SDK_VERSION}}"
|
|
WORKFLOW_HASH=$(tr -dc a-z0-9 </dev/urandom | head -c 10)
|
|
|
|
## We need that hash to separate the artifacts
|
|
echo "WORKFLOW_HASH=${WORKFLOW_HASH}" >> ${GITHUB_OUTPUT}
|
|
|
|
echo "BUILD_ID=$(echo ${BUILD_ID} | sed 's/ /_/g')" >> ${GITHUB_OUTPUT}
|
|
echo "MOBILE_SHA=$(git rev-parse HEAD)" >> ${GITHUB_OUTPUT}
|
|
echo "MOBILE_REF=$(git rev-parse --abbrev-ref HEAD)" >> ${GITHUB_OUTPUT}
|
|
|
|
- name: Generate Test Specs
|
|
id: generate-specs
|
|
uses: ./.github/actions/generate-specs
|
|
with:
|
|
parallelism: ${{ inputs.parallelism }}
|
|
search_path: ${{ inputs.search_path }}
|
|
device_name: ${{ env.AVD_NAME }}
|
|
device_os_version: ${{ env.SDK_VERSION }}
|
|
|
|
e2e-android:
|
|
name: machine-${{ matrix.runId }}-api-${{ matrix.deviceOsVersion }}
|
|
runs-on: ubuntu-latest-8-cores
|
|
continue-on-error: true
|
|
timeout-minutes: 240
|
|
env:
|
|
ANDROID_HOME: /usr/local/lib/android/sdk
|
|
ANDROID_SDK_ROOT: /usr/local/lib/android/sdk
|
|
needs:
|
|
- generate-specs
|
|
strategy:
|
|
fail-fast: false
|
|
matrix: ${{ fromJSON(needs.generate-specs.outputs.specs) }}
|
|
steps:
|
|
- name: Checkout Repository
|
|
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
|
with:
|
|
ref: ${{ inputs.MOBILE_VERSION }}
|
|
|
|
# Each platform gets two provisioned test servers. SITE_1_URL is where a spec
|
|
# does most of its setup, so pinning every shard to the same server would put
|
|
# all parallel shards on one instance while the second sits mostly idle —
|
|
# concurrent setup on one server is what drives API rate limiting and
|
|
# cross-shard interference. Alternating SITE_1/SITE_2 on shard parity spreads
|
|
# that load evenly while still giving each shard two distinct sites for the
|
|
# multi-server specs.
|
|
- name: Rotate logical test sites by shard
|
|
env:
|
|
SERVER_A: ${{ inputs.MM_TEST_SERVER_URL }}
|
|
SERVER_B: ${{ inputs.MM_TEST_SERVER_URL_2 }}
|
|
SERVER_SHARED: ${{ inputs.MM_TEST_SERVER_URL_3 }}
|
|
SHARD_ID: ${{ matrix.runId }}
|
|
run: |
|
|
# Single-server topologies leave SERVER_B unset; without this the even
|
|
# shards would export an empty SITE_1_URL and test_config would silently
|
|
# fall back to localhost instead of the provisioned server.
|
|
SERVER_B="${SERVER_B:-$SERVER_A}"
|
|
if (( 10#$SHARD_ID % 2 == 1 )); then
|
|
SITE_1_URL="$SERVER_A"
|
|
SITE_2_URL="$SERVER_B"
|
|
else
|
|
SITE_1_URL="$SERVER_B"
|
|
SITE_2_URL="$SERVER_A"
|
|
fi
|
|
{
|
|
echo "SITE_1_URL=$SITE_1_URL"
|
|
echo "SITE_2_URL=$SITE_2_URL"
|
|
echo "SITE_3_URL=$SERVER_SHARED"
|
|
} >> "$GITHUB_ENV"
|
|
|
|
- name: Install Dependencies
|
|
run: |
|
|
sudo apt-get update
|
|
sudo apt-get install -y libpulse0
|
|
sudo apt-get install -y scrot ffmpeg xvfb
|
|
|
|
- name: Enable KVM
|
|
run: |
|
|
echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules
|
|
sudo udevadm control --reload-rules
|
|
sudo udevadm trigger --name-match=kvm
|
|
|
|
- name: ci/prepare-node-deps
|
|
uses: ./.github/actions/prepare-node-deps
|
|
|
|
- name: Cache detox node_modules
|
|
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
|
|
with:
|
|
path: detox/node_modules
|
|
key: ${{ runner.os }}-detox-deps-${{ hashFiles('detox/package-lock.json') }}
|
|
restore-keys: ${{ runner.os }}-detox-deps-
|
|
|
|
- name: Install Detox Dependencies
|
|
run: |
|
|
cd detox
|
|
npm install
|
|
|
|
- name: Set .env with RUNNING_E2E=true
|
|
run: |
|
|
cat > .env <<EOF
|
|
RUNNING_E2E=true
|
|
EOF
|
|
|
|
# Must precede the download: download-artifact consumes ARTIFACT_RUN_ID and
|
|
# fails with an opaque API error when it is empty.
|
|
- name: Verify artifact_run_id was passed by the parent workflow
|
|
env:
|
|
ARTIFACT_RUN_ID: ${{ env.ARTIFACT_RUN_ID }}
|
|
run: |
|
|
if [ -z "${ARTIFACT_RUN_ID}" ]; then
|
|
echo "❌ ARTIFACT_RUN_ID is empty — parent workflow must pass artifact_run_id from build-android-apk"
|
|
exit 1
|
|
fi
|
|
|
|
- name: Download Detox Android build from parent workflow
|
|
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
|
|
with:
|
|
name: android-build-files-${{ env.ARTIFACT_RUN_ID }}
|
|
repository: ${{ github.repository }}
|
|
run-id: ${{ env.ARTIFACT_RUN_ID }}
|
|
github-token: ${{ github.token }}
|
|
path: android/app/build
|
|
|
|
- name: Verify Detox Android app from parent build
|
|
run: |
|
|
APP_APK="android/app/build/outputs/apk/debug/app-debug.apk"
|
|
TEST_APK="android/app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk"
|
|
if [ ! -f "$APP_APK" ]; then
|
|
echo "❌ Missing ${APP_APK} from android-build-files-${ARTIFACT_RUN_ID}"
|
|
find android/app/build -name '*.apk' 2>/dev/null || true
|
|
exit 1
|
|
fi
|
|
if [ ! -f "$TEST_APK" ]; then
|
|
echo "❌ Missing ${TEST_APK} from android-build-files-${ARTIFACT_RUN_ID}"
|
|
find android/app/build -name '*.apk' 2>/dev/null || true
|
|
exit 1
|
|
fi
|
|
echo "✅ Detox will use Android APKs from parent workflow run ${ARTIFACT_RUN_ID}:"
|
|
ls -lh "$APP_APK" "$TEST_APK"
|
|
env:
|
|
ARTIFACT_RUN_ID: ${{ env.ARTIFACT_RUN_ID }}
|
|
|
|
- name: Set up Android SDK
|
|
run: |
|
|
export ANDROID_HOME=/usr/local/lib/android/sdk
|
|
export PATH=$ANDROID_HOME/cmdline-tools/latest/bin:$ANDROID_HOME/emulator:$ANDROID_HOME/tools/bin:$ANDROID_HOME/platform-tools:$PATH
|
|
echo "ANDROID_HOME=$ANDROID_HOME" >> $GITHUB_ENV
|
|
echo "PATH=$PATH" >> $GITHUB_ENV
|
|
|
|
- name: Start Xvfb
|
|
run: |
|
|
Xvfb :99 -screen 0 1920x1080x24 &
|
|
export DISPLAY=:99
|
|
echo "DISPLAY=:99" >> $GITHUB_ENV
|
|
|
|
- name: Accept Android licenses
|
|
run: |
|
|
mkdir -p ~/.android
|
|
yes | sdkmanager --licenses > sdk_licenses_output.txt || true
|
|
|
|
# Check if "All SDK package licenses accepted" appears in output
|
|
if grep -q "All SDK package licenses accepted" sdk_licenses_output.txt; then
|
|
echo "✅ All licenses accepted successfully."
|
|
else
|
|
echo "❌ Licenses not fully accepted."
|
|
cat sdk_licenses_output.txt
|
|
exit 1
|
|
fi
|
|
|
|
# Cache SDK under the workspace (not /usr/local) — tar restore into system
|
|
# SDK dirs often fails with exit code 2 when paths already exist on the image.
|
|
- name: Cache Android SDK components
|
|
id: android-sdk-cache
|
|
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
|
|
with:
|
|
path: .android-sdk-cache
|
|
key: android-sdk-${{ inputs.android_api_level }}-google-apis-x86_64-v2
|
|
|
|
- name: Apply cached Android SDK components
|
|
if: steps.android-sdk-cache.outputs.cache-hit == 'true'
|
|
env:
|
|
API_LEVEL: ${{ inputs.android_api_level }}
|
|
run: |
|
|
for component in platform-tools emulator; do
|
|
if [ -d ".android-sdk-cache/${component}" ]; then
|
|
rm -rf "${ANDROID_HOME:?}/${component}"
|
|
cp -a ".android-sdk-cache/${component}" "${ANDROID_HOME}/"
|
|
fi
|
|
done
|
|
if [ -d ".android-sdk-cache/platforms/android-${API_LEVEL}" ]; then
|
|
rm -rf "${ANDROID_HOME}/platforms/android-${API_LEVEL}"
|
|
mkdir -p "${ANDROID_HOME}/platforms"
|
|
cp -a ".android-sdk-cache/platforms/android-${API_LEVEL}" "${ANDROID_HOME}/platforms/"
|
|
fi
|
|
if [ -d ".android-sdk-cache/system-images/android-${API_LEVEL}" ]; then
|
|
rm -rf "${ANDROID_HOME}/system-images/android-${API_LEVEL}"
|
|
mkdir -p "${ANDROID_HOME}/system-images"
|
|
cp -a ".android-sdk-cache/system-images/android-${API_LEVEL}" "${ANDROID_HOME}/system-images/"
|
|
fi
|
|
|
|
- name: Install Android SDK components
|
|
if: steps.android-sdk-cache.outputs.cache-hit != 'true'
|
|
env:
|
|
API_LEVEL: ${{ inputs.android_api_level }}
|
|
JAVA_HOME: ${{ env.JAVA_HOME_17_X64 }}
|
|
run: |
|
|
if ! [[ "$API_LEVEL" =~ ^[0-9]+$ ]]; then
|
|
echo "Invalid android_api_level: $API_LEVEL"
|
|
exit 1
|
|
fi
|
|
for attempt in 1 2 3; do
|
|
echo "SDK install attempt $attempt..."
|
|
if yes | sdkmanager --install "platform-tools" "emulator" "platforms;android-${API_LEVEL}" "system-images;android-${API_LEVEL};google_apis;x86_64"; then
|
|
echo "SDK install succeeded on attempt $attempt"
|
|
break
|
|
fi
|
|
echo "Attempt $attempt failed, retrying..."
|
|
if [ "$attempt" = "3" ]; then
|
|
echo "SDK install failed after 3 attempts"
|
|
exit 1
|
|
fi
|
|
sleep 10
|
|
done
|
|
mkdir -p .android-sdk-cache/platforms .android-sdk-cache/system-images
|
|
cp -a "${ANDROID_HOME}/platform-tools" .android-sdk-cache/
|
|
cp -a "${ANDROID_HOME}/emulator" .android-sdk-cache/
|
|
cp -a "${ANDROID_HOME}/platforms/android-${API_LEVEL}" .android-sdk-cache/platforms/
|
|
cp -a "${ANDROID_HOME}/system-images/android-${API_LEVEL}" .android-sdk-cache/system-images/
|
|
|
|
# CI AVDs live under detox/.android/avd (see create_android_emulator.sh), not ~/.android/avd.
|
|
- name: Cache Android Emulator & AVD
|
|
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
|
|
with:
|
|
path: |
|
|
detox/.android
|
|
detox/${{ env.AVD_NAME }}_api_${{ inputs.android_api_level }}
|
|
key: android-emulator-${{ runner.os }}-${{ runner.arch }}-${{ env.AVD_NAME }}-api-${{ inputs.android_api_level }}-v3
|
|
restore-keys: |
|
|
android-emulator-${{ runner.os }}-${{ runner.arch }}-${{ env.AVD_NAME }}-api-${{ inputs.android_api_level }}-
|
|
android-emulator-${{ runner.os }}-${{ runner.arch }}-
|
|
|
|
# mm_blocks specs are sharded alone (split-tests.js). Script bounds curls and
|
|
# exits 0 on tunnel failure; prefer WEBHOOK_PUBLIC_BASE_URL over trycloudflare.
|
|
- name: Start mm_blocks webhook sidecar
|
|
if: contains(matrix.specs, 'mm_blocks_')
|
|
continue-on-error: true
|
|
env:
|
|
SITE_1_URL: ${{ env.SITE_1_URL }}
|
|
WEBHOOK_PUBLIC_BASE_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_PUBLIC_BASE_URL }}
|
|
CLOUDFLARED_TUNNEL_TOKEN: ${{ secrets.MM_MOBILE_E2E_CLOUDFLARED_TUNNEL_TOKEN }}
|
|
run: bash detox/scripts/start_webhook_sidecar.sh
|
|
|
|
- name: Create and run Android Emulator
|
|
env:
|
|
MATRIX_SPECS: ${{ matrix.specs }}
|
|
SITE_1_URL: ${{ env.SITE_1_URL }}
|
|
SITE_2_URL: ${{ env.SITE_2_URL }}
|
|
SITE_3_URL: ${{ env.SITE_3_URL }}
|
|
MAX_ATTEMPTS: "2"
|
|
ATTEMPT_TIMEOUT_MIN: "60"
|
|
shell: bash
|
|
run: |
|
|
set +e
|
|
attempt=1
|
|
while (( attempt <= MAX_ATTEMPTS )); do
|
|
echo "==> Attempt ${attempt}/${MAX_ATTEMPTS} (timeout ${ATTEMPT_TIMEOUT_MIN}m)"
|
|
timeout "${ATTEMPT_TIMEOUT_MIN}m" bash -c '
|
|
cd detox
|
|
chmod +x ./create_android_emulator.sh
|
|
CI=true ./create_android_emulator.sh "$SDK_VERSION" "$AVD_NAME" $MATRIX_SPECS
|
|
'
|
|
rc=$?
|
|
if (( rc == 0 )); then
|
|
echo "==> Attempt ${attempt} succeeded"
|
|
exit 0
|
|
fi
|
|
echo "==> Attempt ${attempt} failed with exit ${rc}"
|
|
if (( attempt < MAX_ATTEMPTS )); then
|
|
echo "==> Cleaning up before retry"
|
|
adb -s emulator-5554 emu kill 2>/dev/null || true
|
|
pkill -9 -f qemu-system 2>/dev/null || true
|
|
pkill -9 -f emulator 2>/dev/null || true
|
|
rm -rf detox/artifacts/* 2>/dev/null || true
|
|
sleep 5
|
|
fi
|
|
attempt=$(( attempt + 1 ))
|
|
done
|
|
echo "==> All ${MAX_ATTEMPTS} attempts failed"
|
|
exit 1
|
|
continue-on-error: true # We want to run all the tests
|
|
|
|
- name: Upload Android Test Report
|
|
if: always() && hashFiles('detox/artifacts/**') != ''
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
|
with:
|
|
name: android-results-${{ needs.generate-specs.outputs.workflow_hash }}-${{ matrix.runId }}
|
|
path: detox/artifacts/
|
|
|
|
generate-report:
|
|
# TSIO displays gh-job-name as the shard label — keep it aligned with tsio-shard-name.
|
|
name: ${{ inputs.tsio-shard-name != '' && inputs.tsio-shard-name || 'generate-report' }}
|
|
if: always()
|
|
runs-on: ubuntu-24.04
|
|
needs:
|
|
- generate-specs
|
|
- e2e-android
|
|
permissions:
|
|
contents: read
|
|
# test-system-io-report-upload authenticates via the workflow's OIDC
|
|
# token and resolves gh_job_id via the GitHub API.
|
|
id-token: write
|
|
actions: read
|
|
# Per-job TSIO finalize posts detox-android (or CMT <shard>).
|
|
statuses: write
|
|
outputs:
|
|
TARGET_URL: ${{ steps.set-url.outputs.TARGET_URL }}
|
|
STATUS: ${{ steps.determine-status.outputs.STATUS }}
|
|
FAILURES: ${{ steps.summary.outputs.FAILURES }}
|
|
PASSES: ${{ steps.summary.outputs.PASSES }}
|
|
TOTAL: ${{ steps.summary.outputs.TOTAL }}
|
|
SKIPPED: ${{ steps.summary.outputs.SKIPPED }}
|
|
ERRORS: ${{ steps.summary.outputs.ERRORS }}
|
|
PERCENTAGE: ${{ steps.summary.outputs.PERCENTAGE }}
|
|
steps:
|
|
- name: Checkout Repository
|
|
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
|
with:
|
|
ref: ${{ inputs.MOBILE_VERSION }}
|
|
|
|
- name: ci/prepare-node-deps
|
|
uses: ./.github/actions/prepare-node-deps
|
|
|
|
- name: Download Android Artifacts
|
|
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
|
|
with:
|
|
path: detox/artifacts/
|
|
pattern: android-results-${{ needs.generate-specs.outputs.workflow_hash }}-*
|
|
continue-on-error: true
|
|
|
|
- name: Generate Report Path
|
|
id: s3
|
|
env:
|
|
BUILD_ID: ${{ needs.generate-specs.outputs.build_id }}
|
|
MOBILE_SHA: ${{ needs.generate-specs.outputs.mobile_sha }}
|
|
MOBILE_REF: ${{ needs.generate-specs.outputs.mobile_ref }}
|
|
run: |
|
|
path="${BUILD_ID}-${MOBILE_SHA}-${MOBILE_REF}"
|
|
echo "path=$(echo "${path}" | sed 's/\./-/g')" >> "$GITHUB_OUTPUT"
|
|
|
|
# ── TSIO report upload (before e2e:save-report) ────────────────────────
|
|
# save-report deletes android-results-* dirs (where Detox PNGs live). Upload
|
|
# while those artifact trees are still on disk.
|
|
# Each matrix leg wrote Jest CLI --json (name + assertionResults). TSIO's
|
|
# extractDetox expects testFilePath + nested testResults — convert while
|
|
# merging. gh-job-name must match this job's rendered `name:` (tsio-shard-name).
|
|
- name: Merge per-shard Jest JSON for TSIO
|
|
if: always() && inputs.tsio-config != '' && vars.TSIO_UPLOAD_ENABLED != 'false'
|
|
id: tsio-merge
|
|
run: |
|
|
set +e
|
|
# TSIO extractDetox expects testFilePath + nested testResults (not Jest CLI
|
|
# name + assertionResults). Convert while merging shard JSON files.
|
|
if ! node detox/utils/merge-jest-results-for-tsio.js \
|
|
--dir detox/artifacts \
|
|
--repo-root "${GITHUB_WORKSPACE}" \
|
|
--output detox/artifacts/jest-results-merged.json; then
|
|
echo "skip=true" >> "$GITHUB_OUTPUT"
|
|
echo "No jest-results.json found or merge failed — skipping TSIO upload."
|
|
exit 0
|
|
fi
|
|
echo "skip=false" >> "$GITHUB_OUTPUT"
|
|
|
|
- name: Write TSIO failure stub when Detox JSON is missing
|
|
if: always() && inputs.tsio-config != '' && vars.TSIO_UPLOAD_ENABLED != 'false' && steps.tsio-merge.outputs.skip == 'true'
|
|
env:
|
|
TSIO_SHARD_NAME: ${{ inputs.tsio-shard-name != '' && inputs.tsio-shard-name || 'generate-report' }}
|
|
run: |
|
|
node detox/utils/write-tsio-failure-stub.mjs \
|
|
--format jest \
|
|
--output detox/artifacts/jest-results-merged.json \
|
|
--job-name "$TSIO_SHARD_NAME" \
|
|
--reason "Detox job failed or was cancelled before jest-results.json was written"
|
|
|
|
- name: Resolve per-job TSIO config
|
|
if: always() && inputs.tsio-config != '' && vars.TSIO_UPLOAD_ENABLED != 'false'
|
|
id: tsio-cfg
|
|
env:
|
|
TSIO_CONFIG: ${{ inputs.tsio-config }}
|
|
TSIO_SHARD_NAME: ${{ inputs.tsio-shard-name }}
|
|
run: |
|
|
CFG=$(bash detox/utils/resolve-tsio-job-config.sh)
|
|
{
|
|
echo "config<<EOF"
|
|
echo "$CFG"
|
|
echo "EOF"
|
|
} >> "$GITHUB_OUTPUT"
|
|
echo "Resolved TSIO job: $(echo "$CFG" | jq -r '.composite_identity.name') context=$(echo "$CFG" | jq -r '.status_context')"
|
|
|
|
- name: Upload report to Test System IO
|
|
if: always() && inputs.tsio-config != '' && vars.TSIO_UPLOAD_ENABLED != 'false' && hashFiles('detox/artifacts/jest-results-merged.json') != '' && steps.tsio-cfg.outputs.config != ''
|
|
uses: mattermost/mattermost-test-system-io/.github/actions/test-system-io-report-upload@0661accd8cc9ab9dde6ec93f6f5ad40d06deb0d3 # release-0.14.0 / main@0661acc (#93 unique release tags per run)
|
|
with:
|
|
composite-identity: ${{ toJSON(fromJSON(steps.tsio-cfg.outputs.config).composite_identity) }}
|
|
total-reports-expected: ${{ fromJSON(steps.tsio-cfg.outputs.config).total_reports_expected }}
|
|
framework: detox
|
|
github-token: ${{ github.token }}
|
|
gh-job-name: ${{ inputs.tsio-shard-name != '' && inputs.tsio-shard-name || 'generate-report' }}
|
|
json-path: detox/artifacts/jest-results-merged.json
|
|
screenshots-dir: detox/artifacts/
|
|
|
|
- name: Post per-job TSIO commit status
|
|
if: always() && inputs.tsio-config != '' && vars.TSIO_UPLOAD_ENABLED != 'false' && steps.tsio-cfg.outputs.config != ''
|
|
env:
|
|
GITHUB_TOKEN: ${{ github.token }}
|
|
TSIO_CONFIG: ${{ steps.tsio-cfg.outputs.config }}
|
|
SPECS_RESULT: ${{ needs.generate-specs.result }}
|
|
E2E_RESULT: ${{ needs.e2e-android.result }}
|
|
JOB_STATUS: ${{ job.status }}
|
|
MATTERMOST_E2E_WEBHOOK_URL: ${{ secrets.MM_MOBILE_E2E_WEBHOOK_URL }}
|
|
MATTERMOST_MASTER_HEALTH_WEBHOOK_URL: ${{ secrets.MM_E2E_MASTER_HEALTH_WEBHOOK_URL }}
|
|
MATTERMOST_CMT_WEBHOOK_URL: ${{ secrets.MM_E2E_RELEASE_WEBHOOK_URL }}
|
|
run: |
|
|
set +e
|
|
UPSTREAM="true"
|
|
# This job runs with always(), so job.status stays success when an upstream
|
|
# job is cancelled — read the needs results to detect that.
|
|
if [ "$SPECS_RESULT" = "cancelled" ] ||
|
|
[ "$E2E_RESULT" = "cancelled" ] ||
|
|
[ "$JOB_STATUS" = "cancelled" ]; then
|
|
UPSTREAM="false"
|
|
fi
|
|
IDENTITY=$(echo "$TSIO_CONFIG" | jq -c '.composite_identity')
|
|
CONTEXT=$(echo "$TSIO_CONFIG" | jq -r '.status_context // "e2e-test/detox-android"')
|
|
TOTAL=$(echo "$TSIO_CONFIG" | jq -r '.total_reports_expected // 1')
|
|
node detox/utils/tsio-report-status.js \
|
|
--composite-identity="$IDENTITY" \
|
|
--total-reports-expected="$TOTAL" \
|
|
--github-token="$GITHUB_TOKEN" \
|
|
--context="$CONTEXT" \
|
|
--upstream-succeeded="$UPSTREAM" \
|
|
--poll-attempts=24
|
|
RC=$?
|
|
echo "tsio-report-status exit=$RC upstream_succeeded=$UPSTREAM context=$CONTEXT"
|
|
if [ "$RC" -ne 0 ] && [ "$UPSTREAM" = "true" ]; then
|
|
echo "::error::TSIO reported test failures for $CONTEXT"
|
|
exit 1
|
|
fi
|
|
exit 0
|
|
|
|
- name: Save report Detox Dependencies
|
|
id: report-link
|
|
if: always()
|
|
continue-on-error: true
|
|
run: |
|
|
cd detox
|
|
npm ci
|
|
npm run e2e:save-report
|
|
env:
|
|
DETOX_AWS_ACCESS_KEY_ID: ${{ secrets.MM_MOBILE_DETOX_AWS_ACCESS_KEY_ID }}
|
|
DETOX_AWS_SECRET_ACCESS_KEY: ${{ secrets.MM_MOBILE_DETOX_AWS_SECRET_ACCESS_KEY }}
|
|
BUILD_ID: ${{ needs.generate-specs.outputs.build_id }}
|
|
REPORT_PATH: ${{ steps.s3.outputs.path }}
|
|
## These are needed for the MM Webhook report
|
|
COMMIT_HASH: ${{ needs.generate-specs.outputs.mobile_sha }}
|
|
BRANCH: ${{ needs.generate-specs.outputs.mobile_ref }}
|
|
|
|
- name: Calculate failures
|
|
id: summary
|
|
if: always()
|
|
run: |
|
|
if [ -f detox/artifacts/summary.json ]; then
|
|
echo "FAILURES=$(cat detox/artifacts/summary.json | jq .stats.failures)" >> ${GITHUB_OUTPUT}
|
|
echo "PASSES=$(cat detox/artifacts/summary.json | jq .stats.passes)" >> ${GITHUB_OUTPUT}
|
|
echo "SKIPPED=$(cat detox/artifacts/summary.json | jq .stats.skipped)" >> ${GITHUB_OUTPUT}
|
|
echo "TOTAL=$(cat detox/artifacts/summary.json | jq .stats.tests)" >> ${GITHUB_OUTPUT}
|
|
echo "ERRORS=$(cat detox/artifacts/summary.json | jq .stats.errors)" >> ${GITHUB_OUTPUT}
|
|
echo "PERCENTAGE=$(cat detox/artifacts/summary.json | jq .stats.passPercent)" >> ${GITHUB_OUTPUT}
|
|
else
|
|
echo "⚠️ No summary.json found — all test machines likely failed before producing results"
|
|
echo "FAILURES=-1" >> ${GITHUB_OUTPUT}
|
|
echo "PASSES=0" >> ${GITHUB_OUTPUT}
|
|
echo "SKIPPED=0" >> ${GITHUB_OUTPUT}
|
|
echo "TOTAL=0" >> ${GITHUB_OUTPUT}
|
|
echo "ERRORS=0" >> ${GITHUB_OUTPUT}
|
|
echo "PERCENTAGE=0" >> ${GITHUB_OUTPUT}
|
|
fi
|
|
|
|
- name: Set Target URL
|
|
id: set-url
|
|
if: always()
|
|
env:
|
|
S3_BUCKET: ${{ env.DETOX_AWS_S3_BUCKET }}
|
|
REPORT_PATH: ${{ steps.s3.outputs.path }}
|
|
run: |
|
|
echo "TARGET_URL=https://${S3_BUCKET}.s3.amazonaws.com/${REPORT_PATH}/jest-stare/android-report.html" >> "$GITHUB_OUTPUT"
|
|
|
|
- name: Determine Status
|
|
id: determine-status
|
|
if: always()
|
|
env:
|
|
FAILURES: ${{ steps.summary.outputs.FAILURES }}
|
|
ERRORS: ${{ steps.summary.outputs.ERRORS }}
|
|
TOTAL: ${{ steps.summary.outputs.TOTAL }}
|
|
run: |
|
|
# Fail on missing/empty reports (infra died before tests) and on real failures.
|
|
if [[ "$FAILURES" == "-1" || "$FAILURES" -gt 0 || "${ERRORS:-0}" -gt 0 || "${TOTAL:-0}" -eq 0 ]]; then
|
|
echo "STATUS=failure" >> ${GITHUB_OUTPUT}
|
|
else
|
|
echo "STATUS=success" >> ${GITHUB_OUTPUT}
|
|
fi
|
|
|
|
- name: Generate Summary
|
|
if: always()
|
|
env:
|
|
TOTAL: ${{ steps.summary.outputs.TOTAL }}
|
|
PASSES: ${{ steps.summary.outputs.PASSES }}
|
|
FAILURES: ${{ steps.summary.outputs.FAILURES }}
|
|
SKIPPED: ${{ steps.summary.outputs.SKIPPED }}
|
|
ERRORS: ${{ steps.summary.outputs.ERRORS }}
|
|
PERCENTAGE: ${{ steps.summary.outputs.PERCENTAGE }}
|
|
TARGET_URL: ${{ steps.set-url.outputs.TARGET_URL }}
|
|
run: |
|
|
echo "| Tests | Passed :white_check_mark: | Failed :x: | Skipped :fast_forward: | Errors :warning: | " >> ${GITHUB_STEP_SUMMARY}
|
|
echo "|:---:|:---:|:---:|:---:|:---:|" >> ${GITHUB_STEP_SUMMARY}
|
|
echo "| ${TOTAL} | ${PASSES} | ${FAILURES} | ${SKIPPED} | ${ERRORS} |" >> ${GITHUB_STEP_SUMMARY}
|
|
echo "" >> ${GITHUB_STEP_SUMMARY}
|
|
echo "You can check the full report [here](${TARGET_URL})" >> ${GITHUB_STEP_SUMMARY}
|
|
echo "There was **${PERCENTAGE}%** success rate." >> ${GITHUB_STEP_SUMMARY}
|
|
|
|
- name: Fail job on test failures
|
|
if: always() && steps.determine-status.outputs.STATUS == 'failure'
|
|
env:
|
|
FAILURES: ${{ steps.summary.outputs.FAILURES }}
|
|
TOTAL: ${{ steps.summary.outputs.TOTAL }}
|
|
ERRORS: ${{ steps.summary.outputs.ERRORS }}
|
|
run: |
|
|
echo "::error::Detox reported test failures (FAILURES=${FAILURES}, TOTAL=${TOTAL}, ERRORS=${ERRORS})"
|
|
exit 1
|