mattermost-mobile-upstream/detox/scripts/run_android_ci_shard.sh
yasser khan dfc94b3c3c
test(e2e): close manual release-test gaps (#10125)
Closes manual release-test gaps for mobile (Detox + Maestro), automating new test cases and hardening flaky/CI-only failures uncovered along the way.

### New coverage automated
- **MM-T416** offline/online post-retry (Android via airplane mode; iOS ultimately marked unsupported — see below)
- **MM-T1194 / MM-T1193** custom ToS accept/decline survives relaunch
- **MM-T3251** status change via account UI (cross-user visibility)
- **MM-T510** channel-wide/keyword mention delivery — later expanded to a full 5-channel mention matrix with exact server-side counts
- **MM-T3261 (_1/_2)** Report a Problem external hand-off (browser link mode, email mode)
- **MM-T2940** iOS managed-config (MDM) pre-fill/autoconnect
- **MM-T3271/3272** opening the app from a push notification (synthetic notification payloads, background/closed states)
- Un-skipped several tests that turned out to already pass (stale skips)

### Root-caused failing tests
- **Android SQLite corruption**: traced to native Kotlin DB connections not opening in WAL mode while JSI did (post #10109) — journal-mode mismatch corrupted the store on push-triggered writes. Fixed properly with a **WAL patch** (#10117) rather than the original destructive "wipe app database" workaround, which was reverted.
- **Fabric crash** (`addViewAt`/reparent) on `SlideUpPanelItem` taps — root-caused to view flattening racing sheet mounts; fixed with `collapsable={false}` (went through a revert/re-revert once CI reproduced the crash without it).
- **Search race**: stale in-flight search results could overwrite a cleared search box — fixed with request-id tagging.
- **iOS offline detection**: multiple iterations (AAAA sampling → Cloudflare-prefix check → final decision) landed on refusing iOS offline simulation outright, since all E2E servers are Cloudflare-fronted/anycast and the address space can't be enumerated/blocked.
- **MM-T3261 config patch** intermittently silently ignored by the server — root-caused to `ReportAProblemType` being environment-managed on PR Spinwicks (config PATCH returns 200 but is silently dropped); now detected and reported as a skip with proven cause, not a failure.
- **MM-T416 failed-post timing**: multiple rounds of tightening the wait bound to match the client's actual retry/backoff/timeout ceiling (ended at ~150s, derived from OkHttp/retry-interceptor math, not guessed).
- **MM-T5603 upload wait**: raised to match the client's real 180s upload timeout.
- **Classification banner suite**: fixed to verify config state directly instead of trusting a PATCH reply that the cloud sometimes drops.

### CI/reporting infra fixes
- Dedicated Maestro steps (MM-T3261, MM-T67856_4) weren't feeding into the merged JUnit report and couldn't gate pass/fail — fixed report merging/naming so they now count.
- Android Maestro logcat capture added (previously iOS-only diagnostics).
- Jest retry logic no longer discards passing results from a shard killed by timeout.
- Sharding raised 24→28.
- Removed dead Detox Pilot/Playbooks scaffolding (unused, uncovered by CI).

### Cleanup
- Comment/docs pass stripping narrative session-archaeology from long explanatory comments, keeping only load-bearing facts and CI references.
2026-09-11 09:48:02 +05:30

170 lines
5.8 KiB
Bash
Executable File

#!/bin/bash
# Run one Android Detox shard. On failure, retry only failed/unrun files.
# Keep the emulator if it is still healthy; cold-boot only when it is not.
set -u
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
DETOX_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
SDK_VERSION="${1:?SDK version required}"
AVD_NAME="${2:?AVD name required}"
shift 2
SHARD_SPECS=("$@")
if (( ${#SHARD_SPECS[@]} == 0 )); then
echo "No specs provided"
exit 1
fi
MAX_ATTEMPTS="${MAX_ATTEMPTS:-2}"
ATTEMPT_TIMEOUT_MIN="${ATTEMPT_TIMEOUT_MIN:-60}"
RESULTS="${DETOX_DIR}/artifacts/jest-results.json"
ATTEMPT1_RESULTS="${DETOX_DIR}/artifacts/jest-results-attempt1.json"
FAILED_SPECS_JS="${DETOX_DIR}/utils/failed-jest-specs.js"
# Jest --outputFile can miss the shutdown flush while jest-stare still wrote
# android-data.json (CI 33915931136 machine-2) — and `timeout` killing an attempt
# always lands there. Recovering it BETWEEN attempts is what makes the retry
# correct: listRetrySpecs treats a missing report as "retry the whole shard", so a
# timed-out attempt 1 re-runs specs that already passed, and mergeJestResultsPreferLater
# then lets the retry's result win. That is how channel_summary and settings — both
# green at 08:47 — were reported as 8 failures (CI 34099282816 machine-2).
# Returns 0 when a usable report exists afterwards, 1 when there is nothing to promote.
promote_jest_stare_results() {
if [ -f "$RESULTS" ]; then
return 0
fi
local stare="${DETOX_DIR}/artifacts/jest-stare/android-data.json"
if [ -f "$stare" ]; then
echo "==> Detox left no jest-results.json — promoting jest-stare/android-data.json"
cp "$stare" "$RESULTS"
return 0
fi
return 1
}
write_missing_results_stub() {
if promote_jest_stare_results; then
return
fi
echo "==> Detox left no jest-results.json — writing shard stub"
node "${DETOX_DIR}/utils/write-tsio-failure-stub.mjs" \
--format jest \
--output "$RESULTS" \
--job-name "${AVD_NAME:-android-shard}" \
--reason "Detox exited before writing jest-results.json (specs: ${SHARD_SPECS[*]})"
}
trap write_missing_results_stub EXIT
emulator_healthy() {
adb devices 2>/dev/null | grep -qE '^emulator-[0-9]+\s+device' || return 1
[[ "$(adb shell getprop sys.boot_completed 2>/dev/null | tr -d '\r')" == "1" ]] || return 1
adb shell pm list packages 2>/dev/null | grep -q 'com.mattermost.rnbeta' || return 1
}
kill_emulator() {
adb -s emulator-5554 emu kill 2>/dev/null || true
pkill -9 -f qemu-system 2>/dev/null || true
pkill -9 -f emulator 2>/dev/null || true
sleep 5
}
run_detox_attempt() {
local tests_only="${1:-false}"
shift
(
cd "${DETOX_DIR}"
chmod +x ./create_android_emulator.sh
if [[ "${tests_only}" == "true" ]]; then
TESTS_ONLY=true CI=true timeout "${ATTEMPT_TIMEOUT_MIN}m" \
./create_android_emulator.sh "${SDK_VERSION}" "${AVD_NAME}" "$@"
else
CI=true timeout "${ATTEMPT_TIMEOUT_MIN}m" \
./create_android_emulator.sh "${SDK_VERSION}" "${AVD_NAME}" "$@"
fi
)
}
merge_attempt_results() {
if [[ ! -f "${ATTEMPT1_RESULTS}" ]]; then
return 0
fi
if [[ ! -f "${RESULTS}" ]]; then
mv "${ATTEMPT1_RESULTS}" "${RESULTS}"
return 0
fi
node "${FAILED_SPECS_JS}" \
--attempt1 "${ATTEMPT1_RESULTS}" \
--attempt2 "${RESULTS}" \
--output "${RESULTS}"
}
attempt=1
while (( attempt <= MAX_ATTEMPTS )); do
echo "==> Attempt ${attempt}/${MAX_ATTEMPTS} (timeout ${ATTEMPT_TIMEOUT_MIN}m)"
if (( attempt == 1 )); then
run_detox_attempt false "${SHARD_SPECS[@]}"
rc=$?
# Recover the results of the suites that did finish before deciding what to
# retry — a `timeout` kill (exit 124) never lets Jest write --outputFile.
promote_jest_stare_results || true
else
retry_out="$(node "${FAILED_SPECS_JS}" \
--results "${RESULTS}" \
--shard-specs "${SHARD_SPECS[*]}")"
node_rc=$?
retry_specs=()
if (( node_rc == 0 )) && [[ -n "${retry_out}" ]]; then
while IFS= read -r line; do
[[ -n "${line}" ]] && retry_specs+=("${line}")
done <<< "${retry_out}"
fi
if (( ${#retry_specs[@]} == 0 )); then
echo "==> Could not determine failed specs — retrying the whole shard"
retry_specs=("${SHARD_SPECS[@]}")
fi
echo "==> Retrying ${#retry_specs[@]} spec(s): ${retry_specs[*]}"
if [[ -f "${RESULTS}" ]]; then
mv "${RESULTS}" "${ATTEMPT1_RESULTS}"
fi
if emulator_healthy; then
echo "==> Emulator healthy — resetting app and retrying failed specs only"
run_detox_attempt true "${retry_specs[@]}"
rc=$?
if (( rc == 2 )); then
echo "==> Emulator went unhealthy during retry — cold boot"
kill_emulator
run_detox_attempt false "${retry_specs[@]}"
rc=$?
fi
else
echo "==> Emulator unhealthy — cold boot, then retrying failed specs only"
kill_emulator
run_detox_attempt false "${retry_specs[@]}"
rc=$?
fi
# Same recovery for the retry: a second timeout must not throw away the
# suites it did finish, which merge_attempt_results would otherwise drop.
promote_jest_stare_results || true
merge_attempt_results
fi
if (( rc == 0 )); then
echo "==> Attempt ${attempt} succeeded"
exit 0
fi
echo "==> Attempt ${attempt} failed with exit ${rc}"
if (( attempt < MAX_ATTEMPTS )); then
echo "==> Will retry failed/unrun specs (not the whole shard)"
fi
attempt=$(( attempt + 1 ))
done
echo "==> All ${MAX_ATTEMPTS} attempts failed"
exit 1