mattermost-mobile-upstream/detox/e2e/support/server_api/pool_guard.ts
yasser khan 75a3cf2bf5
Some checks are pending
ci / lint-typecheck (push) Waiting to run
ci / detox-lint-typecheck (push) Waiting to run
ci / test (push) Waiting to run
E2E: apply one server baseline at provisioning (#10258)
* MM-71189 Preserve feature flags in Detox config patches

Co-authored-by: mattermost-code <matty-code@mattermost.com>

* Guard getFeatureFlags against empty config responses

Return an error instead of treating a missing config body as an empty
flag set, so a dropped GET does not patch only the requested flags.

Co-authored-by: mattermost-code <matty-code@mattermost.com>

* Retrigger CI after Detox iOS and iPad E2E flakes

Co-authored-by: mattermost-code <matty-code@mattermost.com>

* test(e2e): apply one declared server baseline at provisioning

Provisioning drives each test server to one declared baseline: the config keys E2E
needs and the full FeatureFlags map, written read-modify-write so no flag is reset.
Env-pinned keys are reported, not patched, and leftover E2E policies, remote clusters
and property fields are removed. Provisioning also disables stray plugins once.

* test(e2e): assert the server baseline instead of patching config in specs

These specs patched server-wide settings (CRT, archived channels, auto-replies, ...) and
restored them while other shards depended on them. They now assert the provisioned
baseline instead.

* test(e2e): split CI servers into pool and mutation roles; remove the locks

Each platform's server A becomes a read-only pool for 27 shards (and iPad and Maestro);
server B (and on iOS part of SITE_3) runs one serial mutation shard with the specs in
detox/e2e/server_mutation_specs.json. The classification, server and SITE_3 locks go.

- split-tests.js places mutation specs by the manifest and packs the pool by duration;
  a spec that needs B never falls back to the shared SITE_3.
- A pool request guard, an ESLint rule and a manifest checker keep server-wide writes
  off the pool. Mutation shards re-apply the baseline before each spec file.
- Mutating tests move to their own spec files; ABAC specs use channel-scoped
  permission policies; MFA and Terms of Service run on the mutation server.
- Pool shards no longer disable provisioned plugins, so the Agents specs run.

* fix(e2e): fail a spec when the baseline cannot be applied locally

* fix(e2e): keep transport retries for feature-flag and baseline reads

Restore the retry in getFeatureFlags, which apiEnableFeatureFlags calls directly, and
retry the baseline assertion's GET requests on transport failures.

* test(e2e): cover retries of dropped config and baseline reads

* test(e2e): apply one declared server baseline at provisioning

Provisioning drives each test server to one declared baseline: the config keys E2E
needs and the full FeatureFlags map, written read-modify-write so no flag is reset.
Env-pinned keys are reported, not patched, and leftover E2E policies, remote clusters
and property fields are removed. Provisioning also disables stray plugins once.

* test(e2e): assert the server baseline instead of patching config in specs

These specs patched server-wide settings (CRT, archived channels, auto-replies, ...) and
restored them while other shards depended on them. They now assert the provisioned
baseline instead.

* fix(e2e): fail a spec when the baseline cannot be applied locally

* fix(e2e): keep transport retries for feature-flag and baseline reads

Restore the retry in getFeatureFlags, which apiEnableFeatureFlags calls directly, and
retry the baseline assertion's GET requests on transport failures.

* test(e2e): cover retries of dropped config and baseline reads

* E2E: stop burn-on-read spec turning other feature flags off

Its partial FeatureFlags patch reset every other flag on the shared server,
which turned ChannelAttributes off under channel_attributes on another shard.
Enable only the missing flags, as redacted_files and upload_permission do,
and do the same for the dialog spec. Also tap the attach button the app
actually renders in MM-T339_1.

* E2E: route flag-off spec to B on iOS; tap the real attach button in the max-size spec

* test(e2e): apply one declared server baseline at provisioning

Provisioning drives each test server to one declared baseline: the config keys E2E
needs and the full FeatureFlags map, written read-modify-write so no flag is reset.
Env-pinned keys are reported, not patched, and leftover E2E policies, remote clusters
and property fields are removed. Provisioning also disables stray plugins once.

* test(e2e): assert the server baseline instead of patching config in specs

These specs patched server-wide settings (CRT, archived channels, auto-replies, ...) and
restored them while other shards depended on them. They now assert the provisioned
baseline instead.

* fix(e2e): fail a spec when the baseline cannot be applied locally

* fix(e2e): keep transport retries for feature-flag and baseline reads

Restore the retry in getFeatureFlags, which apiEnableFeatureFlags calls directly, and
retry the baseline assertion's GET requests on transport failures.

* test(e2e): cover retries of dropped config and baseline reads

* E2E: stop burn-on-read spec turning other feature flags off

Its partial FeatureFlags patch reset every other flag on the shared server,
which turned ChannelAttributes off under channel_attributes on another shard.
Enable only the missing flags, as redacted_files and upload_permission do,
and do the same for the dialog spec. Also tap the attach button the app
actually renders in MM-T339_1.

* E2E: wait for the results service before retrying a shard upload

upload-artifact does not retry network errors (ENOTFOUND, ETIMEDOUT), and the
retry fired ~30 ms after the first failure, inside the same DNS outage. A shard
whose specs all passed was then reported as "spec did not run". Each retry now
waits (bounded, 120 s) for the results host to answer, with a second retry.

* E2E: run the specs a killed iOS Jest never started; record why it died

When Jest is killed between spec files (exit code = null) it writes no report, and the
specs after that point never run. Run only those once in a fresh Jest and merge the
results; specs that ran keep their own result, so a spec that crashes Jest stays red.
Also sample memory and collect jetsam/crash reports so the next kill names its cause.

* E2E: keep the demo plugin off the shared servers; 32 shards

The demo plugin posts into every team on every server event and inside every config
save, and on our servers each of those posts fails. Only the two dialog specs use it,
so install it only on mutation server B, disable it everywhere else (failing
provisioning if it stays on), and route both dialog specs to B. Drop the
DialogOnlyMode setting, which v0.12.1 does not have. Raise Detox shards 28 -> 32.

* E2E: give the demo plugin a settings entry on the servers that install it

Demo plugin v0.12.1's ConfigurationWillBeSaved hands the whole config back when it
has no settings entry, and that round trip resets settings it does not know, or that
are false, to their defaults. With DialogOnlyMode gone from the baseline, fresh B
servers had no entry, so ChannelAttributesRequired=true and EnableMobileUpload=false
were reverted inside the save. An empty entry keeps the hook out of the save.

---------

Co-authored-by: mattermost-code <matty-code@mattermost.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: yasserfaraazkhan <yasser.khan@mattermost.com>
2026-10-21 23:36:31 +05:30

135 lines
5.9 KiB
TypeScript

// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
// See LICENSE.txt for license information.
// Blocks server-wide writes from shards on the shared pool servers. Specs that must change
// server-wide state run on the mutation shard instead (detox/e2e/server_mutation_specs.json).
export type PoolGuardMode = 'off' | 'enforce';
type Env = Record<string, string | undefined>;
export const poolGuardMode = (env: Env = process.env): PoolGuardMode => {
if (env.CI !== 'true' || env.E2E_SERVER_ROLE !== 'pool') {
return 'off';
}
return env.E2E_POOL_GUARD === 'enforce' ? 'enforce' : 'off';
};
const originOf = (url?: string) => {
if (!url) {
return '';
}
try {
return new URL(url).origin;
} catch {
return '';
}
};
const protectedOrigins = (env: Env = process.env) => new Set(
[env.SITE_1_URL, env.SITE_2_URL, env.SITE_3_URL].map(originOf).filter(Boolean),
);
const parseBody = (body: unknown): any => {
if (typeof body === 'string') {
try {
return JSON.parse(body);
} catch {
return undefined;
}
}
return body;
};
type Rule = {methods: string[]; path: RegExp; reason: string; allow?: (body: any, path: string) => boolean};
// Channel policies created through this guard; activating or deleting them stays channel-scoped.
const channelPolicyIds = new Set<string>();
type PolicyRequest = {type?: string; id?: string; entries?: Array<{id?: string}>} | undefined;
const isChannelPolicyCreate = (body: PolicyRequest) => {
if (body?.type !== 'channel') {
return false;
}
if (typeof body.id === 'string' && body.id) {
channelPolicyIds.add(body.id);
}
return true;
};
const activatesOnlyChannelPolicies = (body: PolicyRequest) => {
const entries = body?.entries;
return Array.isArray(entries) && entries.length > 0 && entries.every((entry) => channelPolicyIds.has(entry?.id ?? ''));
};
const WRITE = ['post', 'put', 'patch', 'delete'];
const RULES: Rule[] = [
{methods: WRITE, path: /^\/api\/v4\/config(\/patch|\/reload)?$/, reason: 'server config'},
{methods: WRITE, path: /^\/api\/v4\/(license|trial-license)(\/|$)/, reason: 'license'},
{methods: ['post'], path: /^\/api\/v4\/plugins(\/install_from_url|\/marketplace)?$/, reason: 'plugin install'},
{methods: ['post', 'delete'], path: /^\/api\/v4\/plugins\/[^/]+(\/(enable|disable))?$/, reason: 'plugin state'},
{methods: WRITE, path: /^\/api\/v4\/(roles\/[^/]+\/patch|schemes)(\/|$)/, reason: 'roles or schemes'},
// A channel policy only affects its own channel; a permission policy applies server-wide even when team-scoped.
{methods: ['put', 'post'], path: /^\/api\/v4\/access_control_policies$/, reason: 'access policy', allow: isChannelPolicyCreate},
{methods: ['put'], path: /^\/api\/v4\/access_control_policies\/activate$/, reason: 'access policy activation', allow: activatesOnlyChannelPolicies},
{methods: ['delete'], path: /^\/api\/v4\/access_control_policies\/[^/]+$/, reason: 'access policy delete', allow: (_, path) => channelPolicyIds.has(path.split('/').pop() ?? '')},
{methods: WRITE, path: /^\/api\/v4\/custom_profile_attributes\/fields(\/|$)/, reason: 'profile attribute fields'},
// Optional fields named with a suite-unique suffix (see channel_attributes_fixtures.ts) only show
// where that suite sets a value; required or fixed-name fields affect every channel on the server.
{methods: ['post', 'put', 'patch'], path: /^\/api\/v4\/properties\/groups\/[^/]+\/[^/]+\/fields(\/|$)/, reason: 'property fields', allow: (body) => body?.attrs?.required !== true && /_[0-9a-f]{6}$/.test(body?.name ?? '')},
{methods: WRITE, path: /^\/api\/v4\/properties\/groups\/[^/]+\/system\/values(\/|$)/, reason: 'system property values'},
{methods: WRITE, path: /^\/api\/v4\/remotecluster(\/|$)/, reason: 'remote clusters'},
{methods: WRITE, path: /^\/api\/v4\/terms_of_service(\/|$)/, reason: 'terms of service'},
{methods: WRITE, path: /^\/api\/v4\/ldap\/(sync|migrateid)(\/|$)/, reason: 'LDAP'},
];
/** Returns why a request writes server-wide state, or null when it is allowed on a pool server. */
export const classifyRequest = (method: string | undefined, url: string | undefined, body?: unknown, env: Env = process.env, baseURL?: string): string | null => {
const m = (method || 'get').toLowerCase();
if (!WRITE.includes(m) || !url) {
return null;
}
let parsed: URL;
try {
parsed = new URL(url, baseURL);
} catch {
// A write we cannot place could be aimed at a pool server, so it is not let through unchecked.
return `${m.toUpperCase()} ${url} (unresolvable URL)`;
}
if (!protectedOrigins(env).has(parsed.origin)) {
return null;
}
// A site URL with a trailing slash yields "//api/v4/...", which the anchored rules would miss.
const path = parsed.pathname.replace(/\/{2,}/g, '/');
const rule = RULES.find((r) => r.methods.includes(m) && r.path.test(path));
if (!rule || rule.allow?.(parseBody(body), path)) {
return null;
}
return `${m.toUpperCase()} ${path} (${rule.reason})`;
};
const violations: string[] = [];
/** Returns and clears the violations recorded since the last call. */
export const drainPoolGuardViolations = () => violations.splice(0, violations.length);
/** Axios request interceptor. */
export const poolGuardInterceptor = <T extends {method?: string; url?: string; baseURL?: string; data?: unknown}>(config: T): T => {
const mode = poolGuardMode();
if (mode === 'off') {
return config;
}
const violation = classifyRequest(config.method, config.url, config.data, process.env, config.baseURL);
if (!violation) {
return config;
}
violations.push(violation);
// eslint-disable-next-line no-console
console.error(`[pool_guard] server-wide write on a pool server: ${violation}`);
throw Object.assign(new Error(`E2E_POOL_GUARD blocked ${violation}`), {code: 'E2E_POOL_GUARD'});
};