-
Notifications
You must be signed in to change notification settings - Fork 0
Plan 2: Sync Engine — pg-boss worker, outbox dispatcher, five sync jobs #2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
28 commits
Select commit
Hold shift + click to select a range
44babe8
docs: plan 2 sync engine implementation plan
guarzo 4600cc0
docs: apply review findings to plan 2 (wanderer contract, transfer re…
guarzo fffb9e8
docs: plan 2 round-2 review fixes (stale-write guards, wanderer unblo…
guarzo 617a7a9
feat: fail-closed validation for Discord OAuth responses
guarzo 00eda9a
feat: sync_run job wrapper and ops webhook
guarzo 000d0f6
feat: throttled ESI client with fail-closed parsing
guarzo 37c4860
fix: classify malformed ESI bodies as permanent EsiError
guarzo 2c2cf4d
feat: affiliation chunk/bisect resolution and tier decision rule
guarzo 6dbd625
feat: token refresh service with permanent/transient classification
guarzo 63533f6
feat: membership verification job with confirmed-read tier transitions
guarzo 0d4c6a9
feat: desired-set query and label-scoped contacts diff
guarzo 7e990bd
feat: per-character contact push with label ownership and abort-on-pa…
guarzo 285710d
feat: wanderer ACL sync with post-mutation observation
guarzo f312d23
feat: discord role sync with permanent-config validation
guarzo 0836e26
feat: daily token health job with transfer reclaim and subject binding
guarzo a4dbec9
feat: purge job for sessions, oauth transactions, and dispatched outb…
guarzo a4e11dc
feat: transactional outbox dispatcher with singleton fan-out
guarzo 2c35b06
feat: pg-boss worker entry with schedules and dead-letter ops alerts
guarzo ba4982f
test: full deprovision-path integration coverage
guarzo b6e779a
fix: never remove blocked ACL entries in diffAcl
guarzo 175337f
fix: contain JWT verification failures in token-health job
guarzo 70233f8
fix: alert ops webhook on permanent wanderer ACL-read failure
guarzo b7be555
fix: isolate one unpushable desired contact from halting removals
guarzo 09f8f8c
fix: satisfy strict typecheck in finding 2/3 test additions
guarzo ca52eb0
fix: harden chunk/role-diff/contacts/purge/tokens/rest/esi/wanderer
guarzo 009ac65
fix: harden worker job/dispatcher lifecycle
guarzo 54f2043
test: cover hardening fixes and dedupe shared test scaffolding
guarzo e5cc3ac
fix: digits-only permission parsing, JSON-inclusive Discord body clas…
guarzo File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
5,482 changes: 5,482 additions & 0 deletions
5,482
docs/superpowers/plans/2026-08-02-authgd-2-sync-engine.md
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| export type AclMember = { characterId: number; role: string }; | ||
|
|
||
| /** | ||
| * Spec job 3: admin-role entries are NEVER removed; manager-role entries are | ||
| * removed like anyone else when they leave the desired set. A desired | ||
| * character whose role is "blocked" has no effective access — presence alone | ||
| * is not convergence — so it is unblocked (reset to viewer); all other roles | ||
| * (admin/manager/member/viewer) are preserved as-is. Blocked entries are also | ||
| * NEVER removed even when undesired: removing a blocked member is equivalent | ||
| * to un-banning them, which could restore access via an inert corp/alliance | ||
| * ACL entry — blocked access must only ever be lifted deliberately. | ||
| */ | ||
| export function diffAcl(input: { desiredIds: number[]; members: AclMember[] }): { | ||
| add: number[]; | ||
| remove: number[]; | ||
| unblock: number[]; | ||
| } { | ||
| const desired = new Set(input.desiredIds); | ||
| const byId = new Map(input.members.map((m) => [m.characterId, m])); | ||
| return { | ||
| add: input.desiredIds.filter((id) => !byId.has(id)), | ||
| unblock: input.desiredIds.filter((id) => byId.get(id)?.role === "blocked"), | ||
| remove: input.members | ||
| .filter( | ||
| (m) => !desired.has(m.characterId) && m.role !== "admin" && m.role !== "blocked", | ||
| ) | ||
| .map((m) => m.characterId), | ||
| }; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| import { chunk } from "@/core/chunk"; | ||
| import { EsiError, type Affiliation } from "@/lib/esi/client"; | ||
|
|
||
| export type AffiliationOutcome = { | ||
| resolved: Map<number, { corporationId: number; allianceId: number | null }>; | ||
| /** Deterministic 400 on a single id — safe to flag affiliation_invalid. */ | ||
| invalid: number[]; | ||
| /** Transient or ambiguous failures — never flagged, retried next run. */ | ||
| unresolved: number[]; | ||
| }; | ||
|
|
||
| const CHUNK_SIZE = 500; | ||
|
|
||
| export async function resolveAffiliations( | ||
| ids: number[], | ||
| post: (ids: number[]) => Promise<Affiliation[]>, | ||
| ): Promise<AffiliationOutcome> { | ||
| const out: AffiliationOutcome = { resolved: new Map(), invalid: [], unresolved: [] }; | ||
| for (const batch of chunk(ids, CHUNK_SIZE)) { | ||
| await resolveChunk(batch, post, out); | ||
| } | ||
| return out; | ||
| } | ||
|
|
||
| async function resolveChunk( | ||
| ids: number[], | ||
| post: (ids: number[]) => Promise<Affiliation[]>, | ||
| out: AffiliationOutcome, | ||
| ): Promise<void> { | ||
| if (ids.length === 0) return; | ||
| try { | ||
| const rows = await post(ids); | ||
| const returned = new Set<number>(); | ||
| for (const r of rows) { | ||
| returned.add(r.characterId); | ||
| out.resolved.set(r.characterId, { | ||
| corporationId: r.corporationId, | ||
| allianceId: r.allianceId, | ||
| }); | ||
| } | ||
| for (const id of ids) if (!returned.has(id)) out.unresolved.push(id); | ||
| } catch (err) { | ||
| // Bisect ONLY deterministic invalid-request responses. Anything else | ||
| // (420/5xx/network, or odd permanent statuses) must never flag characters. | ||
| if (err instanceof EsiError && err.status === 400) { | ||
| if (ids.length === 1) { | ||
| out.invalid.push(ids[0]); | ||
| return; | ||
| } | ||
| const mid = Math.ceil(ids.length / 2); | ||
| await resolveChunk(ids.slice(0, mid), post, out); | ||
| await resolveChunk(ids.slice(mid), post, out); | ||
| return; | ||
| } | ||
| out.unresolved.push(...ids); | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| export function chunk<T>(items: T[], size: number): T[][] { | ||
| if (!Number.isInteger(size) || size <= 0) { | ||
| throw new Error("chunk: size must be a positive integer"); | ||
| } | ||
| const out: T[][] = []; | ||
| for (let i = 0; i < items.length; i += size) out.push(items.slice(i, i + size)); | ||
| return out; | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| export type ContactState = { | ||
| contactId: number; | ||
| standing: number; | ||
| labelIds: number[]; | ||
| }; | ||
|
|
||
| export type ContactsDiff = { | ||
| add: number[]; | ||
| update: Array<{ contactId: number; labelIds: number[] }>; | ||
| remove: number[]; | ||
| }; | ||
|
|
||
| /** | ||
| * Label-ownership policy (accepted-destructive, aa-standingssync precedent): | ||
| * the app owns `labelId` outright. Desired ids are added, or taken over if | ||
| * they already exist as personal contacts (standing re-asserted, our label | ||
| * added while PRESERVING existing labels — ESI PUT replaces label_ids | ||
| * wholesale). Contacts carrying our label that leave the desired set are | ||
| * deleted entirely. Contacts never carrying our label are never modified. | ||
| * `desiredIds` must already exclude the target character itself. | ||
| */ | ||
| export function diffContacts(input: { | ||
| desiredIds: number[]; | ||
| standing: number; | ||
| labelId: number; | ||
| contacts: ContactState[]; | ||
| }): ContactsDiff { | ||
| const desired = new Set(input.desiredIds); | ||
| const byId = new Map(input.contacts.map((c) => [c.contactId, c])); | ||
| const add: number[] = []; | ||
| const update: Array<{ contactId: number; labelIds: number[] }> = []; | ||
| for (const id of input.desiredIds) { | ||
| const existing = byId.get(id); | ||
| if (!existing) { | ||
| add.push(id); | ||
| continue; | ||
| } | ||
| const hasLabel = existing.labelIds.includes(input.labelId); | ||
| if (!hasLabel || existing.standing !== input.standing) { | ||
| update.push({ | ||
| contactId: id, | ||
| labelIds: hasLabel ? existing.labelIds : [...existing.labelIds, input.labelId], | ||
| }); | ||
| } | ||
| } | ||
| const remove = input.contacts | ||
| .filter((c) => c.labelIds.includes(input.labelId) && !desired.has(c.contactId)) | ||
| .map((c) => c.contactId); | ||
| return { add, update, remove }; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,83 @@ | ||
| export type ManagedRoleIds = { flygd: string; blue: string; green: string }; | ||
|
|
||
| /** Ensure exactly the tier's role among the three managed roles; all other roles untouched. */ | ||
| export function diffRoles(input: { | ||
| tier: "flygd" | "blue" | "green"; | ||
| managed: ManagedRoleIds; | ||
| memberRoleIds: string[]; | ||
| }): { add: string[]; remove: string[] } { | ||
| const want = input.managed[input.tier]; | ||
| const managedAll = [input.managed.flygd, input.managed.blue, input.managed.green]; | ||
| const have = new Set(input.memberRoleIds); | ||
| return { | ||
| add: have.has(want) ? [] : [want], | ||
| remove: managedAll.filter((r) => r !== want && have.has(r)), | ||
| }; | ||
| } | ||
|
|
||
| /** The managed roles a member currently carries (unlinked-user deprovision). */ | ||
| export function stripManagedRoles( | ||
| managed: ManagedRoleIds, | ||
| memberRoleIds: string[], | ||
| ): string[] { | ||
| const managedAll = new Set([managed.flygd, managed.blue, managed.green]); | ||
| return memberRoleIds.filter((r) => managedAll.has(r)); | ||
| } | ||
|
|
||
| const MANAGE_ROLES = 1n << 28n; | ||
| const ADMINISTRATOR = 1n << 3n; | ||
|
|
||
| /** | ||
| * Config validation: three distinct managed role ids that exist in the guild; | ||
| * bot has Manage Roles (or Administrator); bot's highest role sits ABOVE | ||
| * every managed role. Failure is permanent-config — no retry loop. | ||
| */ | ||
| /** Malformed permissions strings must never grant access — treat as zero. | ||
| * Digits-only: BigInt would also accept hex ("0x...") and padded input, which | ||
| * Discord never sends and which must not sneak permissions in. */ | ||
| function parsePermissions(permissions: string): bigint { | ||
| if (!/^\d+$/.test(permissions)) return 0n; | ||
| return BigInt(permissions); | ||
| } | ||
|
|
||
| export function validateRoleConfig(input: { | ||
| managed: ManagedRoleIds; | ||
| guildRoles: Array<{ id: string; position: number; permissions: string }>; | ||
| botRoleIds: string[]; | ||
| /** Discord omits @everyone (id === guild id) from member role arrays; when | ||
| * provided, its guild role is folded into the bot's permission union. */ | ||
| everyoneRoleId?: string; | ||
| }): { ok: true } | { ok: false; error: string } { | ||
| const ids = [input.managed.flygd, input.managed.blue, input.managed.green]; | ||
| if (new Set(ids).size !== 3) { | ||
| return { ok: false, error: "managed role ids are not distinct" }; | ||
| } | ||
| const byId = new Map(input.guildRoles.map((r) => [r.id, r])); | ||
| const missing = ids.filter((id) => !byId.has(id)); | ||
| if (missing.length > 0) { | ||
| return { ok: false, error: `managed roles missing from guild: ${missing.join(", ")}` }; | ||
| } | ||
| const botRoleIds = input.everyoneRoleId | ||
| ? [...new Set([...input.botRoleIds, input.everyoneRoleId])] | ||
| : input.botRoleIds; | ||
| const botRoles = botRoleIds.flatMap((id) => { | ||
| const role = byId.get(id); | ||
| return role ? [role] : []; | ||
| }); | ||
| const canManage = botRoles.some( | ||
| (r) => (parsePermissions(r.permissions) & (MANAGE_ROLES | ADMINISTRATOR)) !== 0n, | ||
| ); | ||
| if (!canManage) return { ok: false, error: "bot lacks Manage Roles" }; | ||
| const botTop = botRoles.reduce((max, r) => Math.max(max, r.position), -1); | ||
| const tooHigh = ids.filter((id) => { | ||
| const role = byId.get(id); | ||
| return role !== undefined && role.position >= botTop; | ||
| }); | ||
| if (tooHigh.length > 0) { | ||
| return { | ||
| ok: false, | ||
| error: `bot's highest role is not above managed roles: ${tooHigh.join(", ")}`, | ||
| }; | ||
| } | ||
| return { ok: true }; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| export type Tier = "flygd" | "blue" | "green"; | ||
|
|
||
| /** | ||
| * Membership rule: unlocked accounts are system-managed — the desired tier is | ||
| * flygd when the main is in the configured alliance, green otherwise (this is | ||
| * how an unlocked Blue converges after "return to auto"). Transitions require | ||
| * a CONFIRMED affiliation read of the main in this run. Returns the tier to | ||
| * set, or null for no change. | ||
| */ | ||
| export function decideTier(input: { | ||
| tier: Tier; | ||
| tierLocked: boolean; | ||
| mainConfirmed: boolean; | ||
| mainInAlliance: boolean; | ||
| }): "flygd" | "green" | null { | ||
| if (input.tierLocked || !input.mainConfirmed) return null; | ||
| const desired = input.mainInAlliance ? "flygd" : "green"; | ||
| return input.tier === desired ? null : desired; | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.