Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -31,4 +31,7 @@ playwright-report/
test-results/

# Generated workflow artifacts
workflow-report.md
workflow-report.md

# Cursor
.cursor/hooks/state/
38 changes: 38 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,44 @@ Previously, single-character scenarios used free-form text with inline translati

---

## Abort / Cancellation Strategy for Audio Requests

### Why this exists
Audio flows can have races: a user may cancel, close/reopen a modal, or start a new 'turn' while a previous Gemini request is still in-flight. If the stale request resolves after the user moved on, it can incorrectly update UI state (wrong transcript/messages/spinners) or throw JSON parsing errors.

This section is a developer-facing rule to prevent that entire class of bug.

### Required strategy (use for every future audio request)
1. Create a new `AbortController` per 'turn/request' and store it in a ref that cancellation/timeout handlers can reach.
2. Pass the per-request `signal` into the Gemini SDK via `config.abortSignal` on *every* relevant SDK call (`ai.models.generateContent(...)` and `chatSession.sendMessage(...)`).
- Do not rely on `Promise.race` / wrapper rejection alone. The Gemini SDK must receive the signal so it can stop internally and reject with `AbortError`.
3. Invalidate/discard stale responses:
- Track a request token (e.g. `requestIdRef.current` captured into `currentRequestId`) and check it before any state updates.
- If a newer request started (token changed) or the relevant UI is no longer open, return early and do not mutate UI state.
4. Preserve JSON enforcement when passing per-request config with `abortSignal`:
- Keep `responseMimeType: 'application/json'` and `responseSchema: ...` set in the same request config.
- This avoids the SDK returning plain text (which breaks downstream JSON parsing/validation).
5. Handle `AbortError` so it doesn’t surface to the UI:
- Treat `AbortError` as intentional cancellation and `return` silently.
- Do not switch to generic ERROR UI or show error flashes for intentional aborts.

### Example: scenario description recording (abort + stale discard)
In the scenario description 'describe by voice' flow:
- Each transcription attempt creates a fresh `AbortController` (`scenarioDescriptionAbortControllerRef`) and increments a request token (`scenarioDescriptionRequestIdRef`).
- The in-flight call passes `abortController.signal` into `transcribeAndCleanupAudio(...)`.
- After awaiting, results are discarded if `currentRequestId !== scenarioDescriptionRequestIdRef.current` or if the modal is closed (`scenarioSetupOpenRef`).
- In `catch`, `AbortError` is ignored, and only non-abort failures show errors / enable retry.
- In `finally`, the transcription spinner is only cleared when the request token still matches (so stale requests can’t affect UI after close+reopen).

This is the same overall strategy used for the main mic audio flow: per-turn `AbortController`, request-token guarded state updates, and explicit `AbortError` suppression to avoid UI regressions.

### Related files
- `App.tsx` (main mic + scenario description cancellation/discard logic)
- `services/geminiService.ts` (`transcribeAndCleanupAudio`, `sendVoiceMessage` per-request `config.abortSignal`, and JSON enforcement config)
- `__tests__/scenarioDescriptionRecordingAbortDiscard.test.tsx` / `__tests__/transcribeAndCleanupAudioAbortSignal.test.ts` (abort + discard + config preservation)

---

## Missing AI Credentials Handling

**Location:** `App.tsx` handlers, `components/ScenarioSetup.tsx`, `components/AdPersuasionSetup.tsx`
Expand Down
65 changes: 61 additions & 4 deletions App.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,9 @@ const App: React.FC = () => {

// TEF Questioning conversation timer (5-minute limit)
const handleTefQuestioningTimeUp = useCallback(() => {
// Treat this as an intentional abort so processAudioMessage doesn't
// fall through into ERROR UI on AbortError.
processingAbortedRef.current = true;
if (abortControllerRef.current) {
abortControllerRef.current.abort();
abortControllerRef.current = null;
Expand Down Expand Up @@ -119,6 +122,10 @@ const App: React.FC = () => {
const scenarioRecordingRef = useRef(false);
const scenarioSetupOpenRef = useRef(false);

// Abort + stale guarding for scenario description transcription requests.
const scenarioDescriptionAbortControllerRef = useRef<AbortController | null>(null);
const scenarioDescriptionRequestIdRef = useRef(0);

// Ref to track if processing was aborted by the user
const processingAbortedRef = useRef(false);
// Request ID counter to detect stale responses from previous requests
Expand Down Expand Up @@ -596,6 +603,13 @@ const App: React.FC = () => {
};

// Scenario mode handlers
const abortScenarioDescriptionTranscription = useCallback(() => {
if (scenarioDescriptionAbortControllerRef.current) {
scenarioDescriptionAbortControllerRef.current.abort();
scenarioDescriptionAbortControllerRef.current = null;
}
}, []);

const handleOpenScenarioSetup = () => {
scenarioSetupOpenRef.current = true;
setScenarioMode('setup');
Expand All @@ -608,6 +622,10 @@ const App: React.FC = () => {
};

const handleCloseScenarioSetup = () => {
// Invalidate any in-flight transcription so late-resolving promises
// cannot overwrite transcript state after close+reopen.
scenarioDescriptionRequestIdRef.current += 1;
abortScenarioDescriptionTranscription();
scenarioSetupOpenRef.current = false;
setScenarioMode('none');
setScenarioDescription('');
Expand Down Expand Up @@ -662,16 +680,29 @@ const App: React.FC = () => {
* Process description audio (from recording or retry) and transcribe it
*/
const processDescriptionAudio = async (audioData: AudioData): Promise<void> => {
// Increment request ID so we can ignore stale results even if the modal
// is closed and then re-opened while a previous transcription is still in-flight.
const currentRequestId = ++scenarioDescriptionRequestIdRef.current;

// Abort any previous transcription attempt (e.g., quick retry or close+reopen races).
abortScenarioDescriptionTranscription();
const abortController = new AbortController();
scenarioDescriptionAbortControllerRef.current = abortController;

setIsTranscribingDescription(true);

try {
const { base64, mimeType } = audioData;

// Single LLM call to transcribe and clean up the audio - using Gemini
const { rawTranscript: rawText, cleanedTranscript: cleanedText } = await transcribeAndCleanupAudio(base64, mimeType);
const { rawTranscript: rawText, cleanedTranscript: cleanedText } = await transcribeAndCleanupAudio(
base64,
mimeType,
abortController.signal
);

// Modal was closed while transcription was in-flight; discard results
if (!scenarioSetupOpenRef.current) {
// Discard results if a newer request started or the modal has been closed.
if (currentRequestId !== scenarioDescriptionRequestIdRef.current || !scenarioSetupOpenRef.current) {
return;
}

Expand All @@ -690,14 +721,32 @@ const App: React.FC = () => {
setLastDescriptionAudio(null);
setShowTranscriptOptions(true);
} catch (error) {
// If this was an intentional cancellation, discard silently.
const errName = error instanceof DOMException ? error.name : (error as any)?.name;
if (errName === 'AbortError') {
return;
}

if (currentRequestId !== scenarioDescriptionRequestIdRef.current || !scenarioSetupOpenRef.current) {
return;
}

console.error('Error transcribing description:', error);
if (scenarioSetupOpenRef.current) {
// Enable retry with the stored audio
setCanRetryDescriptionAudio(true);
showErrorFlash('Failed to transcribe audio. Please try again.');
}
} finally {
setIsTranscribingDescription(false);
// Only clear the spinner for the latest attempt; stale requests must not
// affect transcript UI state after a close+reopen race.
if (currentRequestId === scenarioDescriptionRequestIdRef.current) {
setIsTranscribingDescription(false);
}

if (scenarioDescriptionAbortControllerRef.current === abortController) {
scenarioDescriptionAbortControllerRef.current = null;
}
}
};

Expand Down Expand Up @@ -752,6 +801,10 @@ const App: React.FC = () => {
const handleCancelRecordingDescription = () => {
scenarioRecordingRef.current = false;
setIsRecordingDescription(false);
// Invalidate any in-flight transcription and clear spinner immediately.
scenarioDescriptionRequestIdRef.current += 1;
setIsTranscribingDescription(false);
abortScenarioDescriptionTranscription();
cancelRecording();
};

Expand Down Expand Up @@ -1086,6 +1139,9 @@ const App: React.FC = () => {
};

const handleExitTefQuestioning = () => {
// Treat this as an intentional abort so processAudioMessage doesn't
// fall through into ERROR UI on AbortError.
processingAbortedRef.current = true;
if (abortControllerRef.current) {
abortControllerRef.current.abort();
abortControllerRef.current = null;
Expand All @@ -1096,6 +1152,7 @@ const App: React.FC = () => {

const handleDismissTefQuestioningSummary = () => {
// Abort any in-flight processing or recording
processingAbortedRef.current = true;
if (abortControllerRef.current) {
abortControllerRef.current.abort();
abortControllerRef.current = null;
Expand Down
Loading