Skip to content

feat: dictation into the composer - #148

Draft
incognitojam wants to merge 151 commits into
mainfrom
t3code/add-desktop-dictation-support
Draft

feat: dictation into the composer#148
incognitojam wants to merge 151 commits into
mainfrom
t3code/add-desktop-dictation-support

Conversation

@incognitojam

@incognitojam incognitojam commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator

Dictation into the composer. Audio never leaves the user's own machines: capture
in the renderer, transcription on their T3 server.

Draft. The feature works end to end and is verified live in Chrome; it ships
off by default under Settings → Experimental, so transcription never starts
unless the user opts in. Desktop artifacts still build and package the sidecar
unconditionally.

Approach

apps/web  getUserMedia -> 16kHz mono f32 PCM
             |  WebSocket, binary frames
             v
apps/server /api/dictation/stream --> warm Parakeet sidecar (native/dictation)
             |                              | JSONL {committed, tentative}
             |<-----------------------------+
             v
apps/web  matcher (packages/shared) -> composer draft

Capture lives in apps/web, so desktop is covered for free (the desktop
renderer is apps/web). Transcription lives on the server so every client can
share one implementation, and because the T3 server is the user's own machine
the audio stays on their infrastructure.

Why these choices

Every decision came from a measured spike (native/dictation-spike/README.md,
17 findings, reproducible harness). The load-bearing ones:

  • Parakeet TDT 0.6B (GGUF, transcribe-cpp) over Apple SpeechAnalyzer. 95% vs
    64-82% identifier recall on the same recordings, and unaffected by the mic and
    room noise that moved Apple 18 points. No macOS/Tahoe/Apple Intelligence
    dependency anywhere in the design.
  • No ASR-level biasing. contextualStrings is ignored by SpeechTranscriber;
    Apple custom language models are ignored entirely (a deliberately corrupted
    model file produced no error and no change).
  • A deterministic matcher, not an LLM, for identifiers. 4ms, cannot
    hallucinate. A small on-device model given the same job invented identifiers
    and corrupted a correct sentence.
  • ~100 context-scoped vocabulary terms. At repo scale, recall fell to 52% and
    9 of 24 utterances were damaged ("and then" -> andThen, "is fast" -> isLast).
  • A WebSocket, not a streaming POST. Browsers only allow streaming fetch
    request bodies over HTTP/2; the server speaks HTTP/1.1.
  • Visible, revertable substitutions are required, not polish. Phrase
    collisions ("the room is empty" -> isEmpty) are deterministic consequences of
    correct transcription — six TTS voices across five locales produced identical
    results — so their rate rises as transcription improves.

Verified

E2E in Chrome against a real server: streaming transcript, identifier recovery
and revert, warm/idle sidecar cycling, stash fallback, and the unavailable path.
Second speaker (independent contributor) scored 91% with Parakeet vs 73% with
Apple, so the numbers are not fitted to one voice.

Unit tests cover the matcher (against real ASR transcripts committed as
fixtures, including one asserting the phrase collisions must fire), the
resampler and framer, and the engine's availability paths.

Desktop artifact builds compile and package the Rust sidecar unconditionally.
The fork nightly matrix builds those artifacts on macOS arm64, Linux x64, and
Windows x64; the Experimental setting remains the sole runtime gate.

Before this is usable day to day

  • Model distribution. The server needs T3CODE_DICTATION_MODEL_PATH or
    dictationModelPath pointing at a Parakeet GGUF. The opt-in ~600MB download
    flow is unbuilt, so today this is a manual step.
  • Electron desktop is untested. Same renderer code, different plumbing
    (session permission handler, t3code-dev:// origin, no Vite proxy).

Also unfinished

  • UX round — corrections should be inline in the composer rather than a
    bubble panel; the server should advertise availability so the button can
    disable ahead of time instead of connect-then-fail; visual pass with Handy/Amp
    prior art; input device picker.
  • Hold-to-talk — the setting exists, the interaction is toggle-only.

Notes for review

  • native/dictation-spike/ is deliberate: it is throwaway code but the findings
    are the justification for every decision above, and the harness reproduces
    them. Corpus audio is gitignored (personal voice data); the ASR transcripts are
    committed as regression fixtures.
  • One pre-existing test failure on this branch
    (skips the primary native probe for cross-architecture Windows payloads)
    also fails on a pristine origin/main and is unrelated.
  • DictationControl uses a raw createPortal, which is not the house overlay
    pattern; migrating it is part of the UX round.
  • This adds the first Experimental settings section, so it sets the precedent
    for whatever lands there next.

Written by an agent (T3 Code, gpt-5.6-sol).

@incognitojam
incognitojam force-pushed the t3code/add-desktop-dictation-support branch 3 times, most recently from c2be37b to e3deccf Compare August 15, 2026 20:52
incognitojam and others added 25 commits August 16, 2026 11:58
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* ci: stabilize fork nightly tests

* ci: isolate image compression test

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(ci): allow Apple notarization to finish

* fix(ci): avoid duplicate Apple notarizations

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* ci: build unsigned Windows x64 in fork nightly

* ci: add dry_run input to test fork nightly

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
- Run fork CI and nightly jobs for yngatech/t3code

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
fix(branding): use yngatech fork identity

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* ci: migrate Ubuntu jobs to Blacksmith

* ci: keep lightweight jobs on GitHub runners

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* feat(web): show GitHub service outages

* fix(web): clarify GitHub status link affordance

* feat(web): make GitHub outage alerts opt-in

* refactor(web): remove GitHub status preview override

* test(desktop): include GitHub alert setting

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* feat(web): show unsent drafts in the sidebar

- Add a draft indicator to sidebar thread rows
- Surface drafts containing text or attachments, excluding settings-only state

* fix(web): brighten the sidebar draft pencil on row hover

The pencil sat at a flat secondary-label colour, so on a settled row it
read brighter than the row's own title and stayed put while the rest of
the row lifted. Match the PR badge's settled treatment: muted at rest,
transitioning with the row on hover. The pencil has no state colour of
its own, so it lifts to the same foreground the title does, and the
active row opts out like the PR badge already does.

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
The terminal guide explains browser-opening behavior but does not help
users diagnose a missing injected launcher environment.

Add focused troubleshooting for inspecting the terminal environment,
refreshing terminals after updates, identifying client/server version
mismatches, respecting custom `BROWSER` values, and finding
helper-install diagnostics.

Validation: documentation-only change; formatting checked.

---
Written by an agent (T3 Code, gpt-5.6-sol).
> [!NOTE]
> TL;DR: New **Copy transcript** thread action copies the conversation
as markdown (`## User` / `## Assistant` under the thread title) — no
tool calls, reasoning, or half-streamed replies — for sharing or pasting
into a new chat as context.

There was no way to get a thread's conversation out of T3 Code. This
adds `copy-transcript` to the shared thread action menu (sidebar
right-click and chat header), with the serializer in
`apps/web/src/lib/threadTranscript.ts`. User prompts are copied as their
visible text, without injected terminal/element/issue context blocks;
image attachments become `[Attached image: …]` placeholders.

Details worth reviewing:

- The action fetches a full unwindowed snapshot through a new
`createThreadSnapshotCommands` factory in client-runtime. The cached
detail atom holds only the last 10 user turns (nothing for never-opened
threads), so reading client state would silently truncate.
- This is the only asynchronous copy action, so clipboard writes follow
"last copy wins": every in-app write bumps a shared epoch (the direct
`navigator.clipboard` call sites now route through
`writeTextToClipboard`), and a transcript fetch whose epoch moved drops
its result. A late-landing transcript is therefore harmless — it only
lands when nothing else was copied since the click. Copies made in other
applications during the fetch are unobservable from a web page (window
bounded by the 6s fetch timeout).
- Not wired: LegacySidebar (hand-curated legacy menu) and mobile (the
shared factory makes later adoption cheap).

Verified with unit tests (serializer, menu builder, clipboard epoch) and
an integrated browser pass against seeded fixtures — including
reproducing both clipboard race directions on an 8000-message thread and
confirming the guard fixes them.

## Before / After

| Before | After |
| --- | --- |
| ![Thread context menu without Copy
transcript](https://raw.githubusercontent.com/yngatech/t3code/assets/pr-143-copy-transcript/before-menu.png)
| ![Thread context menu with Copy
transcript](https://raw.githubusercontent.com/yngatech/t3code/assets/pr-143-copy-transcript/after-menu.png)
|

![Transcript copied toast reporting 4
messages](https://raw.githubusercontent.com/yngatech/t3code/assets/pr-143-copy-transcript/after-toast.png)

---
Written by an agent (T3 Code, claude-fable-5).
> [!NOTE]
> Shows GitHub repository requirements directly in the Checks tab and
promotes Auto-merge when a pull request is blocked but otherwise
mergeable.

## What changed

- Added GitHub repository-policy-aware merge readiness to pull request
details.
- Marked reported required checks and synthesized waiting rows for
required checks that have not reported yet.
- Read effective repository and organization rulesets alongside classic
branch protection, with graceful GitHub Enterprise degradation.
- Replaced the primary Merge action with Auto-merge when GitHub says the
pull request is blocked and the viewer may enable auto-merge.
- Kept unknown merge states and behind-base branches on their existing
merge/update flows.

## Why

A green or red job list does not explain whether GitHub will actually
allow a merge. Required checks and GitHub’s merge-readiness verdict make
the Checks tab answer that question directly, while Auto-merge becomes
the obvious action when requirements are still outstanding.

## UI changes

Captured against this pull request while its required checks were
running.

| Before: reported jobs only | After: repository requirements |
| --- | --- |
| ![Before: the Checks tab shows generic job progress and a Merge
button](https://raw.githubusercontent.com/yngatech/t3code/c71d13890d511e75fb6928c7ced92f653c586115/assets/pr/required-checks-before.png)
| ![After: the Checks tab shows merge readiness, Required badges, and an
Auto-merge
button](https://raw.githubusercontent.com/yngatech/t3code/c71d13890d511e75fb6928c7ced92f653c586115/assets/pr/required-checks-after.png)
|

## Verification

- `vp test run apps/server/src/pullRequest/GitHubPullRequestCli.test.ts
apps/server/src/pullRequest/GitHubPullRequestProvider.test.ts
apps/server/src/pullRequest/PullRequestService.test.ts
apps/server/src/pullRequest/gitHubPullRequestJson.test.ts
apps/server/src/pullRequest/pullRequestChecks.test.ts
apps/web/src/components/pullRequest/PullRequestChecksTab.test.tsx
apps/web/src/components/pullRequest/pullRequestChecks.test.tsx
apps/web/src/components/pullRequest/pullRequestDetail.logic.test.ts`
- `vp run t3#typecheck`
- `vp run @t3tools/web#typecheck`
- `vp run @t3tools/contracts#typecheck`
- Targeted formatting and lint checks

## Checklist

- [x] This PR is focused on GitHub merge requirements and the action
they imply
- [x] I explained what changed and why
- [x] I included before/after screenshots for the UI change
- [x] No video is needed because this change adds no motion or timing
behavior

---
Written by an agent (T3 Code, gpt-5.6-sol).
The web client could start restoring the most recent project from cached
thread shells while the server was still resolving its launch-directory
bootstrap project. If those automatic navigations overlapped, a late
bootstrap route could switch projects after the composer became
editable, leaving the user's text behind as a draft in the first
project.

Coordinate index-draft and server-bootstrap navigation through a
one-shot in-memory owner. A bootstrap target that is already known can
open normally, but a late bootstrap welcome cannot interrupt another
launch navigation. Bootstrap targets are marked handled per environment
and thread so later visits to Home continue to open a normal draft.

Verification:
- `vp test run apps/web/src/launchNavigationStore.test.ts`
- `vp run typecheck` in `apps/web`
- targeted formatter check

Browser evidence was not captured because this is launch-timing behavior
and computer-use permission was not requested.

---
Written by an agent (T3 Code, gpt-5.6-sol).
> [!NOTE]
> TL;DR: Clicking an `http://localhost:5173` link in chat now opens the
integrated browser instead of the system browser. Every other host is
unchanged. Meta/ctrl-click and the right-click menu still reach the
system browser.

## The problem

A loopback link an agent writes into chat names a port on the machine
the thread runs on. In a remote or SSH environment that is not the
machine the system browser runs on, so following the link opened the
reader's own empty port instead of the dev server. Locally it worked,
which is why it survived this long.

## What changed

`apps/web/src/components/chat/loopbackLinkPreview.ts` is a pure
predicate: open in the integrated browser when the href passes
`isPreviewableUrl`, a thread and preview runtime are available, and no
meta/ctrl modifier is held. The anchor renderer in `ChatMarkdown.tsx`
consults it after the existing change-request branch; everything else
keeps the `target="_blank"` the shell already handles.

Two details worth reviewing:

- **The loopback test is reused, not rewritten.** `isPreviewableUrl`
(`packages/shared/src/preview.ts`) is the same check the terminal-link
right-click gate uses, so the two paths cannot drift into disagreeing
about what "previewable" means. The open path is the same
`openUrlInPreview` the link context menu already used, which brings
thread history, right-panel opening, and the remote-environment loopback
rewrite in `browserTargetResolver.ts` along unchanged.
- **A failed open falls back to the system browser.** The click asked
for the link, not for the integrated browser specifically, so leaving it
doing nothing would be worse than the old behavior. Interrupted
(superseded) commands do not trigger the fallback.

Non-loopback hosts deliberately stay external. The integrated browser
runs in its own persisted partition, so routing github.com or Vercel
there would land the reader on a logged-out page while their real
session sits in the system browser.

## Surfaces

- **Desktop:** new behavior.
- **Web in a plain browser:** unchanged. `isPreviewSupportedInRuntime()`
is `Boolean(window.desktopBridge?.preview)`, so `canOpenInPreview` is
false and links fall through exactly as before.
- **Mobile:** untouched — there is no integrated browser to route to.
- **Reverse state:** the right-click menu still offers both
destinations, and meta/ctrl-click escapes to the system browser, so
every click is overridable in both directions.

## Verification

- `vp test run apps/web/src/components/chat/loopbackLinkPreview.test.ts
packages/shared/src/preview.test.ts
apps/web/src/components/ChatMarkdown.test.tsx`
- `vp run @t3tools/web#typecheck`
- Targeted lint on the three changed files

## Checklist

- [x] This PR is focused on where loopback chat links open
- [x] I explained what changed and why
- [ ] No before/after image: the link is visually identical before and
after, and only its destination changes. Verified manually instead.
- [x] No video is needed because this change adds no motion or timing
behavior

## Follow-up, not in this PR

Terminal links still pop an "Open in preview / Open in browser" menu on
every loopback click rather than defaulting to preview. Aligning the two
paths changes existing behavior in a different feature and belongs in
its own PR.

---
Written by an agent (Claude Code, claude-opus-5).
> [!NOTE]
> TL;DR: Cmd-clicking an `http://localhost:5173` link in the terminal
now opens the integrated browser directly instead of popping a two-item
chooser first. Non-loopback links are unchanged. Follow-up to #153, but
independent of it.

## The problem

Activating a loopback link in the terminal showed a menu at the cursor —
"Open in preview" / "Open in browser" — on every single click.
Activation is already a deliberate modifier gesture (cmd-click on macOS,
ctrl-click elsewhere), so the chooser added a second click to every
visit to a dev server you are iterating on, for a decision that is
nearly always the same one.

Non-loopback URLs never showed the menu; they already went straight to
the system browser. So the menu was the only place in the app where
following a link asked a question first.

## What changed

`openTerminalLinkInPreview` opens loopback links in the integrated
browser and hands everything else to the system browser, with no menu in
between. That deletes the menu plumbing:
`TerminalLinkContextMenuShowError`, and the `localApi` and `position`
arguments that only existed to position and show it — 107 lines out, 63
in.

The loopback test, the failure fallback, the interrupt handling, and the
origin-only error logging (so a link carrying a token in its query is
not logged) are all unchanged.

## Reverse state

The way back out is the integrated browser's own open-in-system-browser
button in `PreviewChromeRow`, which is one click and already there. I
deliberately did not invent a modifier for "system browser instead":
cmd/ctrl is already spent on activation itself, and a shift variant
would be undiscoverable. If it turns out people want the choice back, a
setting is the honest shape for it, not a menu on every click.

## Relationship to #153

Same reasoning applied to a different surface, and the two together make
the rule uniform: a loopback link opens in the integrated browser
because it is the only browser that can reach the environment's ports;
every other host opens in the system browser, where the reader is signed
in.

They touch disjoint files and can land in either order, so this is a
separate PR rather than a stacked one.

## Surfaces

- **Desktop:** new behavior.
- **Web in a plain browser:** unchanged. `isPreviewSupportedInRuntime()`
is false, so links fall through to the system browser exactly as before.
- **Mobile:** no terminal drawer, nothing to change.

## Verification

- `vp test run
apps/web/src/components/preview/openTerminalLinkInPreview.test.ts`
- `vp run @t3tools/web#typecheck`
- `vp check` on the three changed files

## Checklist

- [x] This PR is focused on removing one chooser from one gesture
- [x] I explained what changed and why
- [ ] No before/after image: the terminal link looks identical, and the
visible difference is the absence of a menu. Verified manually.
- [x] No video is needed because this change adds no motion or timing
behavior

---
Written by an agent (Claude Code, claude-opus-5).
> [!NOTE]
> Adds a project-level **Additional instructions** preference and
applies it whenever T3 Code starts an agent session.

## Problem

Projects cannot currently provide durable guidance to every coding-agent
session, so users must repeat the same instructions in each thread.

## What changed

- Add an **Additional instructions** textarea to Project Settings,
shared across grouped checkouts.
- Persist the preference through orchestration events, projections,
snapshots, and a fork migration.
- Pass instructions through native channels for Codex, Claude, Grok, and
OpenCode.
- Prefix the first prompt of fresh Cursor sessions because Cursor ACP
does not expose a native system-instruction channel.
- Retain instructions when provider sessions are recovered.
- Document the behavior and reset flow.

## Verification

- Focused orchestration, projection, migration, recovery, and provider
adapter tests pass.
- Contracts, server, and web typechecks pass.
- Targeted lint and formatting checks pass.
- Verified in an isolated T3 preview that Project Settings renders the
new textarea and explanatory copy. Before/after capture is omitted
because the shared preview screenshot endpoint repeatedly timed out.

---
Written by an agent (T3 Code, gpt-5.6-sol).
Settled turns currently collapse their work behind a duration-only
divider, which makes it hard to tell whether a turn mostly inspected
files, ran commands, edited code, used external tools, or compacted
context.

This adds a compact, end-aligned activity summary to the existing
`Worked for …` fold. Counts are derived from the already-collapsed work
entries, grouped by activity category, and remain accessible through
singular/plural labels. Context compaction uses the semantic theme color
and stays visible when ordinary activity metadata clips in narrow panes.

Web and desktop receive the change through the shared web client. Mobile
remains unchanged in this PR.

## Screenshots

### Before

<img width="768" alt="Before: duration-only folded turn divider"
src="https://github.com/yngatech/t3code/blob/65ec96f9d26571abbe17a6a4747ab1c19104e066/pr-158-before.png?raw=true">

### After

<img width="768" alt="After: folded turn divider with end-aligned
activity counts"
src="https://github.com/yngatech/t3code/blob/65ec96f9d26571abbe17a6a4747ab1c19104e066/pr-158-after.png?raw=true">

Tests:

- `vp test run
apps/web/src/components/chat/MessagesTimeline.logic.test.ts
apps/web/src/components/chat/MessagesTimeline.test.tsx`
- `vp run --filter @t3tools/web typecheck`
- Targeted `vp lint` on the four changed files

---
Written by an agent (T3 Code, gpt-5.6-sol).
The Summary tab currently shows check health without the repository's
merge verdict, so an optional failing check can make a ready pull
request look blocked. The inverse is also possible when checks pass but
repository policy still blocks the merge.

This puts the host-provided merge verdict first in the existing Summary
checks control, keeps non-green check health as supporting context, and
preserves the previous checks-only fallback when the host has no
verdict. The condensed title row uses the same policy-first presentation
with shorter copy.

## Screenshot

![Pull request Summary showing Ready to merge alongside one optional
failed
check](https://raw.githubusercontent.com/yngatech/t3code/034cb67b1dec01b8b73c7db251131f9a662329a8/summary.png)

## Verification

- `vp test run src/components/pullRequest/PullRequestChecksTab.test.tsx
--project unit`
- `vp run --filter @t3tools/web typecheck`
- Preview: Summary chip reads `Ready to merge · 1 of 9 failing`
- Preview: condensed title row reads `Ready · 1 failing`
- Preview: the Summary chip opens the policy-aware Checks view

---
Written by an agent (T3 Code, gpt-5.6-sol).
> [!NOTE]
> Adds a clearly labeled operational CO₂ estimate to Usage on web,
desktop, and mobile, with familiar comparisons and the assumptions kept
one click away.

## What changed

- estimates operational emissions from generated tokens using a
documented 0.43 g CO₂ per 1,000 output-token factor
- shows phone-charge comparisons below 1 kg and driving-distance
comparisons at 1 kg and above
- adds a methodology popover on web/desktop and a native information
alert on mobile
- shares calculation and formatting logic across clients, with focused
unit coverage
- documents the estimate, exclusions, and source methodology

## Screenshots

### Web / desktop

| Before | After |
| --- | --- |
| ![Usage totals before the estimated carbon
metric](https://raw.githubusercontent.com/yngatech/t3code/3b6238066c0eee39f8233e54a61d414a92bea19a/assets/pr/usage-carbon-web-before.png)
| ![Usage totals with the estimated carbon
metric](https://raw.githubusercontent.com/yngatech/t3code/3b6238066c0eee39f8233e54a61d414a92bea19a/assets/pr/usage-carbon-web-after.png)
|

### Mobile

| Before | After |
| --- | --- |
| ![Mobile Usage totals before the estimated carbon
metric](https://raw.githubusercontent.com/yngatech/t3code/3b6238066c0eee39f8233e54a61d414a92bea19a/assets/pr/usage-carbon-mobile-before.jpg)
| ![Mobile Usage totals with the estimated carbon
metric](https://raw.githubusercontent.com/yngatech/t3code/3b6238066c0eee39f8233e54a61d414a92bea19a/assets/pr/usage-carbon-mobile-after.jpg)
|

All captures use an isolated environment with synthetic usage data.

## Verification

- `vp test run packages/shared/src/usageFormat.test.ts`
- `vp run --filter @t3tools/shared typecheck`
- `vp run --filter @t3tools/web typecheck`
- `vp run --filter @t3tools/mobile typecheck`
- integrated web and iOS passes against the same synthetic usage fixture

---
Written by an agent (T3 Code, gpt-5.6-sol).
The pull request panel can already know that a thread’s pull request
merged while the sidebar still holds its older open snapshot, leaving
the thread active until the separate VCS poll catches up.

Feed a matching panel merge into the existing thread change-request
snapshot, seeding it from the active thread when the sidebar row has not
mounted. The shared snapshot reconciliation treats merged as
irreversible, so stale VCS open state cannot undo the update.
Open/closed transitions remain owned by VCS because cached panel detail
cannot safely establish their ordering.

No server cache, persistence, polling, or contract changes are required.

Verification:
- `vp run --filter @t3tools/web typecheck`
- Web unit suite: 2,741 tests passed

---
Written by an agent (T3 Code, gpt-5.6-sol).
## What Changed

Updated the nightly workflow to use `actions/cache@v5` and
`softprops/action-gh-release@v3`.

## Why

Keeps the nightly release workflow aligned with the latest action
versions and current CI dependencies.

## Checklist

- [x] This PR is small and focused
- [x] I explained what changed and why
- [x] I included before/after screenshots for any UI changes
- [x] I included a video for animation/interaction changes
## What Changed

Replaced the Discord webhook action in the fork nightly workflow with a
direct `curl` request that builds the payload using `jq`, retries
transient failures, and fails when Discord rejects the notification.

## Why

The nightly failure notification was not reliable enough. The new
implementation makes the request and payload construction explicit,
preserves the failure details and workflow link, and adds retry handling
for transient errors.

## Checklist

- [x] This PR is small and focused
- [x] I explained what changed and why
- [x] I included before/after screenshots for any UI changes
- [x] I included a video for animation/interaction changes
## What Changed

Result toasts for opened pull requests now include the pull request
number and open the in-app pull request panel when available. They
continue to open the external pull request URL when the panel callback
is unavailable.

## Why

Previously, clicking “View PR” from a result toast always opened the
pull request externally, preventing users from viewing it in the app.

## UI Changes

The result toast action now opens the corresponding pull request in the
in-app panel.

## Checklist

- [x] This PR is small and focused
- [x] I explained what changed and why
- [ ] I included before/after screenshots for any UI changes
- [ ] I included a video for animation/interaction changes
The sidebar's project filter narrowed the thread list but had no say in
where new threads went, so picking a project still left the New thread
button asking which one you wanted.

The filter now decides where the sidebar's button creates, because that
button belongs to the filtered list and a draft made outside the scope
would have no visible row in it. chat.newLocal keeps the opposite
precedence — it starts a thread beside the one on screen, and only falls
back to the filter when nothing is open — so a narrowed sidebar never
removes the way to work beside your current thread. chat.new still opens
the chooser. The button keeps one behavior per state rather than letting
a modifier divert it.

The filter moved out of component state into a small store, since the
shortcuts live in the chat route and the palette mounts at the root. Its
self-healing reset now waits while the project list is empty instead of
treating "not loaded yet" as "stale filter", which previously let any
remount or reconnect drop the filter silently.

Also corrects the command palette's new-thread shortcut labels, which
have advertised the chooser's shortcut on the direct-create item since
chat.new started opening a picker.

---
Written by an agent (Claude Code, claude-opus-5).
## What Changed

Load the upstream rebase policy script from the immutable `main`
snapshot when running a `source_ref` nightly rebase. The workflow now
validates that both required policy scripts exist on `main`, writes the
rebase script to a temporary file, and sources it from there.

Direct runs continue to use the repository's checked-in rebase script.

## Why

A stale `source_ref` could otherwise control or roll back the rebase
policy script. Loading the script from immutable `main` ensures nightly
rebases use the current trusted policy and fail clearly when `main` is
missing the required script.

## Checklist

- [x] This PR is small and focused
- [x] I explained what changed and why
- [x] I included before/after screenshots for any UI changes
- [x] I included a video for animation/interaction changes
incognitojam and others added 10 commits August 16, 2026 13:04
## What changed

Fork Nightly now reapplies upstream-equivalent commits and stops when a
fork patch becomes empty, making patch retirement a maintainer-reviewed
decision. Successful rebases add a compact patch range-diff to the
workflow summary.

When a rebased candidate has the same tree as the released nightly but
newer ancestry or a reviewed patch retirement, the workflow now promotes
that candidate to `main` without publishing another release. It no
longer restores the older nightly history.

Focused Git fixtures cover clean replay, upstream patch adoption,
content conflicts, range-diff reporting, and tree-equivalent promotion
through the real backup and lease script.

## Verification

- `vp test run scripts/rebase-onto-upstream.test.ts
scripts/check-source-stack.test.ts`
- `vp run --filter @t3tools/scripts typecheck`
- `vp lint scripts/rebase-onto-upstream.test.ts
--report-unused-disable-directives`
- `shellcheck .github/scripts/rebase-onto-upstream.sh
.github/scripts/check-source-stack.sh`
- `git diff --check`

---
Written by an agent (T3 Code, gpt-5.6-sol).
…168)

The compaction marker in the folded-turn work log row was tinted
`text-primary/70`. `--primary` maps to the theme's solid-control fill —
the send-button background, which pairs with `--primary-foreground` on
top of it — so read as text on the chat canvas it lands dimmer than the
muted stats it is meant to outrank. That holds in all 10 shipped
palettes; three fall below the 3:1 floor outright, and T3 Chat dark is
effectively invisible at 1.59:1.

The marker now takes `text-foreground` while its siblings stay
`text-muted-foreground`, keeping its emphasis by being the brightest
thing in the row plus its pinned position beside the chevron. That puts
it between 12.4:1 and 16.7:1. The `transition-colors` / `group-hover`
pair went with it, since there is nothing left to transition and the
row's own `hover:text-foreground` already covers hover. The test
asserted the literal `text-primary/70` string; it now asserts the marker
carries `text-foreground` and not a fill-role colour.

Introduced in #158. Roughly nine other sites read `--primary` as text on
the canvas with the same flaw — fixing that class properly needs a
contrast-solved accent-text role in
`packages/shared/src/themePalettes.ts`, which I left alone. No
before/after image: capture kept failing on this view, and the change is
one colour token on a 12px glyph.

---
Written by an agent (Claude Code, claude-opus-5).
Old timestamps currently remain day counts forever, producing labels
such as `1093d ago`.

This updates the shared web and mobile relative-time formatters to
switch to months after 30 days and years after 12 30-day months. Focused
boundary coverage verifies `29d`, `1mo`, `11mo`, and `1y`.

## Screenshots

The affected surfaces were rendered locally with public/synthetic
fixtures across pull request rows, recent threads, and pull request
activity. The collaborative preview screenshot exporter failed during
capture, so no image is attached rather than substituting generated
evidence.

## Test plan

- `vp test run apps/web/src/timestampFormat.test.ts
apps/mobile/src/lib/time.test.ts`

---
Written by an agent (T3 Code, gpt-5.6-sol).
## Summary

Background preview tabs remain parked far outside the Electron
compositor. Their DOM automation stays available, but
`WebContents.capturePage()` can reject with `UnknownVizError` or never
settle because the guest has no captureable surface.

This change gives snapshot requests a short-lived, reference-counted
compositor lease. Only the target webview moves in-window with `opacity:
0`, the host waits for guest render frames, and the lease is always
released after capture. Native snapshot capture is also bounded and
single-flight so a stuck promise cannot start overlapping compositor
copies.

This is the screenshot portion of #170. Recording and preview visibility
semantics remain separate follow-ups.

## Testing

- `vp test run apps/desktop/src/preview/Manager.test.ts
apps/web/src/browser/browserSurfaceStore.test.ts
apps/web/src/browser/hostedBrowserWebviewStyle.test.ts`
- `vp run --filter @t3tools/web typecheck`
- `vp run --filter @t3tools/desktop typecheck`
- Targeted lint for all changed files
- Electron 41.5.0 compositor probe: five parked-to-leased captures
returned non-empty 1600x1200 images in 5-76 ms
- Dev desktop MCP verification: visible capture succeeded; two fresh
`open: false` tabs stayed `visible: false` before and after valid
1280x800 PNG snapshots; fresh `open: true, show: true` background tabs
also captured successfully

---
Written by an agent (T3 Code, gpt-5.6-sol).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
> [!NOTE]
> Adds the first incremental capability ledger and uses it to flag
upstream overlap during Fork Nightly.

## What changed

- adds a validated, capability-oriented fork ledger with six initial
high-risk or conflict-prone entries
- distinguishes fork implementation evidence from upstream contact paths
- makes Fork CI validation blocking and Nightly overlap reporting
advisory and rename-aware
- documents incremental maintenance, review, and retirement of fork
capabilities

## Verification

- `vp run --filter @t3tools/scripts ledger:check`
- `vp test run scripts/fork-feature-ledger.test.ts
scripts/rebase-onto-upstream.test.ts` (13 tests)
- `vp run --filter @t3tools/scripts typecheck`
- targeted lint for the ledger scripts and tests
- `git diff --check`
- confirmed all 22 initial upstream contact paths exist on
`upstream/main`

---
Written by an agent (T3 Code, gpt-5.6-sol).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Seventeen findings from a working spike (native/dictation-spike/README.md)
drive every design decision: Parakeet GGUF over Apple SpeechAnalyzer (95% vs
64-82% identifier recall, noise-robust), no ASR-level biasing (contextualStrings
ignored by SpeechTranscriber; custom LMs ignored entirely), deterministic
matcher over LLM post-processing (4ms, cannot hallucinate), ~100-term
context-scoped vocabulary (repo-scale damaged a third of utterances), and
marked substitutions as a requirement since phrase collisions are deterministic
consequences of correct transcription.

Corpus audio is gitignored (personal voice data); the ASR transcripts are
committed as regression fixtures.
Deterministic recovery of code identifiers from ASR output ("work tree path"
-> worktreePath). Ported from the spike's measured Swift implementation; every
rule cites the finding that paid for it: acronym-aware spoken forms, boundary
stopwords waived for the candidate's own words, length-banded edit budget
(threshold-insensitive, unlike a ratio), punctuation and line boundaries,
0.05 ambiguity margin, and spoken-form collision collapse at vocabulary build.

Substitutions carry output offsets for the revert UI. Tests embed real
Parakeet transcripts of a second speaker as fixtures, including one asserting
the isEmpty/isValid phrase collisions must fire — losing them means recall
broke; the product mitigation is visible, revertable substitutions.
…oint

native/dictation: Rust sidecar (transcribe-cpp, Parakeet GGUF), PCM on stdin,
JSONL events with cumulative RTF on stdout. Metal on macOS, static CPU on
Linux (0.39 RTF single-stream; no GPU required for the single-user case).
Staged via the resource-monitor pattern, generalized to stageRustSidecar.

DictationEngine keeps one warm sidecar (model load ~10s cold, ~0.6s from page
cache), replaces it per utterance, unloads after 5 idle minutes, and ties the
process to the request scope so client disconnects cannot orphan it.

The endpoint is a WebSocket at /api/dictation/stream: browsers only allow
streaming fetch bodies over HTTP/2, so a POST cannot work against the HTTP/1.1
server; binary frames carry PCM without base64 inflation, a text frame
finalizes, JSON text frames stream back. Auth via wsTicket or same-origin
cookie. 503 with a reason before upgrade when no model is configured
(T3CODE_DICTATION_MODEL_PATH or dictationModelPath).

macOS packaging gains the audio-input entitlement and
NSMicrophoneUsageDescription via extendInfo — without both, packaged builds
record digital silence with no error anywhere (spike finding 4).
Capture in the renderer (shared by web and desktop): getUserMedia at the
microphone's native rate — forcing 16kHz anywhere in the chain renders
silence in Chrome — with an AudioWorklet kept in the render graph through a
muted gain, JS resampling to the 16kHz wire format, and a framer that flushes
its partial frame on stop. Stop lingers 350ms so the final word survives the
click; Escape discards without inserting. Session is an app-wide singleton
and the hook guards its async startup window, so overlapping utterances are
structurally impossible.

Composer: mic button in the footer, live bubble (portalled — the footer
cluster is overflow-x-auto and clips absolute children), level meter with a
no-signal warning after 2.5 silent seconds naming the diagnosis while it
happens, matcher substitutions listed with one-tap revert via a new
ChatComposerHandle.replaceRange, stash fallback when the composer refuses
input, and composer.dictate on mod+shift+d.

Desktop grants media permission on the main window session only, scoped to
the app's own origin — never the preview partitions. Vite forwards WebSocket
upgrades on /api (ws:true), without which the dictation socket hangs at
pending forever. Settings: dictationEnabled (off by default),
dictationVocabularyEnabled, dictationMode (hold mode still to come).
Three real defects the PR run surfaced:

- DictationEngine tests resolved the sidecar from disk, so they passed
  locally (where native/dictation is built) and failed in CI with the wrong
  reason. Both the sidecar and model paths are now set explicitly per test,
  via a new dictationSidecarPath config field alongside dictationModelPath.
- An explicitly configured sidecar path is now authoritative rather than the
  first entry in a fallback chain: a wrong path should say so, not silently
  run a different binary found on disk.
- process.platform in Effect code, the node: import convention in the spike
  harness scripts, and repo formatting.

Also fmt-checks native/dictation in CI, matching resource-monitor.
@incognitojam
incognitojam force-pushed the t3code/add-desktop-dictation-support branch from 40f2b5e to 14062c2 Compare August 16, 2026 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant