fix(i18n): don't treat mustache-style {{tokens}} as named substitutions - #2486
fix(i18n): don't treat mustache-style {{tokens}} as named substitutions#2486thribhuvan003 wants to merge 1 commit into
Conversation
✅ Deploy Preview for creative-fairy-df92c4 ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
friendly bump — still happy to tweak anything if this needs changes. no rush. |
@wxt-dev/analytics
@wxt-dev/auto-icons
@wxt-dev/browser
@wxt-dev/i18n
@wxt-dev/is-background
@wxt-dev/module-react
@wxt-dev/module-solid
@wxt-dev/module-svelte
@wxt-dev/module-vue
@wxt-dev/runner
@wxt-dev/storage
@wxt-dev/unocss
@wxt-dev/webextension-polyfill
wxt
commit: |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2486 +/- ##
==========================================
+ Coverage 79.19% 79.25% +0.06%
==========================================
Files 134 134
Lines 3994 3996 +2
Branches 921 923 +2
==========================================
+ Hits 3163 3167 +4
+ Misses 735 734 -1
+ Partials 96 95 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
aklinker1
left a comment
There was a problem hiding this comment.
Here's the results from bun bench:
Simple
┌───┬───────────┬──────────────────┬──────────────────┬────────────────────────┬────────────────────────┬─────────┐
│ │ Task name │ Latency avg (ns) │ Latency med (ns) │ Throughput avg (ops/s) │ Throughput med (ops/s) │ Samples │
├───┼───────────┼──────────────────┼──────────────────┼────────────────────────┼────────────────────────┼─────────┤
│ 0 │ Vanilla │ 2004.0 ± 1.99% │ 0.00 ± 0.00 │ 489248 ± 0.04% │ 499013 ± 0 │ 499013 │
│ 1 │ main │ 2092.5 ± 1.94% │ 0.00 ± 0.00 │ 468105 ± 0.04% │ 477890 ± 0 │ 477938 │
│ 2 │ PR │ 2162.0 ± 1.94% │ 0.00 ± 0.00 │ 452770 ± 0.04% │ 462541 ± 0 │ 462541 │
└───┴───────────┴──────────────────┴──────────────────┴────────────────────────┴────────────────────────┴─────────┘
Substitution
┌───┬───────────┬──────────────────┬──────────────────┬────────────────────────┬────────────────────────┬─────────┐
│ │ Task name │ Latency avg (ns) │ Latency med (ns) │ Throughput avg (ops/s) │ Throughput med (ops/s) │ Samples │
├───┼───────────┼──────────────────┼──────────────────┼────────────────────────┼────────────────────────┼─────────┤
│ 0 │ Vanilla │ 2712.6 ± 1.94% │ 0.00 ± 0.00 │ 358932 ± 0.05% │ 368645 ± 0 │ 368645 │
│ 1 │ main │ 2835.0 ± 1.93% │ 0.00 ± 0.00 │ 343024 ± 0.05% │ 352732 ± 0 │ 352732 │
│ 2 │ PR │ 3124.4 ± 1.93% │ 0.00 ± 0.00 │ 310389 ± 0.06% │ 320058 ± 0 │ 320058 │
└───┴───────────┴──────────────────┴──────────────────┴────────────────────────┴────────────────────────┴─────────┘
Named substitution
┌───┬───────────┬──────────────────┬──────────────────┬────────────────────────┬────────────────────────┬─────────┐
│ │ Task name │ Latency avg (ns) │ Latency med (ns) │ Throughput avg (ops/s) │ Throughput med (ops/s) │ Samples │
├───┼───────────┼──────────────────┼──────────────────┼────────────────────────┼────────────────────────┼─────────┤
│ 0 │ Vanilla │ 2261.7 ± 1.94% │ 0.00 ± 0.00 │ 432378 ± 0.04% │ 442136 ± 0 │ 442180 │
│ 1 │ main │ 3145.5 ± 2.03% │ 0.00 ± 0.00 │ 308278 ± 0.06% │ 317919 ± 0 │ 317919 │
│ 2 │ PR │ 3260.7 ± 2.01% │ 0.00 ± 0.00 │ 297040 ± 0.06% │ 306678 ± 0 │ 306678 │
└───┴───────────┴──────────────────┴──────────────────┴────────────────────────┴────────────────────────┴─────────┘
Plural (singular form)
┌───┬───────────┬──────────────────┬──────────────────┬────────────────────────┬────────────────────────┬─────────┐
│ │ Task name │ Latency avg (ns) │ Latency med (ns) │ Throughput avg (ops/s) │ Throughput med (ops/s) │ Samples │
├───┼───────────┼──────────────────┼──────────────────┼────────────────────────┼────────────────────────┼─────────┤
│ 0 │ Vanilla │ 2354.9 ± 1.94% │ 0.00 ± 0.00 │ 414908 ± 0.05% │ 424651 ± 0 │ 424693 │
│ 1 │ main │ 3157.7 ± 2.03% │ 0.00 ± 0.00 │ 307037 ± 0.06% │ 316682 ± 0 │ 316682 │
│ 2 │ PR │ 3216.2 ± 2.02% │ 0.00 ± 0.00 │ 301295 ± 0.06% │ 310927 ± 0 │ 310958 │
└───┴───────────┴──────────────────┴──────────────────┴────────────────────────┴────────────────────────┴─────────┘
Plural (plural form)
┌───┬───────────┬──────────────────┬──────────────────┬────────────────────────┬────────────────────────┬─────────┐
│ │ Task name │ Latency avg (ns) │ Latency med (ns) │ Throughput avg (ops/s) │ Throughput med (ops/s) │ Samples │
├───┼───────────┼──────────────────┼──────────────────┼────────────────────────┼────────────────────────┼─────────┤
│ 0 │ Vanilla │ 2305.2 ± 2.08% │ 0.00 ± 0.00 │ 424092 ± 0.04% │ 433793 ± 0 │ 433793 │
│ 1 │ main │ 3148.4 ± 2.04% │ 0.00 ± 0.00 │ 307977 ± 0.06% │ 317622 ± 0 │ 317622 │
│ 2 │ PR │ 3244.3 ± 2.01% │ 0.00 ± 0.00 │ 298591 ± 0.06% │ 308233 ± 0 │ 308264 │
└───┴───────────┴──────────────────┴──────────────────┴────────────────────────┴────────────────────────┴─────────┘
| /** | ||
| * Matches single-brace `{name}` placeholders only. Double-brace tokens like | ||
| * `{{name}}` (mustache-style application templates) are left as literals. | ||
| */ | ||
| const NAMED_SUBSTITUTION_RE = /(?<!\{)\{([A-Za-z0-9_]+)\}(?!\})/g; |
There was a problem hiding this comment.
Shouldn't the solution be to allow escaping them? So instead of only handling {{, we should look for \{.
So you would define a string as "{\\{example}}"
| it('should leave mustache-style tokens unchanged', () => { | ||
| expect( | ||
| applyNamedSubstitutions( | ||
| 'Translate {{selection}} into {{targetLanguage}}', |
There was a problem hiding this comment.
So the tests would look like:
| 'Translate {{selection}} into {{targetLanguage}}', | |
| 'Translate {\\{selection}} into {\\{targetLanguage}}', |
12a3f7e to
77688ce
Compare
|
done, went with your escaping suggestion. |
aklinker1
left a comment
There was a problem hiding this comment.
Nice yeah, benchmarks look good. Seems comparable or faster. I ran the latest commit and main twice, there's some variation, and this is a different computer. Not sure what happened for the simple PR 2 benchmark lol doesn't seem right.
Simple
┌───┬───────────┬──────────────────┬──────────────────┬────────────────────────┬────────────────────────┬─────────┐
│ │ Task name │ Latency avg (ns) │ Latency med (ns) │ Throughput avg (ops/s) │ Throughput med (ops/s) │ Samples │
├───┼───────────┼──────────────────┼──────────────────┼────────────────────────┼────────────────────────┼─────────┤
│ 0 │ Vanilla │ 4867.0 ± 10.88% │ 0.00 ± 0.00 │ 197430 ± 0.09% │ 205465 ± 0 │ 205486 │
│ 1 │ main #1 │ 6787.5 ± 4.15% │ 0.00 ± 0.00 │ 138763 ± 0.12% │ 147329 ± 0 │ 147344 │
│ 2 │ main #2 │ 6073.1 ± 2.01% │ 0.00 ± 0.00 │ 155396 ± 0.11% │ 164662 ± 0 │ 164678 │
│ 3 │ PR #1 │ 6555.4 ± 2.16% │ 0.00 ± 0.00 │ 143534 ± 0.12% │ 152547 ± 0 │ 152547 │
│ 4 │ PR #2 │ 931.51 ± 1.95% │ 0.00 ± 0.00 │ 1063616 ± 0.02% │ 1073523 ± 0 │ 1073630 │
└───┴───────────┴──────────────────┴──────────────────┴────────────────────────┴────────────────────────┴─────────┘
Substitution
┌───┬───────────┬──────────────────┬──────────────────┬────────────────────────┬────────────────────────┬─────────┐
│ │ Task name │ Latency avg (ns) │ Latency med (ns) │ Throughput avg (ops/s) │ Throughput med (ops/s) │ Samples │
├───┼───────────┼──────────────────┼──────────────────┼────────────────────────┼────────────────────────┼─────────┤
│ 0 │ Vanilla │ 7471.3 ± 2.97% │ 0.00 ± 0.00 │ 125099 ± 0.14% │ 133845 ± 0 │ 133845 │
│ 1 │ main #1 │ 9010.5 ± 4.25% │ 0.00 ± 0.00 │ 102606 ± 0.16% │ 110982 ± 0 │ 110993 │
│ 2 │ main #2 │ 7104.7 ± 9.07% │ 0.00 ± 0.00 │ 132726 ± 0.12% │ 140752 ± 0 │ 140780 │
│ 3 │ PR #1 │ 8408.1 ± 3.46% │ 0.00 ± 0.00 │ 110510 ± 0.15% │ 118933 ± 0 │ 118945 │
│ 4 │ PR #2 │ 8498.3 ± 4.30% │ 0.00 ± 0.00 │ 109344 ± 0.15% │ 117671 ± 0 │ 117683 │
└───┴───────────┴──────────────────┴──────────────────┴────────────────────────┴────────────────────────┴─────────┘
Named substitution
┌───┬───────────┬──────────────────┬──────────────────┬────────────────────────┬────────────────────────┬─────────┐
│ │ Task name │ Latency avg (ns) │ Latency med (ns) │ Throughput avg (ops/s) │ Throughput med (ops/s) │ Samples │
├───┼───────────┼──────────────────┼──────────────────┼────────────────────────┼────────────────────────┼─────────┤
│ 0 │ Vanilla │ 6818.6 ± 2.34% │ 0.00 ± 0.00 │ 137849 ± 0.12% │ 146658 ± 0 │ 146658 │
│ 1 │ main #1 │ 9700.9 ± 3.90% │ 0.00 ± 0.00 │ 94679 ± 0.17% │ 103083 ± 0 │ 103083 │
│ 2 │ main #2 │ 8999.5 ± 2.03% │ 0.00 ± 0.00 │ 102192 ± 0.17% │ 111117 ± 0 │ 111128 │
│ 3 │ PR #1 │ 10025 ± 3.00% │ 0.00 ± 0.00 │ 91082 ± 0.18% │ 99751 ± 0 │ 99771 │
│ 4 │ PR #2 │ 8797.9 ± 7.96% │ 0.00 ± 0.00 │ 105439 ± 0.15% │ 113663 ± 0 │ 113663 │
└───┴───────────┴──────────────────┴──────────────────┴────────────────────────┴────────────────────────┴─────────┘
Plural (singular form)
┌───┬───────────┬──────────────────┬──────────────────┬────────────────────────┬────────────────────────┬─────────┐
│ │ Task name │ Latency avg (ns) │ Latency med (ns) │ Throughput avg (ops/s) │ Throughput med (ops/s) │ Samples │
├───┼───────────┼──────────────────┼──────────────────┼────────────────────────┼────────────────────────┼─────────┤
│ 0 │ Vanilla │ 7465.6 ± 2.89% │ 0.00 ± 0.00 │ 125470 ± 0.13% │ 133948 ± 0 │ 133961 │
│ 1 │ main #1 │ 9743.4 ± 4.41% │ 0.00 ± 0.00 │ 94243 ± 0.17% │ 102634 ± 0 │ 102644 │
│ 2 │ main #2 │ 9252.2 ± 2.40% │ 0.00 ± 0.00 │ 99238 ± 0.17% │ 108082 ± 0 │ 108093 │
│ 3 │ PR #1 │ 9526.3 ± 2.30% │ 0.00 ± 0.00 │ 96542 ± 0.17% │ 104973 ± 0 │ 104983 │
│ 4 │ PR #2 │ 8369.8 ± 3.90% │ 0.00 ± 0.00 │ 110833 ± 0.15% │ 119477 ± 0 │ 119489 │
└───┴───────────┴──────────────────┴──────────────────┴────────────────────────┴────────────────────────┴─────────┘
Plural (plural form)
┌───┬───────────┬──────────────────┬──────────────────┬────────────────────────┬────────────────────────┬─────────┐
│ │ Task name │ Latency avg (ns) │ Latency med (ns) │ Throughput avg (ops/s) │ Throughput med (ops/s) │ Samples │
├───┼───────────┼──────────────────┼──────────────────┼────────────────────────┼────────────────────────┼─────────┤
│ 0 │ Vanilla │ 6797.9 ± 2.23% │ 0.00 ± 0.00 │ 138301 ± 0.12% │ 147103 ± 0 │ 147118 │
│ 1 │ main #1 │ 8604.1 ± 4.34% │ 0.00 ± 0.00 │ 107954 ± 0.15% │ 116223 ± 0 │ 116235 │
│ 2 │ main #2 │ 9080.1 ± 1.99% │ 0.00 ± 0.00 │ 101217 ± 0.17% │ 110131 ± 0 │ 110153 │
│ 3 │ PR #1 │ 10370 ± 3.06% │ 0.00 ± 0.00 │ 88218 ± 0.18% │ 96432 ± 0 │ 96519 │
│ 4 │ PR #2 │ 9103.0 ± 2.59% │ 0.00 ± 0.00 │ 101082 ± 0.17% │ 109854 ± 0 │ 109865 │
└───┴───────────┴──────────────────┴──────────────────┴────────────────────────┴────────────────────────┴─────────┘
|
Wait, does escaping also prevent |
|
yeah, it's general. |
Problem
Since
0.2.6, messages containing mustache-style application tokens ({{targetLanguage}}) are scanned as named substitutions, so the generated types require a substitution object and a plaini18n.t("key")call fails withnot assignable to parameter of type never— even when the app intentionally wants the raw localized template back (e.g. AI prompt templates rendered later by the app itself).Fix
NAMED_SUBSTITUTION_REnow matches single-brace{name}placeholders only, using lookarounds to skip brace groups wrapped in additional braces:The regex is shared by the type scanner (
getNamedSubstitutionNames) and the runtime (applyNamedSubstitutions), so both stay consistent:{{selection}}/{{targetLanguage}}→ left as literals, not scanned, not substituted{name}→ still scanned and substituted exactly as before{greeting}, keep {{token}} as-is) → only{greeting}is a substitutionThis is the first option proposed in the issue (single-brace only), which keeps
0.2.6behavior for all existing single-brace users and restores0.2.5behavior for mustache templates.Verification
Added 4 regression tests (scanner + runtime, pure-mustache and mixed cases), using the reproduction strings from the issue.
bun run test: 49 passed (45 existing + 4 new)bun run check: Oxlint ✔ Publint ✔ TypeScript ✔Fixes #2482