Skip to content

Backend rocm-qwen3-tts-cpp using CPU instead of GPU HIPBLAS gfx1201 #10666

Closed
#10667
Closed
Backend rocm-qwen3-tts-cpp using CPU instead of GPU HIPBLAS gfx1201#10666
#10667
Labels
bugSomething isn't workingunconfirmed

Description

@Jpc-Automation
Jpc-Automation
opened on Jul 3, 2026

LocalAI version:
v4.5.6
localai/localai:latest-gpu-hipblas

Environment, CPU architecture, OS, and Version:
CPU: AMD Ryzen 9 5900X
GPU: gigabyte Radeon™ AI PRO R9700 AI TOP 32G (Rev. 1.1)
ubuntu 24.04
docker: localai/localai:latest-gpu-hipblas

Describe the bug
When running TTS with backend rocm-qwen3-tts-cpp it will use the CPU instead of the AMD GPU.
The same issue happen when using: rocm-omnivoice-cpp.
When using rocm-qwen-tts it will be processed by the GPU but it seems that the processing of this is extremely slow. A sentence of 5 words takes sometime a minute to process even with 0.6b models.
llama-cpp is just working fine and all request are handled bij the GPU.
This issues might apply to other rocm backends as well. If further testing is required let me know!

To Reproduce
Install model: qwen3-tts-cpp, backends install automatically.
run the TTS model in studio and see the GPU is not used when processing the request.

Expected behavior
Processing of rocm backends request trough the AMD GPU instead of the CPU.

Logs
6:19:33 AM.613
stderr
+++ realpath run.sh
6:19:33 AM.613
stderr
++ dirname /backends/rocm-qwen3-tts-cpp/run.sh
6:19:33 AM.614
stderr

  • CURDIR=/backends/rocm-qwen3-tts-cpp
    6:19:33 AM.614
    stderr
  • cd /
    6:19:33 AM.614
    stderr
  • echo 'CPU info:'
    6:19:33 AM.614
    stdout
    CPU info:
    6:19:33 AM.614
    stderr
    ++ uname
    6:19:33 AM.615
    stderr
  • '[' Linux '!=' Darwin ']'
    6:19:33 AM.615
    stderr
  • grep -e 'model\sname' /proc/cpuinfo
    6:19:33 AM.615
    stderr
  • head -1
    6:19:33 AM.616
    stdout
    model name : AMD Ryzen 9 5900X 12-Core Processor
    6:19:33 AM.616
    stderr
  • grep -e flags /proc/cpuinfo
    6:19:33 AM.616
    stderr
  • head -1
    6:19:33 AM.617
    stdout
    flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves user_shstk clzero xsaveerptr wbnoinvd arat npt lbrv nrip_save tsc_scale vmcb_clean flushbyasid pausefilter pfthreshold v_vmsave_vmload vgif umip pku ospke vaes vpclmulqdq rdpid overflow_recov succor fsrm
    6:19:33 AM.618
    stderr
    ++ uname
    6:19:33 AM.618
    stderr
  • '[' Linux = Darwin ']'
    6:19:33 AM.618
    stderr
  • LIBRARY=/backends/rocm-qwen3-tts-cpp/libgoqwen3ttscpp-fallback.so
    6:19:33 AM.618
    stderr
  • grep -q -e '\savx\s' /proc/cpuinfo
    6:19:33 AM.620
    stdout
    CPU: AVX found OK
    6:19:33 AM.620
    stderr
  • echo 'CPU: AVX found OK'
    6:19:33 AM.620
    stderr
  • '[' -e /backends/rocm-qwen3-tts-cpp/libgoqwen3ttscpp-avx.so ']'
    6:19:33 AM.620
    stderr
  • LIBRARY=/backends/rocm-qwen3-tts-cpp/libgoqwen3ttscpp-avx.so
    6:19:33 AM.620
    stderr
  • grep -q -e '\savx2\s' /proc/cpuinfo
    6:19:33 AM.620
    stdout
    CPU: AVX2 found OK
    6:19:33 AM.620
    stderr
  • echo 'CPU: AVX2 found OK'
    6:19:33 AM.620
    stderr
  • '[' -e /backends/rocm-qwen3-tts-cpp/libgoqwen3ttscpp-avx2.so ']'
    6:19:33 AM.620
    stderr
  • LIBRARY=/backends/rocm-qwen3-tts-cpp/libgoqwen3ttscpp-avx2.so
    6:19:33 AM.620
    stderr
  • grep -q -e '\savx512f\s' /proc/cpuinfo
    6:19:33 AM.622
    stderr
  • export LD_LIBRARY_PATH=/backends/rocm-qwen3-tts-cpp/lib:
    6:19:33 AM.622
    stderr
  • LD_LIBRARY_PATH=/backends/rocm-qwen3-tts-cpp/lib:
    6:19:33 AM.622
    stderr
  • export QWEN3TTS_LIBRARY=/backends/rocm-qwen3-tts-cpp/libgoqwen3ttscpp-avx2.so
    6:19:33 AM.622
    stderr
  • QWEN3TTS_LIBRARY=/backends/rocm-qwen3-tts-cpp/libgoqwen3ttscpp-avx2.so
    6:19:33 AM.622
    stderr
  • '[' -f /backends/rocm-qwen3-tts-cpp/lib/ld.so ']'
    6:19:33 AM.622
    stderr
  • echo 'Using lib/ld.so'
    6:19:33 AM.622
    stderr
  • echo 'Using library: /backends/rocm-qwen3-tts-cpp/libgoqwen3ttscpp-avx2.so'
    6:19:33 AM.622
    stderr
  • exec /backends/rocm-qwen3-tts-cpp/lib/ld.so /backends/rocm-qwen3-tts-cpp/qwen3-tts-cpp --addr 127.0.0.1:43981
    6:19:33 AM.622
    stdout
    Using lib/ld.so
    6:19:33 AM.622
    stdout
    Using library: /backends/rocm-qwen3-tts-cpp/libgoqwen3ttscpp-avx2.so
    6:19:33 AM.627
    stderr
    2026/07/03 04:19:33 gRPC Server listening at 127.0.0.1:43981
    6:19:35 AM.615
    stderr
    [qwen3-tts-cpp] Load talker=/models/qwen3-tts-cpp/qwen-talker-0.6b-base-Q8_0.gguf codec=/models/qwen3-tts-cpp/qwen-tokenizer-12hz-Q8_0.gguf use_fa=1 clamp_fp16=0
    6:19:35 AM.619
    stderr
    [qwen3-tts-cpp] Loading talker=/models/qwen3-tts-cpp/qwen-talker-0.6b-base-Q8_0.gguf codec=/models/qwen3-tts-cpp/qwen-tokenizer-12hz-Q8_0.gguf
    6:19:35 AM.619
    stderr
    [Qwen] qwentts.cpp 9dbe7ea (2026-06-24)
    6:19:35 AM.623
    stderr
    [Load] Talker backend: CPU (CPU threads: 4)
    6:19:35 AM.651
    stderr
    [GGUF] /models/qwen3-tts-cpp/qwen-talker-0.6b-base-Q8_0.gguf: 478 tensors, data at offset 5954368
    6:19:36 AM.099
    stderr
    [WeightCtx] Loaded 316 tensors, 774.6 MB into backend
    6:19:36 AM.099
    stderr
    [Talker] Loaded: 28 layers, hidden 1024, heads 16/8, head_dim 128, FFN 3072, RoPE theta 1000000, mrope sections [24,20,20] interleaved=0
    6:19:36 AM.177
    stderr
    [WeightCtx] Loaded 86 tensors, 143.5 MB into backend
    6:19:36 AM.177
    stderr
    [CodePredictor] Loaded: 5 layers, hidden 1024, heads 16/8, head_dim 128, FFN 3072, RoPE theta 1000000, 15 acoustic codebooks (vocab 2048 each), mtp_proj identity
    6:19:36 AM.190
    stderr
    [WeightCtx] Loaded 76 tensors, 22.9 MB into backend
    6:19:36 AM.190
    stderr
    [SpeakerEncoder] Loaded: enc_dim=1024 sr=24000 mel_dim=128 hidden=512 mfa=1536 asp_attn=128 se=128 scale=8
    6:19:36 AM.191
    stderr
    [GGUF] /models/qwen3-tts-cpp/qwen-tokenizer-12hz-Q8_0.gguf: 398 tensors, data at offset 29760
    6:19:36 AM.208
    stderr
    [WeightCtx] Loaded 18 tensors, 33.0 MB into backend
    6:19:36 AM.208
    stderr
    [Quantizer] Loaded: 16 codebooks (1 semantic + 15 acoustic), 2048 entries x 256 dim, hidden 512
    6:19:36 AM.225
    stderr
    [WeightCtx] Loaded 93 tensors, 30.9 MB into backend
    6:19:36 AM.225
    stderr
    [Transformer] Loaded: 8 layers, hidden 512, heads 16/16, head_dim 64, FFN 1024, RoPE theta 10000, sliding window 72
    6:19:36 AM.267
    stderr
    [WeightCtx] Loaded 22 tensors, 33.1 MB into backend
    6:19:36 AM.268
    stderr
    [Upsample] Loaded: 2 blocks (2x ratio per block), channels 1024, dwconv kernel 7
    6:19:36 AM.470
    stderr
    [WeightCtx] Loaded 118 tensors, 142.9 MB into backend
    6:19:36 AM.474
    stderr
    [DAC] Loaded: 4 blocks (strides 8/5/4/3), 24 kHz mono out, weights 142.9 MB
    6:19:36 AM.475
    stderr
    [WeightCtx] Loaded 2 tensors, 3.0 MB into backend
    6:19:36 AM.489
    stderr
    [WeightCtx] Loaded 28 tensors, 24.1 MB into backend
    6:19:36 AM.489
    stderr
    [SEANet] Loaded: 4 stages (ratios 4/5/6/8), 64 -> 512 channels, weights 24.1 MB
    6:19:36 AM.503
    stderr
    [WeightCtx] Loaded 96 tensors, 25.6 MB into backend
    6:19:36 AM.503
    stderr
    [EncTransformer] Loaded: 8 layers, hidden 512, heads 8/8, head_dim 64, FFN 2048, RoPE theta 10000
    6:19:36 AM.504
    stderr
    [WeightCtx] Loaded 1 tensors, 2.0 MB into backend
    6:19:36 AM.504
    stderr
    [EncDownsample] Loaded: k=4 stride=2, 512 -> 512 channels, weights 2.0 MB
    6:19:36 AM.521
    stderr
    [WeightCtx] Loaded 20 tensors, 34.0 MB into backend
    6:19:36 AM.521
    stderr
    [EncQuantizer] Loaded: 16 codebooks (1 semantic + 15 acoustic), 2048 entries x 256 dim, hidden 512, weights 34.0 MB
    6:19:36 AM.521
    stderr
    [Pipeline] Ready: hop 1920 samples @ 24000 Hz mono, 16 codebooks @ 12.5 Hz
    6:19:37 AM.034
    stderr
    [KVCache] Allocated: 28 layers, 8 KV heads, head_dim 128, max_seq_len 4096 -> 896 MB
    6:19:37 AM.034
    stderr
    [KVCache] Allocated: 5 layers, 8 KV heads, head_dim 128, max_seq_len 16 -> 0 MB
    6:19:37 AM.034
    stderr
    [Pipeline] Loaded: arch=0b6 variant=base tokenizer=qwen3_tts_tokenizer_12hz codebooks=16 speaker_encoder=loaded speakers=0 fa=off clamp_fp16=off
    6:19:37 AM.104
    stderr
    [BPE] Loaded from GGUF: 151676 vocab, 151291 merges, eos_id=151643
    6:19:37 AM.128
    stderr
    [BPE] Registered 5 arch special tokens (total specials=6)
    6:19:37 AM.128
    stderr
    [qwen3-tts-cpp] Model loaded (9dbe7ea (2026-06-24))
    6:19:37 AM.130
    stderr
    [Prompt] Built: 9 ids, N_text=1, N_instruct=0, T_ctx=10, hidden=1024, lang=auto (id=-1), speaker=none (id=-1) ref_spk_emb=no icl=no
    6:19:37 AM.176
    stderr
    [Sample] step=0 c0=1995 u=0.7798883915 subseq=0
    6:19:37 AM.180
    stderr
    [Sample-CP] g=0 c=532 u=0.2387637496 subseq=1
    6:19:37 AM.183
    stderr
    [Sample-CP] g=1 c=875 u=0.3543468118 subseq=2
    6:19:37 AM.186
    stderr
    [Sample-CP] g=2 c=1008 u=0.3918431401 subseq=3
    6:19:37 AM.188
    stderr
    [Sample-CP] g=3 c=1106 u=0.7294880152 subseq=4
    6:19:37 AM.191
    stderr
    [Sample-CP] g=4 c=328 u=0.2628061473 subseq=5
    6:19:37 AM.193
    stderr
    [Sample-CP] g=5 c=1776 u=0.8389458060 subseq=6
    6:19:37 AM.196
    stderr
    [Sample-CP] g=6 c=1900 u=0.8923861980 subseq=7
    6:19:37 AM.198
    stderr
    [Sample-CP] g=7 c=1190 u=0.6531952024 subseq=8
    6:19:37 AM.201
    stderr
    [Sample-CP] g=8 c=329 u=0.1972885132 subseq=9
    6:19:37 AM.204
    stderr
    [Sample-CP] g=9 c=1657 u=0.9375275970 subseq=10
    6:19:37 AM.206
    stderr
    [Sample-CP] g=10 c=552 u=0.2702071071 subseq=11
    6:19:37 AM.209
    stderr
    [Sample-CP] g=11 c=706 u=0.3710687160 subseq=12
    6:19:37 AM.211
    stderr
    [Sample-CP] g=12 c=795 u=0.4238485396 subseq=13
    6:19:37 AM.214
    stderr
    [Sample-CP] g=13 c=227 u=0.2331452817 subseq=14
    6:19:37 AM.216
    stderr
    [Sample-CP] g=14 c=1695 u=0.9195791483 subseq=15
    6:19:37 AM.235
    stderr
    [Sample] step=1 c0=215 u=0.2402582169 subseq=16
    6:19:37 AM.238
    stderr
    [Sample-CP] g=0 c=1348 u=0.6951002479 subseq=17
    6:19:37 AM.241
    stderr
    [Sample-CP] g=1 c=875 u=0.4189025462 subseq=18
    6:19:37 AM.244
    stderr
    [Sample-CP] g=2 c=1034 u=0.5119064450 subseq=19
    6:19:37 AM.246
    stderr
    [Sample-CP] g=3 c=1027 u=0.6456944346 subseq=20
    6:19:37 AM.249
    stderr
    [Sample-CP] g=4 c=801 u=0.7522907853 subseq=21
    6:19:37 AM.251
    stderr
    [Sample-CP] g=5 c=761 u=0.3625926375 subseq=22
    6:19:37 AM.254
    stderr
    [Sample-CP] g=6 c=727 u=0.3523568213 subseq=23
    6:19:37 AM.257
    stderr
    [Sample-CP] g=7 c=1190 u=0.3910799026 subseq=24
    6:19:37 AM.259
    stderr
    [Sample-CP] g=8 c=1658 u=0.5923764110 subseq=25
    6:19:37 AM.262
    stderr
    [Sample-CP] g=9 c=223 u=0.0938707888 subseq=26
    6:19:37 AM.264
    stderr
    [Sample-CP] g=10 c=183 u=0.1247528717 subseq=27
    6:19:37 AM.267
    stderr
    [Sample-CP] g=11 c=1093 u=0.6197788119 subseq=28
    6:19:37 AM.269
    stderr
    [Sample-CP] g=12 c=322 u=0.1885991395 subseq=29
    6:19:37 AM.272
    stderr
    [Sample-CP] g=13 c=243 u=0.1877557486 subseq=30
    6:19:37 AM.274
    stderr
    [Sample-CP] g=14 c=58 u=0.0933234990 subseq=31
    6:19:37 AM.631
    stderr
    [Pipeline] Generated 8 frames
    6:19:37 AM.824
    stderr
    [Pipeline] EOS at step 11, stopping
    6:19:37 AM.824
    stderr
    [Pipeline] Generation done : 11 frames
    6:19:38 AM.236
    stderr
    [Perf] PromptBuild 0.8 ms
    6:19:38 AM.236
    stderr
    [Perf] Prefill 45.6 ms (T_ctx prefill)
    6:19:38 AM.236
    stderr
    [Perf] TTFA 87.1 ms (first frame codes)
    6:19:38 AM.236
    stderr
    [Perf] TalkerDecode 201.2 ms (11 frames, 18.29 ms/frame)
    6:19:38 AM.236
    stderr
    [Perf] CodePredictor 445.9 ms (40.53 ms/frame)
    6:19:38 AM.236
    stderr
    [Perf] HostCompose 0.8 ms (c0 sample + next emb)
    6:19:38 AM.236
    stderr
    [Perf] CodecDecode 412.6 ms
    6:19:38 AM.236
    stderr
    [Perf] Total 1107.1 ms (11 frames, 58.90 ms/frame AR, audio 0.88 s, RTF 1.258)
    6:19:41 AM.160
    stderr
    [Prompt] Built: 9 ids, N_text=1, N_instruct=0, T_ctx=10, hidden=1024, lang=auto (id=-1), speaker=none (id=-1) ref_spk_emb=no icl=no
    6:19:41 AM.220
    stderr
    [Sample] step=0 c0=1995 u=0.7182165980 subseq=0
    6:19:41 AM.224
    stderr
    [Sample-CP] g=0 c=911 u=0.3298845887 subseq=1
    6:19:41 AM.230
    stderr
    [Sample-CP] g=1 c=676 u=0.3991713524 subseq=2
    6:19:41 AM.236
    stderr
    [Sample-CP] g=2 c=1884 u=0.9513853192 subseq=3
    6:19:41 AM.242
    stderr
    [Sample-CP] g=3 c=149 u=0.7036579847 subseq=4
    6:19:41 AM.247
    stderr
    [Sample-CP] g=4 c=1966 u=0.9560351372 subseq=5
    6:19:41 AM.252
    stderr
    [Sample-CP] g=5 c=1744 u=0.9182322025 subseq=6
    6:19:41 AM.258
    stderr
    [Sample-CP] g=6 c=486 u=0.4160580635 subseq=7
    6:19:41 AM.263
    stderr
    [Sample-CP] g=7 c=1694 u=0.9401233196 subseq=8
    6:19:41 AM.269
    stderr
    [Sample-CP] g=8 c=1783 u=0.9532976151 subseq=9
    6:19:41 AM.275
    stderr
    [Sample-CP] g=9 c=1012 u=0.5419293642 subseq=10
    6:19:41 AM.280
    stderr
    [Sample-CP] g=10 c=963 u=0.5117388964 subseq=11
    6:19:41 AM.285
    stderr
    [Sample-CP] g=11 c=1609 u=0.8891870975 subseq=12
    6:19:41 AM.290
    stderr
    [Sample-CP] g=12 c=1931 u=0.9562736750 subseq=13
    6:19:41 AM.295
    stderr
    [Sample-CP] g=13 c=479 u=0.2501362264 subseq=14
    6:19:41 AM.299
    stderr
    [Sample-CP] g=14 c=596 u=0.2665247023 subseq=15
    6:19:41 AM.329
    stderr
    [Sample] step=1 c0=215 u=0.8607317805 subseq=16
    6:19:41 AM.334
    stderr
    [Sample-CP] g=0 c=911 u=0.6095188260 subseq=17
    6:19:41 AM.339
    stderr
    [Sample-CP] g=1 c=676 u=0.5627208352 subseq=18
    6:19:41 AM.344
    stderr
    [Sample-CP] g=2 c=2026 u=0.9906225204 subseq=19
    6:19:41 AM.349
    stderr
    [Sample-CP] g=3 c=1807 u=0.8873546124 subseq=20
    6:19:41 AM.354
    stderr
    [Sample-CP] g=4 c=1331 u=0.7623772621 subseq=21
    6:19:41 AM.358
    stderr
    [Sample-CP] g=5 c=406 u=0.3736266792 subseq=22
    6:19:41 AM.363
    stderr
    [Sample-CP] g=6 c=240 u=0.1509320438 subseq=23
    6:19:41 AM.368
    stderr
    [Sample-CP] g=7 c=457 u=0.1992857009 subseq=24
    6:19:41 AM.374
    stderr
    [Sample-CP] g=8 c=1351 u=0.7859373093 subseq=25
    6:19:41 AM.379
    stderr
    [Sample-CP] g=9 c=1386 u=0.7201685309 subseq=26
    6:19:41 AM.385
    stderr
    [Sample-CP] g=10 c=16 u=0.0451930761 subseq=27
    6:19:41 AM.390
    stderr
    [Sample-CP] g=11 c=655 u=0.4000560641 subseq=28
    6:19:41 AM.394
    stderr
    [Sample-CP] g=12 c=692 u=0.5621209741 subseq=29
    6:19:41 AM.398
    stderr
    [Sample-CP] g=13 c=306 u=0.2973393202 subseq=30
    6:19:41 AM.403
    stderr
    [Sample-CP] g=14 c=167 u=0.3096304238 subseq=31
    6:19:42 AM.021
    stderr
    [Pipeline] Generated 8 frames
    6:19:42 AM.246
    stderr
    [Pipeline] EOS at step 10, stopping
    6:19:42 AM.246
    stderr
    [Pipeline] Generation done : 10 frames
    6:19:42 AM.609
    stderr
    [Perf] PromptBuild 0.8 ms
    6:19:42 AM.609
    stderr
    [Perf] Prefill 59.4 ms (T_ctx prefill)
    6:19:42 AM.609
    stderr
    [Perf] TTFA 139.5 ms (first frame codes)
    6:19:42 AM.609
    stderr
    [Perf] TalkerDecode 295.0 ms (10 frames, 29.50 ms/frame)
    6:19:42 AM.609
    stderr
    [Perf] CodePredictor 730.4 ms (73.04 ms/frame)
    6:19:42 AM.609
    stderr
    [Perf] HostCompose 0.8 ms (c0 sample + next emb)
    6:19:42 AM.609
    stderr
    [Perf] CodecDecode 362.5 ms
    6:19:42 AM.609
    stderr
    [Perf] Total 1448.9 ms (10 frames, 102.61 ms/frame AR, audio 0.80 s, RTF 1.811)
    6:20:12 AM.797
    stderr
    [Prompt] Built: 14 ids, N_text=6, N_instruct=0, T_ctx=15, hidden=1024, lang=auto (id=-1), speaker=none (id=-1) ref_spk_emb=no icl=no
    6:20:12 AM.882
    stderr
    [Sample] step=0 c0=1342 u=0.4767470360 subseq=0
    6:20:12 AM.887
    stderr
    [Sample-CP] g=0 c=2017 u=0.9607661963 subseq=1
    6:20:12 AM.893
    stderr
    [Sample-CP] g=1 c=947 u=0.5719247460 subseq=2
    6:20:12 AM.899
    stderr
    [Sample-CP] g=2 c=1983 u=0.9872956872 subseq=3
    6:20:12 AM.904
    stderr
    [Sample-CP] g=3 c=1967 u=0.9809268117 subseq=4
    6:20:12 AM.909
    stderr
    [Sample-CP] g=4 c=182 u=0.0581271574 subseq=5
    6:20:12 AM.914
    stderr
    [Sample-CP] g=5 c=330 u=0.2087856829 subseq=6
    6:20:12 AM.919
    stderr
    [Sample-CP] g=6 c=153 u=0.1030211523 subseq=7
    6:20:12 AM.924
    stderr
    [Sample-CP] g=7 c=268 u=0.4272388816 subseq=8
    6:20:12 AM.928
    stderr
    [Sample-CP] g=8 c=476 u=0.2915150523 subseq=9
    6:20:12 AM.934
    stderr
    [Sample-CP] g=9 c=145 u=0.2511431277 subseq=10
    6:20:12 AM.939
    stderr
    [Sample-CP] g=10 c=16 u=0.0903148279 subseq=11
    6:20:12 AM.945
    stderr
    [Sample-CP] g=11 c=15 u=0.0317194872 subseq=12
    6:20:12 AM.951
    stderr
    [Sample-CP] g=12 c=1256 u=0.7276576757 subseq=13
    6:20:12 AM.957
    stderr
    [Sample-CP] g=13 c=182 u=0.1178383008 subseq=14
    6:20:12 AM.963
    stderr
    [Sample-CP] g=14 c=90 u=0.0842224211 subseq=15
    6:20:12 AM.995
    stderr
    [Sample] step=1 c0=574 u=0.0926388949 subseq=16
    6:20:13 AM.000
    stderr
    [Sample-CP] g=0 c=1198 u=0.6761132479 subseq=17
    6:20:13 AM.006
    stderr
    [Sample-CP] g=1 c=1900 u=0.8454701900 subseq=18
    6:20:13 AM.011
    stderr
    [Sample-CP] g=2 c=1187 u=0.6273615360 subseq=19
    6:20:13 AM.016
    stderr
    [Sample-CP] g=3 c=540 u=0.4124435782 subseq=20
    6:20:13 AM.022
    stderr
    [Sample-CP] g=4 c=1653 u=0.8014432192 subseq=21
    6:20:13 AM.026
    stderr
    [Sample-CP] g=5 c=774 u=0.1943364739 subseq=22
    6:20:13 AM.031
    stderr
    [Sample-CP] g=6 c=1427 u=0.6654624939 subseq=23
    6:20:13 AM.036
    stderr
    [Sample-CP] g=7 c=117 u=0.3085294962 subseq=24
    6:20:13 AM.041
    stderr
    [Sample-CP] g=8 c=83 u=0.0507058166 subseq=25
    6:20:13 AM.047
    stderr
    [Sample-CP] g=9 c=678 u=0.2670448720 subseq=26
    6:20:13 AM.053
    stderr
    [Sample-CP] g=10 c=1931 u=0.9657084942 subseq=27
    6:20:13 AM.058
    stderr
    [Sample-CP] g=11 c=530 u=0.3683357239 subseq=28
    6:20:13 AM.064
    stderr
    [Sample-CP] g=12 c=1550 u=0.8526765704 subseq=29
    6:20:13 AM.069
    stderr
    [Sample-CP] g=13 c=706 u=0.4725142717 subseq=30
    6:20:13 AM.074
    stderr
    [Sample-CP] g=14 c=389 u=0.2776201367 subseq=31
    6:20:13 AM.677
    stderr
    [Pipeline] Generated 8 frames
    6:20:14 AM.420
    stderr
    [Pipeline] Generated 16 frames
    6:20:14 AM.811
    stderr
    [Pipeline] EOS at step 20, stopping
    6:20:14 AM.811
    stderr
    [Pipeline] Generation done : 20 frames
    6:20:15 AM.624
    stderr
    [Perf] PromptBuild 0.8 ms
    6:20:15 AM.624
    stderr
    [Perf] Prefill 84.1 ms (T_ctx prefill)
    6:20:15 AM.624
    stderr
    [Perf] TTFA 166.2 ms (first frame codes)
    6:20:15 AM.624
    stderr
    [Perf] TalkerDecode 580.5 ms (20 frames, 29.03 ms/frame)
    6:20:15 AM.624
    stderr
    [Perf] CodePredictor 1347.1 ms (67.35 ms/frame)
    6:20:15 AM.624
    stderr
    [Perf] HostCompose 1.5 ms (c0 sample + next emb)
    6:20:15 AM.624
    stderr
    [Perf] CodecDecode 813.3 ms
    6:20:15 AM.624
    stderr
    [Perf] Total 2827.5 ms (20 frames, 96.45 ms/frame AR, audio 1.60 s, RTF 1.767)
    6:20:19 AM.343
    stderr
    [Prompt] Built: 14 ids, N_text=6, N_instruct=0, T_ctx=15, hidden=1024, lang=auto (id=-1), speaker=none (id=-1) ref_spk_emb=no icl=no
    6:20:19 AM.420
    stderr
    [Sample] step=0 c0=1995 u=0.9816307425 subseq=0
    6:20:19 AM.424
    stderr
    [Sample-CP] g=0 c=1889 u=0.8898526430 subseq=1
    6:20:19 AM.429
    stderr
    [Sample-CP] g=1 c=1114 u=0.7009730339 subseq=2
    6:20:19 AM.434
    stderr
    [Sample-CP] g=2 c=629 u=0.2441880107 subseq=3
    6:20:19 AM.439
    stderr
    [Sample-CP] g=3 c=913 u=0.4219421148 subseq=4
    6:20:19 AM.443
    stderr
    [Sample-CP] g=4 c=13 u=0.0277327411 subseq=5
    6:20:19 AM.448
    stderr
    [Sample-CP] g=5 c=1579 u=0.5101181269 subseq=6
    6:20:19 AM.452
    stderr
    [Sample-CP] g=6 c=1569 u=0.7806637287 subseq=7
    6:20:19 AM.457
    stderr
    [Sample-CP] g=7 c=1069 u=0.6292861104 subseq=8
    6:20:19 AM.461
    stderr
    [Sample-CP] g=8 c=773 u=0.4647079706 subseq=9
    6:20:19 AM.465
    stderr
    [Sample-CP] g=9 c=327 u=0.1908932924 subseq=10
    6:20:19 AM.470
    stderr
    [Sample-CP] g=10 c=1635 u=0.8981161118 subseq=11
    6:20:19 AM.474
    stderr
    [Sample-CP] g=11 c=1398 u=0.7333962321 subseq=12
    6:20:19 AM.479
    stderr
    [Sample-CP] g=12 c=1328 u=0.6595309377 subseq=13
    6:20:19 AM.483
    stderr
    [Sample-CP] g=13 c=1478 u=0.7444941998 subseq=14
    6:20:19 AM.488
    stderr
    [Sample-CP] g=14 c=1405 u=0.6595103741 subseq=15
    6:20:19 AM.517
    stderr
    [Sample] step=1 c0=215 u=0.2738833129 subseq=16
    6:20:19 AM.521
    stderr
    [Sample-CP] g=0 c=496 u=0.1133521497 subseq=17
    6:20:19 AM.526
    stderr
    [Sample-CP] g=1 c=604 u=0.5278705359 subseq=18
    6:20:19 AM.531
    stderr
    [Sample-CP] g=2 c=1859 u=0.9305798411 subseq=19
    6:20:19 AM.535
    stderr
    [Sample-CP] g=3 c=76 u=0.0309558958 subseq=20
    6:20:19 AM.540
    stderr
    [Sample-CP] g=4 c=497 u=0.4998494685 subseq=21
    6:20:19 AM.544
    stderr
    [Sample-CP] g=5 c=1579 u=0.5240754485 subseq=22
    6:20:19 AM.548
    stderr
    [Sample-CP] g=6 c=1886 u=0.9134004116 subseq=23
    6:20:19 AM.553
    stderr
    [Sample-CP] g=7 c=1422 u=0.7844681740 subseq=24
    6:20:19 AM.557
    stderr
    [Sample-CP] g=8 c=1360 u=0.5350474715 subseq=25
    6:20:19 AM.562
    stderr
    [Sample-CP] g=9 c=288 u=0.2289391607 subseq=26
    6:20:19 AM.566
    stderr
    [Sample-CP] g=10 c=175 u=0.0609402657 subseq=27
    6:20:19 AM.571
    stderr
    [Sample-CP] g=11 c=1452 u=0.7853969932 subseq=28
    6:20:19 AM.575
    stderr
    [Sample-CP] g=12 c=1290 u=0.6003324986 subseq=29
    6:20:19 AM.579
    stderr
    [Sample-CP] g=13 c=297 u=0.2743382752 subseq=30
    6:20:19 AM.584
    stderr
    [Sample-CP] g=14 c=1540 u=0.8637158871 subseq=31
    6:20:20 AM.164
    stderr
    [Pipeline] Generated 8 frames
    6:20:20 AM.931
    stderr
    [Pipeline] Generated 16 frames
    6:20:21 AM.698
    stderr
    [Pipeline] Generated 24 frames
    6:20:22 AM.115
    stderr
    [Pipeline] EOS at step 28, stopping
    6:20:22 AM.115
    stderr
    [Pipeline] Generation done : 28 frames
    6:20:23 AM.337
    stderr
    [Perf] PromptBuild 0.9 ms
    6:20:23 AM.337
    stderr
    [Perf] Prefill 76.9 ms (T_ctx prefill)
    6:20:23 AM.337
    stderr
    [Perf] TTFA 145.8 ms (first frame codes)
    6:20:23 AM.337
    stderr
    [Perf] TalkerDecode 813.9 ms (28 frames, 29.07 ms/frame)
    6:20:23 AM.337
    stderr
    [Perf] CodePredictor 1879.2 ms (67.12 ms/frame)
    6:20:23 AM.337
    stderr
    [Perf] HostCompose 2.1 ms (c0 sample + next emb)
    6:20:23 AM.337
    stderr
    [Perf] CodecDecode 1222.0 ms
    6:20:23 AM.337
    stderr
    [Perf] Total 3995.2 ms (28 frames, 96.26 ms/frame AR, audio 2.24 s, RTF 1.784)
    6:21:14 AM.553
    stderr
    [qwen3-tts-cpp] ERROR: synthesize failed (rc=-2): --speaker is only valid for custom_voice models (loaded: base)
    6:21:46 AM.058
    stderr
    [qwen3-tts-cpp] ERROR: synthesize failed (rc=-2): --speaker is only valid for custom_voice models (loaded: base)
    6:22:06 AM.879
    stderr
    [Prompt] Built: 76 ids, N_text=68, N_instruct=0, T_ctx=77, hidden=1024, lang=auto (id=-1), speaker=none (id=-1) ref_spk_emb=no icl=no
    6:22:07 AM.242
    stderr
    [Sample] step=0 c0=1221 u=0.6049761176 subseq=0
    6:22:07 AM.246
    stderr
    [Sample-CP] g=0 c=698 u=0.3298092484 subseq=1
    6:22:07 AM.252
    stderr
    [Sample-CP] g=1 c=1854 u=0.8592889309 subseq=2
    6:22:07 AM.257
    stderr
    [Sample-CP] g=2 c=1583 u=0.7638835311 subseq=3
    6:22:07 AM.263
    stderr
    [Sample-CP] g=3 c=149 u=0.1931425333 subseq=4
    6:22:07 AM.268
    stderr
    [Sample-CP] g=4 c=840 u=0.4952968061 subseq=5
    6:22:07 AM.273
    stderr
    [Sample-CP] g=5 c=714 u=0.4680204690 subseq=6
    6:22:07 AM.277
    stderr
    [Sample-CP] g=6 c=289 u=0.1935870200 subseq=7
    6:22:07 AM.282
    stderr
    [Sample-CP] g=7 c=1562 u=0.8367241025 subseq=8
    6:22:07 AM.288
    stderr
    [Sample-CP] g=8 c=157 u=0.1005397514 subseq=9
    6:22:07 AM.292
    stderr
    [Sample-CP] g=9 c=560 u=0.5509231091 subseq=10
    6:22:07 AM.297
    stderr
    [Sample-CP] g=10 c=1073 u=0.7382171154 subseq=11
    6:22:07 AM.302
    stderr
    [Sample-CP] g=11 c=284 u=0.2733994722 subseq=12
    6:22:07 AM.308
    stderr
    [Sample-CP] g=12 c=622 u=0.6002939939 subseq=13
    6:22:07 AM.313
    stderr
    [Sample-CP] g=13 c=968 u=0.6454140544 subseq=14
    6:22:07 AM.319
    stderr
    [Sample-CP] g=14 c=440 u=0.2408389747 subseq=15
    6:22:07 AM.349
    stderr
    [Sample] step=1 c0=387 u=0.0191695448 subseq=16
    6:22:07 AM.353
    stderr
    [Sample-CP] g=0 c=1790 u=0.7556315064 subseq=17
    6:22:07 AM.358
    stderr
    [Sample-CP] g=1 c=923 u=0.5343030095 subseq=18
    6:22:07 AM.362
    stderr
    [Sample-CP] g=2 c=932 u=0.4377578199 subseq=19
    6:22:07 AM.366
    stderr
    [Sample-CP] g=3 c=1255 u=0.8561711311 subseq=20
    6:22:07 AM.371
    stderr
    [Sample-CP] g=4 c=1273 u=0.8795647621 subseq=21
    6:22:07 AM.376
    stderr
    [Sample-CP] g=5 c=1776 u=0.7007859945 subseq=22
    6:22:07 AM.381
    stderr
    [Sample-CP] g=6 c=347 u=0.3213272691 subseq=23
    6:22:07 AM.385
    stderr
    [Sample-CP] g=7 c=420 u=0.2964567840 subseq=24
    6:22:07 AM.391
    stderr
    [Sample-CP] g=8 c=1186 u=0.5781899691 subseq=25
    6:22:07 AM.395
    stderr
    [Sample-CP] g=9 c=75 u=0.1047787219 subseq=26
    6:22:07 AM.401
    stderr
    [Sample-CP] g=10 c=668 u=0.4848715961 subseq=27
    6:22:07 AM.406
    stderr
    [Sample-CP] g=11 c=1399 u=0.8239454031 subseq=28
    6:22:07 AM.411
    stderr
    [Sample-CP] g=12 c=918 u=0.5567985773 subseq=29
    6:22:07 AM.416
    stderr
    [Sample-CP] g=13 c=1178 u=0.6187819242 subseq=30
    6:22:07 AM.420
    stderr
    [Sample-CP] g=14 c=1933 u=0.9639543891 subseq=31
    6:22:08 AM.058
    stderr
    [Pipeline] Generated 8 frames
    6:22:08 AM.870
    stderr
    [Pipeline] Generated 16 frames
    6:22:09 AM.670
    stderr
    [Pipeline] Generated 24 frames
    6:22:10 AM.462
    stderr
    [Pipeline] Generated 32 frames
    6:22:11 AM.288
    stderr
    [Pipeline] Generated 40 frames
    6:22:12 AM.107
    stderr
    [Pipeline] Generated 48 frames
    6:22:12 AM.925
    stderr
    [Pipeline] Generated 56 frames
    6:22:13 AM.816
    stderr
    [Pipeline] Generated 64 frames
    6:22:14 AM.618
    stderr
    [Pipeline] Generated 72 frames
    6:22:15 AM.425
    stderr
    [Pipeline] Generated 80 frames
    6:22:16 AM.232
    stderr
    [Pipeline] Generated 88 frames
    6:22:17 AM.053
    stderr
    [Pipeline] Generated 96 frames
    6:22:17 AM.887
    stderr
    [Pipeline] Generated 104 frames
    6:22:18 AM.689
    stderr
    [Pipeline] Generated 112 frames
    6:22:19 AM.468
    stderr
    [Pipeline] Generated 120 frames
    6:22:20 AM.249
    stderr
    [Pipeline] Generated 128 frames
    6:22:21 AM.098
    stderr
    [Pipeline] Generated 136 frames
    6:22:21 AM.838
    stderr
    [Pipeline] EOS at step 143, stopping
    6:22:21 AM.838
    stderr
    [Pipeline] Generation done : 143 frames
    6:22:27 AM.679
    stderr
    [Perf] PromptBuild 5.2 ms
    6:22:27 AM.679
    stderr
    [Perf] Prefill 363.0 ms (T_ctx prefill)
    6:22:27 AM.679
    stderr
    [Perf] TTFA 445.1 ms (first frame codes)
    6:22:27 AM.679
    stderr
    [Perf] TalkerDecode 4688.2 ms (143 frames, 32.78 ms/frame)
    6:22:27 AM.679
    stderr
    [Perf] CodePredictor 9896.5 ms (69.21 ms/frame)
    6:22:27 AM.679
    stderr
    [Perf] HostCompose 10.5 ms (c0 sample + next emb)
    6:22:27 AM.679
    stderr
    [Perf] CodecDecode 5840.3 ms
    6:22:27 AM.679
    stderr
    [Perf] Total 20805.0 ms (143 frames, 102.06 ms/frame AR, audio 11.44 s, RTF 1.819)
    6:23:12 AM.004
    stderr
    [qwen3-tts-cpp] ERROR: synthesize failed (rc=-2): --speaker is only valid for custom_voice models (loaded: base)
    6:25:10 AM.707
    stderr
    [qwen3-tts-cpp] ERROR: synthesize failed (rc=-2): --speaker is only valid for custom_voice models (loaded: base)
    8:20:30 PM.947
    stderr
    +++ realpath run.sh
    8:20:30 PM.947
    stderr
    ++ dirname /backends/rocm-qwen3-tts-cpp/run.sh
    8:20:30 PM.948
    stderr
  • CURDIR=/backends/rocm-qwen3-tts-cpp
    8:20:30 PM.948
    stderr
  • cd /
    8:20:30 PM.948
    stderr
  • echo 'CPU info:'
    8:20:30 PM.948
    stdout
    CPU info:
    8:20:30 PM.949
    stderr
    ++ uname
    8:20:30 PM.950
    stderr
  • '[' Linux '!=' Darwin ']'
    8:20:30 PM.950
    stderr
  • grep -e 'model\sname' /proc/cpuinfo
    8:20:30 PM.950
    stderr
  • head -1
    8:20:30 PM.951
    stdout
    model name : AMD Ryzen 9 5900X 12-Core Processor
    8:20:30 PM.951
    stderr
  • grep -e flags /proc/cpuinfo
    8:20:30 PM.951
    stderr
  • head -1
    8:20:30 PM.952
    stdout
    flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves user_shstk clzero xsaveerptr wbnoinvd arat npt lbrv nrip_save tsc_scale vmcb_clean flushbyasid pausefilter pfthreshold v_vmsave_vmload vgif umip pku ospke vaes vpclmulqdq rdpid overflow_recov succor fsrm
    8:20:30 PM.953
    stderr
    ++ uname
    8:20:30 PM.953
    stderr
  • '[' Linux = Darwin ']'
    8:20:30 PM.953
    stderr
  • LIBRARY=/backends/rocm-qwen3-tts-cpp/libgoqwen3ttscpp-fallback.so
    8:20:30 PM.953
    stderr
  • grep -q -e '\savx\s' /proc/cpuinfo
    8:20:30 PM.955
    stdout
    CPU: AVX found OK
    8:20:30 PM.955
    stderr
  • echo 'CPU: AVX found OK'
    8:20:30 PM.955
    stderr
  • '[' -e /backends/rocm-qwen3-tts-cpp/libgoqwen3ttscpp-avx.so ']'
    8:20:30 PM.955
    stderr
  • LIBRARY=/backends/rocm-qwen3-tts-cpp/libgoqwen3ttscpp-avx.so
    8:20:30 PM.955
    stderr
  • grep -q -e '\savx2\s' /proc/cpuinfo
    8:20:30 PM.956
    stdout
    CPU: AVX2 found OK
    8:20:30 PM.956
    stderr
  • echo 'CPU: AVX2 found OK'
    8:20:30 PM.956
    stderr
  • '[' -e /backends/rocm-qwen3-tts-cpp/libgoqwen3ttscpp-avx2.so ']'
    8:20:30 PM.956
    stderr
  • LIBRARY=/backends/rocm-qwen3-tts-cpp/libgoqwen3ttscpp-avx2.so
    8:20:30 PM.956
    stderr
  • grep -q -e '\savx512f\s' /proc/cpuinfo
    8:20:30 PM.957
    stderr
  • export LD_LIBRARY_PATH=/backends/rocm-qwen3-tts-cpp/lib:
    8:20:30 PM.957
    stderr
  • LD_LIBRARY_PATH=/backends/rocm-qwen3-tts-cpp/lib:
    8:20:30 PM.957
    stderr
  • export QWEN3TTS_LIBRARY=/backends/rocm-qwen3-tts-cpp/libgoqwen3ttscpp-avx2.so
    8:20:30 PM.957
    stderr
  • QWEN3TTS_LIBRARY=/backends/rocm-qwen3-tts-cpp/libgoqwen3ttscpp-avx2.so
    8:20:30 PM.957
    stderr
  • '[' -f /backends/rocm-qwen3-tts-cpp/lib/ld.so ']'
    8:20:30 PM.957
    stderr
  • echo 'Using lib/ld.so'
    8:20:30 PM.957
    stderr
  • echo 'Using library: /backends/rocm-qwen3-tts-cpp/libgoqwen3ttscpp-avx2.so'
    8:20:30 PM.957
    stderr
  • exec /backends/rocm-qwen3-tts-cpp/lib/ld.so /backends/rocm-qwen3-tts-cpp/qwen3-tts-cpp --addr 127.0.0.1:33023
    8:20:30 PM.957
    stdout
    Using lib/ld.so
    8:20:30 PM.957
    stdout
    Using library: /backends/rocm-qwen3-tts-cpp/libgoqwen3ttscpp-avx2.so
    8:20:30 PM.965
    stderr
    2026/07/03 18:20:30 gRPC Server listening at 127.0.0.1:33023
    8:20:32 PM.949
    stderr
    [qwen3-tts-cpp] Load talker=/models/qwen3-tts-cpp/qwen-talker-0.6b-base-Q8_0.gguf codec=/models/qwen3-tts-cpp/qwen-tokenizer-12hz-Q8_0.gguf use_fa=1 clamp_fp16=0
    8:20:32 PM.954
    stderr
    [qwen3-tts-cpp] Loading talker=/models/qwen3-tts-cpp/qwen-talker-0.6b-base-Q8_0.gguf codec=/models/qwen3-tts-cpp/qwen-tokenizer-12hz-Q8_0.gguf
    8:20:32 PM.954
    stderr
    [Qwen] qwentts.cpp 9dbe7ea (2026-06-24)
    8:20:32 PM.959
    stderr
    [Load] Talker backend: CPU (CPU threads: 4)
    8:20:32 PM.993
    stderr
    [GGUF] /models/qwen3-tts-cpp/qwen-talker-0.6b-base-Q8_0.gguf: 478 tensors, data at offset 5954368
    8:20:35 PM.371
    stderr
    [WeightCtx] Loaded 316 tensors, 774.6 MB into backend
    8:20:35 PM.371
    stderr
    [Talker] Loaded: 28 layers, hidden 1024, heads 16/8, head_dim 128, FFN 3072, RoPE theta 1000000, mrope sections [24,20,20] interleaved=0
    8:20:35 PM.832
    stderr
    [WeightCtx] Loaded 86 tensors, 143.5 MB into backend
    8:20:35 PM.832
    stderr
    [CodePredictor] Loaded: 5 layers, hidden 1024, heads 16/8, head_dim 128, FFN 3072, RoPE theta 1000000, 15 acoustic codebooks (vocab 2048 each), mtp_proj identity
    8:20:35 PM.909
    stderr
    [WeightCtx] Loaded 76 tensors, 22.9 MB into backend
    8:20:35 PM.909
    stderr
    [SpeakerEncoder] Loaded: enc_dim=1024 sr=24000 mel_dim=128 hidden=512 mfa=1536 asp_attn=128 se=128 scale=8
    8:20:35 PM.911
    stderr
    [GGUF] /models/qwen3-tts-cpp/qwen-tokenizer-12hz-Q8_0.gguf: 398 tensors, data at offset 29760
    8:20:36 PM.036
    stderr
    [WeightCtx] Loaded 18 tensors, 33.0 MB into backend
    8:20:36 PM.036
    stderr
    [Quantizer] Loaded: 16 codebooks (1 semantic + 15 acoustic), 2048 entries x 256 dim, hidden 512
    8:20:36 PM.158
    stderr
    [WeightCtx] Loaded 93 tensors, 30.9 MB into backend
    8:20:36 PM.158
    stderr
    [Transformer] Loaded: 8 layers, hidden 512, heads 16/16, head_dim 64, FFN 1024, RoPE theta 10000, sliding window 72
    8:20:36 PM.270
    stderr
    [WeightCtx] Loaded 22 tensors, 33.1 MB into backend
    8:20:36 PM.271
    stderr
    [Upsample] Loaded: 2 blocks (2x ratio per block), channels 1024, dwconv kernel 7
    8:20:36 PM.781
    stderr
    [WeightCtx] Loaded 118 tensors, 142.9 MB into backend
    8:20:36 PM.785
    stderr
    [DAC] Loaded: 4 blocks (strides 8/5/4/3), 24 kHz mono out, weights 142.9 MB
    8:20:36 PM.794
    stderr
    [WeightCtx] Loaded 2 tensors, 3.0 MB into backend
    8:20:36 PM.870
    stderr
    [WeightCtx] Loaded 28 tensors, 24.1 MB into backend
    8:20:36 PM.870
    stderr
    [SEANet] Loaded: 4 stages (ratios 4/5/6/8), 64 -> 512 channels, weights 24.1 MB
    8:20:36 PM.965
    stderr
    [WeightCtx] Loaded 96 tensors, 25.6 MB into backend
    8:20:36 PM.965
    stderr
    [EncTransformer] Loaded: 8 layers, hidden 512, heads 8/8, head_dim 64, FFN 2048, RoPE theta 10000
    8:20:36 PM.970
    stderr
    [WeightCtx] Loaded 1 tensors, 2.0 MB into backend
    8:20:36 PM.970
    stderr
    [EncDownsample] Loaded: k=4 stride=2, 512 -> 512 channels, weights 2.0 MB
    8:20:37 PM.057
    stderr
    [WeightCtx] Loaded 20 tensors, 34.0 MB into backend
    8:20:37 PM.057
    stderr
    [EncQuantizer] Loaded: 16 codebooks (1 semantic + 15 acoustic), 2048 entries x 256 dim, hidden 512, weights 34.0 MB
    8:20:37 PM.057
    stderr
    [Pipeline] Ready: hop 1920 samples @ 24000 Hz mono, 16 codebooks @ 12.5 Hz
    8:20:37 PM.776
    stderr
    [KVCache] Allocated: 28 layers, 8 KV heads, head_dim 128, max_seq_len 4096 -> 896 MB
    8:20:37 PM.776
    stderr
    [KVCache] Allocated: 5 layers, 8 KV heads, head_dim 128, max_seq_len 16 -> 0 MB
    8:20:37 PM.776
    stderr
    [Pipeline] Loaded: arch=0b6 variant=base tokenizer=qwen3_tts_tokenizer_12hz codebooks=16 speaker_encoder=loaded speakers=0 fa=off clamp_fp16=off
    8:20:37 PM.917
    stderr
    [BPE] Loaded from GGUF: 151676 vocab, 151291 merges, eos_id=151643
    8:20:37 PM.948
    stderr
    [BPE] Registered 5 arch special tokens (total specials=6)
    8:20:37 PM.948
    stderr
    [qwen3-tts-cpp] Model loaded (9dbe7ea (2026-06-24))
    8:20:37 PM.953
    stderr
    [Prompt] Built: 9 ids, N_text=1, N_instruct=0, T_ctx=10, hidden=1024, lang=auto (id=-1), speaker=none (id=-1) ref_spk_emb=no icl=no
    8:20:38 PM.025
    stderr
    [Sample] step=0 c0=1221 u=0.1990894824 subseq=0
    8:20:38 PM.033
    stderr
    [Sample-CP] g=0 c=990 u=0.3109774888 subseq=1
    8:20:38 PM.040
    stderr
    [Sample-CP] g=1 c=1176 u=0.6611105204 subseq=2
    8:20:38 PM.046
    stderr
    [Sample-CP] g=2 c=125 u=0.0324732251 subseq=3
    8:20:38 PM.052
    stderr
    [Sample-CP] g=3 c=1490 u=0.6024870872 subseq=4
    8:20:38 PM.057
    stderr
    [Sample-CP] g=4 c=1863 u=0.9201316237 subseq=5
    8:20:38 PM.062
    stderr
    [Sample-CP] g=5 c=661 u=0.2027195245 subseq=6
    8:20:38 PM.068
    stderr
    [Sample-CP] g=6 c=90 u=0.1273398697 subseq=7
    8:20:38 PM.075
    stderr
    [Sample-CP] g=7 c=151 u=0.0900337100 subseq=8
    8:20:38 PM.082
    stderr
    [Sample-CP] g=8 c=2046 u=0.9974240065 subseq=9
    8:20:38 PM.089
    stderr
    [Sample-CP] g=9 c=1919 u=0.9346252680 subseq=10
    8:20:38 PM.095
    stderr
    [Sample-CP] g=10 c=1269 u=0.5214393735 subseq=11
    8:20:38 PM.102
    stderr
    [Sample-CP] g=11 c=1701 u=0.7901484966 subseq=12
    8:20:38 PM.109
    stderr
    [Sample-CP] g=12 c=1230 u=0.4048664570 subseq=13
    8:20:38 PM.115
    stderr
    [Sample-CP] g=13 c=914 u=0.5316267014 subseq=14
    8:20:38 PM.122
    stderr
    [Sample-CP] g=14 c=1197 u=0.6179689169 subseq=15
    8:20:38 PM.162
    stderr
    [Sample] step=1 c0=765 u=0.1550706774 subseq=16
    8:20:38 PM.170
    stderr
    [Sample-CP] g=0 c=1999 u=0.9800673723 subseq=17
    8:20:38 PM.176
    stderr
    [Sample-CP] g=1 c=1842 u=0.7948336601 subseq=18
    8:20:38 PM.182
    stderr
    [Sample-CP] g=2 c=1626 u=0.8524477482 subseq=19
    8:20:38 PM.188
    stderr
    [Sample-CP] g=3 c=305 u=0.1475155205 subseq=20
    8:20:38 PM.195
    stderr
    [Sample-CP] g=4 c=11 u=0.0132146822 subseq=21
    8:20:38 PM.202
    stderr
    [Sample-CP] g=5 c=1690 u=0.9015900493 subseq=22
    8:20:38 PM.209
    stderr
    [Sample-CP] g=6 c=1801 u=0.9599460959 subseq=23
    8:20:38 PM.215
    stderr
    [Sample-CP] g=7 c=668 u=0.5660899878 subseq=24
    8:20:38 PM.223
    stderr
    [Sample-CP] g=8 c=1695 u=0.7820879817 subseq=25
    8:20:38 PM.230
    stderr
    [Sample-CP] g=9 c=1077 u=0.5643745065 subseq=26
    8:20:38 PM.237
    stderr
    [Sample-CP] g=10 c=273 u=0.0492468700 subseq=27
    8:20:38 PM.242
    stderr
    [Sample-CP] g=11 c=845 u=0.6441421509 subseq=28
    8:20:38 PM.246
    stderr
    [Sample-CP] g=12 c=320 u=0.5415108800 subseq=29
    8:20:38 PM.253
    stderr
    [Sample-CP] g=13 c=58 u=0.0344317332 subseq=30
    8:20:38 PM.260
    stderr
    [Sample-CP] g=14 c=949 u=0.7003046870 subseq=31
    8:20:38 PM.985
    stderr
    [Pipeline] EOS at step 7, stopping
    8:20:38 PM.985
    stderr
    [Pipeline] Generation done : 7 frames
    8:20:39 PM.379
    stderr
    [Perf] PromptBuild 1.6 ms
    8:20:39 PM.379
    stderr
    [Perf] Prefill 72.3 ms (T_ctx prefill)
    8:20:39 PM.379
    stderr
    [Perf] TTFA 171.1 ms (first frame codes)
    8:20:39 PM.379
    stderr
    [Perf] TalkerDecode 271.4 ms (7 frames, 38.77 ms/frame)
    8:20:39 PM.379
    stderr
    [Perf] CodePredictor 687.4 ms (98.20 ms/frame)
    8:20:39 PM.379
    stderr
    [Perf] HostCompose 0.7 ms (c0 sample + next emb)
    8:20:39 PM.379
    stderr
    [Perf] CodecDecode 394.3 ms
    8:20:39 PM.379
    stderr
    [Perf] Total 1427.9 ms (7 frames, 137.07 ms/frame AR, audio 0.56 s, RTF 2.550)

Additional context
qwen3-tts-cpp installation logs:
Jul 03 18:17:57 INFO HTTP request method="POST" path="/api/backends/install/qwen3-tts-cpp" status=200
Jul 03 18:17:57 WARN installing OCI backend without signature verification backend="rocm-qwen3-tts-cpp" gallery="localai" uri="quay.io/go-skynet/local-ai-backends:latest-gpu-rocm-hipblas-qwen3-tts-cpp"
Jul 03 18:18:02 INFO Downloading fileName="Downloading 1/1 quay.io/go-skynet/local-ai-backends:latest-gpu-rocm-hipblas-qwen3-tts-cpp" current="514.9 MiB" total="1.4 GiB" percentage=37.1556048400352 eta=8.459097829s
Jul 03 18:18:07 INFO Downloading fileName="Downloading 1/1 quay.io/go-skynet/local-ai-backends:latest-gpu-rocm-hipblas-qwen3-tts-cpp" current="1.3 GiB" total="1.4 GiB" percentage=95.95750090253145 eta=421.334447ms
Jul 03 18:19:01 INFO Downloading fileName="Extracting quay.io/go-skynet/local-ai-backends:latest-gpu-rocm-hipblas-qwen3-tts-cpp" current="512 B" total="4.7 GiB" percentage=1.0136580282729497e-05 eta=174032h30m10.522014208s
Jul 03 18:19:06 INFO Downloading fileName="Extracting quay.io/go-skynet/local-ai-backends:latest-gpu-rocm-hipblas-qwen3-tts-cpp" current="4.2 GiB" total="4.7 GiB" percentage=89.33489842132899 eta=8.178650008s
Jul 03 18:19:08 INFO HTTP request method="GET" path="/api/backends/upgrades" status=200
Jul 03 18:19:08 INFO HTTP request method="GET" path="/api/backends" status=200

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingunconfirmed

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions