Skip to content

Eval bug: Qwen3.6 27B segfaults when using -sm tensor and --kv-tail-tokens #107

Description

@dinerburger

Name and Version

version: 10855 (176c1a1)
built with GNU 14.2.0 for Linux x86_64

Operating systems

Linux

GGML backends

CPU, CUDA

Hardware

Intel(R) Xeon(R) w5-3435X + 3090 Ti + A4000

Models

Qwen3.6 27B

Problem description & steps to reproduce

llama-server can be invoked with -sm tensor, but adding --kv-tail-tokens causes a segfault. Looks like, unsurprisingly, related to cache context creation.

First Bad Commit

No response

Relevant log output

Logs
0.03.976.118 I llama_context: constructing llama_context
0.03.976.124 I llama_context: n_seq_max     = 4
0.03.976.124 I llama_context: n_ctx         = 65536
0.03.976.125 I llama_context: n_ctx_seq     = 65536
0.03.976.125 I llama_context: n_batch       = 2048
0.03.976.125 I llama_context: n_ubatch      = 512
0.03.976.126 I llama_context: causal_attn   = 1
0.03.976.127 I llama_context: flash_attn    = enabled
0.03.976.127 I llama_context: kv_unified    = true
0.03.976.131 I llama_context: freq_base     = 10000000.0
0.03.976.131 I llama_context: freq_scale    = 1
0.03.976.132 I llama_context: n_rs_seq      = 0
0.03.976.132 I llama_context: n_outputs_max = 4
0.03.976.132 I llama_context: n_ctx_seq (65536) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
0.04.239.980 I llama_context:  CUDA_Host  output buffer size =     3.79 MiB
0.04.241.298 I llama_kv_cache:     Meta() KV buffer size =  1040.00 MiB
[1]    53361 segmentation fault (core dumped)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions