vtm randomly segfaults on manycore system for no apparent reason

[itskenny0]

Hiya!

I use vtm a lot, but recently I had a lot of crashing issues which made me run back to tmux. Sessions will stay alive for days at a time, no problem at all, but inevitably, one day I come back to this screen:

root@vtm:~# vtm
  os: Running with elevated privileges
 tty: Pseudoterminal /dev/pts/4
  os: Terminal type: xterm-256color
  os: Color mode: xterm truecolor
  os: Mouse mode: VT-style
 vtm: v2026.06.29
args: vtm
apps: Update settings source files from
      /etc/vtm/settings.xml
      ~/.config/vtm/settings.xml
apps: Loading settings from "/etc/vtm/settings.xml"...
      Not found
apps: Loading settings from "/root/.config/vtm/settings.xml"...
      Not found
root@vtm:~# 

When this occurs, vtm has been completely torn down and so have the terminal sessions that were running inside - very frustrating. Looking around, the only trace I could find of this is this dmesg entry:

[13126285.564934] vtm[342364]: segfault at 75f7bf0e520b ip 00000000007369b8 sp 000075f10b3fed90 error 4 in vtm[401000+82f000] likely on CPU 32 (core 18, socket 1)
[13126285.570477] Code: 0f 29 a4 24 a0 00 00 00 66 48 0f 6e e0 48 8d 84 24 b0 01 00 00 48 89 44 24 40 66 0f 6c ec 0f 29 ac 24 90 00 00 00 0f 1f 40 00 <4d> 8b 67 18 4d 8b 77 10 4d 85 e4 74 08 4c 89 e7 e8 b3 17 e6 ff 66

The aliveness of the terminals are quite important to me, so I can't continue using vtm for the moment. Would you be so kind whether you can find out why this is?

A bit more context to my setup:
vtm is running on Ubuntu 24.04, inside LXC on Proxmox
Kernel: Linux srv3 6.8.4-2-pve #1 SMP PREEMPT_DYNAMIC PMX 6.8.4-2 (2024-04-10T17:36Z) x86_64 GNU/Linux

Thanks lots!
kenny

[itskenny0]

Some more context on my system:

root@srv3:~# lscpu
Architecture:             x86_64
  CPU op-mode(s):         32-bit, 64-bit
  Address sizes:          46 bits physical, 48 bits virtual
  Byte Order:             Little Endian
CPU(s):                   80
  On-line CPU(s) list:    0-79
Vendor ID:                GenuineIntel
  Model name:             Intel(R) Xeon(R) Gold 6138 CPU @ 2.00GHz
    BIOS Model name:      Intel(R) Xeon(R) Gold 6138 CPU @ 2.00GHz  CPU @ 2.0GHz
[... truncated ...]
root@srv3:~# free -h
               total        used        free      shared  buff/cache   available
Mem:           503Gi       373Gi       129Gi       190Mi       4.0Gi       129Gi
Swap:          8.0Gi       8.0Gi       1.7Mi

[o-sdn-o]

Hi! Thank you for your feedback!

I'll try to reproduce this bug on my system. Do you have any additional information about a previous stable release or a crash dump? That would help me localize this bug faster.

[itskenny0]

Unfortunately I've been having this for a while - I only now discovered the segfault - but I know that at minimum, I remember this happening from the May releases onward. However, I don't know when exactly it might've been introduced - sorry :(

[o-sdn-o]

Okay! I'll find this bug either way!

[o-sdn-o]

@itskenny0 I fixed some bugs in the next releases. One of them might have been the one that was causing the crash on your system. If you have time, you might want to check it out.

If you decide to test, you can run the following before starting vtm:

ulimit -c unlimited

or temporarily install systemd-coredump:

sudo apt install systemd-coredump

to obtain a crash dump to identify the crash location.