lsp_format_on_save deleting entire file contents

[staticf0x]

When hitting ctrl+s to save a file, the LSP action to format a file is invoked. I found the ruff plugin to be the cause, here's the log file from python-lsp-server:

2023-10-18 08:41:40,113 CEST - ERROR - pylsp_ruff.plugin - Error running ruff: error: unexpected argument '--format' found

  tip: to pass '--format' as a value, use '-- --format'

Usage: ruff check <--verbose|--quiet|--silent> [FILES]...

For more information, try '--help'.

Seems like this option is deprecated in 0.1.0: astral-sh/ruff#7984

[staticf0x]

My sublime text was using ruff from .cache/sublime-text/Package Storage/LSP-pylsp/bin/ which is already on 0.1.0 thanks to sublime automatically updating packages. I had to manually set pylsp.plugins.ruff.executable to ~/.local/bin/ruff which is on 0.0.292.

[jhossbach]

Yes, this is an issue with the latest ruff version, I will release a new version later today (see #48)

[staticf0x]

Great, thanks!

[staticf0x]

Although one thing I don't know if the PR fixes is: why does it attempt to format the file even when pylsp.plugins.ruff.format is []? And why does it delete the file contents when the format fails?

[jhossbach]

The LSP receives a signal asking for formatting on the current file, but because of this error ruff will return this empty array, which the editor interprets as an empty file (as it should)
The return value usually contains the complete contents of the file, this is specified in the LSP specs as far as I know

[znd4]

@jhossbach , assuming that this plugin is currently using the first ruff on PATH, would it be possible to switch to [sys.executable, "-m", "ruff", ...]?

restatement: is the ruff version pin in pyproject.toml doing its job?

[jhossbach]

I need to check if sys.executable respects being called in an environment, but we can certainly do this. I don't see a problem though with the versioning, can you give an explicit example of it not working as intended? Make sure that the client calls the version pylsp from the correct virtual environment and that ruff with the correct version exists in that environment, otherwise there will be issues.

[znd4]

edit: this is enough that I'll probably create a separate issue if I remember

status quo

I'm not sure how reproducible this will be on other environments (although it might not be too hard to replicate with a Dockerfile, especially if there's a pipx - ready image), but currently I'm running into this (on MacOS, fwiw), after

pipx install --force "ruff>=0.1"

And it goes away after

pipx install --force "ruff<0.1"

Some info about versions:

$ pipx list | rg 'ruff|lsp'
   package python-lsp-server 1.8.2, installed using Python 3.11.6
    - pylsp
   package ruff 0.0.292, installed using Python 3.11.6
    - ruff
$ pipx runpip python-lsp-server freeze | rg ruff
python-lsp-ruff==1.5.2
ruff==0.1.0

Based on the above, even though I've got the incompatible ruff version in my python-lsp-server environment, I can avoid this issue if ruff --version is compatible. Conversely (?), if I had a good version range in the python-lsp-server venv, it wouldn't matter if my globally-installed ruff had breaking changes.

change to sys.executable

Limited testing with my pipx environments:

at 10:12:30 🍎 ❯ ~/.local/pipx/venvs/python-lsp-server/bin/python -c 'import sys; import os; os.system(sys.executable + " -m ruff --version")'
ruff 0.1.0

at 10:12:54 🍎 ❯ ~/.local/pipx/venvs/ruff/bin/python -c 'import sys; import os; os.system(sys.executable + " -m ruff --version")'
ruff 0.0.292

I'd hope that testing in a CPython python -m venv environment would be sufficient, but it's possible that different python distributions (e.g. circuitpython) or project tools (pdm, poetry, etc.) might behave differently.

Based on personal anecdotal evidence, I haven't run into issues with using sys.executable like this in the past, but there are a lot of environments I haven't worked in yet (embedded, WASM, etc.)

[jhossbach]

Hmm I see, I think this is related to pipx exposing the binaries to ~/.local/bin.
I will create a new issue.

[staticf0x]

By the way, Sublime Text plugin does something similar, it prepends PATH with .cache/sublime-text/Package Storage/LSP-pylsp/lib/python3.11 (or something along the way) so it avoids system paths or in my case ~/.local/lib/.... The only way to use ruff from my ~/.local/bin is to set pylsp.plugins.ruff.executable to a absolute path to the correct one.