Skip to content

Add logprobs support to chat completions #44

@raullenchai

Description

@raullenchai

The OpenAI API spec supports logprobs: true in chat completion requests, returning token log-probabilities in the response.

Currently Rapid-MLX accepts the parameter silently but doesn't return logprobs data. The response choices[].logprobs field is absent.

What needs to happen:

  • When logprobs: true, capture per-token log-probabilities from mlx-lm's generate_step()
  • Return them in the choices[].logprobs field per the OpenAI spec
  • Add a test case

Relevant files: vllm_mlx/server.py (chat completion endpoint)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions