Commit 6a0200c
Cap transformers <5.13 to keep mlx-lm importable (macOS), bump to 0.3.19 (#316)
transformers 5.13.0 tightened AutoModel/AutoTokenizer.register() to require a
class (it does key.__module__). mlx-lm 0.31.3 registers a custom tokenizer by
string name (tokenizer_utils.py: AutoTokenizer.register("NewlineTokenizer", ...)),
so `import mlx_lm` — and therefore `import optillm` — crashes with
"AttributeError: 'str' object has no attribute '__module__'".
This only affects Apple silicon, where optillm installs mlx-lm
(platform_machine=="arm64" and sys_platform=="darwin"); Linux CI never installs
mlx-lm so it was unaffected. The transformers 5.13.0 HF path itself is fine
(dhara loads and generates); the incompatibility is purely mlx-lm's.
Pin transformers>=5.0.0,<5.13.0 (resolves to 5.12.1) until mlx-lm ships a
transformers-5.13-compatible release, then lift the cap.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>1 parent beca974 commit 6a0200c
3 files changed
Lines changed: 7 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | | - | |
| 2 | + | |
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
7 | | - | |
| 7 | + | |
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
8 | | - | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
9 | 13 | | |
10 | 14 | | |
11 | 15 | | |
| |||
0 commit comments