feat: add Tencent Hy-MT2 series support#5029
Conversation
![gemini-code-assist[bot]](https://avatars.github.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds support for Tencent Hunyuan Hy-MT2 multilingual translation models (HunYuanDenseV1ForCausalLM and HYV3ForCausalLM) across SGLang, vLLM, and Transformers backends. Feedback includes adding proper type annotations and default values to pytorch_model_config in the new model's initializer, and returning a descriptive error message instead of a plain boolean when architecture matching fails in match_json.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
|
Addressed Gemini Code Assist's review comments in
|
Register Tencent Hunyuan Hy-MT2 multilingual translation models for deployment via Transformers, vLLM, SGLang, and llama.cpp engines. Models registered (Hy-MT2-1.8B / 7B / 30B-A3B): - pytorch (none, fp8) and ggufv2 specs with HF + ModelScope sources - 33 supported languages, 262144 context length - Embedded the official chat_template.jinja into llm_family.json - Correct stop_token_ids per model: 1.8B/7B [120020], 30B [120025,120026] Engine adapters: - New transformers/hy_mt2.py adapter forces torch_dtype=bfloat16 to avoid the MPSFloatType embedding error on Apple Silicon and enables trust_remote_code for the custom HunYuanDenseV1/HYV3 modeling code. - vllm/core.py: add HunYuanDenseV1ForCausalLM and HYV3ForCausalLM to the vLLM (>=0.22.0) supported chat models list. - sglang/core.py: add the same architectures to SGLANG_SUPPORTED_CHAT_MODELS. Verified locally on macOS via the llama.cpp engine (Q4_K_M GGUF) end-to-end translation; Transformers engine loads successfully on MPS (bfloat16 fix confirmed) but full inference is constrained by host RAM. Review fixups: - Type-annotate pytorch_model_config as Optional[PytorchModelConfig] in HyMT2PytorchModel.__init__ to align with PytorchChatModel. - Return a descriptive tuple from match_json when the architecture does not match, matching other adapters' behavior. - Add the #sglang_dependencies# ; #engine# == "sglang" marker to the virtualenv packages of all three Hy-MT2 families so the SGLang engine passes check_engine_by_spec_parameters_with_virtual_env when virtualenv filtering is enabled (matches the SGLang registration in sglang/core.py).
3 participants
Register Tencent Hunyuan Hy-MT2 multilingual translation models for deployment via Transformers, vLLM, SGLang, and llama.cpp engines.
Models registered (Hy-MT2-1.8B / 7B / 30B-A3B):
Engine adapters: