Ai

LlamaFactory v0.9.4 Brings OFT, Megatron-LM Training, FP8 and Transformers v5 Support

LlamaFactory v0.9.4 Brings OFT, Megatron-LM Training, FP8 and Transformers v5 Support

LlamaFactory v0.9.4 is a significant release for teams building and fine-tuning large language models, combining important platform modernization with a broad set of new training and inference capabilities. This update renames the project from LLaMA-Factory to LlamaFactory, raises the Python requirement to 3.11 through 3.13, switches package installation to uv, and adds new support for Orthogonal Fine-Tuning, Megatron-LM training, FP8, KTransformers, and Transformers v5.

What Changed

The biggest structural changes in v0.9.4 are the repository rebrand and the updated runtime requirements. The project is now officially named LlamaFactory, replacing the previous LLaMA-Factory branding. At the same time, Python 3.9 and 3.10 have been deprecated, with the framework now targeting Python 3.11 to 3.13. Installation guidance has also shifted from pip-first workflows to uv, reflecting a push toward faster and more modern Python dependency management.

On the feature side, the release adds support for Orthogonal Fine-Tuning, a new method aimed at improving adaptation efficiency, along with semantic initialization for newly added tokens. That gives practitioners more flexibility when extending vocabularies or experimenting with model adaptation strategies.

LlamaFactory v0.9.4 also broadens infrastructure and training compatibility. New support includes Megatron-LM training through MCoreAdapter, the KTransformers backend, MPO, FP8 training, DeepSpeed AutoTP, efficient NPU fused kernels, TRL 0.24, and compatibility with Transformers v5. The release also adds support for reasoning and plaintext handling in function call messages, which may help teams working on agentic and tool-calling workflows.

In addition, the project launched its official blog, giving the community a new channel for release updates, guidance, and ecosystem communication heading into 2026.

Why It Matters

This release matters because it makes LlamaFactory more production-aligned for current AI engineering stacks. Requiring newer Python versions and adopting uv signals a focus on maintainability and modern developer tooling, even though it may force some teams to update existing environments before upgrading.

The added support for OFT, FP8, Megatron-LM, and DeepSpeed AutoTP is especially notable for enterprises and research teams optimizing training efficiency, scaling workloads, or targeting new hardware backends. Compatibility with Transformers v5 and TRL 0.24 also helps reduce integration friction for teams already building on the Hugging Face ecosystem.

Overall, LlamaFactory v0.9.4 is not just a maintenance release. It is a meaningful platform update that expands fine-tuning options, improves ecosystem compatibility, and sets a clearer technical direction for the framework in 2026.

Official Source: https://github.com/hiyouga/LlamaFactory/releases/tag/v0.9.4