Ai

LocalAI v4.3.5 Patches Critical Tool-Call Bugs in Streaming and Tokenizer-Template Models

LocalAI v4.3.5 Patches Critical Tool-Call Bugs in Streaming and Tokenizer-Template Models

LocalAI's incremental update v4.3.5 addresses two critical bugs affecting tool-call functionality. The first fix prevents tool-call JSON from leaking into content when streaming is combined with tools on tokenizer-template models. The second stops double-emission of tool-calls during streaming when the autoparser is active. These patches improve stability for developers building AI agents that rely on real-time streaming and structured tool outputs.

What Changed

Bug fix #10057 resolves a leak where the JSON representation of a tool call would bleed into the regular content stream. This happened only on tokenizer-template models when both streaming and tools were enabled. The result was garbled output with raw JSON mixed in—a real headache for anyone parsing responses.

Bug fix #10055 tackles a double-emission issue in the OpenAI-compatible streaming endpoint. When the autoparser was active, the server would send the tool-call event twice. That could cause client-side logic to fire actions repeatedly. Contributor bozhouDev identified the source and patched the streaming logic to emit only once.

Why It Matters

Tool-calling is the backbone of agentic AI workflows. If the JSON leaks or duplicates, downstream applications break. This is especially painful in enterprise setups where reliability is non-negotiable. The fixes are small but impactful—they remove two landmines for anyone building on LocalAI. Also, the fact that the community caught and fixed these quickly shows the project's maturity. It's not just about speed; it's about getting the details right.

Official Source: https://github.com/mudler/LocalAI/releases/tag/v4.3.5