v4.3.6

What's Changed

Other Changes

• chore: ⬆️ Update ggml-org/llama.cpp to 22d66b567eef11cf2e9832f04db64ee0323a0fd0 by @localai-bot in #10080
• security(http): refuse redirects on outbound clients via hardened pkg/httpclient by @richiejp in #10087
• feat(parakeet-cpp): add NVIDIA NeMo Parakeet ASR backend (parakeet.cpp) by @localai-bot in #10084
• chore: ⬆️ Update antirez/ds4 to e16ead1e29c81a67bbb64e5b001117679cf9ce6e by @localai-bot in #10076
• chore: ⬆️ Update mudler/parakeet.cpp to 30a307553f1965ceb38a1a922069a71e7dd67bf3 by @localai-bot in #10092

Full Changelog: v4.3.5...v4.3.6

v4.3.5

What's Changed

Bug fixes 🐛

• fix: tool-call JSON leaks into content with stream+tools on tokenizer-template models (#10052) by @localai-bot in #10057
• fix(openai): stop streaming tool-call double-emission when autoparser is active by @bozhouDev in #10055
• fix(application): stop backend processes synchronously on shutdown by @richiejp in #10058
• fix(functions): validate auto-detected XML tool-call names — robust glm-4.5/Hermes guard (#9722, supersedes #9940) by @localai-bot in #10059
• fix(model): track intentional stops, stop misreading clean shutdowns as crashes by @richiejp in #10060

Exciting New Features 🎉

• feat(reasoning): honor per-request reasoning_effort on chat completions by @localai-bot in #10082

Other Changes

• chore: ⬆️ Update mudler/rf-detr.cpp to ecf64d7f7f20d73ebd906a983f398ed287256320 by @localai-bot in #10035
• docs: ⬆️ update docs version mudler/LocalAI by @localai-bot in #10046
• chore: ⬆️ Update antirez/ds4 to 22393e770ea8eb7501d8718d6f66c6374004e03f by @localai-bot in #10047
• chore: ⬆️ Update leejet/stable-diffusion.cpp to 0e4ee04488159b81d95a9ffcd983a077fd5dcb77 by @localai-bot in #10048
• chore: ⬆️ Update ggml-org/llama.cpp to 751ebd17a58a8a513994509214373bb9e6a3d66c by @localai-bot in #10049
• chore: ⬆️ Update ikawrakow/ik_llama.cpp to 6eff055a0cc0e427a6849cfcb5de531b4b82d667 by @localai-bot in #10050
• chore: ⬆️ Update ggml-org/whisper.cpp to c932729a304f7d9eb5354afa38624cfa86a780cf by @localai-bot in #10051
• test(react-ui): cover models gallery empty-state reset flow by @Oceankj in #10019
• test(utils): cover path verification, sanitization, and unique naming by @TLoE419 in #9978
• chore(model gallery): 🤖 add 1 new models via gallery agent by @localai-bot in #10061
• chore: ⬆️ Update ikawrakow/ik_llama.cpp to 8960c5ba5ee9db30ba838304373aa4dbec9f7cbd by @localai-bot in #10077
• chore: ⬆️ Update vllm-project/vllm cu130 wheel to 0.22.0 by @localai-bot in #10079
• chore(model-gallery): ⬆️ update checksum by @localai-bot in #10081
• docs: ⬆️ update docs version mudler/LocalAI by @localai-bot in #10074
• chore: ⬆️ Update mudler/rf-detr.cpp to 65c0ffcc9a9bc9dae38252f63d0417c9845a6cf7 by @localai-bot in #10075
• chore: ⬆️ Update ggml-org/whisper.cpp to f24588a272ae8e23280d9c220536437164e6ed28 by @localai-bot in #10078

New Contributors

• @bozhouDev made their first contribution in #10055
• @Oceankj made their first contribution in #10019
• @TLoE419 made their first contribution in #9978

Full Changelog: v4.3.4...v4.3.5

v4.3.4

What's Changed

Other Changes

• fix(turboquant): guard upstream-only grpc-server fields for fork by @localai-bot in #10043

Full Changelog: v4.3.3...v4.3.4

v4.3.3

What's Changed

Other Changes

• chore: ⬆️ Update ikawrakow/ik_llama.cpp to 3bf7e836c2c5a895e8d12d3eb7e398ae7ab2f9ce by @localai-bot in #10037
• chore(model-gallery): ⬆️ update checksum by @localai-bot in #10038
• chore: ⬆️ Update ggml-org/llama.cpp to aa50b2c2ae91326d5aad956ceeb015d1d48e626b by @localai-bot in #10034
• chore: ⬆️ Update leejet/stable-diffusion.cpp to 29ab511fc75f89fbab148665eab1a8e10a139a72 by @localai-bot in #10033
• chore: ⬆️ Update ggml-org/whisper.cpp to 6dcdd6536456158667747f724d6bd3a2ceaa8d88 by @localai-bot in #10032
• chore: ⬆️ Update antirez/ds4 to 072bc0feb187be5f374c08b16d0045e1ad7bc9bc by @localai-bot in #10036
• fix(openresponses): populate Content and accept bare {role,content} items (#10039) by @Anai-Guo in #10040
• perf(react-ui): code-split bundle, speed up coverage suite by @richiejp in #10042

Full Changelog: v4.3.2...v4.3.3

v4.3.2

What's Changed

👒 Dependencies

• chore(deps): bump github.com/nats-io/nats.go from 1.50.0 to 1.52.0 by @dependabot[bot] in #10003
• chore(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.19.15 to 1.19.17 by @dependabot[bot] in #10008
• chore(deps): bump actions/stale from 10.2.0 to 10.3.0 by @dependabot[bot] in #10002
• chore(deps): bump sentence-transformers from 5.5.0 to 5.5.1 in /backend/python/transformers by @dependabot[bot] in #10007
• chore(deps): update transformers requirement from >=5.8.1 to >=5.9.0 in /backend/python/transformers by @dependabot[bot] in #10005
• chore(deps): bump protobuf from 6.33.5 to 7.35.0 in /backend/python/transformers by @dependabot[bot] in #10004

Other Changes

• feat(middleware): Model routing, PII filtering, Cloud model proxies by @richiejp in #9802
• fix(intel): VRAM detection by @richiejp in #9944
• feat(distributed): gated X-LocalAI-Node response header (middleware + wrapper) by @localai-bot in #9976
• fix(distributed): persist per-model load info so reconciler survives frontend restart by @localai-bot in #9981
• feat(stablediffusion-ggml): LTX-2 support + LTX-2.3 GGUF gallery entries by @localai-bot in #9980
• fix(distributed): sync gallery OpCache + caches across frontend replicas by @localai-bot in #9983
• fix(gallery/ltx-2.3): add diffusion_model flag to all variants by @mudler in #9986
• fix(gallery/ltx-2.3): add vae_decode_only:false for i2v / flf2v by @mudler in #9987
• fix(reasoning): stop leaking into content when autoparser is in pure-content mode by @localai-bot in #9991
• fix(stablediffusion-ggml): mux LTX-2 audio into output MP4 by @localai-bot in #9990
• feat(swagger): update swagger by @localai-bot in #9992
• docs: ⬆️ update docs version mudler/LocalAI by @localai-bot in #9993
• fix(streaming/tools): stop healing-marker stubs from gating off content by @localai-bot in #9999
• chore: ⬆️ Update antirez/ds4 to ad0209f6a4b067574d2b4afe896c08c177156b31 by @localai-bot in #9996
• chore: ⬆️ Update ikawrakow/ik_llama.cpp to b4e1d916c5ec7e75ea3c124dd090425a99fc613f by @localai-bot in #9995
• chore: ⬆️ Update leejet/stable-diffusion.cpp to 1ceb5bd9df7784bcdf67dd9ed8bf0198b542ebc9 by @localai-bot in #9994
• chore: ⬆️ Update ggml-org/whisper.cpp to e0fd1f6787a5bd4a4957dd97c5b64df882ee7b0c by @localai-bot in #9997
• fix(streaming/tools): don't leak prefill-misclassified content as trailing reasoning chunk by @localai-bot in #10000
• chore: ⬆️ Update ggml-org/llama.cpp to 35c9b1f39ebe5a7bb83986d64415a079218be78d by @localai-bot in #9998
• chore(model gallery): 🤖 add 1 new models via gallery agent by @localai-bot in #10011
• fix(dockerignore): exclude local-only artifacts from build context by @richiejp in #10015
• chore(model gallery): 🤖 add 1 new models via gallery agent by @localai-bot in #10016
• test: add Go + React UI coverage gates and fill test gaps by @richiejp in #9989
• fix(qwen-asr): enable timestamp output when forced_aligner is configured by @fqscfqj in #10013
• fix(nemo): extract Hypothesis.text for TDT/RNNT ASR models by @fqscfqj in #10012
• chore: ⬆️ Update ikawrakow/ik_llama.cpp to d2da6da05c73aeb658a3d1751f386c24e6963856 by @localai-bot in #10020
• chore: ⬆️ Update ggml-org/whisper.cpp to 27101c01dcac1676e2b6422256233cd0f1f9ae28 by @localai-bot in #10021
• chore: ⬆️ Update ggml-org/llama.cpp to 0d18aaa9d1a8af3df9abccd828e22eeaac7f840b by @localai-bot in #10022
• chore: ⬆️ Update leejet/stable-diffusion.cpp to 92dc7268fc4ffb0c0cc0bd52dfcefea91326e797 by @localai-bot in #10023
• chore: ⬆️ Update antirez/ds4 to e8e8779b261c10f36ad6270ba732c8f0be5b62e3 by @localai-bot in #10024
• UI: add 'Fits in my GPU' filter on Install Models by @siddimore in #10017
• fix(react-ui): share single /api/operations poller across consumers by @localai-bot in #10029
• feat(backend): rfdetr-cpp native object detection + segmentation backend by @localai-bot in #10028
• fix(react-ui): polish 'Fits in my GPU' filter to use design-system Toggle by @localai-bot in #10030
• fix(react-ui): force .check() on hidden Toggle input in fits-filter e2e by @localai-bot in #10031

New Contributors

• @fqscfqj made their first contribution in #10013

Full Changelog: v4.3.1...v4.3.2

v4.3.1

What's Changed

Other Changes

• Fix kokoros backend build break from Backend trait drift by @Copilot in #9972
• chore: ⬆️ Update antirez/ds4 to f91c12b50a1448527c435c028bfc70d1b00f6c33 by @localai-bot in #9975
• chore: ⬆️ Update ikawrakow/ik_llama.cpp to 9f7ba245ab41e118f03aa8dd5134d18a81159d02 by @localai-bot in #9973
• chore: ⬆️ Update ggml-org/llama.cpp to 549b9d84330c327e6791fa812a7d60c0cf63572e by @localai-bot in #9974

Full Changelog: v4.3.0...v4.3.1

v4.3.0

🎉 LocalAI 4.3.0 Release! 🚀

LocalAI 4.3.0 is out!

This release hardens the trust boundary and improves defaults for speed. Backend OCI images now ship with keyless cosign signatures and a per-gallery verification: policy, with an opt-in strict mode that fails closed.
The llama-cpp server-side prompt cache works by default: repeated system prompts (agents, OpenAI/Anthropic-compatible CLIs, coding assistants) collapse from minutes to seconds without touching YAML. Distributed mode gets rounds of optimizations. Usage tracking grows a per-API-key + per-user Sources view so admins can finally answer "who is burning the GPU?". And, for everyone on a Jetson/DGX box, the L4T13 (cu130/aarch64) backends are back.

---

📌 TL;DR

Feature	Summary
🔐 Signed Backends	Keyless cosign + sigstore-go verification for backend OCI images, OCI 1.1 referrers, not_before revocation, opt-in strict mode.
⚡ Prompt Cache by Default	llama-cpp server-side prompt cache works out of the box. Repeated system prompts go from 5-8 min to seconds.
📊 Usage per API Key	New Sources tab attributes traffic to keys and users. Revoked keys stay readable in history.
🛰️ Distributed v3	Per-request replica routing, cached probeHealth, async per-node installs with streaming progress, unified backend-logs entry point.
🩺 Traces UI Stays Snappy	LOCALAI_TRACING_MAX_BODY_BYTES caps API + backend trace payloads. Admin Traces page stops drowning in 40 MB embeddings.
🧊 Nix Flake	Dockerless setup for NixOS users via flake.nix + dev shell.
🦾 Jetson Thor Restored	vllm / sglang / vllm-omni L4T13 backends switched to PyPI aarch64+cu130 wheels (torch 2.10 ABI fix).

---

🚀 New Features & Major Enhancements

🔐 Signed Backends with Keyless Cosign

LocalAI now verifies that backend OCI images came from our CI, not a compromised registry or MITM. This closes a real trust gap: the gallery YAML told LocalAI which image to pull, but nothing checked the bytes.

The producer side (.github/workflows/backend_merge.yml) signs every merged backend image (and every per-arch entry under the manifest list) with sigstore/cosign keyless via Fulcio + Rekor, using OCI 1.1 referrers (no legacy :tag.sig). The consumer side (pkg/oci/cosignverify, built on sigstore-go) verifies signatures against a per-gallery verification: policy:

verification:
  issuer_regex: "^https://token\\.actions\\.githubusercontent\\.com$"
  identity_regex: "^https://github\\.com/mudler/LocalAI/\\.github/workflows/backend_merge\\.yml@.*$"
  not_before: "2026-05-22T00:00:00Z"

• TUF trusted root cached process-wide, so N backends from one gallery do 1 fetch, not N.
• not_before is the revocation lever: keyless Fulcio certs are ephemeral, so revocation is policy-side. Advance the date in the gallery YAML and every signature predating the cutoff is invalidated.
• Digest pinning closes the TOCTOU window between verify and pull.
• Strict mode: --require-backend-integrity (or LOCALAI_REQUIRE_BACKEND_INTEGRITY=true) escalates missing policy / empty SHA256 from warn to hard-fail.

Rollout is backward-compatible: until a gallery ships a verification: block, installs proceed with a warning. The default backend/index.yaml will be populated next, and strict mode is opt-in. See .agents/backend-signing.md for the full producer + consumer story.

🔗 PRs: #9823 (consumer + producer + plumbing), #9957 (fix for current cosign releases).

---

⚡ Prompt Cache: On by Default

llama-cpp ships with a server-side prompt cache, but until now LocalAI was not enabling it by default. Repeated system prompts (agents, Claude-Code-style coding assistants, OpenAI-compatible CLIs with long instructions) were re-prefilled on every call. With this release, the same workload collapses to seconds without no specific configuration on your side.

Two changes, one default flip each:

1. kv_unified=true by default in grpc-server.cpp. The previous false was silently force-disabling cache_idle_slots at server init (the host prompt cache was being allocated but never written across requests).
2. prompt_cache_all defaults to true at the YAML layer, matching upstream llama.cpp's own common.h default. The per-request cache_prompt knob is now on out of the box.

You can still opt out with options: ["kv_unified:false"] or prompt_cache_all: false, and there are new option keys (cache_idle_slots, checkpoint_every_nt) for tuning. Docs in docs/content/advanced/model-configuration.md got a worked example for the repeated-system-prompt workload and a proper explanation of how kv_unified, cache_ram, and cache_idle_slots interact.

🔗 PRs: #9925 (kv_unified + cache_idle_slots defaults + docs), #9951 (prompt_cache_all tristate default).

---

📊 Per-API-Key Usage Tracking

Closes #9862. The usage page now answers "who spent these tokens?", not just "how many tokens were spent".

• usage_records gained Source (apikey / web / legacy), APIKeyID, APIKeyName, plus an idempotent backfill of pre-feature rows on InitDB.
• Auth middleware plumbs the resolved *UserAPIKey and the request source through the Echo context. Usage middleware snapshots the key id + name, so revoked keys stay readable in history (rendered as (revoked)).
• New endpoints: GET /api/auth/usage/sources (self, no legacy) and GET /api/auth/admin/usage/sources (admin, with user_id / api_key_id filters, 200-key truncation).
• React Usage page gains a Sources tab with a source-mix ribbon, a top-7 + Other time chart, and a searchable/sortable table with drill-in chip.
• Admin view (follow-up in #9935) also rolls up (source, user_id, user_name) so Web UI session traffic is split per user instead of lumped into one global "Web UI" row, and every named-key row shows the owning account.

Docs: features/authentication.md gained a full Usage Tracking section with the new tab, endpoints, response shape and migration notes.

🔗 PRs: #9920 (core + Sources tab), #9935 (per-user attribution in admin view).

---

🛰️ Distributed Mode v3

Distributed mode keeps hardening. This release fixes the two things that bit operators hardest in practice and lays the groundwork for the next round of UX.

Per-request routing across replicas (#9968) restores cross-node load balancing. The bug: ModelLoader.Load cached a *Model whose embedded InFlightTrackingClient was bound to a single (nodeID, replicaIndex). After the first request, every subsequent call reused that wrapper and pinned to whichever node won the first pick, even after the reconciler scaled the model out. The reproducer from the report:

dgx-spark1     loaded   in_flight=6
nvidia-thor1   loaded   in_flight=0       (← idle, never gets traffic)

Now SmartRouter.Route runs per request, the existing in_flight ASC, last_used ASC, available_vram DESC round-robin actually fires, and the replica-selection rule lives in one place (PickBestReplica) with a mirror spec asserting the SQL ORDER BY and the Go picker agree on a seeded dataset. probeHealth is now memoized per (nodeID, addr) with a 30s TTL and singleflight coalescing, so a burst of new requests doesn't stall on a HealthCheck that llama.cpp serializes against in-flight Predict.

Async per-node installs via the gallery job queue (#9928). POST /api/nodes/:id/backends/install used to block the request for up to 3 minutes while the worker pulled the image, freezing the React UI's Backends picker. It now returns HTTP 202 + jobID immediately, scoped to a one-element targetNodeIDs allowlist, with a node-scoped opcache row so concurrent installs on different nodes don't collide. The Operations panel surfaces a nodeID field for attribution.

Resilient backend installs with streaming progress (#9958). Two phases:

• Phase 1: LOCALAI_NATS_BACKEND_INSTALL_TIMEOUT / LOCALAI_NATS_BACKEND_UPGRADE_TIMEOUT env vars (default 15m, previously hardcoded 3m). A NATS round-trip timeout while the worker is still pulling no longer reports as a hard failure: per-node status becomes running_on_worker, the queue row stays alive without bumping Attempts, and ListBackends proactively clears install rows whose intent is satisfied (so the UI updates instantly instead of waiting up to 15m for the next reconciler tick).
• Phase 2: workers publish debounced (~250ms) BackendInstallProgressEvent values on a transient nodes.<nodeID>.backend.install.<opID>.progress subject. The master subscribes for the duration of the request and forwards each event into OpStatus.UpdateStatus, so the admin UI gets per-byte progress for distributed installs the same way local-mode does, with no UI changes. Backward compatible: old workers stay silent, new masters tolerate silence.

Unified backend-logs entry point (#9949). /app/backend-logs/:modelId is now a single, mode-aware route. In standalone it's the local WebSocket view, unchanged. In distributed it probes nodesApi.getModels, filters by model_name, then routes: 0 hits → empty state with a link to Nodes; 1 hit → <Navigate replace> to the per-node logs URL preserving the ?from= deep-link timestamp; N hits → a picker listing each hosting worker with node id, replica index and load state. Every view that links to backend logs now points at the same URL.

Bug-hunt harness. A new distributed test harness landed in tests/distributed/ to catch the kind of regressions the #9968 reproducer surfaced.

🔗 P...

v4.2.6

What's Changed

Other Changes

• feat(llama-cpp): bump to MTP-merge SHA and automatically set MTP defaults by @localai-bot in #9852
• docs: ⬆️ update docs version mudler/LocalAI by @localai-bot in #9853
• chore: ⬆️ Update antirez/ds4 to ef0a4905d05263df8e63689f2dd1efac618a752c by @localai-bot in #9857
• chore: ⬆️ Update ikawrakow/ik_llama.cpp to 3e573cfea6e0a332eff822ffbdb1dd3b112e9051 by @localai-bot in #9856
• chore: ⬆️ Update leejet/stable-diffusion.cpp to bd17f53b7386fb5f60e8587b75e73c4b2fed3426 by @localai-bot in #9854

Full Changelog: v4.2.5...v4.2.6

v4.2.5

What's Changed

Bug fixes 🐛

• fix(ollama): guard nil filter in galleryop.ListModels (#9817) by @localai-bot in #9836
• realtime: honor output_modalities to skip TTS in text-only mode by @localai-bot in #9838
• fix(ollama): accept float-encoded integer options (fixes #9837) by @localai-bot in #9849

Other Changes

• chore: ⬆️ Update ggml-org/llama.cpp to 7f3f843c31cd32dc4adc10b393342dfee071c332 by @localai-bot in #9809
• feat(llama-cpp): expose 12 missing common_params via options[] by @localai-bot in #9814
• fix(streaming): comply with OpenAI usage / stream_options spec by @localai-bot in #9815
• Close Hugging Face scan response body by @massy-o in #9818
• Validate video image URLs before download by @massy-o in #9819
• feat(swagger): update swagger by @localai-bot in #9824
• chore: ⬆️ Update antirez/ds4 to 04b6fda2be395094cbf2d20d921e7a705a4166ef by @localai-bot in #9830
• chore: ⬆️ Update ggml-org/whisper.cpp to 46ca43d6399fdeada1b49fb2126ba373bd9ebc38 by @localai-bot in #9829
• chore: ⬆️ Update ikawrakow/ik_llama.cpp to 0fcffdb64d21e57f0778f342415754156e01adfa by @localai-bot in #9828
• docs: ⬆️ update docs version mudler/LocalAI by @localai-bot in #9825
• chore: ⬆️ Update leejet/stable-diffusion.cpp to 0b8296915c4094090cff6bd2e09a5e98288c3c7d by @localai-bot in #9827
• chore: ⬆️ Update ggml-org/llama.cpp to 834a243664114487f99520370a7a7b00fc7a486f by @localai-bot in #9826
• Validate archive member paths before extraction by @massy-o in #9820
• fix(deps): bump gomarkdown/markdown for GHSA-77fj-vx54-gvh7 by @richiejp in #9841
• chore: ⬆️ Update vllm-project/vllm cu130 wheel to 0.21.0 by @localai-bot in #9846
• chore: ⬆️ Update ikawrakow/ik_llama.cpp to 5cc0d86c760e9858e4bed4418400bb39dbe025f2 by @localai-bot in #9845
• chore: ⬆️ Update antirez/ds4 to 950e8e6474a1c9fabe04e669d607606a7ef8824f by @localai-bot in #9844
• chore: ⬆️ Update ggml-org/whisper.cpp to 968eebe77225d25e57a3f981da7c696310f0e881 by @localai-bot in #9843
• chore: ⬆️ Update ggml-org/llama.cpp to 1348f67c58f561808136e8a152a9eddec168f221 by @localai-bot in #9842

New Contributors

• @massy-o made their first contribution in #9818

Full Changelog: v4.2.4...v4.2.5

v4.2.4

What's Changed

Bug fixes 🐛

• fix(distributed): cascade-clean stale node_models rows + filter routing by healthy status by @localai-bot in #9754
• fix(http): honor X-Forwarded-Prefix when proxy strips the prefix by @Dennisadira in #9614
• fix(agentpool): close truncate-then-read race in agent_jobs.json persistence by @localai-bot in #9811
• fix(middleware): parse OpenAI-spec tool_choice in /v1/chat/completions by @Anai-Guo in #9559

Exciting New Features 🎉

• feat: also parse VRAM budget/usage from vulkaninfo by @eglia in #9800
• feat(realtime): Add Liquid Audio s2s model and assistant mode on talk page by @richiejp in #9801

Other Changes

• chore: ⬆️ Update ggml-org/llama.cpp to a9883db8ee021cf16783016a60996d41820b5195 by @localai-bot in #9796
• chore: ⬆️ Update TheTom/llama-cpp-turboquant to 5aeb2fdbe26cd4c534c6fa15de73cb5749bd0403 by @localai-bot in #9740
• docs: ⬆️ update docs version mudler/LocalAI by @localai-bot in #9805
• chore: ⬆️ Update antirez/ds4 to 0cba357ca1bc0e7510421cc26888e420ea942123 by @localai-bot in #9806
• chore: ⬆️ Update ikawrakow/ik_llama.cpp to 949bb8f1d660fc1264c137a6f3dbd619375f6134 by @localai-bot in #9807
• chore: ⬆️ Update ggml-org/whisper.cpp to 3e9b7d0fef3528ee2208da3cdb873a2c53d2ae2f by @localai-bot in #9808
• ci(image): publish missing :latest-* and :v-* singleton image tags by @localai-bot in #9812

Full Changelog: v4.2.3...v4.2.4