Jeff Open-Source Project Drops 0.8B Decision Model: Deterministic Tool Orchestration in 30ms at the Edge

The Three-Question Framework

What Happened?

Open-source AI engineer Felix 'firelex' released Jeff, an 800-million parameter decision model trained specifically for low-latency JSON-RPC tool routing, autonomous agent state machines, and system calls. Designed as a local, lightweight counterpart to TypeSafe AI's Jev system, Jeff executes complete multi-parameter function dispatches in approximately 30 milliseconds on standard Apple Silicon and laptop GPUs.

Why It Matters

Standard frontier LLMs (such as GPT-6 or Claude Opus) are computationally excessive and prohibitively slow for simple intermediate agent decisions, often adding 800–1,500ms of cloud latency and substantial API costs to check a file or parse an HTTP status code. Jeff proves that specialized 0.8B models trained strictly on deterministic execution graphs match frontier models on tool accuracy while slashing latency by 95% and running fully offline.

What Could Happen Next

The Jeff weights and training pipelines were released under an MIT license on Hugging Face and GitHub, with initial quantization adapters published for llama.cpp, Ollama, and WebGPU runtimes.

Executive Key Takeaways

Sub-30ms Latency: Eliminates cloud API roundtrips for intermediate tool selection and parameter schema validation.

0.8B Parameter Scale: Fits into less than 600MB of RAM using 4-bit GGUF quantization, allowing seamless background operation on mobile devices.

Jev Protocol Compatibility: Drop-in replacement for enterprise agent frameworks adhering to TypeSafe AI's System One decision standards.

100% Deterministic Output: Constrained grammar decoding ensures zero JSON formatting hallucinations during high-frequency execution loops.

In autonomous agent architectures, one of the most glaring inefficiencies is the reliance on massive, cloud-hosted frontier foundation models to execute trivial intermediate logic, such as determining whether to invoke a file-read tool or route a search query.

On September 28, 2026, open-source engineer Felix 'firelex' released Jeff, an 800-million parameter edge decision model designed specifically to solve this latency and cost bottleneck.

Named as a lighthearted homage to TypeSafe AI's enterprise Jev System One model released earlier this month, Jeff is engineered strictly for deterministic tool selection, argument parameter extraction, and JSON-RPC dispatching.

Rather than training on broad conversational text, Jeff was fine-tuned on millions of synthetic tool execution trajectories filtered through Direct Preference Optimization (DPO). The model rejects conversational fluff entirely, emitting strictly structured JSON payloads compliant with user-defined JSON Schemas.

Running locally via 4-bit GGUF quantization on llama.cpp, Jeff consumes less than 600 megabytes of memory and executes complete tool routing decisions in 28 to 32 milliseconds on an M3 MacBook Air—roughly 40 times faster than calling a cloud frontier API.

By decoupling high-frequency state machine routing from expensive cognitive reasoning, Jeff allows developers to build responsive local agents where simple bash commands, file lookups, and browser navigation happen instantaneously, reserving costly frontier models only for deep analytical synthesis.

The Three-Question Framework

What Happened?

Why It Matters

What Could Happen Next

Executive Key Takeaways

Primary Sources & Citations