What is Needle?
Needle is a foundation model built by Cactus Compute for applications on phones, wearables, robots, smart-home devices, automotive systems and microcontrollers. It prioritizes tool calling and structured extraction over general conversation. Given application-defined functions, it selects tools and fills their arguments, returning ordered calls or an empty list when no tool applies.
The model uses a schema-derived decoding grammar to constrain output and provides text embeddings for local search, matching and routing. Its Python package exposes function decorators and an execution loop, while prebuilt engines load model weights from a local binary container. Developers can select model depth and fine-tune locally or through the Cactus platform.
Whistle, a separate speech-to-text model sharing the same container format and engine, adds audio input to the tool-calling workflow. Browser and Node.js execution is also documented through a community-maintained WebAssembly package. Telemetry is enabled by default in the binary and can be disabled through environment variables.
Key capabilities
- Tool calling Selects application-defined functions, fills arguments and returns ordered calls, with an empty list for unsupported requests.
- Structured extraction Converts unstructured text into typed records using schema-constrained decoding, including classification through enumerated values.
- Local embeddings Produces sentence vectors for search, matching and routing on the device.
- Speech input Uses Whistle in the shared runtime to transcribe audio and produce tool calls in a single engine call.
- Customisation Supports local LoRA fine-tuning and platform-based full-model training, with selectable deployment depth.
- Developer interfaces Provides a Python API, command-line workflows and a C API, with prebuilt deployment engines.
Where Needle fits
Needle fits inside applications that map natural-language requests to a bounded set of actions, such as smart-home controls or device commands. Its extraction and embedding functions also support local record processing, classification and search. Developers supply the tool definitions, schemas and executable functions rather than relying on a ready-made catalogue of business connectors.
Before deployment, teams should test tool selection and argument accuracy against their own requests, verify the engine and model depth on the target hardware, and review telemetry settings. Local LoRA fine-tuning returns no calibrated confidence value. Always-on speech applications should place voice activity detection or a wake word before transcription, since music and other sounds can still produce text.
Install
pip install cactus-needleReviews of Needle
Be the first to share your experience with Needle.
Write a review
Loading review form…
Alternatives to Needle
DwarfStarDwarfStar runs selected DeepSeek, GLM and Qwen models locally, with a conversational CLI, a native coding agent and an HTTP server. It targets specific Metal, CUDA and ROCm hardware configurations.
FreeTokenFreeToken is an Apache-2.0 serving engine for open-weight MoE models on consumer hardware, with a Windows and Linux desktop app, a CLI and compatible APIs.PMProject MayaProject Maya runs large mixture-of-experts models locally, spreading experts across GPU memory, RAM and SSD, with browser chat and OpenAI-compatible APIs.