Hi Hugging Face community — I’m the developer of Vyact, an AGPL-3.0 desktop AI workspace built with Electron, React, and FastAPI.
I’m working on the step after downloading a local model: using it with documents and notes that remain useful across conversations. A typical workflow is to index a PDF, ask a question, inspect the supporting passages, and save project decisions as searchable memos.
How the pieces fit together
- Local inference: GGUF models run through llama.cpp with llama-swap. On Apple Silicon, MLX models run through oMLX. Model search and downloads are available inside the app, alongside model metadata and hardware-aware memory estimates. These estimates help with selection; they are not a guarantee that every configuration will fit.
- Retrieval: Indexed documents, memos, and email threads can be grouped into knowledge collections. Chat can retrieve relevant passages from the selected collections and return sources for inspection. This helps users check the context behind an answer; it does not guarantee that the answer is correct.
- Provider integration: The workspace also supports hosted providers and custom OpenAI-compatible endpoints. MCP connections and reusable skills extend the available tools and instructions.
Measuring local performance
There is a built-in performance test for comparing supported settings on your own hardware. It runs short input, long input, and a follow-up conversation, reporting time to first token, generation speed, total response time, and reused prefix tokens. Separately reported prefill metrics are shown when the engine provides them.
For MLX, oMLX provides prefix caching. A matching follow-up can reuse prompt state, so the follow-up workload is useful for examining cache reuse rather than treating every request as a cold start. The benchmark ranks tested settings by speed; it is not an answer-quality benchmark.
Local-first scope
With a Vyact-managed local model selected, AI chat context is not sent to an external AI provider. Optional Google integrations, external tools, and hosted providers communicate with their respective services. Local inference does not make every connected workflow offline.
The README includes setup instructions for Apple Silicon Macs, Windows, and Linux x64, plus screenshots and platform-specific dependencies.
Source, screenshots, and installation instructions: GitHub - vyact/vyact: Vyact is an open-source, local-first personal AI workspace for llama.cpp, RAG, AI agents, document intelligence, and Google Workspace. · GitHub
I’d appreciate feedback on model compatibility and retrieval workflows. If you try it, which model/runtime and hardware are you using, and where does the workflow break down or need more control?