# \[Project\] QitOS: A research-first framework for building and evaluating LLM agents

**URL:** <https://discuss.huggingface.co/t/project-qitos-a-research-first-framework-for-building-and-evaluating-llm-agents/175037>\
**Category:** Beginners\
**Created:** [April 7, 2026, 5:47am UTC](https://discuss.huggingface.co/t/project-qitos-a-research-first-framework-for-building-and-evaluating-llm-agents/175037 "2026-04-07T05:47:14Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![morinoppp](https://avatars.discourse-cdn.com/v4/letter/m/bbe5ce/32.png) [@morinoppp](https://discuss.huggingface.co/u/morinoppp)\
**Post date:** [April 7, 2026, 5:47am UTC](https://discuss.huggingface.co/t/project-qitos-a-research-first-framework-for-building-and-evaluating-llm-agents/175037/1 "2026-04-07T05:47:14Z")

</div>

# [Project] QitOS: A research-first framework for building and evaluating LLM agents

Hey everyone,

I wanted to share QitOS, a new framework I’ve been working on that’s built specifically for LLM agent researchers.

After working on several agent projects, I found that most existing frameworks didn’t really fit the research workflow:

- It was too hard to quickly iterate on new agent architectures without rewriting the entire execution stack
- Strategy (how the agent thinks) and execution (tool calling, tracing, evaluation) were always tangled together
- Getting set up to evaluate on standard benchmarks took way longer than the actual research
- Debugging agent trajectories was a mess without proper tooling

QitOS was built to solve all these problems:

## Key Features

- **Clean architecture** : Separation between `AgentModule` (your strategy/innovation) and `Engine` (orchestration, tool execution, tracing). You focus on the research, the framework handles the rest.
- **Research-friendly** : Supports all common agent patterns out of the box: ReAct, Plan-Act, Tree-of-Thought, Reflexion, and makes it extremely easy to implement custom scaffolds.
- **Benchmark-native** : Built-in adapters for GAIA, Tau-Bench, and CyBench so you can get your evaluation up and running in minutes.
- **Great observability** : The `qita` CLI lets you browse, inspect, replay, and export full agent trajectories — no more digging through raw log files.
- **Ecosystem compatible** : Works naturally with any OpenAI-compatible model API, so you can use whatever models you prefer.

## Minimal Example

Here’s what a minimal SWE agent looks like in QitOS:

```python
from dataclasses import dataclass, field
from qitos import AgentModule, Engine, Task, ToolRegistry
from qitos.kit.parser import ReActTextParser
from qitos.kit.tool import EditorToolSet, RunCommand

@dataclass
class SWEState(StateSchema):
    scratchpad: list[str] = field(default_factory=list)

class MySWEAgent(AgentModule[SWEState, ...]):
    def __init__ (self, llm, workspace_root):
        reg = ToolRegistry()
        reg.include(EditorToolSet(workspace_root))
        reg.register(RunCommand(workspace_root))
        super(). __init__ (
            tool_registry=reg,
            llm=llm,
            model_parser=ReActTextParser()
        )
    
    # Implement your strategy logic here...

# Run it
agent = MySWEAgent(llm=my_llm, workspace_root="./playground")
result = Engine(agent=agent).run(my_task)
print(result.state.final_result)

```

## Get Started

- GitHub: [GitHub - Qitor/qitos: Let's Qitos! A torch-like agent-native framework for researchers. · GitHub](https://github.com/Qitor/qitos)
- Documentation: [Home - QitOS](https://qitor.github.io/qitos/)
- Install: `pip install qitos`

I’m really interested to hear what the community thinks — what do you find most frustrating about building LLM agents for research? Are there any features you’d like to see added to QitOS?

All feedback and contributions are very welcome!