Tool calling is easily the most important extension point of any harness. A tool is how the LLM reach out to the world - access a service, interact with a IoT device, load data, write data, search knowledge, talk to an API, run shell scripts - it’s all just tool calls.
This is an area that’s hard to get right first time because of how many features it unlocks, so focus on making it flexible, abstract well and make sure code is modular and easy to change as your harness grows.
Choosing a model to use for most cases requires one trained to do tool use - some models are quite bad at this and will favor inventing their own data over tool calls and others are really great at it. Bigger is not always better either because bigger models will lean on their training data more. A recent released model openbmb/MiniCPM5-2B-GGUF is excellent at calling tools rather than inventing data, and it’s great at understanding what it received from knowledge bases while only being 2 billion parameters big.
Likewise, some models are great at dealing with many tools and some will just not be able to figure out what is the right tool for the job should there be many tools configured. It’s a balancing act. But this is more or less the only extension point that exist so it’s one that requires a good, flexible, design.
NOTE: Shown here is not the exact text that goes to the LLM rather a representation encapsulated in my harness specific protocol, but this is derived from what the model APIs provide.
In the most basic sense, when you have tools configured and the model wish to run one or more tools it will answer your prompt with a message like this:
[
{
"protocol": "io.choria.fisk-ai.v1.session.assistant",
"stop_reason": "tool_use",
"in_tokens": 3538,
"out_tokens": 121,
"cache_read_tokens": 256,
"thinking_tokens": 52,
"assistant": {
"iteration": 0,
"message": {
"role": "assistant",
"content": [
{
"thinking": {
"text": "The user is asking about work done to speed up testing, and the final outcome for the full suite.This is project-specific knowledge, so I should search the knowledge base first."
}
},
{
"tool_use": {
"id": "call_4824d540e4454b3ab4d81396",
"name": "knowledge_search",
"input": {
"query": "speed up testing full test suite outcome results"
}
}
},
{
"tool_use": {
"id": "call_629fd83516a94d43a730d46f",
"name": "knowledge_enumerate",
"input": {
"query": "test suite speed"
}
}
}
]
}
}
}
]
Here we see a few important things:
stop_reason: tool_use- the turn resulted in output but it’s not the final output, it wants to use tools- we have 3 blocks here, one
thinkingand 2tool_use - each
tool_usehas a unique ID and function to call and the input to pass to the function. Later we will see how the agent knows what the inputs are. - these are as far as the LLM is concerned executed in parallel and, you must send the result of all parallel calls in one response. By calling several tools it needs like this it’s often hedging it will need some information over the latency costs of round trips. I often see it read memories in case another tool call returns data that might require the memory.
At this point your harness just look for tool_use blocks and turn that into a function call. The LLM does not know
the implementation details the call might be a REST call, Request-Reply over NATS, shell out to a CLI tool or
anything really that will return some data.
After the tool is called the harness returns a result, here just showing the first call result, note we include the
same tool_use_id as in the request this is how the model correlates what result belongs to what tool call:
{
"protocol": "io.choria.fisk-ai.v1.session.tool_result",
"tool_result": {
"result": {
"tool_use_id": "call_4824d540e4454b3ab4d81396",
"content": "{\"tier\":\"tier: hybrid (FTS5 + vectors, RRF) - model=text-embedding-qwen3-embedding-0.6b.....\"}]}"
}
}
}
The agent will interpret this and decide how to follow up - more tool calls, more thinking or the final answer. It can be any string but JSON is the conventional output format.
You can also set is_error on the tool_result blocks which will cause the model to retry or adjust its input.
Also, important to note that tool output goes directly into your context and stays there, do not output 2MB of data from a tool call.
This is not particularly complex thing to do and the libraries from Anthropic, OpenAI etc. make this a very easy thing to do. This is foundational block that enables more or less everything else - Memory, RAG, MCP, API calls and more. It’s all built on this one capability.
There’s a lot of details to make this work, especially with many tools, so read the full entry for the details.
[]