AI Harness Design Series

I recently released a AI Harness called Fisk AI and I want to do a series of posts on how the major components is designed. There’s a lot of mystery around Agent Harnesses but really it is not magic or particular hard, it’s a different paradigm that enables some novel new behaviors but untimely it’s just a API client.

The purpose and background of the Fisk harness is best captured by the project announcement post, but, in short, it’s a harness that lets you use LLMs in a safe and constrained manner. Rather than all-powerful agents like Claude, Codex, Hermes etc I want to assemble an agent by starting with nothing and adding to it what I need via tools, guardrails and limits. In the end giving me complete visibility in what it can do in order to make it safe, secure, privacy respecting and potentially local only.

To build an agent you need no code at all, just a YAML file, like this one that lets me build an agent that can answer questions about OpenVox. Here is a slightly longer than typical example that creates a full chat experience where you are essentially talking with https://docs.openvoxproject.org. You get discussion grounded in documentation with full citations of every source linking directly back to the source material on the OpenVox website.

identity: OpenVOX
icon_url:  https://raw.githubusercontent.com/voxpupuli/logos/refs/heads/master/images/OpenVox/Sticker/HexagonSticker.svg

harness:
  knowledge:
    enabled: true
    expand_to_section: true
    read_tool: true
    paths:
      - openvox-docs/docs/_openvox_8x
      - openvox-docs/docs/_openvox-server_8x
      - puppet-specifications/language
      - types.md
    citations:
      - pattern: '^openvox-docs/docs/_openvox_8x/(.+).(markdown|md)$'
        replace: 'https://docs.openvoxproject.org/openvox/latest/$1.html#${anchor}'
      - pattern: '^openvox-docs/docs/_openvox-server_8x/(.+).(markdown|md)$'
        replace: 'https://docs.openvoxproject.org/openvox-server/latest/$1.html#${anchor}'

llm:
  model: deepseek/deepseek-v4-flash
  budget:
    max_tokens: 100000
    max_iterations: 50

system_prompt: |
  You help users based on a knowledge base of OpenVox information

  Every answer must be researched using the RESEARCH STRATEGY, presented using RESPONSE FORMAT and validated using RESPONSE CHECKLIS.

  RESEARCH STRATEGY:
  - Use must use `knowledge_search` tool to find the direct answer
  - For broad topics, always perform at least 2-3 searches with different keyword angles to ensure comprehensive coverage

  OUTPUT FORMATS:
  - The target output is Markdown.
  - You can generate SVG images, display them in a code block with the language `svg`
  - You can make mermaid diagrams in code blocks with language `mermaid`

  RESPONSE FORMAT:

  Show citations to sources in the paragraph text in the form `(#ref)` or `(#ref)(#ref)` so that users know exactly where the
  information comes from. Citations may only include entries from `knowledge_search`. Citations matching the
  `ref` in the `knowledge_search` output.

Here I create a knowledge base that can answer questions about OpenVox by simply ingesting their documentation git repository. This is the sweet spot for this tool - just quickly assemble a agent that can bring some agentic abilities to your exiting files.

Here I use only Full Text Search and BM25 so it would work well without a local embeddings model - but you can opt into that if needed.

Fisk Web Interface

When running in the web frontend we can see on the left many agents each with a unique but limited ability. We see the agent answering a question grounded in the documentation including citations that take you direct to the website. While I was using a hosted model here via Open Router you would get very good results from even a small local LLM like the 8B Qwen models - runnable on most modern laptops.

This is the start of a series of posts that discuss how I designed the components of this system in mid 2026, starting first with what a Harness is.

Read on to the full entry for that.

[]
ai  choria  fisk