Skip to content

Taming AI agents

Six months ago I thought I was simply going to learn a bit more about AI agents.

Instead, I ended up diving head first into the ecosystem: trying frameworks, integrating them into real projects, contributing to existing tools, and eventually building my own.

It was one of the most enjoyable technical rabbit holes I’ve explored in years. It also completely changed the way I think about building agentic systems.

This post is a summary of that journey: the lessons I learned, the ideas that stuck with me, and the open-source projects that naturally came out of it.

What I discovered

I learned those lessons the hard way.

Lesson #1 – LLMs will always fail from time to time

Don’t fight it, you’ll lose. Embrace it and use them for what they’re good at.

Deterministic processes should not be based on LLMs.

Lesson #2 – We often use agentic systems where a simple LLM call would be enough

Understanding the various levels of an agentic system is key.

Make sure you are able to use either a simple provider with your prompt, or use a complex agentic system with skills, tools, hooks if needed. Be able to combine and compose all of them.

Lesson #3 – Orchestrate with code, not LLMs

Guiding an agent using another LLM is wrong: if you need strict workflows or guidance, use external guardrails like code, not skills nor trust in a manager LLM.

At first I was writing skills to do TDD. It was failing as soon as context was growing, or unless I was paying for huge frontier models.

I want to trust my TDD workflow, and won’t accept that it fails 10% of the time. Then I realized that what I was trying to enforce could be written in 10 lines of Ruby code 🤣

Lesson #4 – Control context explicitly

Most agentic frameworks handle context sharing in a simple way, just concatenating and compressing history over time. Most of your agents don’t need that much of context to perform their tasks.

It’s like overloading a developer with all the internal discussions between stakeholders that took place to write a feature’s requirements. This is mostly noise to the developer who only needs a proper requirements document to work on.

Lesson #5 – Mainstream agentic systems (like Copilot, ClaudeCode, Cline…) are really development specialized

Using them outside of their comfort zone (development tasks) is tricky. Even orchestration can be difficult because of simple concepts like Plan/Act. If you need more orchestration than what those tools provide, consider using less orchestration features from those tools, and more from your code.

For example today I use Cline only in Act and “auto-approve all” modes, but each task will be very small, fully guard-railed using commits and tools that are code-driven, and I will sometimes completely rewrite its system prompt.

What next?

With that in mind, I want safe ways of using agents to help me in my daily developer tasks. I want a system that is easy to use, deterministic in its workflow, and uses AI agents for design, plan, coding, documentation.

Deterministic doesn’t mean “never use AI.” It means that the structure of the workflow should always be predictable, even if one of the steps is creative. I know exactly which agent will run next, which inputs it will receive, and what outputs it is expected to produce. The only non-deterministic part is the content generated by the LLM—not the workflow itself.

As always, I want this system to produce quality outputs, easy to review and fix iteratively, and Open Source.

Here it is!

The main concept: Composable Agents with Artifacts

My conclusion was surprisingly simple: stop thinking of agents as autonomous workers coordinating with each other.

Instead, think of them as functions. Every agent consumes explicit inputs, produces explicit outputs, and exposes a stable interface.

Once you do that, you can compose them exactly like any other software component.

  • Instead of using LLMs to orchestrate other agents (the “crew” principle), use code. Keep LLMs for creativity, content generation and non-deterministic tasks that allow varying output quality levels.
  • Think of agents as software components that you can compose, like LEGOs. 1 agent could be using code, LLMs, or whatever internally. Some of them could be 100% deterministic, others could generate content based on LLMs.
  • Compose agents using simple interfaces that give you control over the context. I named those interfaces the artifacts: agents get input artifacts, and produce output artifacts. This is the context shared between agents and is fully under control. Composable agents happen to behave very much like pure functions because they consume explicit inputs and produce explicit outputs.

Artifacts: explicit interfaces between agents

Most agentic frameworks communicate through a shared conversation. Every agent receives the entire history: prompts, previous answers, tool outputs, corrections… As the workflow grows, so does the context, making prompts harder to understand and agents increasingly dependent on information they don’t actually need.

flowchart LR
    A[Planner] --> H[(Conversation History)]
    H --> B[Developer]
    B --> H
    H --> C[Reviewer]

I took a different approach inspired by traditional software engineering: agents shouldn’t communicate through conversations, they should communicate through interfaces.

In Composable Agents, those interfaces are called artifacts. An artifact can be anything: a design document, a JSON object, source code, a test report… Each agent explicitly declares which artifacts it consumes and which artifacts it produces.

flowchart LR
    P[Planner]
    D[Developer]
    R[Reviewer]

    PLAN[/plan.md/]
    CODE[/source_code/]
    REVIEW[/review.md/]

    P --> PLAN --> D --> CODE --> R --> REVIEW

This has two important consequences. First, every agent receives exactly the information it needs—nothing more. Second, agents become truly composable. Whether an agent is implemented in pure Ruby, calls an LLM directly, or wraps a complete agentic system like Cline becomes an implementation detail. As long as it consumes and produces artifacts, it can participate in the workflow.

Composable agents in Ruby

I released the composable_agents Rubygem for that purpose. Here is how you can compose deterministic code agents, full agentic systems and even simple LLM calls with it:

# The preferences agent is pure Ruby code
# => Output the user travel preferences.
preferences_agent = ComposableAgents::RubyAgent.new(
  proc do
    puts 'What kind of holidays are you looking for?'
    {  preferences: $stdin.gets.strip }
  end
)

# The itinerary agent is a full agentic system with a prompt (Cline)
# - Input the user travel preferences.
# => Output a list of matching cities.
itinerary_agent = ComposableAgents::Cline::Agent.new(
  role: 'You are a travel planner',
  objective: 'Find cities that would be the best destinations for the user\'s holidays',
  input_artifacts_contracts: { preferences: 'The user travel preferences' },
  output_artifacts_contracts: { cities: 'The best cities matching the user travel preferences' }
)

# The budget agent is pure code.
# - Input a list of cities.
# => Output a price.
budget_agent = ComposableAgents::RubyAgent.new(
  proc do |input_artifacts|
    # Compute the budget from the cities list
    { budget: input_artifacts[:cities].size * 1000 }
  end
)

# Compose them.
# Each agent only receives the artifacts from previous ones, not a growing context.
preferences = preferences_agent.run
itinerary = itinerary_agent.run(**preferences)
budget = budget_agent.run(**itinerary)

puts "Budget for the cities #{itinerary[:cities]} is $#{budget[:budget]}"

With this you have full control over how artifacts are being produced, and you can create all the harness, guardrails and deterministic workflows that you need to help your agents produce the desired output. Full example here.

For now this Rubygem supports AI agents served by the wonderful ai-agents library (can be used to target any LLM provider), and the Cline ecosystem.

Then what about the daily developer job?

The answer: a nice CLI, built on top of composable_agents. I shipped it as the x_aeon_agents Rubygem.

For example, ask it to implement a new feature and create a PR from it on Github using xaa implement --pr "Add authentication middleware". See what will happen under the hood with composable_agents:

flowchart LR
    U["$ xaa implement --pr #quot;Add authentication middleware#quot;"]

    P["Planner"]
    PLAN[/plan.md/]

    D["Developer"]
    DIFF[/code changes/]

    T["Tester"]

    DOC["Documenter"]
    README[/README.md/]

    C["PR creator"]

    U --> P
    P --> PLAN
    PLAN --> D
    D --> DIFF
    DIFF --> T
    T --> DIFF
    DIFF --> DOC
    DOC --> README
    README --> C
    DIFF --> C

Here are the currently supported commands:

➜  xaa --help
Commands:
  xaa commit                                 # Commit staged changes with an ...
  xaa create-pr                              # Push on Github and create a Pu...
  xaa generate-readme                        # Generate a comprehensive READM...
  xaa generate-skills                        # Generate skill files from ERB ...
  xaa help [COMMAND]                         # Describe available commands or...
  xaa implement REQUIREMENTS                 # Implement given requirements w...
  xaa implement-issue ISSUE_NUMBER           # Implement a GitHub issue using AI
  xaa install-skills                         # Install skills and their depen...
  xaa interpret-diffs [BASE]                 # Summarize git diffs relative t...
  xaa prompt PROMPT                          # Send a one-shot prompt to the ...
  xaa review-comments [PULL_REQUEST_NUMBER]  # Address review comments on a G...
  xaa start-task                             # Open a new git worktree for a ...
  xaa tree                                   # Print a tree of all available ...

Options:
  [--session-id=SESSION_ID]                # Session ID for persistence
  [--debug], [--no-debug], [--skip-debug]  # Enable debug mode
                                           # Default: false

Still this CLI is to be considered in beta stage, as it is quite opinionated around the development workflows that I am using, it is still lacking some customization and polishing.

cline-rb – Using the Cline ecosystem in Ruby

To implement Composable Agents in Ruby, I wanted to provide a nice Ruby API for Cline.

cline-rb is a Ruby interface to the Cline ecosystem, supporting both the legacy and current concepts used by the VSCode extension and the CLI. If you’ve ever wanted to automate Cline workflows from Ruby instead of driving everything through the UI or CLI, that’s exactly what this gem is for. With it you can:

  • Integrate Cline into larger Ruby-based agentic systems.
  • Read and edit skills, tasks, sessions and logs.
  • Monitor sessions in real time.
  • Launch and control the Cline CLI programmatically, including interactive workflows.

pty_compat – Using Ruby’s PTY interface on Windows

I always try to be OS-independent in my development. The Cline ecosystem is working pretty well on both Linux and Windows, so I also wanted my CLI and composable agents to do the same.

For that purpose I wanted to use Ruby::PTY under Windows. That was a challenge that I overcame using Microsoft’s node-pty library, wrapped in a nice Rubygem that exposes the standard Ruby::PTY interface on Windows: pty_compat. Now Ruby::PTY is completely OS-agnostic from a user point-of-view.

Next steps

After months of experimentation, I realized we often give AI far too much credit for its autonomy and decision quality. It was simply oversold. Now it’s time to use it for what it’s best.

I think it belongs to software engineers building deterministic systems that use LLMs where they’re strongest and traditional software everywhere else.

That’s the philosophy behind composable_agents, x_aeon_agents and the other projects released during this exploration.

They’re all open source, and I’m looking forward to seeing where this approach leads next.

Tags:

Leave a Reply

Your email address will not be published. Required fields are marked *