Everything Before the Click

Every tool my agent had until this one was reversible by doing nothing. Search a product, read a price, fetch the details: run any of them twice and the second run costs a little time and changes nothing else. The worst outcome was a stale answer.

Then I gave it a tool that adds an item to my cart.

That is a different kind of tool, and the difference is not the code inside it. The handler is short and dull. What changes is everything around it: what may be retried, what has to be written down, and what it may do at all when I am only rehearsing.

Rehearsal

The first property I wanted was the ability to run the whole thing with the last step disarmed. Not a mock, not a test double: the real tool, the real arguments, the real dispatch, and nothing bought.

Every tool in my registry carries a flag:

defstruct [:name, :description, :input_schema, :output_schema, :handler, mutating: false]

and the dispatcher branches on it before the handler ever runs:

defp run(%Tool{mutating: true} = tool, params) do
  outcome =
    if Config.dry_run?() do
      {:ok, params |> Map.put("dry_run", true) |> Map.put("simulated", true)}
    else
      safe_execute(tool, params)
    end

  audit(tool, params, outcome)
  outcome
end

defp run(tool, params), do: execute(tool, params)

Two clauses. A read goes straight through and is never audited, because there is nothing to answer for. A write is rehearsed or performed, and either way it leaves a trace.

The answer a rehearsal returns says so in its own body, dry_run and simulated both true, rather than looking like a success. A model reading that result should be able to tell that nothing happened, and so should I, three weeks later, reading the row it left behind.

The record that almost was not written

The audit call sits after the outcome, which looks obviously correct and was not.

Both guards on the mutating path raise rather than return. Guard raises when Amazon serves a challenge page, SessionGuard raises when the session has expired. So the first version of this code had a hole exactly where a record matters most: let Amazon answer a cart request with a challenge page, and the exception unwinds straight past the audit call. The attempt happened, it failed for a reason worth knowing, and nothing anywhere wrote it down.

The fix is a rescue, and one idea:

defp safe_execute(tool, params) do
  execute(tool, params)
rescue
  error -> {:error, Exception.message(error)}
end

The idea is that an attempt is a fact. Whether the cart changed is a separate question, answered by the row’s result. What must never happen is a request that reached the point of trying and left no trace, because that is precisely the request I will want to find later.

Where a retry is allowed to reach

This is the part I would get wrong again if I were not careful, and it is where the flag on the tool stops being enough. mutating: true tells the dispatcher to rehearse and to keep a record. It says nothing about how far a retry may go, because that boundary does not live in the dispatcher at all.

Driving a browser is flaky. Navigation times out, the sidecar hiccups, a page is not ready yet. Reads have a retry wrapper for exactly that, and it is the right answer: try again, three times, with a small delay.

Applied to a cart, that same wrapper buys two of everything.

So the retry boundary stops before the click, and the code says so out loud in both places. The shared preparation sequence carries the rule in its documentation:

@moduledoc """
Shared retryable-phase sequence for the Amazon mutating page objects
(`CartPage`, `WishlistPage`): navigate, read, guard against a challenge,
guard against an expired session, then wait for the control the click
phase needs.

Deliberately excludes the click and everything after it. Callers wrap only
this function in `Resilience.with_retry/2` -- the click is never retried.
"""

and the caller splits itself in two along that line:

# Retryable phase: navigation and waiting for the button. Nothing has been clicked yet.
with {:ok, :ready} <-
       Resilience.with_retry(fn -> MutatingPage.prepare(page, url, hd(@add_button)) end) do
  if quantity > 1, do: set_quantity(page, quantity)

  # Non-retryable phase: a click failure here is reported as-is, never
  # retried, and never raised -- it must reach ToolCall.run/2 as a
  # {:error, _} outcome so the failed attempt still gets audited.
  with :ok <- Page.click(page, hd(@add_button)) do
    {:ok, %{"cart_item_count" => read_count(page)}}
  end
end

Everything before the click can be attempted again safely, because none of it has touched anything. Everything from the click onward gets one attempt, and its failure is reported rather than raised, so it still reaches the audit.

What I like about this arrangement is that the boundary is a real seam in the code, not a comment asking someone to be careful. A future tool that wants retries gets them by calling prepare. It cannot accidentally acquire them for its click, because the wrapper is not there.

What the trail is allowed to know

An audit trail is a tempting place to put everything, on the theory that more context is better when something goes wrong. It is the wrong theory. This one stores what identifies the action and its outcome, and refuses the rest:

@moduledoc """
Append-only trail of every mutating tool invocation, simulated or real.

Stores only what identifies the action and its outcome: tool name, params,
result and the dry-run flag. Never page HTML, cookies or session state.
"""

The row is five fields, and the discipline is in what is missing from them. The page that was on screen when this happened is not there. Neither is the session. A trail that captured those would be a second copy of my Amazon account sitting in a table nobody thinks about, growing, being backed up.

The one thing I cannot test

Every Amazon read tool in this project has a recorded page behind it. I capture a real response, commit it, and the parser is tested offline against the same bytes forever.

There is no such fixture for adding to a cart, and there cannot be. Recording one would mean actually adding something to my cart, on the real account, and whatever came back would describe that one purchase attempt rather than the shape of the interaction. So the selectors for the add button carry an honest admission:

# Selector candidates are informed guesses from the brief, unverified against a
# real logged-in Amazon page (no fixture can be recorded for one here). The
# owner's first live run confirms or corrects them.

Which is uncomfortable, and clarifying. The action I am least able to verify in advance is the one that changes something in the world. The tests cannot carry that weight, so the structure has to: rehearse it before arming it, write down that it was attempted, and never let a retry reach past the point of no return.

None of those three needs a test to be true. They are true because of where the code is divided.