get in touch

use cases

How loopmath can help you

loopmath is a CLI your agent calls around each coding task. It predicts how each way of running the task will do, on success and on cost, and it learns from every run. Here are four ways that pays off.

  1. Switch models to save costs
  2. Find the efficient frontier across your workflows
  3. Optimize your harness
  4. Know whether a change really helps

01 / switch models

Switch to a cheaper model where it does the job as well

Most teams send every task to their strongest model at high effort, because nobody knows which tasks would do just as well on less. loopmath predicts, for each task, how likely each model and effort level is to give an accepted result, and what each will cost. Where the cheaper option is predicted to do as well, it says so. Where it is unsure, it suggests a try, priced against what the switch would save on every similar task after.

Every run that comes back updates the estimates. The kinds of task where the cheaper model holds up move over to it; the kinds where it falls short stay where they were. Your cost per task goes down, and the results you accept stay as good.

Tasks go to loopmath, which predicts success and cost on a large and a smaller model, tries the smaller one where it is unsure, and moves the kinds of task where it holds up. Cost falls while quality stays level. tasks large model smaller model loopmath predicts successcost large $$$ smaller $ smaller model:unsure docs, tests:smaller is enough bug fix docs feature tests refactor config quality holds cost falls
  1. Tasks arrive.
  2. loopmath predicts each model's chance of success and its cost. On the smaller model it is unsure.
  3. It tries the smaller model where it is unsure, and keeps the rest on the large one.
  4. Results come back: docs and tests pass, config does not.
  5. The belief updates. Docs and tests stay on the smaller model; config goes back.
  6. From now on those tasks run cheaper. Cost falls, quality holds.
Illustrative: it shows the idea, not measured results.

02 / the frontier

Find the Pareto-efficient frontier across your workflows

A workflow is how the work is split between agents: code only, plan and code, plan, code and review, a second reviewer, or a cheaper model in one of the roles. Each workflow lands somewhere on cost and quality, and most are beaten on both by another one. loopmath puts every workflow you run on one success-cost curve, with ranges that narrow as runs come in, so you can see which workflows are on the frontier and which ones cost you more for nothing.

Then you choose by your own limit: the best result within a budget, or the cheapest workflow that clears your bar. Before each task, loopmath recommend shows the curve with your usual workflow on it and suggests the workflows worth a try.

Agent workflows appear as points on cost against quality with wide ranges: code only, plan and code, plan, code and review, two reviews, a small reviewer. Runs move them to where they land and shrink the ranges. A frontier line joins the workflows nothing beats on both, the others fade, and a budget line picks plan, code and a small reviewer. quality cost per task code, then two reviews plan, code, two reviews code only plan and code plan, code, small reviewer plan, code, review your budget best within budget
  1. Each workflow (who plans, codes and reviews, and on which model) starts as a guess, with wide ranges.
  2. Runs come in. Each point moves to where it really lands, and its range shrinks.
  3. The frontier draws itself through the workflows nothing beats on both cost and quality.
  4. The others fade: for each one, something on the frontier is cheaper and at least as good.
  5. You pick the workflow that fits your budget: here, a small model as the reviewer.
Illustrative: it shows the idea, not measured results.

03 / your harness

Optimize your harness at a quarter of the cost

Harness optimizers such as harness2 improve an agent's instruction file, skills and hooks: a strong model proposes edits, and the edits are tried on the task. They work, and they are expensive. We paired harness2 with loopmath: the same editors propose edits, and loopmath tests them in a small, designed set of runs and estimates which edits help. On 40 coding tasks from WorkBuddy-Bench, with Claude Haiku 5.5 doing the tasks and Claude Opus 5.5 editing, harness2 with loopmath kept about 95% of harness2's gain at about a quarter of its cost: 2.64 against 10.18 dollars per task. harness2 cut down to the same budget kept much less.

Mean score on 40 tasks, with list cost per task: baseline 0.746 at 0.02 dollars, harness² as published 0.798 at 10.18 dollars, harness² with loopmath 0.796 at 2.64 dollars, harness² at the same budget 0.773 at 2.61 dollars. Adding loopmath cuts the cost to 26% of harness² as published. 0.82 0.78 0.74 0.70 0.746 0.798 0.796 0.773 adding loopmath cuts the cost to 26% of harness²'s baseline harness² as published harness² + loopmath harness² at the same budget $0.02 $10.18 $2.64 $2.61 list cost per task
Mean score on 40 WorkBuddy-Bench Code tasks, 0 to 1 (the axis starts at 0.70). Cost is list dollars per task.

Read the study →

04 / check a change

Know whether a change really helps

A new skill, a rewritten prompt, a plugin: each looks better on the two tasks you tried it on. loopmath check-change runs the base and the change in pairs, the same task on both, in a balanced order. After each pair it updates its estimate of the gain, with a range, and it tells you when to stop: adopt once the range clears zero, keep the base if the evidence has not decided by your cap or budget, and in between, what the next pair is worth.

The same gate works inside optimization loops, so a prompt that scored well on three examples by luck is not kept. It plugs into DSPy's GEPA and MIPROv2, and it reads claude plugin eval results as they are.

A base harness and a change are run in pairs. The estimated gain and its range are drawn pair by pair. The range crosses zero at first, then narrows, and once it clears zero the verdict is adopt. base harness base + your change gain from the change no difference paired runs cap range crosses zero: run another pair adopt range clears zero
  1. You change the harness: a new skill, a rewritten prompt, a plugin.
  2. loopmath plans paired runs: the same task on the base and on the change, in a balanced order.
  3. After each pair it updates the estimated gain and its range.
  4. While the range crosses zero, the answer is: run another pair, and here is what it is worth.
  5. Once the range clears zero, the verdict is adopt. If the cap or the budget comes first, the base stays.
Illustrative: it shows the idea, not measured results.

Try it on your own runs

loopmath is open source and runs on your machine. Install it, let your agent onboard it, and it starts learning from the runs you already make.

$ pipx install loopmath

How it works →GitHub →Get in touch →