Adaptive Auto-Harness: Sustained Self-Improvement for Agentic System Deployment on Open-Ended Task Streams — part 4/4

Cleaned-text code points [46948, 50453). Bibliography omitted here; source.md includes it.

Per-batch view. Table 12 reports the same series broken out by batch on the replay subset (only batches with full cross-branch replays are shown, so e.g. CTF-Dojo includes batches 2 and 3 only). CTF-Dojo’s branch tree improves between these two batches (Oracle → 65 45\!\to\!65 %) as new specialisations come online; PolyBench shows a quiet batch 4 followed by a sports-led recovery in batch 5; FutureX’s batch 3 has the widest Oracle − - Adapt headroom, where the router’s level-based branches were less reliable than main on the same tasks.

## Appendix H Human-in-the-Loop Event Log

Table 13 reports the full HITL event log from the FutureX RQ5 run (§ 4.6 ). The run uses the curated 5-batch FutureX stream (Design C: 100 tasks, 5 × \times 20) with two engineered regime shifts, a Sonnet 4.6 solver, an Opus 4.6 evolver, and hitl_enabled=true . The Analyst and Builder decide when to invoke each hook; the human responds via Telegram from a pre-authored cheat-sheet. Two P2 (research-phase) events fire at cycle 1 to bootstrap the search pipeline, and one substantive P3 (task-board) event fires at cycle 3 to steer the evolver toward Western and Chinese specialty endpoints. The remaining P3 prompts (cycles 1, 2, 4, and 5) return skip , matching the cheat-sheet protocol that the human only intervenes when the cheat-sheet has a relevant entry. The slice-level lift on each regime ( 0 0 , + 5 +5 , + 20 +20 , + 15 +15 , 0 0 on regimes 1–5) is reported in Figure 9 .

## Appendix I Run-Detail Analysis

Table 14 reports per-task turn counts and elapsed seconds, complementing the aggregate cost summary in Table 5 . Both distributions are right-skewed on CTF-Dojo and FutureX, where a small number of long-running tasks pull the mean above the median; we therefore report both. CTF-Dojo’s solver budget is consistently saturated — Sonnet’s mean of 89.4 turns reflects a large mass of tasks that loop until the cap, whereas the evolved variants typically reach a flag (or give up) earlier. PolyBench is dominated by direct decisions: most tasks submit immediately (turn count 1) because the prompt already contains the market context the solver needs.

## Appendix J System Prompts

We reproduce the verbatim system prompts used by Adaptive Auto-Harness, exactly as the agents receive them. Curly-brace placeholders such as {benchmark_context} , {regime} , {workspace_extras} , and {categories} are substituted by the framework at runtime per benchmark or per regime; we leave them in place so the templating is visible.

⬇

⬇

⬇

⬇

⬇

## Instructions for reporting errors

We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:

Click the "Report Issue" ( ) button, located in the page header.

Tip: You can select the relevant text first, to include it in your report.

Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.

Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .