Self-Programming AI: Code-Learning Agents for Autonomous Refactoring and Architectural Evolution — part 2/7
Cleaned-text code points [13768, 26943). Bibliography omitted here; source.md includes it.
motivates integrating code generation with feedback loops (e.g. executing tests) so that an agent can validate and refine its code beyond the one-shot capabilities of standalone LMs. Automated Program Repair Parallel to code synthesis, the field of Automated Program Repair (APR) has explored methods for fixing bugs with minimal human input. GenProg is a seminal example that uses genetic programming to evolve program variants until a failing test is repaired. Starting from a buggy program and its test suite, GenProg applies stochastic mutations and crossover on the code’s abstract syntax tree, searching for a patch that makes all tests pass. Notably, GenProg requires no formal specification, using tests as the correctness criterion, and demo nstrated repairs for multiple defect classes in large C programs. Another influential system, Prophet, took a data-driven approach: it learned a probabilistic model of correct code by mining hundreds of human-written patches, then used that model to rank candidate fixes generated by a template-based search. Prophet could repair many more bugs than earlier methods (15–18 vs. only 1–2 in a benchmark of 69 errors) by recognizing patterns of successful patches. These classical APR techniques, however, are limited by the search space of simple edits and often produce “plausible” patches that merely satisfy the given tests without truly eliminating the root cause (patch overfitting is a known issue). Recently, large language models have been applied to program repair to overcome these limitations. LLM-based repair can generate free-form fixes conditioned on bug descriptions, often handling more complex edits than templated methods. For example, AlphaRepair showed that a pre-trained code model (CodeBERT) can be used in a zero-shot fashion to fill in a masked buggy line with a correct fix, achieving state- of-the-art results on several benchmarks without specialized training data. Moreover, studies have found that generative code models like Codex can fix substantially more bugs from standard test suites than prior APR systems; one study reported Codex resolving 32 more Defects4J bugs than the best traditional repair tool. Interactive systems like ChatGPT have also been used to repair code via conversational refinement, yielding high success rates on automated bug-fixing challenges at very low cost. Limitations: LLM-driven repair still relies on the model’s static knowledge and often benefits from sampling or user feedback to get a correct fix. They lack an internal mechanism to verify the patch – the model might propose a change that looks plausible but fails unseen tests or breaks other functionality. This suggests that coupling LLMs with an automated test-and-debug loop could combine the strengths of both approaches: the creativity of learned fixes with the rigor of execution-based validation. LLMs with Execution Feedback Loops A promising direction is to have AI agents that utilize runtime feedback to guide their outputs. The ReAct framework is a general paradigm where an LLM produces Reasoning traces and Action commands interleaved. Rather than answering a query in one step, the model can iteratively reason about the problem, perform actions such as calling an external tool or executing code, observe the results, and then update its reasoning. Yao et al. demonstrated that ReAct greatly improves performance on knowledge-intensive and decision-making tasks: for instance, when answering questions, a ReAct agent can avoid hallucinations by issuing search queries and using the retrieved facts to correct its intermediate reasoning. This integration of chain-of-thought prompting with tool use yields solutions that are more grounded and interpretable than vanilla LMs. Along similar lines, Reflexion introduces a self-reflective loop for autonomous agents. Shinn et al. propose giving the agent a dynamic memory and letting it analyze its own mistakes after each trial. The agent generates an action (or solution), monitors the outcome (e.g. whether it succeeded or failed), and then appends a natural language “reflection” to its memory explaining the error and how to avoid it. This simple self-feedback mechanism led to striking improvements – e.g. success rates up to 97% in interactive game tasks – by allowing the agent to learn from failures without any weight updates. Another notable example is OpenAI’s Code Interpreter, which augments an LLM (ChatGPT) with a sandboxed Python execution environment. With Code Interpreter enabled, the model can write and run code to solve user prompts, using the program’s output as new information. This empowers the system to handle tasks like data analysis, visualization, and complex math by actively testing its code and observing results, rather than relying purely on the internal language model. Limitations: These approaches illustrate the benefit of closing the loop between thinking and acting, but they have not yet been fully applied to an agent that continuously improves its own code. ReAct and Reflexion were demonstrated on question-answering or game env ironments, not on the problem of a code-generating agent that refines a software project over time. Code Interpreter, while capable of running generated code, treats execution as a one-off step to produce an answer for a given query – it does not autonomously rewrite its underlying codebase. This leaves an open opportunity: an agent that uses execution feedback (test results, performance measures, etc.) to iteratively improve the code it writes, merging the above ideas into a sustained self-improvement cycle. Self-Improving and Self-Modifying Agents The concept of an AI that modifies its own algorithms has long been a topic of theoretical inquiry. Alan Turing and others envisioned that a sufficiently advanced machine could “learn to learn” by updating its own instructions, and later researchers formalized this as a blueprint for self-referential AI. Schmidhuber’s Gödel Machine is a notable formulation: a hypothetical universal problem-solver that can rewrite any part of its co de once it mathematically proves that the change will improve its performance towards a given goal. This idea provides an optimality framework for self-improvement, but it assumes a tractable way to prove the benefits of code modifications. In practice, no full Gödel Machine has been implemented due to the difficulty of the proof search and the risk of incorrect self-modifications. Nonetheless, the vision inspired decades of meta-learning research and discussions on how an agent might achieve open-ended improvement. Recent work has started translating these ideas into concrete systems. For example, Sheng and Padmanabhan (2023) demonstrated a Self-Programming Transformer that actually edits its own source code using an embedded code-generation model. In their approach, a language model is prompted with its current code and tasked to generate improvements, effectively treating its own implementation as mutable data. They showed empirically that this self-programming agent could modify aspects of its architecture and learning hyperparameters, and even spawn new “sub-models” to handle auxiliary task s. This result is arguably the first practical realization of a self-modifying AI, an idea that had previously been confined to theory. There have also been anecdotal reports of experimental agents in which an LLM, given access to its run-time code, will attempt to edit it to achieve some extended goal – underlining both the potential and the unpredictability of such systems. Limitations: The field of self-improving agents is still in its infancy. Early implementations are often limited to toy domains or controlled conditions, and they lack robust safeguards (e.g., an agent could just as easily self-modify in a detrimental way). Crucially, existing self-programming agents do not yet incorporate external feedback like tests or metrics in deciding how to evolve themselves – the improvements are generated based on the model’s internal logic or a limited reward. This gap points to the need for agents that combine self-modification with grounded feedback from their environment, such as actual code quality or perf ormance criteria, to ensure that “improvements” are both safe and beneficial. Test-Driven and Metric-Guided Code Optimization Integrating software engineering feedback (test outcomes, coverage, complexity metrics) into the code generation loop has shown promise in various contexts. In fact, traditional program repair and genetic improvement methods treat the test suite as a fitness function: a candidate patch or code variant is only acceptable if it passes all tests, thereby preserving required functionality. Search-based software engineering has extended this idea to optimize programs for non-functional objectives. For example, genetic improvement framework s view source code as “genetic material” that can be evolved to improve a given objective (speed, memory, maintainability) while tests ensure behavioral correctnesseecs.qmul.ac.uk. Researchers have used evolutionary algorithms to automatically refactor code and reduce code smells or complexity metrics, under the guidance of multi-objective fitness functions that balance correctness and quality metrics. However, these techniques typically operate by randomly mutating existing code and require many iterations of trial-and-error. Recent learning-based approaches aim to be more directed: CodeRL (Le et al., 2022) is one such framework that combines unit test feedback with reinforcement learning to improve code generation. During training, CodeRL’s critic network learns to predict the proportion of tests a generated program will pass, and uses this signal to reward the LM for producing correct and robust code. At inference time, CodeRL can iteratively regenerate solutions, guided by sample test results (pass/fail) and the critic’s judgments, which effectively prioritizes candidates that satisfy more of the specification. This led to state-of-the-art accuracy on challenging benchmarks by explicitly optimizing for functional correctness. Beyond tests, researchers have proposed incorporating runtime performance measurements or static analyzers into the loop – for instance, an agent could prefer code variants that not only pass tests but also improve coverage, execution speed, or maintainability scores. Limitations: Prior work in this vein often tackles one aspect at a time (e.g. just passing given tests, or just minimizing complexity with constraint of not breaking tests). There is no unified agent yet that can take full-spectrum feedback – i.e. simultaneously consider tests, coverage, performance, style metrics – to drive code evolution in an autonomous manner. The need for such integration is clear: an AI software agent should be able to treat failing tests as cues to fix bugs, low coverage as a hint to generate more tests or diversify code, high complexity as a prompt to refactor, and performance/profile data as guidance for optimization. Motivation: Bringing these strands together, our work aims to create an agent capable of refining code with comprehensive feedback. By learning from the successes and shortcomings of code LM generation, APR, execution-integrated prompting, self- modification, and metric-driven improvement, we seek to develop a self-pro gramming AI that continuously improves its codebase using test-driven development and objective metrics as first-class signals. This approach addresses the limitations of prior work by empowering the agent to not just generate code, but to validate, critique, and evolve that code autonomously toward higher levels of correctness, efficiency, and quality. 3. System Architecture and Proposed Framework Overview: The proposed system is implemented as a closed-loop self-programming agent pipeline that iteratively writes, tests, and refines code with minimal human intervention. It consists of modular components that mirror a human developer’s workflow: a planning phase to generate initial code and tests, an execution phase to run tests and gather quality metrics, and a refinement phase to improve the code based on feedback. All these phases are orchestrated around a single large language model (LLM) agent, creating an autonomous cycle that continues until the code meets predefined quality criteria. This closed-loop design allows the agent to continuously refine its output, producing robust and correct code; iterative code-generation frameworks have demonstrated significantly higher success rates and reliability compared to single-pass generation. In spirit, our f ramework aligns with recent multi- agent coding systems that decompose tasks into planning, coding, and debugging roles, but here these roles are integrated within one coordinated agent pipeline for efficiency and simplicity. Figure 1 outlines the architecture, highlighting the core modules and their interactions in the self-programming lo op. Task Planning Module (TPM)