STOP: When the Code Improver Becomes Its Own Optimization Target

Review date: 2026-10-18
Paper: Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation
Authors: Eric Zelikman, Eliana Lorch, Lester Mackey, Adam Tauman Kalai
Reading version: arXiv:2310.02304v3, revised August 16, 2024; first submitted October 3, 2023
Resources: Full paper, official repository

1. The interesting object is the improver

Most code-generation loops optimize a solution. They ask a model for a program, measure its quality, and ask for a revision. STOP moves the optimization target up one level: the program being rewritten is the program that organizes those requests and evaluations. A better search procedure can then be reused to improve other programs. The attraction is not merely a stronger answer to one problem; it is an improvement in the process that produces answers.

This distinction also limits the claim. The language model remains fixed. STOP changes a code scaffold around it: candidate generation, search control, temperature selection, evaluation, and stopping. It does not train new model weights, prove that every iteration improves, or establish unbounded recursive self-improvement. Its empirical contribution is a concrete demonstration that a sufficiently capable fixed model can help modify its own surrounding optimizer, and that some resulting optimizers transfer to other tasks.

This review follows the full 46-page paper, including its appendices. Numerical results below are attributed to the paper. Analytical examples and suggested experiments are identified separately. The central question is: when the optimizer becomes editable, what prevents a better-looking optimizer from simply exploiting noisy or incomplete evaluation?

Figure 1. The current improver rewrites an improver, while meta-utility evaluates its ability to improve downstream programs. Cropped from Figure 3 of STOP, arXiv:2310.02304v3. The language model and evaluation objective remain external to the mutable scaffold.

2. Prerequisites: programs, utilities, and two kinds of search

Represent a candidate program by a string $s$. A utility function $u(s)$ measures its quality on a particular task. A useful utility must cover more than an attractive-looking answer: invalid output, execution failure, time limits, and task-specific scoring all affect the result. STOP’s theoretical discussion uses bounded utilities in $[0,1]$; its practical tasks supply measurable scores and resource restrictions.

An improver $I$ receives a utility, an initial solution, and access to a language model $L$. It returns another solution:

$$s'=I(u,s,L). \tag{1}$$

The word “improver” names an intended role, not a mathematical guarantee. Nothing in this signature requires $u(s')\geq u(s)$. A program can discard a good incumbent, misread the objective, or spend the budget before returning. This matters especially when the output is itself an improver: a single bad revision can damage the search process used by the next round.

There are therefore two searches. The inner search explores solutions for a downstream problem. The outer search explores implementations of that inner search. The same language model can participate in both, but the objects being evaluated differ. A downstream solver is judged by task utility. An improver is judged by the quality of the solvers it produces.

This is close to automated algorithm design, with an unusually expressive proposal mechanism. Instead of selecting from a small menu of hand-written optimizer settings, the model proposes code. The space includes new prompts, loops, caches, selection rules, and combinations of familiar heuristics. That flexibility is the source of both opportunity and fragility: the search can alter useful control logic, but it can also break assumptions that a conventional optimizer would keep fixed.

A second prerequisite is the difference between task diversity and repeated randomness. Running the same task five times measures how stochastic generation behaves on that task. Sampling five different task families measures something else. Neither number can be substituted for the other in a generalization argument. This distinction will become important when we examine the small meta-training distribution and the finite-class bound.

3. From solution utility to meta-utility

Suppose downstream tasks are drawn from a distribution $D$. Each task supplies a utility $u$ and an initial solution $s$. The population quality of an improver is

$$U(I)=\mathbb{E}_{(u,s)\sim D} \left[u\!\left(I(u,s,L)\right)\right]. \tag{2}$$

Any randomness in generation or evaluation can be included in this expectation. In practice, the population distribution is unavailable. STOP evaluates candidates using a finite collection of downstream problems. A simplified empirical objective is

$$\widehat U(I)=\frac{1}{n}\sum_{i=1}^{n} u_i\!\left(I(u_i,s_i,L)\right). \tag{3}$$

The change of level is now explicit. Equation (1) takes a solution as its second argument. In the outer loop, that argument becomes the source string of the current improver, and the utility becomes $\widehat U$:

$$I_t=I_{t-1}\!\left(\widehat U,I_{t-1},L\right). \tag{4}$$

The two occurrences of $I_{t-1}$ have different jobs. The first is an executing search procedure. The second is the code being edited. A model-generated candidate $J$ is useful only if running $J$ on downstream tasks yields better solutions, not because its own prose explains optimization convincingly.

STOP exposes a general description of the meta-utility while withholding the detailed downstream tasks during meta-level generation. This gray-box arrangement encourages improvements to the search method rather than direct specialization to a visible task definition. It does not make the objective unknowable. Repeated scores still leak information about what the evaluator rewards, and the search can adapt to that feedback.

There is no monotonicity theorem hidden in Equation (4). To obtain even an empirical non-decrease, the candidate set would need to retain the incumbent and compare it fairly. To obtain a population non-decrease, the evaluation error would also need control. Recursion changes who proposes the next candidate; it does not remove the need for trustworthy selection.

4. The seed: generate several candidates and measure them

The starting improver is deliberately simple. It describes the utility and initial code to the model, requests nontrivial improvements, evaluates the resulting candidates, and returns the highest-scoring one. Figure 2 of the paper uses a sampling temperature of 0.7. The following pseudocode abstracts that seed without reproducing its prompt or implementation.

Algorithm 1. Seed candidate improver

Input: utility $u$, initial program $s$, language model $L$, generation budget $B$.

1. Describe the objective and initial program to L.
2. Generate a batch of candidate programs within budget B.
3. Evaluate each candidate using u.
4. Select a candidate with the largest observed utility.
5. Return that candidate as the proposed improved program.

The seed makes an important design choice: quality is ultimately determined by measured utility, not by the model’s self-rating. The model supplies a proposal distribution over programs. The evaluator decides which proposal survives. This separation allows the optimizer to benefit from a model that occasionally generates excellent code even when it cannot reliably recognize its best answer.

To see why multiple candidates can help, consider an illustrative model. Suppose each independently sampled candidate has probability $p$ of satisfying a chosen success criterion. The chance of no success in $k$ samples is $(1-p)^k$. Taking the complement gives

$$P(\text{at least one success})=1-(1-p)^k. \tag{5}$$

The incremental gain from one more independent sample is $p(1-p)^k$, so returns diminish. Figure 2 plots this elementary relationship; these are not measured STOP success rates. Real proposals are correlated, “success” may be graded rather than binary, and a noisy evaluator may not identify the successful candidate even when one exists.

Figure 2. Analytical best-of-$k$ curves under independent sampling and perfect identification of success. The figure explains why sampling can help and why it eventually saturates; it does not estimate STOP’s empirical behavior.

A stronger improver can change much more than $k$. It can use feedback to shift the proposal distribution, preserve several distinct promising programs, or spend additional evaluation on uncertain candidates. Conversely, a larger candidate pool can make noisy selection worse: the maximum observed score becomes increasingly likely to include a favorable error. More search is useful only when proposal quality and evaluation reliability improve together.

5. Closing the recursive loop

At the meta-level, the seed’s task is to rewrite itself. A candidate improver must be run on downstream problems before its meta-score can be computed. This creates a nested expense: one outer candidate can trigger many inner model calls and many utility evaluations.

Algorithm 2. STOP outer loop with empirical meta-utility

Input: seed improver $I_0$, downstream sample $\{(u_i,s_i)\}_{i=1}^{n}$, fixed model $L$, number of rounds $T$.

1. Define MetaScore(J):
2.     For each downstream task i, run J(u_i, s_i, L).
3.     Score the returned solution with u_i.
4.     Return the average score across the sampled tasks.
5. Set current = I_0.
6. For t = 1, ..., T:
7.     proposed = current(MetaScore, source(current), L).
8.     Set current = the proposed improver.
9. Return current.

This is an abstraction of the paper’s recursion, not an additional acceptance algorithm. In particular, line 8 does not assert an independent validation test. An incumbent-retention rule would be an extra design decision and would need its own budget accounting.

The recursive mechanism differs from asking a fixed editor to improve many programs. In a fixed-editor system, the proposal procedure stays constant while the edited artifact changes. Here, a successful revision can also change how later revisions are generated. A new selection rule can influence both future downstream solutions and future search-procedure designs.

That feedback creates path dependence. If the first useful change encourages diversity, later rounds may discover strategies that a narrow initial search would miss. If an early change breaks resource accounting or makes the prompt brittle, later rounds inherit that weakness. Averages over complete search chains are therefore more informative than a collection of attractive individual programs.

The gray-box objective is a reasonable compromise. Fully exposing downstream tasks would make task-specific shortcuts easier. Hiding everything except a scalar score would remove much of the model’s ability to reason about a program’s purpose. STOP gives a description of the general evaluation role while keeping task-specific details out of the meta-level prompt. The resulting inductive bias is toward reusable search logic, although empirical transfer still has to be measured.

6. Why parity with noise is a useful test bed

The main downstream problem is learning parity with noise. A hidden subset of input bits determines a label through XOR. For a binary vector $x$ and hidden binary mask $a$, a clean label can be written as

$$y=\left(\sum_{j=1}^{d}a_jx_j\right)\bmod 2, \qquad \widetilde y=y\oplus\eta. \tag{6}$$

Here $\eta$ is a label-flip variable. An algorithm must infer the hidden rule from examples rather than simply memorize every observed label. Figure 3 uses four bits to make the rule visible; the paper’s main experiments use ten-bit inputs.

Figure 3. A four-bit teaching example, not experimental data. The clean rule depends on bits 1 and 3. One observed label is deliberately flipped to illustrate why fitting every training label is not the same as recovering the underlying rule.

The paper describes a 30% prior probability for each bit to belong to the hidden subset, with 100 training and 20 test examples per instance. Its illustrative task in Appendix F uses 5% training-label noise. The main evaluation averages over 20 independently generated LPN instances, while the meta-objective uses repeated copies of the task to average stochastic improver behavior. Held-out evaluation uses 50 newly sampled LPN instances.

One detail needs care: the illustrative utility description in Appendix F contains a shorter time limit and a different example test count. Appendix K clarifies that these descriptions are not exact statements of the runtime settings and reports a two-second timeout for the LPN experiment. Reading the example as the definitive experimental configuration would produce the wrong interpretation.

This test bed is attractive because utility is cheap enough to measure repeatedly, the initial solver has room to improve, and candidate algorithms can trade sophistication against runtime. It is also narrow. Ten-bit synthetic parity does not represent the long feedback delays, changing dependencies, or ambiguous correctness criteria of a production coding task. Success here supports the feasibility of the mechanism; it does not settle its usefulness across arbitrary software work.

7. Main results: capability is a prerequisite

The main GPT-4 experiment runs five independent self-improvement chains for four rounds. Figure 4 of the paper shows average held-out meta-utility rising from roughly 61% to roughly 72% before flattening. These approximate values are read from the plotted trajectory, not reconstructed raw measurements. The shaded bands are one standard error.

Figure 4. Original Figure 4 from STOP, arXiv:2310.02304v3. GPT-4 improves on average; GPT-3.5 and Mixtral decline on average. The vertical scales differ across panels, so compare numerical axes rather than slopes alone.

The model versions matter. The experiments use GPT-4-0314, GPT-3.5-turbo-0613, and Mixtral-8x7B-Instruct-v0.1. These results establish behavior for those models under the reported setup. They do not supply a ranking of current models or a timeless parameter threshold for self-improvement.

For the two weaker models, the paper runs 25 chains. Both average trajectories deteriorate. Only 12% of GPT-3.5 chains achieve the paper’s stated improvement of at least 3%. The existence of occasional successful chains therefore coexists with a negative average result. Selecting only those successes would reverse the practical conclusion a user should draw from the experiment.

The seed’s initial GPT-3.5 score is higher than GPT-4’s in this setting. The paper attributes this to GPT-4 proposing more sophisticated but slower solutions, which interact badly with the short execution limit. This is a useful reminder that a capable model can lose under a particular runtime objective even when its algorithmic suggestions are more elaborate. “Better reasoning” and “better utility under this budget” are not interchangeable.

The paper also compares a chain-of-thought baseline and a fixed greedy iterative improver. It reports 57.7% ± 3.0% for chain-of-thought when considering non-error runs, but 49.6% ± 3.5% when errors are included. The greedy iterative baseline reaches 64.2% ± 0.9%. The error-inclusive result is the relevant view when a system must return a usable solution on every request. Conditional success tells us about answer quality after a successful run; it does not describe end-to-end reliability.

These comparisons show room beyond a simple fixed loop. They do not isolate a single reason for the gain. Recursive editing can simultaneously change prompting, sampling, caching, candidate selection, and effective compute allocation. Without matched ablations, it is premature to attribute the full improvement specifically to recursion rather than to the useful search mechanisms discovered during recursion.

8. Transfer: useful evidence with a selection caveat

The paper takes a better-performing evolved improver from the LPN search and applies it to five other tasks without further meta-training. This is stronger evidence than improvement only on the task family used for search. It asks whether the scaffold has learned a reusable way to use model calls.

Task	Initial solution	Seed improver	Selected evolved improver
String-grid distance	43.9	44.3	56.7
Modified quadratic assignment	20.4	20.6	22.1
3-SAT	0.0	21.2	75.1
Max-Cut	0.0	58.7	74.2
Parity without noise	50.0	59.3	81.7

The entries are the task-specific utility percentages in STOP Table 1. They are not five measurements of the same accuracy definition.

Figure 5. Table 1 redrawn with all three conditions. The selected evolved improver is stronger than the seed on each listed task. Different utility definitions prevent treating the bars as a common cross-task accuracy scale.

Relative to the seed, the gains are 12.4, 1.5, 53.9, 15.5, and 22.4 percentage points on their respective utility scales. Those differences are simple arithmetic from the table, not new experiments. Averaging them into a single “overall improvement” would obscure the different task meanings and baseline difficulties.

Three qualifications matter. First, this is a selected improver, not the average transfer performance of all self-improvement chains. The selection itself is part of the experimental pipeline. Second, the table does not provide uncertainty intervals for these entries. Third, all five tasks have synthetic, executable scoring rules. The results do not demonstrate transfer to open-ended repository maintenance or tasks with subjective feedback.

Appendix H gives the selected transfer improver. At a conceptual level, it samples with several temperatures, avoids exact duplicate candidates, caches evaluations, and keeps the best-scoring result. These are plausible reusable improvements. But the paper does not provide an ablation proving that any one component explains the transfer gains.

Algorithm 3. Conceptual structure of the selected transfer improver

This is a reading abstraction of Appendix H, not a new implementation.

1. Prepare candidate-generation requests at several temperatures.
2. Generate candidate programs while resources remain.
3. Remove exact duplicate program strings.
4. Evaluate candidates not already present in the score cache.
5. Track the candidate with the best observed utility.
6. Return the best candidate found.

Exact deduplication saves repeated work, but two different strings can implement the same algorithm. Conversely, caching a stochastic score can preserve a lucky measurement indefinitely. A cache is computationally useful without necessarily being statistically sufficient.

9. What kinds of strategies emerged?

The paper and Appendix B contain examples resembling beam search, simulated annealing, genetic search, multi-armed bandits, component-wise editing, and temperature schedules. A model can use its semantic understanding to propose a meaningful mutation rather than randomly changing tokens. It can also suggest combining portions of promising candidates.

These examples should not all be read as successful discoveries in the final experiment. Some are proposed candidates rather than selected improvements; some come from earlier versions of the seed setup. The ten-iteration example in Figure 5 is also distinct from the four-round improver used for the transfer table. Combining all attractive snippets into one fictional “final STOP algorithm” would exaggerate what was measured.

The search mechanisms nevertheless reveal useful design dimensions:

• Breadth: keep several strong candidates instead of committing immediately to one.
• Exploration: vary prompts or sampling temperature to reach different program families.
• Exploitation: refine a promising candidate using measured feedback.
• Resource control: stop after stagnation, avoid duplicate evaluations, or allocate calls adaptively.

Two temperatures must be kept conceptually separate. A language-model sampling temperature changes the probability of tokens. An annealing temperature changes whether a search process accepts a worse candidate. For a utility difference $\Delta=u_{\mathrm{new}}-u_{\mathrm{old}}$, a conventional annealing rule is

$$P(\text{accept})=\min\{1,\exp(\Delta/T_a)\}. \tag{7}$$

At high $T_a$, a negative $\Delta$ is less strongly penalized. By contrast, changing the model’s temperature changes which candidates are proposed before scoring. The two controls can work together, but one is not an implementation of the other. Equation (7) explains the general heuristic; it is not a claim that the transfer result isolates this rule.

The examples are most persuasive as evidence that natural-language models can search over familiar algorithmic structures. They are weaker evidence for algorithmic novelty. Familiar heuristics may already be represented in training data, and the paper does not establish that the generated variants are historically new.

10. A generalization bound, derived carefully

Appendix A asks whether optimizing empirical meta-utility can generalize. A finite program class gives a simple route. Suppose every improver has at most $l$ symbols from an alphabet of size $q\geq2$, and each independent evaluation contributes a value in $[0,1]$. The number of possible strings, including the empty string, is

$$N_l=\sum_{j=0}^{l}q^j =\frac{q^{l+1}-1}{q-1}. \tag{8}$$

Not every string is a valid program. Counting all of them is conservative: it bounds the number of valid improvers from above. Invalid programs can be assigned the designated failure utility. This is a counting argument, not a claim that the search visits all strings.

For a fixed improver $I$, Hoeffding’s inequality gives

$$P\!\left(\left|\widehat U(I)-U(I)\right|\geq\epsilon\right) \leq 2\exp(-2n\epsilon^2). \tag{9}$$

Now apply a union bound across the entire class. The chance that any candidate has a deviation larger than $\epsilon$ is at most

$$P\!\left(\sup_I|\widehat U(I)-U(I)|\geq\epsilon\right) \leq 2N_l\exp(-2n\epsilon^2). \tag{10}$$

Setting the right-hand side equal to a failure probability $\delta$ and solving for $\epsilon$ yields

$$\epsilon= \sqrt{\frac{\log(2N_l/\delta)}{2n}}. \tag{11}$$

This is an independently re-derived standard finite-class bound. It expresses the same counting-and-concentration idea as the appendix. The printed proof contains an intermediate exponential substitution whose reciprocal factors do not match the preceding expression; Equations (9)-(11) avoid carrying that algebraic error forward.

The uniform statement matters because the selected candidate depends on the observed scores. A confidence interval for one candidate chosen in advance does not automatically apply to the winner of a large adaptive search. A bound that holds simultaneously for the whole pre-specified class can cover selection within that class.

Let $I^*$ maximize true utility and $\widehat I$ maximize empirical utility over the class. On the uniform-concentration event:

$$\begin{aligned} U(I^*)&\leq \widehat U(I^*)+\epsilon\\ &\leq \widehat U(\widehat I)+\epsilon\\ &\leq U(\widehat I)+2\epsilon. \end{aligned} \tag{12}$$

The first and third steps use concentration. The middle step uses empirical optimality. STOP’s search is not guaranteed to find that empirical optimum. If it returns a candidate within $\eta$ of the best empirical score, the same argument gives a $2\epsilon+\eta$ bound. A statistical guarantee and an optimization guarantee are separate obligations.

Figure 6. Equation (11) evaluated analytically for an alphabet of 128 symbols and $\delta=0.05$. The three curves use different maximum program lengths. Bounds above 1 are uninformative for a utility confined to $[0,1]$; these are not empirical confidence intervals for STOP.

The practical limitation is severe. Long programs imply enormous classes. A small collection of meta-training tasks can make the bound vacuous. The plot makes this visible instead of treating the existence of a theorem as evidence of a tight experimental guarantee.

Independence also needs the right unit. Five repetitions of a stochastic improver on one task do not create five independent draws from a broad task distribution. They can reduce uncertainty about generation on that task while leaving uncertainty about other task families untouched. The theorem clarifies what would need to be controlled; the reported experiments provide a much narrower empirical check.

11. Random stopping defines recursion; it does not certify progress

Appendix A also considers alternative mathematical formulations, including maximizers that need not take an initial solution and a recursively defined objective with random stopping. These arguments help describe self-reference without requiring an infinite computation.

Suppose a recursive evaluation stops at each level with probability $\lambda$, independently of earlier stopping decisions. Let $K$ count calls, including the final one. To stop at call $k$, the process must continue $k-1$ times and then stop:

$$P(K=k)=\lambda(1-\lambda)^{k-1}, \qquad k\geq1. \tag{13}$$

The expectation can be derived without summing the series. There is always one call; with probability $1-\lambda$, the process starts an equivalent continuation:

$$\mathbb{E}[K]=1+(1-\lambda)\mathbb{E}[K] \quad\Longrightarrow\quad \mathbb{E}[K]=\frac{1}{\lambda}. \tag{14}$$

Figure 7. Geometric stopping-depth distributions for three stopping probabilities. This is an illustration of the appendix’s mathematical device, separate from the finite four-round empirical experiment.

Finite expected depth makes an objective more tractable to define. It does not imply that successive programs improve, that the process reaches an optimal fixed point, or that its execution cost is small. A fixed point can be an optimizer that repeatedly returns itself and performs poorly. If each call has widely varying cost, controlling the number of calls is also weaker than controlling total runtime.

This separation is useful throughout the paper. One can have a well-defined recursive objective, a statistically meaningful empirical estimator, and an ineffective search procedure. Establishing one property does not establish the others.

12. Compute accounting and the cost of reuse

Appendix K estimates approximately 3,000 GPT-4 calls per iteration per run. Multiplying by four iterations gives roughly 12,000 calls per chain; multiplying again by five chains gives roughly 60,000. These are arithmetic estimates from the reported rate, not independently metered totals. They do not establish token counts, dollar costs, or elapsed time.

The nested structure explains why expense grows quickly. If an outer round evaluates $m$ candidate improvers, each candidate is tested on $n$ tasks, and each downstream run uses $b$ model calls, the leading call count scales like

$$C_{\mathrm{calls}}\approx Tmnb, \tag{15}$$

before accounting for outer proposal calls, retries, caching, or early stopping. This expression is a budget model, not a fitted description of the paper. A small increase in the apparent thoroughness of each layer can multiply total cost.

Reuse is the economic justification for paying that cost. If meta-search costs $C_{\mathrm{meta}}$, a learned improver costs $C_{\mathrm{use}}$ per deployment, and it is reused $N$ times, the amortized cost per use is

$$\overline C(N)=\frac{C_{\mathrm{meta}}}{N}+C_{\mathrm{use}}. \tag{16}$$

Against a baseline costing $C_{\mathrm{base}}>C_{\mathrm{use}}$, cost break-even requires $N>C_{\mathrm{meta}}/(C_{\mathrm{base}}-C_{\mathrm{use}})$. This comparison assumes comparable quality and workload. If the learned improver is more expensive per use, its justification must instead come from better outcomes.

Figure 8. A hypothetical example with meta-search cost 1,000 units, learned per-use cost 1, and baseline per-use cost 2. Break-even occurs at 1,000 uses. The assumed units are illustrative and are not STOP’s measured costs.

Transfer results are therefore relevant to efficiency as well as generalization. A scaffold useful only for one narrow problem may never amortize its search cost. A reusable scaffold can distribute that cost across many tasks, but only if its advantage survives changes in model, task distribution, and resource limits.

13. What the sandbox experiment actually measures

The paper separately studies generated text that attempts to disable an evaluation constraint. It uses 10,000 independent generation attempts in each condition and checks the resulting text syntactically. Those candidate programs are not executed in this experiment. Accordingly, the result is a frequency of flagged generations, not a measured rate of successful sandbox escapes.

Model condition	Flagged text	Reported 95% Wilson interval
GPT-4	0.42%	0.31%-0.57%
GPT-4 with warning	0.46%	0.35%-0.61%
GPT-3.5	0.12%	0.07%-0.21%
GPT-3.5 with warning	0.17%	0.11%-0.27%

Figure 9. STOP Table 2 redrawn with its reported Wilson intervals. The programs were checked as text rather than executed. Adding the warning did not produce a statistically significant difference at the paper’s stated 0.05 threshold.

The narrow conclusion is still relevant: the editable proposal process can produce programs that conflict with evaluation constraints. It would be incorrect to infer intent, successful compromise, or a general failure probability for deployed systems from these counts.

For optimization, the broader issue is whether the measured score continues to mean what it is supposed to mean. The appendix describes a malformed-output case that produced an impossible apparent accuracy above 1,000% under an earlier evaluator. A high number is not evidence of task progress when the scoring contract has broken.

The methodological response is to keep the measurement definition separate from the artifact being optimized, give malformed outputs explicit failure semantics, and distinguish task success from evaluation failure. These are properties of a credible experiment. They do not require interpreting every odd generated string as an adversarial strategy.

14. Limitations

A narrow empirical distribution. The main meta-training task is synthetic LPN, and transfer covers five compact, executable problems. The work does not establish performance on long-running software tasks, delayed feedback, changing environments, or objectives that require human judgment.

Selection and uncertainty. The transfer improver is chosen from the stronger LPN outcomes. Table 1 does not report the transfer distribution over all chains or uncertainty for each entry. A selected successful scaffold demonstrates possibility; it does not estimate how reliably a new run will produce one.

No guaranteed improvement. Candidate programs can fail, selection can exploit noise, and later rounds can inherit earlier mistakes. The weaker-model experiments directly show average degradation. Calling the program an improver does not remove this failure mode.

Compute is not fully disentangled. Better results may come from different prompts, more effective sampling, caching, or a different distribution of calls. The paper does not isolate recursion under a complete matched-compute comparison across all these choices.

The formal class is much larger than the data support. A bound over all bounded-length strings is mathematically convenient but often loose. It says little about the small subset the model actually proposes unless additional structure is introduced.

Descriptions and measurements can diverge. The appendix explicitly distinguishes illustrative utility descriptions from actual experimental settings. A model can optimize what a description suggests while being evaluated under a different constraint. Gray-box access reduces direct task exposure but also removes details that may be necessary for good resource decisions.

Mutable control logic has its own reliability burden. Exact-score caching, output parsing, stopping rules, and candidate retention are all possible optimization targets. A change that looks locally efficient can reduce robustness or distort the meaning of repeated evaluation.

15. Critical analysis: the evaluator is the scarce resource

My reading is that STOP’s most durable contribution is the choice of optimization object. The scaffold is a reusable artifact with enough structure to encode search strategies, yet it is compact enough for a language model to rewrite. The experiments show that this object can improve under a fixed model. The central unresolved question is whether the evaluator can distinguish reusable progress from noise and specialization as the search becomes more adaptive.

15.1 Separate recursive search from a good discovered heuristic

A decisive comparison would hold downstream resources fixed and vary two factors: whether the editor itself changes, and whether the best validated incumbent is retained. This yields four conditions rather than one ambiguous comparison between a simple seed and an evolved system.

The fixed-editor condition should be allowed to search over the same scaffold space. Otherwise, a gain could come from access to richer candidate structures rather than from changing the editor. The retention condition should use a separate validation sample, and its evaluation cost should count toward the same overall budget.

This proposed experiment would answer a causal question: does recursive control add value beyond searching for a strong static controller? It is not an experiment reported in STOP.

15.2 Evaluate every chain, not just the attractive artifact

The right deployment unit is a complete meta-search run. A run consumes resources and returns one scaffold, possibly a bad one. Reporting only the strongest selected scaffold leaves out the probability of obtaining it.

A stronger study would show held-out transfer for every independent chain, retain failed runs with their designated failure utility, and report the distribution of quality and cost. Paired random seeds can reduce variance when comparing controllers, but final evaluation still needs independent samples to avoid turning the comparison seeds into another training set.

15.3 Split at the level where generalization is claimed

Holding out LPN instances tests generalization within LPN. Holding out a different combinatorial task tests a broader property. Holding out an entire domain, evaluator style, or model backend tests something broader still.

If the intended artifact is a general coding harness, the split should include unseen task families and different resource regimes. A controller that wins only at one short timeout may encode a useful local heuristic without learning a robust allocation strategy. Reporting quality-versus-cost curves would reveal that distinction more clearly than one scalar score.

15.4 Let noise influence allocation, not just selection

When two candidates have similar observed scores, spending another model call on a new candidate is not always the best use of resources. Re-evaluating an uncertain incumbent may change the selection decision more. A cache that stores only one score misses this opportunity.

A useful next design would track repeated measurements and uncertainty while preserving a fixed external test set. That proposal concerns measurement quality, not an assumption that a more complicated optimizer is automatically better. Its benefit must be measured against the extra evaluations it consumes.

15.5 Avoid turning self-reference into the success criterion

The equation that calls an improver on itself is elegant, but self-reference is not the outcome users need. They need a higher probability of obtaining a good solution under a known budget. A recursive system that rediscovers a beam-search controller may be valuable even if it creates no novel algorithm. A system with impressive self-modification traces may be unhelpful if its failure rate or cost is too high.

The practical standard should therefore be reusable quality per resource, with transparent selection and failure accounting. STOP makes that research direction concrete; it does not close the evaluation problem.

16. Conclusion

STOP turns a code improver into an editable search target. Its two-level objective explains how a fixed language model can participate in recursive scaffold improvement without changing its own weights. The GPT-4 experiments and selected transfer results provide evidence that this can produce useful search procedures.

The negative results are equally informative. Weaker models often damage the loop, empirical winners can exploit evaluation noise, and the finite-class theory does not guarantee that the search finds a good program. The strongest takeaway is not that recursion guarantees progress. It is that the surrounding optimizer is a meaningful artifact to search over-and that trustworthy evaluation becomes more important as that artifact gains control over its own search.

References and figure provenance

1. Eric Zelikman, Eliana Lorch, Lester Mackey, Adam Tauman Kalai. Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation. Version 3, August 16, 2024. Main method: Sections 3-4; experiments: Sections 5-6; theory: Appendix A; strategies: Appendix B; task descriptions: Appendices F-G; selected transfer improver: Appendix H; experimental details: Appendix K.
2. STOP full paper, 46 pages. Review Figures 1 and 4 reproduce cropped paper Figures 3 and 4. Review Figures 5 and 9 redraw paper Tables 1 and 2. Figures 2, 3, 6, 7, and 8 are explanatory illustrations constructed for this review.
3. Official STOP resource repository, linked by the paper. This link is provided as a resource for readers.