Skip a Layer or Loop It? Learning Program-of-Layers in LLMs — part 4/4

Cleaned-text code points [46994, 53428). Bibliography omitted here; source.md includes it.

Across all models and difficulty levels, PoLar achieves strong in-distribution performance and generally outperforms standard inference and prior dynamic-depth baselines. In particular, increasing p ​ @ ​ k p@k leads to monotonic accuracy improvements for PoLar , demonstrating effective test-time computation scaling through execution-program exploration. At p ​ @ ​ 5 p@5 , PoLar achieves substantial absolute gains over Base (sampling), with improvements of +15.6 / +17.5 / +15.0 / +10.1 / +12.2 on Qwen1.5-MoE-A2.7B-Chat, +40.4 / +19.2 / +17.5 / +14.8 / +13.5 on Qwen2.5-3B-Instruct, and +14.1 / +23.4 / +15.5 / +10.6 / +9.9 on Qwen3-8B for DM-1 to DM-5, respectively.

Notably, methods that rely solely on layer skipping (e.g., ShortGPT, MindSkip, FlexiDepth) often suffer accuracy degradation, especially on harder difficulty levels. In contrast, PoLar jointly supports layer skipping and recurrence, allowing it to retain or improve accuracy while exploring diverse execution programs. Compared to DR.LLM, which performs layer-level routing, PoLar achieves stronger performance in most settings, particularly at larger p ​ @ ​ k p@k , indicating the benefit of structured, program-level execution prediction.

### C.2 Out-of-Distribution Generalization

We further evaluate the out-of-distribution (OOD) generalization of PoLar on benchmarks that differ substantially from DART-Math in both format and domain. Tables 8 , 9 , and 10 report pass@1 accuracy on ASDiv, MAWPS, and subject-wise subsets of MMLU-Pro using LLaMA-3.2-3B-Instruct, Qwen2.5-3B-Instruct, and Qwen3-8B, respectively.

Across all evaluated models, PoLar consistently improves over standard inference on arithmetic word problem benchmarks (ASDiv and MAWPS), indicating strong transfer from structured mathematical reasoning to natural language problem settings. On MMLU-Pro, which spans diverse domains including mathematics, natural sciences, social sciences, and humanities, PoLar achieves broad and consistent gains across most subject areas.

These results suggest that the execution programs learned by PoLar capture general, transferable computation control strategies rather than dataset-specific heuristics. Despite being trained on mathematical reasoning data, PoLar generalizes effectively to heterogeneous domains, highlighting the robustness of program-of-layers inference and its applicability beyond the original training distribution.

## Appendix D Empirical Details

### D.1 Dataset Details

All in-distribution experiments are conducted on DART-Math , a structured mathematical reasoning benchmark consisting of five difficulty levels, denoted as DM-1 to DM-5 .

We construct the in-distribution data from the DART-Math MATH pool. Since the released pool may contain multiple response records associated with the same underlying question, we deduplicate examples within each difficulty level by question text before splitting. We then adopt a difficulty-wise data split, where each difficulty level is split independently into training, validation, and test sets.

After deduplication, the five difficulty levels contain 565, 1,349, 1,579, 1,537, and 1,577 examples for DM-1 to DM-5, respectively. For each difficulty level, we use approximately 62.5% of the examples for training, 12.5% for validation, and 25% for testing.

Overall, this results in 4,130 training examples, 826 validation examples, and 1,651 test examples. All in-distribution results are reported on the held-out test sets corresponding to the same difficulty level used for training.

#### Revision note.

This arXiv version updates the DART-Math in-distribution evaluation protocol by removing duplicate questions prior to data splitting. We have rerun the affected in-distribution analyses, and the qualitative conclusions remain unchanged. The out-of-distribution (OOD) results are unaffected.

### D.2 Training Configuration

We train the PoLar prediction network using supervised learning on the training splits described above. All hyperparameters are selected via validation tuning.

#### Optimization.

We use the AdamW optimizer and tune the learning rate from {1e-4, 3e-4, 5e-4, 8e-4, 1e-3, 3e-3}, the batch size from {32, 128, 256}, and the number of training epochs from {3, 10} based on validation performance. We adopt a cosine learning rate schedule with linear warmup, and tune the number of warmup steps on the validation split together with the other hyperparameters.

### D.3 Predictor Size

The learned PoLar predictor is lightweight compared with the frozen base LLM. Across the evaluated models, the predictor contains approximately 2.1M parameters, corresponding to only 0.01%–0.06% of the base model size. This small footprint makes training and inference inexpensive relative to standard LLM fine-tuning or full-model execution.

### D.4 Direct Prompting

We adopt a direct prompting strategy throughout all experiments, without eliciting chain-of-thought or intermediate reasoning. The model is explicitly instructed to output only the final answer in a strictly formatted form.

Given a math problem instance question , the input prompt is constructed as follows: Solve the following math problem and output ONLY the final answer directly, formatted strictly as \boxed{ANSWER}. ### Problem Start {question} ### Problem End Answer:

This prompt design enforces concise answer generation and isolates the effect of latent execution programs from token-level reasoning strategies.

## Instructions for reporting errors

We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:

Click the "Report Issue" ( ) button, located in the page header.

Tip: You can select the relevant text first, to include it in your report.

Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.

Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .