RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments — part 4/5

Cleaned-text code points [45964, 60527). Bibliography omitted here; source.md includes it.

(1) ABProgramSimulation , (2) AddMultiple_Divisible_Counting , (3) AdditionTable , (4) AlmostCompleteGraphCycleCounting , (5) AndOr_Sequence_Counting , (6) AntiPalindromicSubstringCounting , (7) Axis_KCenter , (8) BAJBytecomputer , (9) BEZMinimalistSecurity , (10) BannedPointSupersetPathCounting , (11) BanyanHeart , (12) BezoutIdentity , (13) Binario , (14) Binario_NoAdjacencyRequirement , (15) BinaryAlternation , (16) BinaryLinearEquation_SolutionCounting , (17) BinaryTreeLeafNumExpectation , (18) BitAndZero_PathCounting , (19) BitEquationCounting , (20) BitwiseOperationSequenceCounting , (21) BlockImage , (22) BoundedAdjacencyDifference_Permutation_Counting , (23) BoundedIntervalIntersection , (24) BoundedMeanSubarrayCounting , (25) BoundedSubarrayCounting , (26) BoxScheduling , (27) Bridge , (28) BubbleSwapLowerBound_PermutationCounting , (29) BucketSorting , (30) CRT , (31) CampfireParty , (32) CampsitePuzzle , (33) Canon , (34) CantorExpansion , (35) CapitalCityEffect , (36) CardColoringCounting , (37) CatalanNumberMod , (38) CheckAllCycleXorZero , (39) ChoHamsters , (40) Cinema , (41) Circuit , (42) CirculatingDecimalCounting , (43) CirculatingGrid , (44) CleaningUp , (45) ClearSymmetry , (46) Clique_IndependentSet_Partitioning_Counting , (47) CoinSquareGame , (48) ColoringCounting , (49) CombinationOddSubsequenceCounting , (50) ConcatenationPartitionCountingSum , (51) CongruentEquation , (52) ConstructHackInterval , (53) ConvexHull , (54) Cornfield , (55) CountdownClose , (56) CountdownEqual , (57) CowDanceShow , (58) Cryptarithmetic , (59) Cube_FixedLocalMaximumCounting , (60) CycleCounting , (61) DecreasingDigitCounting , (62) DegreeFixed_SpanningTree , (63) DeltaMinPopcount , (64) DeltaNimGame , (65) DerangementExtension , (66) DifferenceConstraintSystem , (67) DifferenceConstraintSystemDAG , (68) DifferentColorPairing , (69) Differentiate , (70) DigitLISCounting , (71) DiscreteLogarithm , (72) Disinfection , (73) DistinctArrayPermutation , (74) DistinctEdgeColoredCompleteGraphCounting , (75) Division , (76) DivisorFlipExpectation , (77) DoubleCrossCounting , (78) DoublePalindromicStringCounting , (79) DoubleStackSorting , (80) DynDynamite , (81) EightDigitPuzzle , (82) EmperorWorries , (83) EnergyStorageMeter , (84) EuclidGame , (85) EvenDegreeGraphPartitioning , (86) Expression_AddingParenthese_Counting , (87) FBI_BinaryTree , (88) FaceRightWay , (89) FactorialTrailingZeroCount , (90) Fibonacci , (91) FibonacciContainingCounting , (92) Fibtrain , (93) FireworkShow , (94) FixedModK_Selection_Counting , (95) FixedOneEdgeNum_SpanningTree , (96) FractionalProgramming , (97) FractionalProgramming_BipartiteGraphMatching , (98) FutoshikiPuzzle , (99) GCDFibonacciProduct , (100) GCDOne_Counting , (101) GCDPrime_Counting , (102) GasFireExtinguishers , (103) GaussianElimination , (104) GcdLcmCounting , (105) GoldWashing , (106) GraMinimaGame , (107) GradeRankingCounting , (108) GraphContainTreeCounting , (109) GraphIsomorphism , (110) GridBFS , (111) GridColoringCounting , (112) GridComponent , (113) GridLocalMinimumCounting , (114) GridParityConstruction , (115) GridTriangleCounting , (116) HURWarehouseStore , (117) HalvingChainCounting , (118) HamiltonianPath , (119) HamiltonianPathExistence , (120) HeapCounting , (121) HitoriPuzzle , (122) HungryRabbit , (123) ImpParty , (124) IndividualSumBounded_SequenceCounting , (125) IntegerFactorizationCounting , (126) IntegerProgramming , (127) Integral , (128) InversionPair , (129) InversionPairK_Counting , (130) Josephus , (131) JugPuzzle , (132) KPartition , (133) KUR , (134) Kakurasu , (135) KiddingMe , (136) KingSorting , (137) KloBlocks , (138) Knapsack , (139) KnightsAndKnaves , (140) KosDicing , (141) KthSubsequence , (142) Kth_BinaryTree , (143) Kth_SemiBalancedBracketSequence , (144) LAS , (145) LASLaser , (146) LCM , (147) LDSTwo_Counting , (148) LIS_LDS_Concatenation , (149) LIZ_Lollipop , (150) LampChanging , (151) LandAcquisition , (152) LandformGenerationCounting , (153) LargestConvexPolygon , (154) LargestRectangle_AmongPoints , (155) LightUpPuzzle , (156) LinkBeads , (157) LongestMaxDiffBoundedInterval , (158) LongestPath , (159) Longest_DoublePalindrome , (160) Longest_MatchingSubsequence , (161) Longest_RepeatedPalindrome , (162) MYJ , (163) MafMafia , (164) MagicSquarePuzzle , (165) MakingGrade , (166) MatrixPermutationEquivalence , (167) MatrixPermutation_BothDiagonalOne , (168) MatrixPermutation_MainDiagonalOne , (169) MatrixPooling , (170) MatrixRMQCounting , (171) Matrix_BinaryExponentiation , (172) MaxDifferentGroupPairDivision , (173) MaxGridPathIntersection , (174) MaxMinimum_AfterIntervalAddition , (175) MaxMultSplit , (176) MaxMultiplicationFixedSum , (177) MaxNoConflictingBombs , (178) MaxPermutation , (179) MaxRMQExpectation , (180) MaxSegmentCoverageConstraint , (181) MaxSumLDS , (182) MaxThreeSquareSum , (183) MaxTreeXorPath , (184) MaxTree_KPathCoverage , (185) MaxWeightPalindromicSubstring , (186) MaxXorPath , (187) MaxXorSet , (188) Max_NonAdjacent_KElementSum , (189) Max_TreeConstrainedPermutation_Weight , (190) MaximumAchromaticNumber , (191) MaximumClique , (192) MaximumDivisor , (193) MaximumIndependentSetGrid , (194) MaximumLexicographicalOrderSubsequence , (195) MaximumPointSegmentMatching , (196) MaximumWeightMatching , (197) Maximum_IndependentSet_Tree , (198) Maximum_SubsequenceNum , (199) Maze , (200) MinConversionToCycleCost , (201) MinCostReducingLNDS , (202) MinCostTreeCoverage , (203) MinCubeAssignment , (204) MinDivisionSumXor , (205) MinInorderBinaryTree , (206) MinKDivisorNumber , (207) MinNoSolutionLinearDiophantineEquation , (208) MinNonsubstring , (209) MinPairSumMultiplicationPermutation , (210) MinPathCover_DAG , (211) MinSumChebyshevDistance , (212) MinSumDistanceSquare , (213) MinSumPreXor , (214) MinSwapTwoPermutations , (215) MinXorPair , (216) Minesweeping , (217) MinimalCyclicShift , (218) MinimumChromaticNumber , (219) MinimumChromaticNumber_SegmentOverlap , (220) MinimumCost_MaximumFlow , (221) MinimumDirectedSpanningTree , (222) MinimumFibonacciRepresentation , (223) MinimumHarmoniousChromaticNumber , (224) MinimumIntervalCoverage , (225) MinimumRatioPath , (226) MinimumSpanningTree , (227) MinimumSpanningTreeCounting , (228) MinimumSteinerTree , (229) MinimumSumDifferenceSubmatrix , (230) MinimumTreeWeightedDominatingAncestor , (231) MinimumUnconflictedGridKMax , (232) MinimumWeightedSpanningTree , (233) Minimum_CrossingEdges_GraphPartition , (234) Minimum_DominatingInterval , (235) Minimum_DominatingSet , (236) Minimum_DominatingSet_Grid , (237) Minimum_MaxAbsSlicer , (238) Minimum_MaxSlicer , (239) Minimum_VertexCover , (240) MitterTransportation , (241) MixedGraphEulerianCircuit , (242) MoneyChargingGame , (243) MonochromeBlockCounting , (244) MonotonicStack , (245) MostComponentTreeRemovingTwoPaths , (246) MostNumEdge_NonSelfIsomorphism , (247) MultiDrink , (248) MultipleFlippingGame , (249) Multiplication , (250) NANDResultCounting , (251) NegativeBase , (252) NewNimGame , (253) NextPalindromic , (254) NinePuzzle , (255) NoAdjacentGirlCounting , (256) NoDoubleTripleCounting , (257) NotContainingStringCounting , (258) NumberPartitionCounting , (259) Numbrix , (260) ODLDistance , (261) OddVisitation , (262) PCPPermutation , (263) POLPolarization , (264) PairMoreOneCounting , (265) PalembangBridges , (266) PalindromePartitionCounting , (267) PalindromicSubstringNumberCounting , (268) PanSolarPanels , (269) Path_NoGoingBack_Counting , (270) Patrol , (271) PipelineArrangement , (272) PolyaModel , (273) PolynomialFactorization , (274) PolynomialInterpolation , (275) PolynomialMinimum , (276) PolynomialRemainder , (277) PowerCycle , (278) PowerNest , (279) PowerShortcut , (280) PrefixConcatenation , (281) PrefixProductMODDistinctPermutation , (282) PrefixSumMODDistinctPermutation , (283) Prefixuffix , (284) PreorderTraversal , (285) PrimeGraph_MinimumChromaticNumber , (286) ProtectingFlowers , (287) PythagoreanGraph_IndependentSetCounting , (288) QuadMagicItems , (289) QuadraticFunctionSegmentation , (290) QuantumLockPuzzle , (291) QueenPlacement , (292) RandomRangeMaxExpectation , (293) RangeConstrained_IncreasingSequence_Counting , (294) RangeFourSequenceConstruction , (295) RangeShrinkingSequenceCounting , (296) RecursiveFunction , (297) RecursiveSequenceSumConstruction , (298) RepeatSequenceLNDS , (299) RootExtraction , (300) RoundRobin , (301) RoundTableAssignment , (302) RoyalLockCounting , (303) SAT , (304) SCC_Sequence_Counting , (305) SLOElephants , (306) STUWell , (307) SaladBar , (308) SalesmanFatigue , (309) SameAdjacencyCounting , (310) SecretCowCode , (311) SegmentMinLengthEqual_Counting , (312) SegmentTreeSortingCounting , (313) SelfPowerSequenceMOD , (314) SetCover , (315) SetSplitting , (316) SharedSubstringCounting , (317) ShortestPath , (318) ShortestPathCountConstruction , (319) ShortestUnicolorSubstring , (320) SingingGirlStory , (321) SingleStackSorting , (322) SkaRockGarden , (323) SkyscraperPuzzle , (324) SkyscraperSumPuzzle , (325) SlidingWindow , (326) SmallestBinaryMultiple , (327) SmallestCircle , (328) Sorting , (329) SpiralMatrix , (330) SplittingGame , (331) SpyNetwork , (332) SquSquarks , (333) SquareUndamagedPointCounting , (334) StarBattle , (335) StirlingSecond , (336) StoneGame , (337) StoneIntervalsGame , (338) StringPartitionShuffle , (339) StringReversalConstruction , (340) StuntFlying , (341) SubarraySumXor , (342) SubarrayXorSum , (343) SubgraphIsomorphism , (344) SubmatrixSumDivisibleCounting , (345) SubsequenceReversalLNDS , (346) SubsetSum , (347) SubsetSumSequence , (348) Sudoku , (349) SumGCD , (350) SumGCDWithIndividual , (351) SumLCM , (352) SumMOD , (353) SumManhattan_CurvedSurface , (354) SumPHIInterval , (355) SumProductDivisorNum , (356) SumPseudoEuclidean , (357) SumSetMultiplication , (358) SumSpanningTreeGCD , (359) SumTriangleArea , (360) SumXorDivisorNum , (361) Sum_DivisorNum , (362) SurvoPuzzle , (363) TakingPrimeGame , (364) TaskArrangement , (365) TetrisAttack , (366) ThreeStringCommonSubsequenceCounting , (367) ThreeVertexCycleCounting , (368) TopologicalSort , (369) TopologicalSort_MinimalLexicographicalOrder , (370) Tournament_LongestPath , (371) TransmissionDelay , (372) TreeAddOneEdgeDiameter , (373) TreeCenter , (374) TreeChangeOneEdgeDiameter , (375) TreeColoring , (376) TreeDynamic_XORZeroPath , (377) TreeElimination_Expectation , (378) TreeEvenPartitioning , (379) TreeMaximumVisitedVertex , (380) TreeRandomWalkExpectation , (381) TreeTopologicalSequenceCounting , (382) Tree_DistanceEqualTriad_Counting , (383) TriumphalArch , (384) TwiddlePuzzle , (385) TwoSAT , (386) TwoSet_AllCoprime_Counting , (387) UndamagedSubmatrixCounting , (388) ValueDiminishingSelection , (389) Vertex_KCenter , (390) VirusSynthesis , (391) VisibleLine , (392) WIL , (393) WYC , (394) WYRLevelingGround , (395) WarehouseConstruction , (396) WeightedBinaryTree , (397) WeightedLIS , (398) WhackAMole , (399) XorEquationCounting , (400) ZeroPrefixSubsetCounting .

## Appendix C RL Training Details

We run our RL training using the slime framework 4 4 4 https://github.com/THUDM/slime and adopt the DAPO algorithm ( Yu et al., 2025 ) , a variant of GRPO ( Shao et al., 2024 ) . Unless otherwise specified, the setup described below is consistent across all runs.

During each rollout step, we employ oversampling combined with dynamic filtering to enable dynamic sampling; we use a training batch size of 128 and an oversampling batch size of 384. We enable the partial-rollout technique ( Kimi Team, 2025b ; GLM-4.5 Team, 2025 ; Zhou et al., 2025 ) , which caches unfinished generations from the current rollout step and subsequently resumes them. Each prompt produces 16 rollouts with a temperature of 1.0. We do not use KL regularization or entropy loss. The clipping range is [ 0.2 , 0.28 ] [0.2,0.28] . We adopt off-policy importance sampling ( Yao et al., 2025 ) to correct for distribution mismatch between the training and inference engines. We perform one parameter update after each rollout step. We use the Adam optimizer ( Kingma & Ba, 2015 ) with a constant learning rate schedule and weight decay of 0.01.

For R1-Distill-Qwen-1.5B , DeepScaleR-1.5B , ProRL-1.5B-v2 , and OpenThinker3-1.5B , we use a learning rate of 2 × 10 − 6 2\times 10^{-6} and a maximum rollout response length of 24,576 tokens. We use these models’ default chat templates and directly insert the input into the user prompt field, following these models’ prompt format for training.

For Qwen2.5-7B-Base , we use a learning rate of 1 × 10 − 6 1\times 10^{-6} and a maximum rollout response length of 8,192 tokens. We adopt a variant of the prompt format from DeepSeek-AI (2025) ; Pan et al. (2025) : ⬇

Every training run is conducted on a single node equipped with 8 × \times NVIDIA H100 (80GB) GPUs. The total training time of one single run varies depending on the number of training steps, the distribution of rollout response lengths, and the model size, ranging from approximately 2 to 8 days, which is equivalent to roughly 350 to 1,500 H100 GPU hours.

Details specific to RLVE are provided in Appendix A .

## Appendix D Evaluation Details

We conduct all evaluations using the SGLang framework ( Zheng et al., 2024 ) as the inference engine for model generation. We set the sampling temperature to 0.6 0.6 and the top- p p parameter to 0.95 0.95 . For R1-Distill-Qwen-1.5B , DeepScaleR-1.5B , ProRL-1.5B-v2 , and OpenThinker3-1.5B , the maximum response length during evaluation is 32,768 tokens; for Qwen2.5-7B-Base , the maximum response length is 16,384 tokens.

For AIME 2024/2025 ( Art of Problem Solving, 2025 ) , we sample 64 outputs per problem and report Avg@64 as the evaluation metric. For OMEGA-500 ( Sun et al., 2025 ) , OlympiadBench ( He et al., 2024 ) , and BBEH ( Kazemi et al., 2025 ) , we sample 4 outputs per problem and report Avg@4 as the evaluation metric. For LiveCodeBench ( Jain et al., 2025 ) , we sample 16 outputs per problem and report Pass@8 as the probability that at least one of 8 uniformly sampled outputs from the 16 outputs passes all test cases; we use the latest version (v6) of LiveCodeBench 5 5 5 https://huggingface.co/datasets/livecodebench/code_generation_lite/blob/main/test6.jsonl available at the time of this work.

To construct the held-out test set 𝒟 ood \mathcal{D}_{\text{ood}} introduced in Section 4 , we randomly sample 50 held-out test environments from the full suite of 400 verifiable environments from RLVE-Gym (listed in Appendix B.2 ), which are: