{"id":"arxiv:2402.01817","title":"LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks","date":"2024-02-02","section":"Reasoning and the \"physics\" of language models","quality":-0.200767,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/d4b5a3d7cb6b8ad1768f1c0a91042bc53b290daaf00e9972dff5b384f5996db3/","source_url":"https://arxiv.org/abs/2402.01817","text_available":true,"authors":["Subbarao Kambhampati","Karthik Valmeekam","Lin Guan","Mudit Verma"],"outgoing":[{"source":"arxiv:2402.01817","target":"doi:10.1038/s41586-023-06924-6","evidence":"title","id":"doi:10.1038/s41586-023-06924-6","title":"Mathematical discoveries from program search with large language models","url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/70a6b427d42fd28c9d538648e148a97db2fb2614c7764f3231e53f4bebdce887/"},{"source":"arxiv:2402.01817","target":"openreview:wUU-7XTL5XO","evidence":"title","id":"openreview:wUU-7XTL5XO","title":"PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change","url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/da39c89a33c7acabf5f152b7d25c596a91b3b44bb2dec90057cd739273338425/"}],"incoming":[],"text_parts":[{"start":0,"end":13662,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/d4b5a3d7cb6b8ad1768f1c0a91042bc53b290daaf00e9972dff5b384f5996db3/part-1.html"},{"start":13662,"end":29032,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/d4b5a3d7cb6b8ad1768f1c0a91042bc53b290daaf00e9972dff5b384f5996db3/part-2.html"},{"start":29032,"end":44038,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/d4b5a3d7cb6b8ad1768f1c0a91042bc53b290daaf00e9972dff5b384f5996db3/part-3.html"}],"contents":[{"title":"Position: LLMs Can’t Plan, But Can Help Planning in LLM-Modulo Frameworks","offset":0,"level":1,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/d4b5a3d7cb6b8ad1768f1c0a91042bc53b290daaf00e9972dff5b384f5996db3/part-1.html#text"},{"title":"Abstract","offset":77,"level":6,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/d4b5a3d7cb6b8ad1768f1c0a91042bc53b290daaf00e9972dff5b384f5996db3/part-1.html#text"},{"title":"Keywords:","offset":1144,"level":6,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/d4b5a3d7cb6b8ad1768f1c0a91042bc53b290daaf00e9972dff5b384f5996db3/part-1.html#text"},{"title":"1 Introduction","offset":1162,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/d4b5a3d7cb6b8ad1768f1c0a91042bc53b290daaf00e9972dff5b384f5996db3/part-1.html#text"},{"title":"2 Planning-centered Limitations of LLMs","offset":7176,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/d4b5a3d7cb6b8ad1768f1c0a91042bc53b290daaf00e9972dff5b384f5996db3/part-1.html#text"},{"title":"2.1 LLMs cannot generate executable plans in autonomous mode","offset":7473,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/d4b5a3d7cb6b8ad1768f1c0a91042bc53b290daaf00e9972dff5b384f5996db3/part-1.html#text"},{"title":"2.2 LLMs cannot verify plans and thus cannot improve by self-critiquing","offset":9784,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/d4b5a3d7cb6b8ad1768f1c0a91042bc53b290daaf00e9972dff5b384f5996db3/part-1.html#text"},{"title":"2.3 Analyzing Claims to the Contrary in the Literature","offset":13260,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/d4b5a3d7cb6b8ad1768f1c0a91042bc53b290daaf00e9972dff5b384f5996db3/part-1.html#text"},{"title":"3 LLM-Modulo Framework for Robust Planning","offset":20184,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/d4b5a3d7cb6b8ad1768f1c0a91042bc53b290daaf00e9972dff5b384f5996db3/part-2.html#text"},{"title":"3.1 Critics/Verifers","offset":23940,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/d4b5a3d7cb6b8ad1768f1c0a91042bc53b290daaf00e9972dff5b384f5996db3/part-2.html#text"},{"title":"3.2 Backprompt (Meta) Controller","offset":28049,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/d4b5a3d7cb6b8ad1768f1c0a91042bc53b290daaf00e9972dff5b384f5996db3/part-2.html#text"},{"title":"3.3 Specification Refinement & Critic/Model Acquisition","offset":28971,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/d4b5a3d7cb6b8ad1768f1c0a91042bc53b290daaf00e9972dff5b384f5996db3/part-2.html#text"},{"title":"3.4 Summary of LLM Roles in LLM-Modulo","offset":30533,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/d4b5a3d7cb6b8ad1768f1c0a91042bc53b290daaf00e9972dff5b384f5996db3/part-3.html#text"},{"title":"3.5 Can LLM-Modulo Frameworks Pay Their Way?","offset":31747,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/d4b5a3d7cb6b8ad1768f1c0a91042bc53b290daaf00e9972dff5b384f5996db3/part-3.html#text"},{"title":"4 Two Case Studies of LLM-Modulo","offset":33844,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/d4b5a3d7cb6b8ad1768f1c0a91042bc53b290daaf00e9972dff5b384f5996db3/part-3.html#text"},{"title":"5 Related Work","offset":36380,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/d4b5a3d7cb6b8ad1768f1c0a91042bc53b290daaf00e9972dff5b384f5996db3/part-3.html#text"},{"title":"6 Conclusion","offset":41087,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/d4b5a3d7cb6b8ad1768f1c0a91042bc53b290daaf00e9972dff5b384f5996db3/part-3.html#text"},{"title":"Impact Statement","offset":42511,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/d4b5a3d7cb6b8ad1768f1c0a91042bc53b290daaf00e9972dff5b384f5996db3/part-3.html#text"},{"title":"Acknowledgments","offset":43455,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/d4b5a3d7cb6b8ad1768f1c0a91042bc53b290daaf00e9972dff5b384f5996db3/part-3.html#text"}],"text_code_points":44038,"source_markdown":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/d4b5a3d7cb6b8ad1768f1c0a91042bc53b290daaf00e9972dff5b384f5996db3/source.md"}