This AI Paper Introduces LLM+P: The First Framework that Incorporates the Strengths of Classical Planners into LLMs

Researchers in AI have been working to develop systems that can talk in natural language with the same elegance and adaptability as people ever since the field’s inception. Even though very simple models, like Eliza from 1966, may provide replies to some plausible prompts, it has always been relatively simple to produce questions that reveal their shortcomings compared to people – their lack of actual “understanding.” Even though large language models (LLMs) like GPT-4 and ChatGPT significantly surpassed expectations from a few years ago, they are the same. The internet is flooded with people who take great pleasure in manipulating ChatGPT to produce output that even a 5-year-old human child would see as unwise. 

This behavior should not be surprising, given how LLMs are created and educated. They are not designed with comprehension in mind. They have been taught to produce word sequences that, given a context, would seem believable to a human. LLMs have mastered the art of linguistic competence, or knowing how to say things, according to Mahowald et al., but they need to be more skilled at functional competence or understanding what to say. In particular, they can be (relatively) readily tricked by, for instance, asking for the answer to a simple math issue not included in their training corpus or asking for the solution to a unique planning problem that necessitates knowledge of how the outside world functions. 

Do they now need to work harder to incorporate all math and planning tasks in their training corpus? That is a fool’s errand. But why should it be necessary, on the other hand? They already have general-purpose symbolic planners and calculators guaranteed to yield accurate results. Connecting LLMs to such technologies is a logical alternative strategy that they are not the first to investigate. With this purpose in mind, the research described in this paper aims to provide LLMs with the first-ever accurate solution to planning difficulties. They want to do this even with finetuning without changing the LLMs themselves. 

Instead, researchers from UT Austin and the State University of New York present a method known as LLM+P that, when given a natural language description of a planning problem, the LLM:

1. Outputs a problem description suitable as input to a general-purpose planner.
2. Solves the problem using the general-purpose planner.
3. Converts the planner’s production back to natural language.

In this work, they do not request that the LLM understand when a prompt has been presented that may be processed by the suggested LLM+P pipeline. Recognizing when LLM+P should handle a prompt will be important for future research. Their thorough empirical analyses show that LLM+P can accurately answer many more planning issues than LLMs alone. This broad technique may be used to respond to any class of cases for which there is a good and comprehensive solver, such as arithmetic problems (by using calculators), even though it was illustrated in this work on planning problems. The code and results are publicly available on GitHub.

Check out the Paper and GitHub link. Don’t forget to join our 20k+ ML SubReddit, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more. If you have any questions regarding the above article or if we missed anything, feel free to email us at Asif@marktechpost.com

🚀 Check Out 100’s AI Tools in AI Tools Club