Meet CodePlan: A Task-Agnostic AI Framework For Repository-Level Coding Using Large Language Models (LLMs) And Planning

A revolution in the field of coding work automation has been brought about by introducing Large Language Models (LLMs), such as GPT-3. These models have extraordinary generative skills and have opened the path for the creation of Replit, GitHub Copilot, and Amazon Code Whisperer. These tools are now frequently used to automate tasks like code modification and completion using natural language inputs and contextual data.

A recent research paper has examined the necessity to carry out ubiquitous code edits across a complete code repository, which is a fundamental problem in software engineering. Changes to the codebase are necessary for tasks like package migration, error repair, and type annotation addition. Due to the fact that these jobs require altering code throughout the entire repository, they are known as repository-level coding tasks. As repository-level coding chores are more difficult and can’t be solved entirely by LLMs, as the repository’s code is interlinked, and the size of the entire repository may be too large for LLMs to manage in a timely manner, the team has come up with a solution.

A group of researchers from Microsoft Research has introduced CodePlan, a task-agnostic framework that has been designed to tackle repository-level coding tasks by framing them as planning problems. It creates a chain of modifications with multiple steps or a plan, where each step involves calling an LLM to update a particular section of code. Each edit’s context is taken from the repository as a whole, earlier code modifications, and task-specific instructions.

The CodePlan structure is dependent on three essential elements.

1. Incremental Dependency Analysis: This feature aids CodePlan in comprehending the complex interdependencies among the many components of the code repository. It reveals where sections of the code are impacted by a specific code update, allowing for efficient planning.

2. Change May-Impact Analysis: CodePlan does this analysis to determine the potential effects of a specific code update on other areas of the codebase. Planning subsequent edits, which ensures that changes are made in the right order, depends heavily on this predictive skill.

3. Algorithm for Adaptive Planning: An adaptive planning algorithm is used to create the plan for editing the code. It takes into account the incremental dependency analysis and change may-impact analysis to lead the LLM effectively.

Practical experiments on two difficult repository-level tasks—package migration in C# and temporal code modifications in Python—have been used to gauge CodePlan’s performance. These jobs call for making interconnected modifications to somewhere between 2 and 97 files in the repositories. 

The evaluation’s findings have shown that CodePlan operates effectively. It outperforms baseline approaches and achieves a close agreement with ground truth expectations. It successfully makes it possible for 5 out of 6 repositories to pass validation tests, including building without issues and applying the proper code updates. In contrast, the baselines, which don’t use planning but make use of similar contextual data to CodePlan, find it difficult to succeed to the same degree.

By fusing the strength of LLMs with a sophisticated planning framework, CodePlan constitutes an amazing method for automating difficult repository-level coding chores. It closes a fundamental hole in software engineering and has the potential to greatly increase the effectiveness and precision of pervasive code modifications across huge codebases.

Check out the Paper. All Credit For This Research Goes To the Researchers on This Project. Also, don’t forget to join our 30k+ ML SubReddit, 40k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.

If you like our work, you will love our newsletter..