Microsoft Researchers Introduce InsightPilot: An LLM-Empowered Automated Data Exploration System

Data exploration is an important step in data analysis that extracts key insights using multiple steps such as filtering, sorting, grouping, etc. It helps uncover patterns in the dataset and reveal potential relationships among the variables. However, this process is generally interactive and requires the user to manually explore the data, making the process time-consuming and necessitating domain expertise. 

Although different tools exist for general data exploration, they often fail to consider user intent and dataset characteristics, leading to irrelevant insights. Additionally, LLM hallucination is an infamous issue that causes LLMs to generate unreliable content. To tackle the shortcomings of existing models, researchers at Microsoft have released InsightPilot, a system that automates the process of data exploration using LLMs. The system provides LLMs with accurate insights to avoid hallucinations and presents a compact abstraction of the dataset to reduce computational costs, which allows the LLM to answer user questions better.

InsightsPilot consists of the following three components:

1. A UI that allows users to ask questions in natural language and also display the analysis results.
2. An LLM that facilitates data exploration by selecting the appropriate analysis on the basis of the context.
3. An insight engine that does the analysis and presents the results in natural language.

A user initially poses a query in the interface, and the insight engine generates preliminary insights. Depending on the context, the LLM identifies the most relevant insights and keeps querying the engine to get more details about them. For example, a user may ask about trends in science scores for students, and then, based on initial insights, the LLM might query the engine for further analysis, such as comparing scores or finding any outliers. As long as the exploration is not complete, the interaction between the LLM and the engine continues, and at the end of the data exploration step, the engine presents the top-K insights in the form of a coherent report, which is then displayed to the user via the interface.

To evaluate its performance, the researchers conducted user studies to simulate real-world use cases of InsightPilot. Four data science participants were asked to raise three questions, and the system was evaluated against metrics like relevance, completeness, and understandability. The results show that InsightPilot consistently outperformed both OpenAI Code Interpreter and Langchain Pandas Agent. 

A case study based on a car sales dataset was also conducted to assess the performance of InsightPilot. When enquiring about the overall trend of Toyota’s car sales, the system not only identified ‘Camry’ as the key driver of Toyota’s sales but also compared Toyota’s sales with that of Honda and provided other interesting insights as well.

Although InsightPilot performs better than other state-of-the-art systems, it often produces vague answers that necessitate manual evaluation. Therefore, it is crucial to test its effectiveness across different real-life datasets. Nonetheless, it is an effective method of deriving insights from a dataset using natural language inquiries and has the potential to streamline the process of exploratory data analysis and save time and effort. Further research is necessary to ensure the method can be deployed in real-world scenarios and bolster efficiency and data-driven decision-making.

Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to join our 34k+ ML SubReddit, 41k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.

If you like our work, you will love our newsletter..