LLMs & Knowledge Graphs – MarkTechPost

Large Language Models (LLMs) are AI tools that can understand and generate human language. They are powerful neural networks with billions of parameters trained on massive amounts of text data. The extensive training of these models gives them a deep understanding of human language’s structure and meaning.

LLMs can perform various language tasks like translation, sentiment analysis, chatbot conversation, etc. LLMs can comprehend intricate textual information, recognize entities and their connections, and produce text that maintains coherence and grammatical correctness.

A Knowledge Graph is a database that represents and connects data and information about different entities. It comprises nodes representing any object, person, or place and edges defining the relationships between the nodes. This allows machines to understand how the entities relate to each other, share attributes, and draw connections between different things in the world around us.

Knowledge graphs can be used in various applications, such as recommended videos on YouTube, insurance fraud detection, product recommendations in retail, and predictive modeling. 

One of the main limitations of LLMs is that they are “black boxes,” i.e., it’s hard to understand how they arrive at a conclusion. Moreover, they frequently struggle to grasp and retrieve factual information, which can result in errors and inaccuracies known as hallucinations. 

This is where knowledge graphs can help LLMs by providing them with external knowledge for inference. However, Knowledge graphs are difficult to construct and are evolving by nature. So, it’s a good idea to use LLMs and knowledge graphs together to make the most of their strengths.

LLMs can be combined with Knowledge Graphs (KGs) using three approaches:

1. KG-enhanced LLMs: These integrate KGs into LLMs during training and use them for better comprehension.
2. LLM-augmented KGs: LLMs can improve various KG tasks like embedding, completion, and question answering.
3. Synergized LLMs + KGs: LLMs and KGs work together, enhancing each other for two-way reasoning driven by data and knowledge.

KG-Enhanced LLMs

LLMs are well-known for their ability to excel in various language tasks by learning from vast text data. However, they face criticism for generating incorrect information (hallucination) and lacking interpretability. Researchers propose enhancing LLMs with knowledge graphs (KGs) to address these issues. 

KGs store structured knowledge, which can be used to improve LLMs’ understanding. Some methods integrate KGs during LLM pre-training, aiding knowledge acquisition, while others use KGs during inference to enhance domain-specific knowledge access. KGs are also used to interpret LLMs’ reasoning and facts for improved transparency.

LLM-augmented KGs

Knowledge graphs (KGs) store structured information crucial for real-world applications. However, current KG methods face challenges with incomplete data and text processing for KG construction. Researchers are exploring how to leverage the versatility of LLMs to address KG-related tasks.

One common approach involves using LLMs as text processors for KGs. LLMs analyze textual data within KGs and enhance KG representations. Some studies also employ LLMs to process original text data, extracting relations and entities to build KGs. Recent efforts aim to create KG prompts that make structural KGs understandable to LLMs. This enables direct application of LLMs to tasks like KG completion and reasoning.

Synergized LLMs + KGs

Researchers are increasingly interested in combining LLMs and KGs due to their complementary nature. To explore this integration, a unified framework called “Synergized LLMs + KGs” is proposed, consisting of four layers: Data, Synergized Model, Technique, and Application. 

LLMs handle textual data, KGs handle structural data, and with multi-modal LLMs and KGs, this framework can extend to other data types like video and audio. These layers collaborate to enhance capabilities and improve performance for various applications like search engines, recommender systems, and AI assistants.

Multi-Hop Question Answering

Typically, when we use LLM to retrieve information from documents, we divide them into chunks and then convert them into vector embeddings. Using this approach, we might not be able to find information that spans multiple documents. This is known as the problem of multi-hop question answering.

This issue can be solved using a knowledge graph. We can construct a structured representation of the information by processing each document separately and connecting them in a knowledge graph. This makes it easier to move around and explore connected documents, making it possible to answer complex questions that require multiple steps.

In the above example, if we want the LLM to answer the question, “Did any former employee of OpenAI start their own company?” the LLM might return some duplicated information or other relevant information could be ignored. Extracting entities and relationships from text to construct a knowledge graph makes it easy for the LLM to answer questions spanning multiple documents.

Combining Textual Data with a Knowledge Graph

Another advantage of using a knowledge graph with an LLM is that by using the former, we can store both structured as well as unstructured data and connect them with relationships. This makes information retrieval easier.

In the above example, a knowledge graph has been used to store:

• Structured data: Past Employees of OpenAI and the companies they started.
• Unstructured data: News articles mentioning OpenAI and its employees.

With this setup, we can answer questions like “What’s the latest news about Prosper Robotics founders?” by starting from the Prosper Robotics node, moving to its founders, and then retrieving recent articles about them.

This adaptability makes it suitable for a wide range of LLM applications, as it can handle various data types and relationships between entities. The graph structure provides a clear visual representation of knowledge, making it easier for both developers and users to understand and work with.

Researchers are increasingly exploring the synergy between LLMs and KGs, with three main approaches: KG-enhanced LLMs, LLM-augmented KGs, and Synergized LLMs + KGs. These approaches aim to leverage both technologies’ strengths to address various language and knowledge-related tasks.

The integration of LLMs and KGs offers promising possibilities for applications such as multi-hop question answering, combining textual and structured data, and enhancing transparency and interpretability. As technology advances, this collaboration between LLMs and KGs holds the potential to drive innovation in fields like search engines, recommender systems, and AI assistants, ultimately benefiting users and developers alike.

Also, don’t forget to join our 30k+ ML SubReddit, 40k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.

If you like our work, you will love our newsletter..

References: