12-05, 14:30–15:00 (UTC), LLM Track
Knowledge graphs are excellent at representing and storing heterogeneous and interconnected information in a structured manner, effectively capturing complex relationships and attributes across different data types.
Structured text generation allows for building knowledge graphs by providing neatly structured outputs, making it an ideal method for extracting structured information.
Similarly, structured text generation enables the creation of agents by defining which tools are allowed and what action inputs are permitted.
In this talk, we first build a graph database from unstructured data and then we create an agent to query the graph database. We will show these capabilities with a demo.
The idea of LLM agents is to provide Large Language Models (LLMs) with the ability to decompose a task into subtasks and the ability to 'use' tools. This works by eliciting step-by-step 'reasoning' and by providing tool definitions, telling the LLM how to call those tools (what parameters the tools accept) if needed. For example, if we provide the tool 'get_temperature_from_location(location: str)', when a user asks a query related to getting the temperature in Paris, the LLM can call the tool 'get_temperature_from_location' with the input 'Paris'.
Structured text generation restricts the LLM to sample only from the set of allowed tokens (while masking everything else) to generate text. This means that we can restrict the LLM to only call from the set of available tools (tool_A, tool_B, tool_C, etc.) and respective parameters, and nothing else will be allowed. Similarly, we can restrict the LLM to only sample from a list of node types and edge types to constrain the generation of a knowledge graph. By using open-weights LLMs, we remove the dependency on an external LLM provider while retaining complete control over the data flows and how the data is being shared and stored.
Our approach consists of two main phases:
-
Graph Database Construction: We use an open-weights LLM and structured text generation techniques to construct a graph database from unstructured data.
-
Query Processing and Response Generation: When a user submits a question, we use the following steps:
a) Task Decomposition: The LLM breaks down the complex query into manageable subtasks.
b) Tool and Parameter Selection: For each subtask, the LLM determines the most appropriate tool and corresponding parameters to retrieve relevant information.
c) Query Generation: Based on the selected tools and parameters, we deterministically generate precise database queries.
d) Information Retrieval: These queries are executed against the graph database to extract pertinent data.
e) Response Synthesis: The LLM integrates the retrieved information to formulate a comprehensive answer to the user's original query.
This workflow ensures efficient information retrieval and accurate response generation, leveraging the strengths of both structured data representation and large language models.
We will show these capabilities with a demo. The tools used in the demo will be Kùzu + Llama.cpp + Outlines.
No previous knowledge expected
Alonso Silva is a Senior Researcher on Generative AI at Nokia Bell Labs. He previously did his Ph.D. at INRIA, a postdoc at UC Berkeley, and worked as an ML researcher at Safran.