Context
Metadata management rarely fails because a database is missing. It fails because the same entity is called three different things in a glossary, a technical document and a DDL file. My M.Sc. thesis, developed as a graduate intern at Data Reply, explored whether production-oriented Generative AI could build and query that alignment automatically.
Problem
An LLM asked to link a business term to a schema column can produce a plausible link that does not exist. The thesis treats that as the central problem: the system has to retrieve evidence, validate the alignment and grade its own answer before returning it. The target was a working pipeline over real relational metadata, not a benchmark-only prototype.
Contribution
I authored the thesis and built the system: the builder graph that extracts triplets, resolves entities, validates with an actor–critic loop and heals failing Cypher; the query graph that combines dense search, BM25 and graph traversal, reranks with a cross-encoder and grades the answer for grounding; and the evaluation protocol across seven datasets.
Approach
Two LangGraph pipelines share one Neo4j ontology. The first writes and repairs the alignment; the second reads it and decides whether an answer is supported.
Builder graph
Query graph
Builder graph
- Triplet extraction. Documents and DDL are turned into candidate entities and relations.
- Entity resolution. Duplicate terms and columns are merged before they reach the graph.
- Actor–critic validation. A second agent checks proposed links and rejects the ones it cannot support.
- Cypher healing. Failed writes are diagnosed and retried rather than dropped.
Query graph
- Hybrid recall. Dense retrieval, BM25 and graph traversal each return candidates over the aligned ontology.
- Cross-encoder reranking. Candidates are reordered by a cross-encoder before generation.
- Hallucination grading. The final answer is checked against the retrieved evidence and graded.
Evaluation & results
The system was evaluated over real relational metadata: seven datasets, 111 tables and 210 questions, with ablations for the retrieval and validation stages.
- 7
- Datasets
- 111
- Tables
- 210
- Questions
- 210/210
- Grounded answers
0 hallucinations · 4.31/5 mean score · 21 ablation studies
Every answer in the evaluation was graded as grounded, with no hallucinations. The mean score was 4.31 out of 5, and 21 ablation studies tested the contribution of individual stages.
Figures are as reported in the thesis evaluation; no additional runs were performed for this page. Limitations were not documented in a form I can publish, so they are not listed here.