Automated Tree Knowledge Graph Construction using Ontology Expansion and Retrieval from Vietnamese History Textbooks

The RAG systems were designed with a generative retriever, and the LLM can produce its answers in retrieved documents [14]. Adding a knowledge graph to the pipeline, GraphRAG [19, 8] enables the system to follow relational paths rather than rely on keyword overlap alone. Existing methods traverse the graph in different ways: ReMindRAG [10] allows the LLM to decide which edges to follow, GFM-RAG [16] trains a graph foundation model for the task on hierarchical structures. RAPTOR [22] builds a retrieval tree through recursive abstractive summarization. Unlike existing works, our approach keeps the original document hierarchy and evaluates multiple traversal directions rather than a single bottom-up clustering strategy.

Recent works in [21, 6, 15] use LLMs to extract knowledge graphs from unstructured documents. Doc-KG [21] and AutoMathKG [6] both combine embedding-based relation filtering with LLM extraction. LLMs can perform zero-shot relation extraction [24] and speed up ontology design [23], the expansion quickly produces duplicate types [12]. The work in [15] uses multi-agent LLMs to enrich existing KGs, though enrichment differs from construction. Our proposed system will combine controlled expansion with dual-LLM validation to prevent the ontology from growing too large. In the education domain, Ain et al. [2, 3] build a KG pipeline for educational content and compare top-down with bottom-up construction. Abu-Rasheed et al. [1] uses LLMs to fill in missing relations in curriculum KGs. To the best of our knowledge, there is a lack of KGs built from Vietnamese textbooks.

CRAG is the most popular RAG benchmark [25], which has 4,409 question-answer pairs and evaluates flat text and entity-level question answering. Graded relevance labels, where a node can be partially relevant rather than simply relevant or not, are absent from existing benchmarks.

Therefore, three open challenges remain and will be addressed in our paper: (i) building a KG with controlled ontology expansion and dual-LLM validation for a low-resource language, (ii) designing a retrieval benchmark with graded relevance labels and query direction categories, and (iii) comparing traversal strategies on a hierarchical structure for Vietnamese history textbooks.

The strategy first identifies the best-matching sections using a semantic summary of each structural node based on embedding similarity. Then it traverses downward to collect all content nodes in the subtree of sections. Three ranking scores are evaluated: E1a uses a cosine similarity score, E1b uses the RRF score given seed_score, and E1c extends E1b with MMR [7] to enhance the diversity.

Many queries require connecting scattered information across different sections. For example, a cause might appear in one section while its consequence events appear in another. The Horizontal strategy addresses the problem by traversing semantic relation edges between branches of the tree. The process starts with a vector search to select seed nodes and the corresponding seed_score based on the cosine similarity score. Then, it filters to select the relevant relation types and traverses one hop to reach cross-branch nodes (i.e., extended nodes). Among all of the selected nodes, the strategy performs the following three rankings to select the top-K relevant nodes. First, E2a uses only cosine similarity between the content nodes, including the seed and extended nodes. Second, E2b uses the RRF score given seed_score, the cosine similarity of the extended content nodes. Third, E2c is a restricted variant when keeping only the top-3 relation types.

Given the query, when the content nodes have a high similarity score, the shared parent structural node is likely relevant. The approach is especially useful when the parent summary is too general to match the query directly. The main factor for ranking is the threshold-based child aggregation. We denote $\mathcal{C}(p)$ as the set of child nodes belonging to the parent node $p$, and $s_{c}=\text{sim}(\mathbf{q},\mathbf{e}_{c})$ is the cosine similarity between the query and child node $c$. The threshold $\tau_{p}$ is computed as the median of all child scores belonging to the parent node $p$. Only children whose score is greater than or equal to $\tau_{p}$ are selected. The process starts with a vector search to retrieve candidate content nodes, then follows inbound edges to identify parent nodes using the following three variants. First, E3a uses cosine similarity based on the semantic summary to select top-K parent nodes. Second, E3b applies the aggregated score of the parent node $p$ based child scores:

The first term captures the average cosine quality of all children under parent $p$. The second term introduces the logarithmic scaling factor based on the number of relevant children, thereby avoiding the explosion of dense parents. Third, E3c utilizes RRF using the parent cosine and agg_score(p).

From the KG stored in ArangoDB, the system selects all of the L5-level structural nodes that have at least one child content node. Each L5 candidate is assembled through three phases:

• •

  SEED: Traversal from the selected L5 node to collect core content nodes.

• •

  ENRICH: From each core content node, one-hop traversal via Tier-2 semantic relations discovers content nodes belonging to other sections and selects at most two sections ranked by the number of relations between content nodes. Then we retrieve the parent nodes and child nodes that belong to them.

• •

  BOUND: We filter to select the relevant relations among collected nodes (i.e., core and enriched nodes), forming a closed subgraph.

The benchmark categorizes queries along five directions reflecting distinct information needs. Neutral (NT) queries perform a single-fact lookup, require only one node serving as a baseline. Top-down (TD) queries synthesize multiple nodes within the same section. Horizontal (HZ) queries relate to the information from different sections. Bottom-up (BU) queries target summaries or themes at the section level. Finally, Cross-section (CS) queries compare content from two distinct sections.

The LLM generates JSON including the query, a list of relevant nodes and their relevant level 0/1/2, direction, and a confidence score. A seven-step QA verification for each output will help to check the nodes existence, content relevance exceeding a cosine threshold, remove the duplicates, remove non-relevant nodes, and validate directions given their quotas. High-confidence outputs receive a 10% spot-check; medium-confidence outputs are reviewed at a 50% rate; every low-confidence item is reviewed in full.