SpecMind: Enabling Spectrum Intelligence via Multi-Agent Hybrid Retrieval-Augmented Generation
Thanks: This work was supported in part by SpectrumX, the National Science Foundation (NSF) Spectrum Innovation Center, through grant AST 2132700 operated under Cooperative Agreement by the University of Notre Dame.Thanks: ^∗The first two authors contributed equally to this work.

The construction process is illustrated in Figure 1. We consider three primary data sources in the spectrum domain: (1) license data, which appear as structured tables with numerical attributes; (2) proceeding documents, released by the FCC, each centered on a specific topic and containing diverse comments from individuals and organizations during regulatory deliberation; and (3) regulatory documents, which codify existing rules and policies. Applying RAG to such heterogeneous sources requires constructing modality-aware databases tailored to their distinct characteristics, rather than relying on a general-purposed semantic retrieval pipeline.

For regulatory texts, we adopt a standard dense retrieval paradigm. Documents are segmented into fixed-length chunks (1000 tokens with 200-token overlap) to preserve semantic continuity. Given a query, we retrieve top-$k$ candidates via embedding similarity and further apply a reranking stage to refine relevance. Specifically, we select the top 20 candidates from initial retrieval and rerank them to obtain the top 5 final contexts. This design aligns with the nature of regulatory texts, which are concise and legally precise, making queries rely heavily on accurate semantic matching.

In contrast, FCC proceeding comments and reply comments exhibit high volume, topic diversity, and complex multi-entity interactions. A single entity may express inconsistent positions across proceedings, and many queries require cross-document synthesis (e.g., identifying agency stances or comparing stakeholder views). Such requirements are poorly served by chunk-level semantic retrieval. We therefore enhance GraphRAG [12], which models documents as graphs with entities as nodes and relations as edges, and construct hierarchical summaries to support multi-hop reasoning. We build a separate graph database for each proceeding to ensure structural consistency (e.g., avoiding cross-document conflicts introduced by merging multiple proceedings into a single graph) and enable modular extensibility without rebuilding the entire corpus when new proceedings are introduced. Detailed statistics of the graph corpora, including node distributions across hierarchy levels, are summarized in Table I.

License data present a fundamentally different challenge, as they are dominated by structured numerical fields with weak semantic signals [13]. Traditional RAG methods perform poorly in this setting due to their reliance on semantic similarity. We instead model license data using relational databases and perform retrieval via SQL queries to enable exact matching over structured attributes. The problem is thus reformulated as generating executable SQL queries from natural language questions, which can be effectively handled by LLMs. Concretely, we extract license records into intermediate JSON representations for preprocessing, and then convert them into a relational SQL database. We further organize the database into service-specific subtables to accommodate heterogeneous schemas across different license types.

We propose SpecMind, a multi-agent hybrid RAG system for reasoning over heterogeneous spectrum data. The overall system design is illustrated in Figure 2. Each task instance is represented as

where $Q$ denotes a user query, $A$ is the ground-truth answer, and $\mathcal{K}$ comprises heterogeneous knowledge sources, including structured license databases, graph-structured proceeding documents, and unstructured regulatory texts. The agent set is defined as

Each agent $a\in\mathcal{A}$ is associated with an action policy $\pi_{a}\in\Pi$. The Supervisor Agent $a_{\text{sup}}$ coordinates task decomposition, agent invocation, and result integration, while specialized agents perform domain-specific actions.

Given a query $Q$, the system maintains a global task state $\mathcal{T}$ that accumulates intermediate evidence and partial results across multiple agent interactions. The Supervisor Agent coordinates the execution of specialized agents and iteratively refines the task state until a final answer $\hat{A}$ is produced.

The objective of the system is to generate an answer $\hat{A}$ that is factually grounded in $\mathcal{K}$ and consistent with the ground-truth answer $A$, while enabling accurate and flexible reasoning across heterogeneous spectrum data sources.

Design principle. We formulate prompt engineering as the explicit specification of agent-level action policies $\Pi=\{\pi_{\text{sup}},\pi_{\text{lic}},\pi_{\text{proc}},\pi_{\text{reg}}\}$, which govern task decomposition, tool usage, and cross-agent coordination. Inspired by the ReAct paradigm [14], which interleaves reasoning and action, our design extends this principle to a multi-agent setting tailored to spectrum-specific tasks. Rather than relying on implicit reasoning in LLMs, we enforce structured decision-making through prompt-level control, enabling reliable interaction with heterogeneous data sources.

Supervisor prompt. The Supervisor Agent is prompted to follow a structured Think–Act–Observe loop for multi-step reasoning. The prompt explicitly instructs the agent to decompose the query into subtasks, decide whether to invoke a specialized agent or perform an internal operation, and iteratively update the global task state. This design enables adaptive planning and dynamic routing across heterogeneous tools, while ensuring that intermediate results are incorporated into subsequent reasoning steps.

License agent prompt. For structured license data, we design a tool-aware prompt that explicitly specifies the usage of four SQL tools, including schema inspection, query execution, and query validation. The action policy $\pi_{\text{lic}}$ enforces a stepwise interaction pattern, where the agent first inspects database structure, then generates executable SQL queries, and finally refines results based on returned outputs. This design enables precise handling of numerical and table-centric queries beyond the capability of semantic retrieval.

Proceeding agent prompt. For graph-structured proceeding data, the prompt encodes a hierarchical retrieval strategy. The action policy $\pi_{\text{proc}}$ first performs proceeding selection via topic matching, and then dynamically chooses among three GraphRAG tools (basic, local, and global) according to query granularity. Specifically, local retrieval is preferred for entity-centric queries, whereas global retrieval is used for higher-level summarization. To improve robustness under incomplete graph construction, the prompt further incorporates a fallback rule that invokes basic retrieval when structured retrieval fails to return sufficient evidence.

Regulation agent prompt. For regulatory documents, we design a minimal high-precision prompt that focuses on retrieval refinement rather than tool selection. The action policy $\pi_{\text{reg}}$ adopts a two-stage retrieve-and-rerank pipeline, where embedding-based retrieval is followed by lightweight reranking with Qwen3-Reranker-0.6B. This design improves evidence precision for regulation-oriented queries, where correctness is critical.

Web-search RAG. This baseline uses Google Search as the retriever, collecting the top-20 results per query as generation context. It serves as a practical baseline for spectrum question answering using publicly accessible resources.

SpectrumRAG. We include SpectrumRAG [8] as a competitive baseline, which is an iterative RAG framework that leverages LLM-based query rewriting to refine retrieval for spectrum policy question answering. This design improves retrieval quality and downstream generation, making it a strong representative of advanced RAG systems in this domain.

Models. To ensure a fair comparison, we evaluate all methods, including the baselines and SpecMind, using two backbone LLMs of different scales, Qwen3-8B and GPT-5.2. Each method uses the same backbone for both RAG and reasoning, and text-embedding-3-small is used for retrieval across all methods.

Metrics. We report success rate and win rate for each query. The success rate evaluates whether a method produces a valid response under one of the two settings: (i) for standard question types, including license, proceeding, regulation, and compound queries, the method must return a factually grounded answer when sufficient supporting evidence exists; and (ii) for negative-rejection questions, where no valid evidence is available, the method must correctly abstain and avoid hallucination by issuing a rejection. The win rate measures comparative answer quality. We use a strong proprietary LLM, Gemini 3.1-Pro-Preview, as a unified evaluator, which is distinct from the backbone models and fixed across all experiments to ensure consistent comparisons. For each query, the evaluator selects the best answer among all candidates; if all answers are incorrect, no method is assigned a win.

As shown in Table III, SpecMind consistently outperforms the baselines, including Web-search RAG and SpectrumRAG, across question types under both backbone LLMs, achieving substantial win-rate gains, especially on license and compound queries, demonstrating effective structured retrieval and cross-source reasoning.

In terms of success rate, SpecMind maintains strong performance across all categories, achieving near-perfect results with GPT-5.2 and consistently high accuracy with Qwen3-8B. In contrast, the baselines achieve significantly lower success rates, particularly on license and compound tasks, highlighting their limitations in precise retrieval and coordinated reasoning. While Web-search RAG performs relatively well on regulation queries due to strong semantic retrieval, it fails to generalize to structured and multi-source settings.

For unanswerable questions, all methods achieve perfect success rates. This is largely because the task primarily requires abstention rather than evidence synthesis, and modern backbone LLMs already exhibit strong intrinsic capabilities in recognizing insufficient evidence and avoiding unsupported generation. As a result, performance on this category is less sensitive to the retrieval framework and more dependent on the underlying model’s calibration.

To analyze the source of performance gains, we conduct ablation studies on the three specialized agents under the GPT-5.2 backbone by individually replacing each sub-agent with a naive RAG module (without re-ranking) while preserving the overall framework. Results are reported in Table IV.

We observe consistent performance degradation across all ablations, confirming that each agent contributes to the overall effectiveness. The impact is most pronounced on the corresponding question types, followed by compound queries, reflecting error propagation in multi-step reasoning.

The degradation is particularly severe for the license agent, with substantial drops in both win rate and success rate (up to $-87.8$ and $-82.8$), highlighting the importance of SQL-based retrieval for structured data. The proceeding agent also plays a critical role, with notable declines on proceeding questions ($-27.3$ win, $-10.3$ success), primarily due to failures in multi-document reasoning, underscoring the benefit of graph-based retrieval. In contrast, the regulation agent shows relatively smaller impact, as its improvements mainly arise from prompt design and reranking over text-based retrieval, which offer more limited gains than structured and graph-based mechanisms.

To illustrate how SpecMind operates, we present a compound Q&A example jointly supported by all three specialized agents: Within the 12.7–13.25 GHz band, which frequencies are allocated to satellite services? Is Intelsat an incumbent in this band, and if so, do Intelsat and other incumbents share the same position on spectrum sharing with terrestrial services?

The reasoning process is illustrated in Fig. 4. The supervisor agent operates in a sequential, state-driven manner, iteratively selecting which agent to invoke and what subtask to issue. It first queries the regulation agent to identify the relevant frequency band (e.g., 12.7–13.25 GHz), establishing a constraint for subsequent steps. Based on this result, it invokes the license agent to retrieve corresponding licensees (e.g., INTELSAT, GLOBECOMM) via SQL queries, grounding the task in structured data. Finally, the proceeding agent performs graph-based retrieval to compare these entities’ positions on spectrum sharing and reveal consistent opposition among incumbents. At each step, intermediate results are incorporated into the task state, allowing the supervisor to adaptively refine subsequent actions and resolve inter-step dependencies. This example demonstrates how SpecMind supports coherent cross-source reasoning through dynamic coordination of heterogeneous retrieval mechanisms.

To better understand the small set of remaining non-optimal cases, we manually inspect queries where SpecMind does not achieve the best result or produces a partially complete answer. We observe three recurring edge conditions. (i) Fine-grained attribution. In some proceeding graphs, entity–filing relations indicate co-occurrence but do not always explicitly encode whether the entity authored the filing or was referenced by another party, which can make fine-grained attribution more challenging. (ii) Sparse evidence for long-tail entities. Stakeholders that appear only a few times in a proceeding naturally have sparser graph representations, and retrieval for such entities may surface broader contextual information rather than the most specific statement of position. (iii) Ambiguity in structured queries. Some identifiers are only unique within a service-specific subtable, so queries that omit the service type can admit multiple reasonable interpretations and lead to broader aggregation than intended. Overall, these cases largely reflect edge conditions in evidence representation and query specification rather than systematic limitations of the coordination framework. They also point to natural extensions, including richer authorship metadata in graph construction and lightweight clarification mechanisms for ambiguous queries.