Triple-Bottom-Line Sustainability of Language Models for Edge AI:
A Comparison between SLMs and Quantized LLMs

The rapid growth of language models has made inference quality easier to observe than inference burden. Public comparisons often emphasize aggregate benchmark accuracy while omitting memory footprint, latency, energy per response and safety behavior. This omission is especially consequential for edge AI, where hardware, power and response-time budgets are fixed. GPU memory determines whether a model can run locally; latency and throughput determine usability and operating cost; energy affects battery life and environmental burden; and unsafe behavior can shift risk to users who operate outside a centrally managed service. The Green AI literature consequently argues that efficiency should be reported alongside predictive quality rather than treated as an implementation detail (Schwartz et al., 2020; Strubell et al., 2019).

Quantization offers a direct intervention. By representing weights with fewer bits, it can reduce memory traffic and model footprint, potentially improving throughput and energy efficiency. However, the result depends on the quantization algorithm, kernels, checkpoint construction, architecture and inference backend. LLM.int8() isolates activation outliers in higher precision (Dettmers et al., 2022); NF4 was designed for normally distributed weights (Dettmers et al., 2023); GPTQ uses approximate second-order information for post-training weight quantization (Frantar et al., 2023) and GGUF Q4 is commonly executed through the separately optimized llama.cpp stack (Gerganov and contributors, 2026). A smaller bit width therefore does not guarantee lower end-to-end latency or energy.

Edge-AI model selection lacks a unified framework that simultaneously evaluates capability, economic deployability, operational environmental cost and behavioral safety. The central research question is therefore - Are natively trained SLMs more sustainable for edge-AI deployment than larger LLMs compressed using post-training quantization? Our working hypothesis is that SLMs should lead raw resource efficiency, whereas a quantized LLM may justify additional cost through higher capability; the winning family must be determined from the complete trade-off rather than assumed from parameter count.

We operationalize this question through three analyses. First, we identify the most balanced SLM and LLM configurations. Second, we rank the five LLM families within each fixed quantization case. Third, we renormalize all SLM and LLM measurements together to test the family-level hypothesis in one pool. The experiment evaluates five BF16 SLMs and a $5\times 5$ grid of LLMs and quantization cases (Figure 1), then constructs an explicitly relative HSS from three equally weighted pillars. The contribution is not a claim that a single scalar captures every societal consequence. Rather, HSS is a transparent, auditable edge-deployment decision aid that exposes trade-offs, preserves the raw metrics and makes its comparison population explicit.

Table 1 presents the principal outcomes; complete rankings appear in Appendix A. In the SLM-only pool, Llama-3.2-3B/BF16 ranks first with HSS 92.42. Its mean capability of 0.556 is lower than Phi-4-mini’s 0.591 but its 9.47 s latency, 34.41 tokens/s throughput and 2.63 J/token create the best overall balance. This illustrates why capability and deployment cost should be reported together.

In the global LLM pool, Qwen3-30B-A3B/GGUF Q4 ranks first (93.38) and Mistral-Small-24B/GGUF Q4 ranks second (92.40). Across the five models, GGUF Q4 preserves mean capability (0.524 versus 0.525 for BF16) while raising mean throughput from 22.74 to 95.50 tokens/s, thus reducing mean peak VRAM from 56.35 to 13.64 GB and reducing mean energy from 12.10 to 5.28 J/token. The result reflects the joint configuration, not only four-bit weights: GGUF uses llama.cpp whereas the other cases use Hugging Face/Transformers-based execution.

Mistral provides the clearest quality-efficiency trade-off. Its BF16 case has the highest LLM capability of 0.658 but its GGUF case retains 0.647 capability while improving throughput from 27.71 to 61.13 tokens/s and reducing peak VRAM from 48.57 to 15.88 GB. Conversely, gpt-oss-20B/GGUF records the highest throughput of 173.86 tokens/s) and only 1.48 J/token, yet its 0.352 capability limits its global HSS to 84.11. A single fastest or most accurate metric therefore does not determine the best balanced case.

Within a fixed quantization case, gpt-oss-20B wins BF16 (91.64), INT8 (87.72) and NF4 (86.83); Mistral wins GPTQ (97.18) and Qwen3 wins GGUF (86.66). These scores differ from the 25-case global values because each five-model block has new extrema. The outcome also rejects a simple ‘fewer bits is faster’ narrative. Mistral INT8 is slower and more energy intensive than its BF16 case while Gemma GPTQ reaches 958.71 s latency, 0.33 tokens/s, 85.01 GB peak VRAM and 493.02 J/token. Kernel support, retained high-precision modules, conversion quality and backend behavior can dominate bit width.

When the five SLMs enter the global pool, Qwen GGUF and Mistral GGUF remain first and second. Phi-4-mini/BF16 becomes third (89.49), Mistral NF4 fourth (89.47), and Llama-3.2-3B/BF16 fifth (89.25). Thus, small models are competitive on the composite even when larger models lead capability, because lower VRAM and energy can offset part of the quality gap. The combined ranking should nevertheless be treated as exploratory, although both workflows used A100-80GB GPUs, SLM and LLM runs occurred on different platforms and software paths.

These conclusions are also bound by a few limitations. First, min-max normalization is sensitive to outliers; the Gemma GPTQ failure expands several ranges and compresses differences among other cases. Second, the social pillar is a five-prompt lexical refusal proxy and is unsuitable as a complete safety assessment; it is especially misleading for the non-aligned LLama base model. Third, operational joules omit data center power usage effectiveness, regional grid carbon intensity, embodied hardware emissions and lifecycle effects. The anomalously low LLama GGUF VRAM delta also shows that external-server memory accounting warrants independent validation. HSS is therefore most useful as a transparent screening score accompanied by raw metrics, sensitivity analyses and deployment-specific weights.

This study evaluates triple-bottom-line edge sustainability as a joint property of model, precision, backend and workload. The central comparison does not yield a universal SLM victory: Qwen3-30B-A3B/GGUF Q4 offers the best balance in the 25- and 30-case pools, while Mistral-Small-24B/GGUF Q4 combines near-leading capability with large system-level gains. At the same time, SLMs remain competitive, three occupy the combined top nine because their lower resource demand can offset part of the capability gap. The within-quantization analysis shows that the preferred model changes with the deployment choice. Most importantly, INT8, NF4, GPTQ, and GGUF cannot be ordered universally, their edge benefits depend on architecture and execution stack.

Future work should run more benchmark examples and repeated trials with confidence intervals; isolate cold-start, prompt-processing and token-generation latency; record CPU and host-memory energy; validate VRAM across in-process and server backends; and report carbon using measured PUE and grid intensity. Safety evaluation should replace lexical refusal detection with a larger adversarial set and human or validated classifier judgments. Finally, HSS should be subjected to weight sweeps, robust normalization, Pareto-front analysis and uncertainty propagation. Those additions would turn the current transparent course-score into a stronger deployment decision framework without hiding the trade-offs behind a single number.