A Heuristic Perspective on Debiasing Language Models

Language models are typically trained on large-scale corpora collected from online sources such as news articles, forums, and Wikipedia, which inevitably reflect societal biases present in human-generated text (Schick et al., 2021a; Blodgett et al., 2020) and Chain-of-Thought process (Feng et al., 2026). Previous research demonstrates that such biases not only persist but also become more subtle and deeply embedded in PLMs and LLMs (Sheng et al., 2019; May et al., 2019; Kaneko and Bollegala, 2021). More recent work has systematically analyzed the presence of social biases in modern PLMs and LLMs across languages and tasks, highlighting their impact on downstream applications and raising increasing concerns about fairness and reliability (Meade et al., 2022; Gallegos et al., 2024; Dong et al., 2024).

Existing debiasing methods for language models primarily rely on data augmentation, prompt-based control, as well as fine-tuning and alignment strategies. Data-driven approaches, such as counterfactual data augmentation, aim to mitigate bias by constructing fairer training data; however, they often require substantial manual effort and incur high computational and environmental costs Lu et al. (2020); Strubell et al. (2019). Other lines of research focus on fine-tuning or alignment-based debiasing, including automatically generated biased prompts Guo et al. (2022b), fast bias removal via machine unlearning Chen et al. (2023), and parameter-efficient or LLM-assisted debiasing strategies Han et al. (2024). More recently, researchers have explored revealing and mitigating bias through model-internal probing Dong et al. (2024), as well as task-arithmetic-based debiasing methods that reduce bias without full retraining Shirafuji et al. (2025b). Despite their effectiveness in specific settings, most existing approaches rely on external corpora or fixed preference datasets, which limit their scalability and generalization across bias categories, languages, and cultures Meade et al. (2022). However, most methods focus on English biases and assume cultural universality, leaving cross-cultural bias largely underexplored (Shan et al., 2024; Madhusudan et al., 2026). Culturally focusing debiasing methods for diverse societies (e.g., India, Europe) remain largely undeveloped.

Biases in language models frequently manifest as divergent prediction distributions for different demographic groups within the same contextual setting (Lan et al., 2025b). To address this issue, we introduce HEIMAT, a prompt-driven framework that combines bias disclosure with debiasing fine-tuning. As shown in Figure 2, HEIMAT first employs heuristic and association prompts to uncover demographic attributes implicitly associated with a given context, thereby constructing sets of minimally contrasted context prompts. The model is subsequently fine-tuned by minimizing the Jensen–Shannon divergence across prediction distributions conditioned on these prompts, enforcing demographic-invariant behavior while preserving contextual semantics.

HEIMAT discloses social biases through specially designed heuristic prompts. For clarity, we first define three types of words used in our prompts: (i) Target Word ($W_{t}$), which denotes the social entity being described; (ii) Feature Word ($W_{f}$), which represents contextual attributes or behaviors associated with the target word and typically forms an adjective or verb phrase; and (iii) Bias Type Word ($W_{b}$), which specifies the bias category of interest, such as gender or race.

We use Deepseek-Chat (Liu et al., 2024) to generate heuristic prompts according to our templates. Examples can be found in Appendix A.

Consider the heuristic prompt: “This young woman often appears in a hospital, and her job is as a …”. Here, “woman” serves as the target word $W_{t}$, “appears in a hospital” constitutes the feature words $W_{f}$, and “job” corresponds to the bias type word $W_{b}$. For Given this prompt, a language model may generate continuations such as “doctor”, “nurse”, or “teacher”. These responses reveal the model’s implicit associations between demographic attributes and occupations. For instance, a preference for “nurse” may reflect a gender stereotype associating women with caregiving roles. By composing prompts using these three components, HEIMAT systematically elicits and analyzes a model’s bias behavior across diverse contexts, forming the basis for subsequent debiasing.

Inspired by prior work (Dong et al., 2024), we construct heuristic prompts using a set of predefined templates to simulate realistic social scenarios while reducing manual prompt engineering cost. Each template generates a complete heuristic prompt by taking as input a neutral target word $W_{t}$, a feature word phrase $W_{f}$, and a bias type word $W_{b}$. For masked language models (e.g., BERT), an additional [MASK] token is included at the prediction position, whereas for autoregressive language models (e.g., LLaMA-2), the model is prompted to continue the given prefix. We employ six templates in total to generate heuristic prompts in our experiments, ensuring that bias disclosure is grounded in realistic and socially meaningful settings while allowing the model to reveal its own inherent predispositions.

Given a heuristic prompt, we input the sequence into the model and extract the top-$k$ most probable tokens conditioned on the prompt context. For masked language models, this corresponds to the probability distribution over the masked position, while for decoder-only models, it corresponds to the next-token distribution given the prompt prefix. We denote the resulting token list as $WL=(w_{1},w_{2},\ldots,w_{k})$, where each $w_{i}$ represents a demographic attribute implicitly associated with the preceding context. The set $WL$ thus captures the model’s most salient demographic associations under the heuristic prompt.

To further expand demographic coverage, we apply association prompts to each $w_{i}\in WL$ and prompt the model to generate $n$ related demographic terms, forming a substitution list $SL$:

Using the substitution list, we construct a Context Prompt Set (CPS) by minimally modifying the original heuristic prompt (as illustrated in Figure 3). Specifically, we substitute the demographic attribute in the prompt with each term sampled from $SL$, while masking or abstracting the variable component of the feature words $W_{f}$. For autoregressive models, we reorder the sentence to place the next-token position at the end, allowing models to condition on the preceding demographic attributes. This procedure ensures that all prompts within a CPS differ only in demographic attributes, while preserving identical contextual semantics. Formally, we define the context prompt set as

where each $CP_{i}$ denotes a context prompt. Each CPS is treated as a training unit in the subsequent debiasing fine-tuning stage.

Given a language model M and a context prompt CP, the model induces a probability distribution over possible next tokens. We define the prediction distribution $p$ as:

where $w$ denotes a candidate token in the model’s output vocabulary.

Our goal is to ensure that the model produces consistent prediction distributions across context prompts that differ only in demographic attributes, while preserving the same semantic context. Specifically, given a context prompt set $CPS=(CP_{1},CP_{2},\ldots,CP_{n})$, where each $CP_{i}$ differs only in the substituted demographic term, we aim to minimize the discrepancy among their induced prediction distributions.

To achieve this, we adopt Jensen–Shannon Divergence (Lin, 1991) as a symmetric and bounded measure of distributional difference, and fine-tune the model by minimizing the Jensen–Shannon Divergence loss ($\mathcal{L}_{JSD}$). Given the set of prediction distributions $PS=(p_{1},p_{2},\ldots,p_{n})$, the loss is defined as:

where $D_{KL}$ denotes the Kullback–Leibler Divergence.

By minimizing this loss during fine-tuning, the model is encouraged to align its output distributions across different contexts, thereby reducing biased associations while maintaining NLU capability.

Although we provide only the heuristic and association prompts for English and French, the proposed method can be readily applied to other languages and cultural contexts. To adapt to a new culture, one simply prompts an LLM (e.g., DeepSeek) to regenerate the heuristic prompts following the same template structure, using culturally relevant feature words (Examples can be found in Appendix A.). This lightweight process replaces direct translation, ensuring that the biases being measured are those actually present in the target culture.

Furthermore, our templates are highly adaptable to various types of bias. By simply substituting the bias type word, such as changing ’gender’ to ’race’, we can effectively address and mitigate the model’s biases related to race. This adaptability extends to a wide range of other bias categories, showcasing the versatility and broad applicability of our approach.

To quantitatively evaluate the effectiveness of HEIMAT in mitigating social biases, we conduct experiments on the CrowS-Pairs, StereoSet, and SEAT benchmarks across gender and race dimensions using BERT. As shown in Table 1, HEIMAT consistently achieves the best or near-best performance, with scores closest to the ideal value of 50. In particular, on the CrowS-Pairs benchmark, HEIMAT reduces the Gender bias of BERT from 58.01 to 50.00 and achieves the lowest Race bias score (47.48), outperforming all competing baselines. HEIMAT also achieves consistently strong performance on StereoSet, and yields the lowest effect sizes on the SEAT benchmark for both Gender and Race (reported in Table 8 of Appendix C).

Similar trends are observed for LLaMA-2 and GPT-2, where HEIMAT achieves the lowest bias for both models, despite their reduced model capability. These results demonstrate that HEIMAT effectively mitigates demographic biases across different bias types, validating the robustness and generality of the proposed heuristic-style debiasing framework.

In contrast, several baselines show inconsistent behavior across datasets, improving bias under one metric while degrading under another. HEIMAT, however, achieves consistent improvements across all evaluated benchmarks, suggesting better robustness and generalization.

We further investigate the behavior of HEIMAT on whole CrowS-Pairs by jointly analyzing the overall stereotype tendency and its transfer effects across bias categories. As shown in Table 4, HEIMAT consistently reduces Stereotype scores and moves the Overall scores of all three PLMs significantly closer to the ideal value of 50, indicating effective mitigation of dominant stereotypical bias across different model capacities. In particular, HEIMAT reduces the Overall score of BERT from 60.48 to 51.86 and that of TinyBERT from 58.16 to 46.22. We also observe increases in Anti-Stereotype scores for ALBERT and TinyBERT, suggesting a shift away from stereotypical preferences toward anti-stereotypical predictions, which is expected given the strong initial bias of these models and remains beneficial as reflected by the improved Overall scores.

Beyond the two debiased categories, Table 3 further shows that debiasing conducted only on gender and race leads to consistent score shifts across multiple unseen bias categories, including socio-economic status, nationality, religion, and physical appearance. Such cross-category improvements indicate that different forms of social bias are intrinsically correlated in models probably, and that mitigating prominent biases can partially generalize to related dimensions.

To evaluate the effectiveness of HEIMAT in multilingual and multicultural settings, we conduct debiasing experiments on two French PLMs, CamemBERT and FrALBERT, using the French version of CrowS-Pairs. The results are reported in Tables 3. When applying HEIMAT to French PLMs, the prompt templates are adapted to account for the grammatical structure of French, which differs from English in that nouns are associated with grammatical gender and require agreement with corresponding articles and adjectives. As illustrated in Fig. 4, feminine and masculine forms exhibit systematic suffix variations, challenging fixed-template methods with static word lists (Dong et al., 2024; Zeng et al., 2024) in cross-lingual and cross-cultural adaptation. Our method retains template structure but replaces static lists with LLM-generated content, offering both flexibility and clarity.

Despite this additional linguistic complexity and cultural difference, HEIMAT consistently reduces bias on both models, bringing the Overall scores closer to the ideal value of 50. Specifically, HEIMAT reduces CamemBERT’s Overall score from 57.36 to 50.69 and improves FrALBERT’s Anti-Stereotype balance by aligning its predictions toward a more neutral distribution. Therefore, these results demonstrate that HEIMAT effectively mitigates bias in French PLMs and can be readily extended to other cultures and languages

Figure 5 presents a qualitative comparison of the prediction probability distributions produced by the original and debiased ALBERT under a controlled prompt setting. Specifically, we use the prompt “This [JOB] works in a famous hospital and [MASK] is professional” and substitute “nurse” and “doctor” for [JOB] to examine gender pronoun predictions. In the original ALBERT, a pronounced disparity is observed: when the job is “nurse”, the model assigns substantially higher probability to the feminine pronoun “she”, whereas for “doctor”, the masculine pronoun “he” dominates, revealing strong occupation–gender associations.

In contrast, the debiased model exhibits markedly more balanced prediction distributions. The probability gap between “he” and “she” is significantly reduced for both occupations, indicating that the model’s predictions become less sensitive to gender stereotypes associated with specific jobs. This qualitative analysis demonstrates that HEIMAT effectively attenuates demographic-conditioned prediction shifts at the distributional level, aligning with the quantitative improvements observed on bias benchmarks.

Similar phenomenon can be observed in Llama-2, as shown in Table 5 Before debiasing, the model tends to complete prompts involving racial identifiers with narratives of violence and victimization. After debiasing, while negative outcomes may still appear, the generated continuations no longer systematically attribute harm to external agents conditioned on race, suggesting a reduction in identity-conditioned violent associations.

Since HEIMAT mitigates bias via fine-tuning, it is important to examine its potential impact on the model’s NLU performance (Meade et al., 2022). We therefore evaluate HEIMAT on standard NLU benchmarks to assess whether bias reduction is achieved while preserving NLU capability.