Right Frame, Wrong Rule: Cultural Cues
Expose the Financial Knowledge Gap They Were Meant to Close

Stereotype benchmarks (Parrish et al., 2022; Nangia et al., 2020; Nadeem et al., 2021) test for demographic-attribute association against a single correct answer, an orthogonal failure mode to normative framework selection. Cultural alignment work (Myung et al., 2024; Durmus et al., 2023; Chiu et al., 2025; Rao et al., 2025; Vo and Koyejo, 2025) confirms LLMs possess less non-Western knowledge, but a model may answer Islamic-finance questions correctly under explicit Shariah framing yet suppress that knowledge when contextual signals should trigger it. CAMeL (Naous et al., 2024) is the closest prior, measuring entity preference via token probabilities; we extend it to framework selection in an expert domain, decomposing lean from correctness.

Islamic knowledge benchmarks (Atif et al., 2025; Elmahjub et al., 2026; Abdelaal et al., 2026; Alwajih et al., 2025) evaluate jurisprudence and scripture but none isolates finance or tests routing; financial bias surveys (Nie et al., 2024; Lee et al., 2025) document that demographic signals shift recommendations without controlling for framework possession. Financial NLP benchmarks (Chen et al., 2024; Xie et al., 2024) and recent systems (Zhou et al., 2026; Xie et al., 2026; Zhang et al., 2026) assume the applicable framework is fixed; our benchmark tests whether models select the appropriate framework under cultural context and remain correct within it.

Expert personas reduce factual performance by 3–5 points (Hu et al., 2026), RLHF alignment trades task performance for safety (Lin et al., 2024), steering interventions incur side effects (Stickland et al., 2024), and counterfactual cultural cues drop medical QA accuracy by 3–7 points (Rezaei and Shakeri, 2026). (Khanuja et al., 2026) show non-default cultural directions require explicit anchor cues. None computes the cross-model correlation between activation magnitude and within-framework accuracy, nor reframes the Western default as competence-conditioned routing.

The benchmark comprises three evaluation sets. The bilateral framework set ($n{=}304$) contains questions admitting valid answers under both Islamic and Western finance; the CI/CW/II/IW taxonomy operates here. The Western-anchor controls ($n{=}64$) contain questions with a valid Western answer but no distinctly Islamic counterpart, isolating within-Western competence. The Islamic-anchor controls ($n{=}41$) contain questions whose underlying construct is unique to Islamic jurisprudence, such as waqf (perpetual charitable endowment), isolating within-Islamic competence and ruling out signal conditioning as the stereotype trap’s source.

Both the bilateral set and the Islamic-anchor controls originate from SAHM (Elbadry et al., 2026), an expert-validated Arabic Islamic-finance corpus spanning 48 topic codes and seven product clusters (Table 1). Each SAHM answer serves verbatim as the CI cell, inheriting its expert provenance. Four stages transform these sources into the evaluation instrument (Figure 2): stem neutralization, Western answer generation, distractor generation, and signal injection. All generation steps use Sonnet 4.5 (Anthropic, 2025b); every stage is independently validated by domain experts ($\kappa=0.71$–$0.85$ across stages; details in Appendix A).

Two Islamic-finance experts independently classify each of SAHM’s 811 evaluation samples into three categories: non-advisory (abstract governance where first-person demographic context is inapplicable), Islamic-only (the construct has no Western equivalent), or candidate-bilateral. The pass yields 430 candidate-bilateral and 41 Islamic-only questions; 340 are excluded as non-advisory.

SAHM stems contain framework-specific terminology (murābaḥa, ijāra, AAOIFI standard numbers) that would prime the model toward the Islamic framework before any cultural signal is applied. Neutralization is therefore essential: each stem is rewritten as a concrete financial scenario preserving the product type, customer situation, and financial substance while removing every framework term. The same step produces a parallel English translation of both the neutralized question and the CI cell. Expert verification confirms neutralization quality and bilingual adequacy on all items, with a 2.4% correction rate on borderline cases (rubrics and interface in Appendix A).

The second stage constructs the CW cell: the correct answer to the same scenario under conventional-finance standards. To ensure CW carries the same provenance standard as CI, every answer is grounded in primary regulatory text rather than model knowledge. For each cluster, the full source documents (one to three per cluster, 12 total; Table 1) are provided in-context alongside the neutralized question. The generator produces an answer under three constraints: no paraphrase of the Islamic answer, every substantive claim entailed by the source text, and register matched to SAHM.

Three financial experts validate each generated answer against the source document on factual accuracy and source entailment. Of 430 candidates, 304 pass: 78 are excluded because the Islamic and Western answers converge on the same economic outcome despite different terminology, and 48 fail accuracy. Two Islamic-finance experts independently confirm that each surviving CI, CW pair recommends substantively different financial products ($\kappa=0.82$). Rubrics and audit prompts are in Appendix E.

Sonnet 4.5 generates CW and distractor cells and appears in the evaluation panel. Excluding its evaluation rows leaves the signal hierarchy unchanged (top-10 rank preserved, $\leq 0.03$ deviation). To control for distractor provenance, we regenerate the four-cell grid for a 50-item subset using GPT-4o; tier-level IFR patterns are preserved (per-model deviation $\leq\pm 0.03$).

Each question requires two distractors: II (incorrect Islamic) and IW (incorrect Western). The design goal is asymmetric difficulty: a model relying on terminological pattern-matching should find the distractor plausible, while a domain expert should identify 2 to 3 semantic errors targeting liability assignment, contract scope, or instrument identity. Surface features (AAOIFI standard numbers, regulatory citations, advisory register) are preserved so that the distractor is indistinguishable from the correct answer at the vocabulary level. Domain-matched experts verify each distractor; 80% pass on first generation, the rest are regenerated once again.

Cultural signals enter as short first-person prefixes prepended to the neutralized question, isolating the signal effect from register changes that full stem rewriting would introduce. The 50 signal codes span nine families (Figure 3), decomposing framework lean along four dimensions: cultural identity (names at six tiers of religious specificity, crossed with gender), declared or implied belief (from explicit declaration to behavioural cues such as Ramadan observance), regulatory jurisdiction (ranked by Islamic-banking mandate strength), and professional context. A keyword ceiling (keyword_sharia: “I want a Shariah-compliant option”) and two stacked composites establish empirical bounds; ten conflict cells compose opposing cues. Stack and conflict signals concatenate atomic prefixes in fixed order, enabling inclusion, exclusion residual analysis of compositional effects (Complete inventory in Appendix C).

Not every question, signal pairing is coherent: a gold-venue signal paired with a lending question, or an institutional-investor prefix on a consumer credit card query, would confound evaluation. An LLM classifies each cell as coherent, awkward, or incoherent; one expert validates on 100 stratified cells ($\kappa=0.79$). Only coherent cells are retained (mean 38.6 per question per language). Coherence filtering is uniform across all twelve evaluation models. The full annotation panel comprises two Islamic-finance experts, three financial experts, and a senior researcher as adjudicator; demographics and compensation are in Appendix G.

For each combination of model, language, and signal, $P(X)$ denotes the observed proportion of responses assigned to category $X$. Table 2 summarizes the five metrics used in our analysis.

The Western default is not uniform across financial topics. Where Islamic products carry recognisable brand names (sukūk, muḍāraba, qarḍ), models show partial Islamic routing at baseline ($p_{\text{islamic}}$ $0.21$–$0.40$). Where the two frameworks differ only in institutional rules (waqf governance, insolvency priority, documentary credit liability), baseline routing falls near zero (Table 3). Only Opus defaults Islamic ($p_{\text{islamic}}=0.84$); the remaining panel falls below $0.41$, with six midsize models below $0.13$. Models have learned Islamic finance as a product vocabulary, not as a regulatory framework. Yet the knowledge is latent: a single Shariah-compliance request lifts every topic above $p_{\text{islamic}}=0.77$.

Cultural signals shift this baseline asymmetrically (Table 4). The strongest Islamic cue (keyword_sharia, $\Delta p_{\text{islamic}}{=}+0.66$) is FDR-significant across the full panel; the strongest Western cue (rel_secular_explicit, $-0.11$) reaches significance in only three model–language cells. The asymmetry is not a coverage artefact: occ_islamic_bank and occ_conventional_bank share the same prompt format and comparable item counts, yet the Islamic-bank cue reaches FDR-significance in ten model–language cells while the conventional-bank cue reaches none. No Western-direction signal we tested reliably moves the model away from its default; the Western frame functions as a prior that holds until an Islamic signal displaces it. The same table exposes a deeper split. Signals the model can pattern-match on identity vocabulary fire reliably: occ_islamic_bank ($+0.62$) and occ_islamic_nonfinance ($+0.32$) both contain the word “Islamic.” Signals that require structural financial knowledge do not: occ_institutional ($-0.02$) and occ_trade_professional ($+0.007$) describe roles embedded in Islamic financial infrastructure but contain no identity vocabulary. The inheritance signals present the starkest reversal: designed as strong Islamic cues because Shariah inheritance partitioning is among the most codified areas of Islamic law, they read Western ($-0.08$, $-0.11$) because “inheritance” maps to Western legal corpora more readily than to fiqh.

Among names (Figure 4), the ordering subverts the intuition that name fame drives activation. Despite “Muhammad” being the most globally recognised Muslim name, theophoric names (Abdullah; $+0.13$) produce the strongest shift, outpacing prophetic names (Muhammad; $+0.11$). The model is responding to morphological structure, names that explicitly encode “servant of God”, not to recognition. Arab cultural names (Khaled, Tarek; $+0.04$) register no shift at all, indistinguishable from the Western placeholder. The most informative case is Christian Arabic names (name_christian_arab, e.g. Boutros; $+0.05$): they cluster with Muslim-coded names, not with Western names, even though they unambiguously code a non-Islamic religion. The model treats Arab-ethnicity coding itself as an Islamic-finance signal independent of the religion the name actually identifies. Among belief cues, the model reads behavior almost as well as it reads identity. Mentioning Ramadan fasting or Zakat giving ($+0.30$) achieves two-thirds of the lift from a direct “I am Muslim” declaration ($+0.49$); a Hijri-calendar date ($+0.23$), designed as a weak control, lands in the same band. The pattern mirrors (Hofmann et al., 2024) implicit/explicit race gap: post-training suppresses what users say outright, not what they reveal through behavior. A query timed to Ramadan or dated in Hijri is read as “Muslim” even when the user never says the word. The same hierarchy holds in Arabic ($\bar{\rho}{=}0.96$), with every baseline shifted roughly $2\times$ higher; the cross-lingual ceiling and floor effects are examined in §5.

Section 4 showed that cultural signals redirect models toward Islamic framing. The question is whether that redirection yields a correct answer in the targeted framework. We measure within-frame accuracy symmetrically. The Islamic Fake Rate $\mathrm{IFR}=P(\mathrm{II})/p_{\text{islamic}}$ is the share of Islamic-frame responses stating a wrong AAOIFI rule; its Western counterpart $\mathrm{WFR}=P(\mathrm{IW})/[P(\mathrm{CW})+P(\mathrm{IW})]$ measures the same on the Western side (Table 5). Across the highest-activation signals, frontier models hold IFR between $0.02$ and $0.11$; open-weight models range from $0.33$ to $0.79$, with the smallest models suffering most (Gemma-3-4b at $0.79$, Llama-8B at $0.75$, falling to $0.57$ for the largest open-weight model, Gemma-3-27b). The gap is absolute: the worst frontier IFR ($0.114$) is three times lower than the best non-frontier IFR ($0.333$). Opus is the only model where forced activation improves correctness (IFR drops from $0.098$ to $0.075$). At the other extreme, Qwen-14B’s IFR rises to $0.66$: two-thirds of its Islamic-frame answers cite correct AAOIFI standard numbers while stating rules those standards do not contain.

The trap is direction-specific (Table 6). Under the Shariah keyword, non-frontier IFR reaches $0.58$–$0.68$ while WFR on the same items stays at $0.10$–$0.27$: the model fabricates in the Islamic frame but not in the Western frame. Two control sets close the remaining exits. On Western-anchor questions, non-frontier tiers retain $80$–$85\%$ correctness, ruling out general financial incompetence. On Islamic-anchor questions: waqf, Zakat, musaqah, ju’āla, with no Western alternative), non-frontier IFR remains $0.52$–$0.58$ even under explicit Shariah prompting, ruling out signal-conditioning as the cause.

The trap is also signal-invariant (Table 7). Across eight signals spanning a $16\times$ range in activation strength, non-frontier IFR stays in the $0.52$–$0.68$ band while WFR stays in $0.21$–$0.29$. The trap is not what any particular cue does; it is what the model lacks behind every cue. The split is categorical, not gradient (Figure 5). Frontier produces 30 clean_lift cells and zero trap cells; large tier produces zero lifts and 26 traps, invariant under three threshold settings (Appendix J). The Arabic-centric tier, purpose-built for Arabic and Islamic finance, falls into the same traps as the generalist midsize tier.

The signal-invariance of IFR (§4) implies that routing and execution are served by separate representations. If they shared a single layer, different signal families would produce different correctness costs. They do not (Figure 11): $\tau$ is near-constant across all cue families within each tier, with tier explaining $71.6\%$ of IFR variance and signal family explaining $2.6\%$. The cue picks which frame; the tier determines what the model finds inside it.Cross-lingual evaluation confirms the separation. Switching from English to Arabic shifts frontier routing by $+0.10$ to $+0.54$ while moving frontier IFR by at most $0.03$. For non-frontier models, IFR moves with routing ($+0.03$ to $+0.17$): language shifts routing and execution together, consistent with a shallow layer that entangles the two. The pattern is not two independent language regimes but one shared surface operating at different baselines: the per-signal AR–EN activation gap follows a saturation curve ($r{=}-0.92$, $R^{2}{=}0.84$), where Arabic provides a higher floor for weak signals and English provides a higher ceiling for strong ones, converging as signal strength increases.

Neither scale nor language specialisation closes the trap. The Shariah keyword increases IFR on six of seven clusters; the exception is Arabic f_sarf (gold trading), where the keyword reduces IFR across all three Arabic-centric models ($\Delta\mathrm{IFR}\approx-0.16$), the only cell where every Arabic-centric model escapes. f_sarf is governed by AAOIFI Standard No. 1, the most codified rule in the corpus. Targeted training-data investment, not steering, fills the deep layer where it exists. On d_securities the large tier gives zero correct-Islamic responses on the institutional cue ($n{=}5$, Table 20); the Shariah keyword on the full cluster ($n{=}48$) unlocks Islamic routing but at $\mathrm{IFR}{=}0.63$–$0.73$. Two pre-registered predictions encoding institutional structure over identity tokens were rejected: occ_trade_professional produces $\Delta p_{\text{islamic}}=+0.007$ while occ_islamic_bank produces $+0.62$. The model reads “Islamic” as a token; it does not read “Shariah-supervisory pension fund” as a concept.

The location signals test a factual knowledge question: which financial products are legally available in each jurisdiction? The model fails on this layer. It orders jurisdictions in the right direction (Gulf, then non-Gulf Arab, then Muslim non-Arab, then Western; Table 8) but cannot distinguish statutory single-system jurisdictions (Iran, Sudan: only Islamic banking exists by law) from dual-system Islamic-dominant jurisdictions (Saudi Arabia: conventional banks fully licensed despite $\approx\!80\%$ Islamic market share) from dual-system mixed jurisdictions (Egypt: $\approx\!40\%$ Islamic). Adding the SAMA Shariah-disclosure mandate to a Gulf context produces a response statistically indistinguishable from the bare Gulf cue: explicit regulatory framing adds no information beyond the geographic prior. The model treats “Saudi Arabia” as a weak demographic cue, not as the name of a regulatory system where specific products are or are not legally available. The conflict cells expose what this costs. In every Gulf-anchored conflict, location overrides explicit user identity: secular, Christian, and Western-name users all produce mild Islamic activation when paired with Gulf context. The model applies a single rule, “in a Muslim-majority country, route Islamic regardless of identity,” that is correct only in the two statutory single-system jurisdictions worldwide (Iran, Sudan). It is the wrong rule everywhere else. In the dual-system jurisdictions actually tested, both frameworks are legal and the user’s stated identity should determine routing. The model has no representation that Saudi differs from Iran in this dimension.

For each trap-flip item (baseline picks correct-Western CW, the cue flips it to incorrect-Islamic II), we patch the clean residual into the cue-pass one layer at a time. Across all $40$ cells the commitment localises to a single gate in the back half (proportional depth $0.55$–$0.84$; Table 9: patching before it does nothing, patching at or after it recovers the Western answer in $59$–$100\%$ of items not a last-layer slip but a deep commitment. Reading the grid two ways separates cause from effect: within a model the five cues commit at nearly identical depth (spread as low as $\Delta{=}0.02$), but across models the same cue lands at very different depths ($\Delta$ up to $0.42$). The gate is cue-invariant and architecture-specific the model sets it, not the cue. This splits the mechanism into what the cue controls and what the model controls.

Why then does keyword_sharia produce five times the routing of a name ($\Delta p_{\text{islamic}}=+0.66$ vs $+0.13$)? Not by activating earlier through the early layers, a Muslim name activates Islamic framing more strongly (Figure 6). The keyword’s advantage is built at the gate: its framing survives (late-layer $p_{\text{islamic}}$ exceeds other cues by $+0.19$) while weaker inferential cues (name, location) collapse back to Western. The cue is a volume knob on routing survival, not on the gate or the answer.

Competence is within-frame correctness given an Islamic answer, the AAOIFI rule (CI) or a stereotype (II) read as the margin $m(L)=p_{\text{CI}}-p_{\text{II}}$ per layer (Figure 7). Its peak sign splits two regimes: in six of eight models the margin is positive early (the correct answer leads) then crosses negative at the gate the model held the answer and suppressed it; in the two Gemma-3 models it is never positive, a genuine knowledge gap. For most models the trap is deletion, not absence. The two axes then meet: gate depth predicts the stereotype rate ($r=-0.78$; deeper commitment lets competence act before the answer locks; Table 9), while baseline routing is uninformative about it ($r=-0.01$; Table 9). Routing (surface) and competence (deep) are orthogonal.

Counting each trap under its own-direction cue Islamic traps (CW$\to$II) under Islamic cues, Western traps (CI$\to$IW) under Western cues Islamic traps outnumber Western $1{,}084$ to $122$, an $8.9\times$ asymmetry: models abandon a correct Western answer for a wrong Islamic one far more readily than the reverse, the mechanistic correlate of IFR$\gg$WFR (§4). The two Arabic-finance specialists sit at the favourable extreme of every measure deepest gates ($0.84$, $0.76$), lowest asymmetry ($1.1\times$, $4.4\times$ vs generalist $23$–$111\times$), largest margins. ALLaM is the sharpest case: it holds the correct answer at a $+0.59$ margin the most confident of any model yet still suppresses it to $-0.31$. Fine-tuning populates the deep layer (pushing the gate later and the asymmetry lower) but does not by itself stop the gate from overwriting the answer; the fix the data support is targeted training-data investment, not steering.

Our results describe a harm that can occur in deployed systems. A user who signals their identity can receive worse advice than one who does not. These outcomes vary together across models: signals can shift framework selection while exposing differences in within-framework accuracy, particularly among non-frontier models; however, our current analysis does not establish a causal or predictive relationship between the two measures. Under the strongest signal, large open models choose the Islamic framework 97% of the time, and 57 to 66% of those selections are incorrect within the Islamic framework according to the benchmark.

Users may not easily identify this failure. Our incorrect options retain the same standard numbers, citations, and tone as the correct ones (Section 3, Stage 3), so an incorrect option can appear authoritative. Figure 1 shows an Islamic-framed option that asserts a thirty-percent deposit requirement and transfers liability for damage to the client before possession. Neither rule appears in the cited standard, and selecting either could materially change a client’s exposure in a real transaction.

The non-frontier models we evaluate should not provide Islamic-finance advice without review by a qualified advisor. Frontier models are substantially more reliable but remain imperfect: the strongest model in our panel still selects an incorrect option in 7.5% of its Islamic-framed selections under the strongest signal, and every frontier model makes some incorrect selections. Cultural cues are often proposed as a way to reduce bias in language models; in this setting, they expose differences in within-framework competence.

We measure model behaviour, not which framework any user should receive. Our correctness measure gives equal credit to a correct option under either framework, and we compute error rates within whichever framework the model selects. Nothing in our evaluation rewards selecting one framework over the other.

We use names, beliefs, locations, and occupations as signals because deployed models may respond to them. We ask how these cues affect model behaviour and who may consequently be exposed to a model’s knowledge gaps. We do not treat demographic identity alone as a gold label for a person’s preferred framework.

Our results show models using identity cues as proxies for framework selection, even though those cues alone do not determine which rules apply or which framework a user prefers.

Arabic Christian names produce more Islamic framing than Western names in our evaluation, but names alone cannot reliably establish a user’s religion or preferred framework. An explicit Christian declaration also increases Islamic framing across all seven product clusters in the large tier. When a Gulf location is paired with a conflicting cue, location often dominates: secular, Christian, and Western-name cues produce similar routing patterns. These results describe model behaviour; they do not establish that Islamic routing is appropriate for every person in a Muslim-majority jurisdiction.

The location analysis shows a related limitation. Models respond differently to location cues, but the executed prompts name cities rather than legal mandates or regulators. The Tehran result therefore measures a Tehran location-cue effect, not demonstrated knowledge of Iran’s banking requirements. Likewise, the Riyadh result measures a Riyadh location-cue effect rather than explicit SAMA knowledge. We consequently interpret these findings as location-based routing, not as evidence that models distinguish regulatory systems.

Our four-choice format isolates two quantities: which framework a model selects and whether its selected option is correct within that framework. Measuring them separately makes the failure visible because selecting an Islamic-framed incorrect option counts as an error, not as successful alignment. A different instrument would be needed to evaluate responses that present both frameworks alongside their sources; that is separate from the question we study here.

Our questions cover products for which two frameworks specify different procedures for the same client. They do not cover rules that assign different entitlements to different people. Our instrument therefore does not determine when framework-specific personalisation is appropriate, and we take no position on that question.

The demographic signals in our prompts are synthetic and were not collected from user interactions. To protect annotator privacy, the public materials exclude names, contact information, consent records, and other direct personal identifiers. Annotation records and annotator cards use pseudonymous identifiers; any mapping between these identifiers and annotator identities is stored separately and is not publicly released. Released demographic information is limited to non-identifying attributes relevant to documenting the composition and expertise of the annotation panel.