A small multilingual model guesses on almost every question it cannot answer. We fine-tuned it to abstain instead, with labels that need no gold answers in the target language. Here are the same 252 test questions, before and after.
Each square is one test question. Counts are the mean rate over 3 training seeds, scaled to 252 questions; the base model is a single deterministic run.
Teacher-written traps (R1) help far less: in English, Spanish and Malay they lower it only from about 0.8 to 0.7.
R4 uses no gold answers in the target language. It is not significantly different from R2 in English, Spanish and Malay.
English to Spanish and Malay helps. Arabic and the other pairs move much less.
Each bar is all 252 test questions. A good abstaining model turns red into grey without losing much green.
Over-refusal is the share of questions the base model answered correctly that the tuned model refused. Abstain recall is the share of questions the base model got wrong that the tuned model refused. Means over 3 seeds; the standard deviation is under hallucination.
Paired bootstrap on the test questions, outcomes averaged over seeds. A filled dot means the 95% interval excludes zero. A hollow dot means we cannot tell the two recipes apart.
R4 matches R2 in English, Spanish and Malay. In Arabic R4 is more conservative: 0.040 less hallucination, but also 0.021 fewer correct answers. Against R1, R4 cuts hallucination by 0.32 to 0.52 in every language.
Rows are the language an adapter was trained on, columns the language it is tested on. The outlined diagonal is same-language. Seed 1 only.
English-trained adapters help most on Spanish and Malay, about 0.35 less hallucination than the base model. Arabic and the other pairs move much less. Base-model hallucination is 0.80 to 0.85 everywhere, so values near that mean no transfer.
AUROC for telling known from unknown questions, with 95% intervals. The probe trains on English gold labels and is transferred; consistency needs at least 4 of 5 sampled answers to agree.
The intervals overlap in every language, so the fair reading is "comparable or better", not "clearly better". Precision of the known class is only 0.40 to 0.48 for both sources.