Publication Type
Conference Proceeding Article
Version
acceptedVersion
Publication Date
8-2025
Abstract
By training on text in various languages, large language models (LLMs) typically possess multilingual support and demonstrate remarkable capabilities in solving tasks described in different languages. However, LLMs can exhibit linguistic discrimination due to the uneven distribution of training data across languages. That is, LLMs are hard to keep the consistency of responses when faced with the same task but depicted in different languages. In this study, we first explore the consistency in the LLMs’ outputs responding to queries in various languages from two aspects: safety and quality. We conduct this analysis with two datasets (AdvBench and NQ) based on four LLMs (Llama2-13b, Gemma7b, GPT-3.5-turbo and Gemini-pro). The results show that LLMs exhibit stronger human alignment capabilities with queries in English, French, Russian, and Spanish (only 1.04% of harmful queries successfully jailbreak on average) compared to queries in Bengali, Georgian, Nepali and Maithili (27.7% of harmful queries jailbreak successfully on average). Moreover, for queries in English, Danish, Czech and Slovenian, LLMs tend to produce responses with a higher quality (with 0.1494 ��1 score on average) compared to the other languages. Upon these findings, we propose LDFighter, a similarity-based voting, to mitigate the linguistic discrimination in LLMs. LDFighter ensures consistent service for different language speakers. We evaluate LDFighter with both benign queries and harmful queries. The results show that LDFighter not only significantly reduces the jailbreak success rate but also improve the response quality on average, demonstrating its effectiveness.
Keywords
Large language models, linguistic discrimination, jailbreak, defense
Discipline
Programming Languages and Compilers | Software Engineering
Research Areas
Software and Cyber-Physical Systems
Areas of Excellence
Digital transformation
Publication
Proceedings of the 34th International Joint Conference on Artificial Intelligence (IJCAI), Montreal, Canada, 2025 August 16-22
First Page
1
Last Page
12
Identifier
10.48550/arXiv.2404.18534
City or Country
USA
Citation
DONG, Guoliang; WANG, Haoyu; SUN, Jun; and WANG, Xinyu.
Evaluating and mitigating linguistic discrimination in large language models: Perspectives on safety equity and knowledge equity. (2025). Proceedings of the 34th International Joint Conference on Artificial Intelligence (IJCAI), Montreal, Canada, 2025 August 16-22. 1-12.
Available at: https://ink.library.smu.edu.sg/sis_research/10283
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.48550/arXiv.2404.18534