This codebase is the official implementation of EfficientXLang: Towards Improving Token Efficiency Through Cross-Lingual Reasoning.

We recommend using uv for package installation. It's significantly faster than pip and provides better dependency resolution.

# Install uv if you don't have it already
curl -LsSf https://astral.sh/uv/install.sh | sh

# Clone the repository
git clone https://github.com/microsoft/EfficientXLang.git
cd EfficientXLang

# Install dependencies using lockfile for consistent environments
uv pip install -e . --lockfile uv.lock

The best way to use Sglang is to download the specific Docker Image corresponding to your GPUs from here and then edit the start_sglang.sh script to select the model accordingly.

To query any of the model hosted using Sglang, please have a look at the run_client.py file. You can edit the EvaluationConfig class in the run_client.py file for any language/model and dataset that is currently supported in utils.py

Setup a .env file with the required OpenAI Azure API parameters described in the openai_client.py as enviornment variables. Then you can run the translate_reasoning_traces.py file by making required changes to the TranslationConfig dataclass for the arguments.

To compare English v/s non-English reasoning traces and judge which is better overall, run an automatic pairwise comparison using GPT-4o as the judge (accounting for positional bias) by making changes to the EvalConfig dataclass for the arguments. A sample command is below:

python run_comparison.py --input_base_dir /root/EfficientXLang --output_base_dir /root/EfficientXLang --model_to_eval qwen --metrics ["aime2024"] --languages ["Arabic"]

Outputs are saved in

1. {output_base_dir}/{model_to_eval}/{metric}/comparison_results/comparison_scores_{lang}.csv which contains the evaluated responses, choice in each direction (A v/s B and B v/s A) along with the corresponding reasoning
2. {output_base_dir}/{model_to_eval}/{metric}/comparison_results/summary_stats_{lang}.json which contain the following fields as in this example:

   {
   "total_examples": 6,
   "eng_wins": 1,
   "xlang_wins": 3,
   "neither_wins": 0,
   "inconsistent_results": 2,
   "eng_win_percent_of_attempts": 16.666666666666664,
   "xlang_win_percent_of_attempts": 50.0,
   "neither_win_percent_of_attempts": 0.0,
   "inconsistent_percent_of_attempts": 33.33333333333333,
   "num_direct_contests": 4,
   "eng_win_rate_h2h": 25.0,
   "xlang_win_rate_h2h": 75.0
   }
   

Next, conduct a binomial test to determine significance & win rates for the comparison by modifying the BinomialTestConfig dataclass for the arguments. A sample command is below:

python binomial_test.py --input_base_dir /root/EfficientXLang --output_base_dir /root/EfficientXLang --model_to_eval qwen --metrics ["aime2024"] --languages ["Arabic"]

Outputs are saved in {output_base_dir}/{model_to_eval}/{metric}/binomial_test/result_{lang}.json and contain binomial test results with fields as in the following example:

{
    "num_xlang_wins_h2h": 15,
    "num_direct_contests": 17,
    "reject_null": true,
    "p_value": 0.0011749267578125,
    "proportion_estimate": 0.8823529411764706
}

If you use this codebase in your research, please cite our paper with the following BibTex entry:

@article{ahuja2025efficientxlang,
  title={EfficientXLang: Towards Improving Token Efficiency Through Cross-Lingual Reasoning},
  author={Ahuja, Sanchit and Vaddamanu, Praneetha and Patra, Barun},
  journal={arXiv preprint arXiv:2507.00246},
  year={2025}
}

This code is released under the MIT License (see LICENSE).

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

We are releasing this code for others to replicate the results in our paper. All the models and datasets used by us, are open-source and easily accessible.

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

You can read more about Microsoft's privacy statement here.