The rapid advancement of global AI translation technology has given birth to TransBench, the first application-oriented AI translation benchmark. Launched by Alibaba's International AI Business team, Shanghai Artificial Intelligence Laboratory, and Beijing Language and Culture University, TransBench aims to provide the industry with a standardized assessment of translation quality.
Unlike traditional translation evaluations, TransBench introduces new metrics such as illusion rate, cultural taboo words, and honorific norms, focusing on critical issues in large model translations. These metrics are derived from real-world usage feedback, striving to reflect the practicality and cultural adaptability of translations. For instance, translations are marked as "illusions" if they contain fabricated information, and the evaluation is affected if translations do not conform to local culture or lack necessary polite expressions.
According to the latest evaluation results, GPT-4o secures its position as the "ceiling" of translation AI, excelling in multilingual translation with the highest composite score. DeepL Translate and GPT-4-Turbo follow closely behind. DeepL Translate, designed specifically for machine translation, has significantly improved translation quality with its latest version released last month. In the e-commerce industry, DeepSeek-R1 also stands out, demonstrating its competitiveness in specific domains.
In terms of cultural characteristics, the Qwen series models shine, with Qwen2.5-0.5B-Instruct and Qwen2.5-1.5B-Instruct taking the top two spots, showcasing their advantages in cross-cultural translation. Developed by multiple research institutions, this series supports multiple languages and aims to enhance the cultural adaptability of translations.
For Chinese translation, GPT-4o again ranks first, with DeepSeek-V3 and Claude-3.5-Sonnet following closely. Particularly in the e-commerce field, DeepSeek-V3 has garnered widespread attention due to its excellent scores.
TransBench's evaluation methods and datasets are now open-source, encouraging major AI translation organizations to participate in horizontal comparisons and performance assessments. This move not only provides a basis for industry standardization but also promotes further development of AI translation technology.
Alibaba's International AI Business team stated that as translation technology continues to advance, the industry's requirements for translation models become increasingly stringent. TransBench is a benchmark launched in response to this demand. In the future, Alibaba International will continue to commit to the application of AI technology, helping more businesses achieve global development.
As competition in the AI translation market intensifies, the release of TransBench undoubtedly provides the industry with a clear benchmark and offers users an additional reliable reference standard when choosing translation services.
