xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detection

16 October 2023

Papers citing "xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detection"

50 / 77 papers shown

Title
Remedy: Learning Machine Translation Evaluation from Human Preferences with Reward Modeling Shaomu Tan Christof Monz 29 0 0 18 Apr 2025
AskQE: Question Answering as Automatic Evaluation for Machine Translation Dayeon Ki Kevin Duh Marine Carpuat 24 0 0 15 Apr 2025
Large Language Models as Span Annotators Zdeněk Kasner Vilém Zouhar Patrícia Schmidtová Ivan Kartáč Kristýna Onderková Ondřej Plátek Dimitra Gkatzia Saad Mahamood Ondrej Dusek Simone Balloccu ALM 27 0 0 11 Apr 2025
Redefining Machine Translation on Social Network Services with Large Language Models Hongcheng Guo Fei Zhao Shaosheng Cao Xinze Lyu Z. Liu ... Boyang Wang Z. Li Chonggang Lu Zhe Xu Yao Hu 23 0 0 10 Apr 2025
DeepSeek vs. o3-mini: How Well can Reasoning LLMs Evaluate MT and Summarization? Daniil Larionov Sotaro Takeshita Ran Zhang Yanran Chen Christoph Leiter Zhipin Wang Christian Greisinger Steffen Eger ReLM ELM LRM 66 0 0 10 Apr 2025
Zero-shot Benchmarking: A Framework for Flexible and Scalable Automatic Evaluation of Language Models José P. Pombal Nuno M. Guerreiro Ricardo Rei André F. T. Martins ALM 66 0 0 01 Apr 2025
MT-RewardTree: A Comprehensive Framework for Advancing LLM-Based Machine Translation via Reward Modeling Zhaopeng Feng Jiahan Ren Jiayuan Su Jiamei Zheng Zhihang Tang Hongwei Wang Zuozhu Liu LRM 46 1 0 15 Mar 2025
Adding Chocolate to Mint: Mitigating Metric Interference in Machine Translation José P. Pombal Nuno M. Guerreiro Ricardo Rei André F. T. Martins 58 0 0 11 Mar 2025
Compositional Translation: A Novel LLM-based Approach for Low-resource Machine Translation A. Zebaze Benoît Sagot Rachel Bawden 70 0 0 06 Mar 2025
QE4PE: Word-level Quality Estimation for Human Post-Editing Gabriele Sarti Vilém Zouhar Grzegorz Chrupała Ana Guerberof Arenas Malvina Nissim Arianna Bisazza 38 0 0 04 Mar 2025
InfiniSST: Simultaneous Translation of Unbounded Speech with Large Language Model Siqi Ouyang Xi Xu Lei Li 48 1 0 04 Mar 2025
SwiLTra-Bench: The Swiss Legal Translation Benchmark Joel Niklaus Jakob Merane Luka Nenadic Sina Ahmadi Yingqiang Gao ... Matthew Guillod Robin Mamié Daniel Brunner Julio Pereyra Niko Grupen AILaw ELM 74 0 0 03 Mar 2025
Enhancing Human Evaluation in Machine Translation with Comparative Judgment Yixiao Song Parker Riley Daniel Deutsch Markus Freitag 60 1 0 25 Feb 2025
Post-edits Are Preferences Too Nathaniel Berger Stefan Riezler M. Exel Matthias Huck 32 0 0 24 Feb 2025
Automatic Input Rewriting Improves Translation with Large Language Models Dayeon Ki Marine Carpuat 38 0 0 23 Feb 2025
M-MAD: Multidimensional Multi-Agent Debate for Advanced Machine Translation Evaluation Zhaopeng Feng Jiayuan Su Jiamei Zheng Jiahan Ren Yan Zhang Jian Wu Hongwei Wang Zuozhu Liu ELM 198 0 0 21 Feb 2025
Varco Arena: A Tournament Approach to Reference-Free Benchmarking Large Language Models Seonil Son Ju-Min Oh Heegon Jin Cheolhun Jang Jeongbeom Jeong Kuntae Kim 39 0 0 20 Feb 2025
BenchMAX: A Comprehensive Multilingual Evaluation Suite for Large Language Models Xu Huang Wenhao Zhu Hanxu Hu Conghui He Lei Li Shujian Huang Fei Yuan ELM 49 3 0 11 Feb 2025
Multilingual Machine Translation with Open Large Language Models at Practical Scale: An Empirical Study Menglong Cui Pengzhi Gao Wei Liu Jian Luan Bin Wang LRM 41 0 0 04 Feb 2025
mHumanEval -- A Multilingual Benchmark to Evaluate Large Language Models for Code Generation Nishat Raihan Antonios Anastasopoulos Marcos Zampieri ELM 36 5 0 28 Jan 2025
Reference-free Evaluation Metrics for Text Generation: A Survey Takumi Ito Kees van Deemter Jun Suzuki ELM 33 2 0 21 Jan 2025
A 2-step Framework for Automated Literary Translation Evaluation: Its Promises and Pitfalls Sheikh Shafayat Dongkeun Yoon Woori Jang Jiwoo Choi Alice H. Oh Seohyon Jung 91 1 0 03 Jan 2025
MT-LENS: An all-in-one Toolkit for Better Machine Translation Evaluation Javier García Gilabert Carlos Escolano Audrey Mash Xixian Liao Maite Melero AIMat ELM 71 0 0 16 Dec 2024
From Jack of All Trades to Master of One: Specializing LLM-based Autoraters to a Test Set M. Finkelstein Dan Deutsch Parker Riley Juraj Juraska Geza Kovacs Markus Freitag 66 0 0 23 Nov 2024
Fine-Grained Reward Optimization for Machine Translation using Error Severity Mappings Miguel Moura Ramos Tomás Almeida Daniel Vareta Filipe Azevedo Sweta Agrawal Patrick Fernandes André F. T. Martins 31 1 0 08 Nov 2024
Mitigating Metric Bias in Minimum Bayes Risk Decoding Geza Kovacs Daniel Deutsch Markus Freitag 23 6 0 05 Nov 2024
Context-Informed Machine Translation of Manga using Multimodal Large Language Models Philip Lippmann Konrad Skublicki Joshua Tanner Shonosuke Ishiwatari Jie-jin Yang 26 0 0 04 Nov 2024
MetaMetrics-MT: Tuning Meta-Metrics for Machine Translation via Human Preference Calibration David Anugraha Garry Kuwanto Lucky Susanto Derry Wijaya Genta Indra Winata OSLM 30 2 0 01 Nov 2024
Speech is More Than Words: Do Speech-to-Text Translation Systems Leverage Prosody? Ioannis Tsiamas Matthias Sperber Andrew Finch Sarthak Garg 21 0 0 31 Oct 2024
Anticipating Future with Large Language Model for Simultaneous Machine Translation Siqi Ouyang Oleksii Hrinchuk Zhehuai Chen Vitaly Lavrukhin Jagadeesh Balam Lei Li Boris Ginsburg 37 0 0 29 Oct 2024
SpeechQE: Estimating the Quality of Direct Speech Translation HyoJung Han Kevin Duh Marine Carpuat 26 0 0 28 Oct 2024
GrammaMT: Improving Machine Translation with Grammar-Informed In-Context Learning Rita Ramos Everlyn Asiko Chimoto Maartje ter Hoeve Natalie Schluter 29 1 0 24 Oct 2024
How Good Are LLMs for Literary Translation, Really? Literary Translation Evaluation with Humans and LLMs Ran Zhang Wei-Ye Zhao Steffen Eger 68 4 0 24 Oct 2024
IntGrad MT: Eliciting LLMs' Machine Translation Capabilities with Sentence Interpolation and Gradual MT Seung-Woo Choi Ga-Hyun Yoo Jay-Yoon Lee 29 0 0 15 Oct 2024
QE-EBM: Using Quality Estimators as Energy Loss for Machine Translation Gahyun Yoo Jay Yoon Lee 24 0 0 14 Oct 2024
Adapters for Altering LLM Vocabularies: What Languages Benefit the Most? HyoJung Han Akiko Eriguchi Haoran Xu Hieu T. Hoang Marine Carpuat Huda Khayrallah VLM 32 2 0 12 Oct 2024
Modeling User Preferences with Automatic Metrics: Creating a High-Quality Preference Dataset for Machine Translation Sweta Agrawal José G. C. de Souza Ricardo Rei António Farinhas Gonçalo Faria Patrick Fernandes Nuno M. Guerreiro Andre Martins 18 5 0 10 Oct 2024
Are Large Language Models State-of-the-art Quality Estimators for Machine Translation of User-generated Content? Shenbin Qian Constantin Orasan Diptesh Kanojia Félix do Carmo ELM 17 0 0 08 Oct 2024
Beyond Correlation: Interpretable Evaluation of Machine Translation Metrics Stefano Perrella Lorenzo Proietti Pere-Lluís Huguet Cabot Edoardo Barba Roberto Navigli 14 2 0 07 Oct 2024
MetricX-24: The Google Submission to the WMT 2024 Metrics Shared Task Juraj Juraska Daniel Deutsch Mara Finkelstein Markus Freitag 31 14 0 04 Oct 2024
A Multi-task Learning Framework for Evaluating Machine Translation of Emotion-loaded User-generated Content Shenbin Qian Constantin Orasan Diptesh Kanojia Félix do Carmo 20 0 0 04 Oct 2024
X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at Scale Haoran Xu Kenton W. Murray Philipp Koehn Hieu T. Hoang Akiko Eriguchi Huda Khayrallah 18 7 0 04 Oct 2024
MetaMetrics: Calibrating Metrics For Generation Tasks Using Human Preferences Genta Indra Winata David Anugraha Lucky Susanto Garry Kuwanto Derry Wijaya 37 7 0 03 Oct 2024
Is Preference Alignment Always the Best Option to Enhance LLM-Based Translation? An Empirical Analysis Hippolyte Gisserot-Boukhlef Ricardo Rei Emmanuel Malherbe C´eline Hudelot Pierre Colombo Nuno M. Guerreiro 20 2 0 30 Sep 2024
MQM-APE: Toward High-Quality Error Annotation Predictors with Automatic Post-Editing in LLM Translation Evaluators Qingyu Lu Liang Ding Kanjian Zhang Jinxia Zhang Dacheng Tao 24 3 0 22 Sep 2024
Improving Statistical Significance in Human Evaluation of Automatic Metrics via Soft Pairwise Accuracy Brian Thompson Nitika Mathur Daniel Deutsch Huda Khayrallah 18 0 0 15 Sep 2024
Guardians of the Machine Translation Meta-Evaluation: Sentinel Metrics Fall In! Stefano Perrella Lorenzo Proietti Alessandro Sciré Edoardo Barba Roberto Navigli 18 3 0 25 Aug 2024
Plug, Play, and Fuse: Zero-Shot Joint Decoding via Word-Level Re-ranking Across Diverse Vocabularies Sai Koneru Matthias Huck M. Exel Jan Niehues 19 0 0 21 Aug 2024
mbrs: A Library for Minimum Bayes Risk Decoding Hiroyuki Deguchi Yusuke Sakai Hidetaka Kamigaito Taro Watanabe 26 3 0 08 Aug 2024
Machine Translation Hallucination Detection for Low and High Resource Languages using Large Language Models Kenza Benkirane Laura Gongas Shahar Pelles Naomi Fuchs Joshua Darmon Pontus Stenetorp David Ifeoluwa Adelani Eduardo Sánchez HILM 23 4 0 23 Jul 2024