HELP: A Dataset for Identifying Shortcomings of Neural Models in Monotonicity Reasoning

International Workshop on Semantic Evaluation (SemEval), 2019

27 April 2019

Papers citing "HELP: A Dataset for Identifying Shortcomings of Neural Models in Monotonicity Reasoning"

45 / 45 papers shown

Title
How to Improve the Robustness of Closed-Source Models on NLI Joe Stacey Lisa Alazraki Aran Ubhi Beyza Ermis Aaron Mueller Marek Rei 244 0 0 26 May 2025
How Hard is this Test Set? NLI Characterization by Exploiting Training DynamicsConference on Empirical Methods in Natural Language Processing (EMNLP), 2024 Adrian Cosma Stefan Ruseti Mihai Dascalu Cornelia Caragea 115 4 0 04 Oct 2024
Self-Evolving GPT: A Lifelong Autonomous Experiential Learner Jinglong Gao Xiao Ding Yiming Cui Jianbai Zhao Hepeng Wang Ting Liu Bing Qin KELM CLL 188 11 0 12 Jul 2024
Reason from Fallacy: Enhancing Large Language Models' Logical Reasoning through Logical Fallacy Understanding Yanda Li Dixuan Wang Jiaqing Liang Guochao Jiang Qi He Yanghua Xiao Deqing Yang LRM ELM 163 15 0 04 Apr 2024
Estimating the Causal Effects of Natural Logic Features in Transformer-Based NLI ModelsInternational Conference on Language Resources and Evaluation (LREC), 2024 Julia Rozanova Marco Valentino André Freitas CML 155 1 0 03 Apr 2024
Task-Oriented Paraphrase Analytics Marcel Gohsen Matthias Hagen Martin Potthast Benno Stein 195 1 0 26 Mar 2024
GLoRE: Evaluating Logical Reasoning of Large Language Models Hanmeng Liu Zhiyang Teng Ruoxi Ning Jian Liu Qiji Zhou Yuexin Zhang Yue Zhang ReLM ELM LRM 310 8 0 13 Oct 2023
Synthetic Dataset for Evaluating Complex Compositional Knowledge for Natural Language Inference Sushma A. Akoju Robert Vacareanu Haris Riaz Eduardo Blanco Mihai Surdeanu CoGe NAI 147 1 0 11 Jul 2023
NatLogAttack: A Framework for Attacking Natural Language Inference Models with Natural LogicAnnual Meeting of the Association for Computational Linguistics (ACL), 2023 Zióu Zheng Xiao-Dan Zhu AAML LRM 214 6 0 06 Jul 2023
SpaceNLI: Evaluating the Consistency of Predicting Inferences in Space Lasha Abzianidze J. Zwarts Yoad Winter 93 4 0 05 Jul 2023
Evaluating Paraphrastic Robustness in Textual Entailment ModelsAnnual Meeting of the Association for Computational Linguistics (ACL), 2023 Dhruv Verma Yash Kumar Lal Shreyashee Sinha Benjamin Van Durme Adam Poliak 240 7 0 29 Jun 2023
ScoNe: Benchmarking Negation Reasoning in Language Models With Fine-Tuning and In-Context LearningAnnual Meeting of the Association for Computational Linguistics (ACL), 2023 Jingyuan Selena She Christopher Potts Sam Bowman Atticus Geiger 208 22 0 30 May 2023
Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds Victoria Basmov Yoav Goldberg Reut Tsarfaty ReLM LRM 192 8 0 24 May 2023
Estimating the Causal Effects of Natural Logic Features in Neural NLI Models Julia Rozanova Marco Valentino André Freitas CML 126 4 0 15 May 2023
Interventional Probing in High Dimensions: An NLI Case StudyFindings (Findings), 2023 Julia Rozanova Marco Valentino Lucas C. Cordeiro André Freitas 83 8 0 20 Apr 2023
Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4 Hanmeng Liu Ruoxi Ning Zhiyang Teng Jian Liu Qiji Zhou Yuexin Zhang ELM ReLM LRM 324 304 0 07 Apr 2023
Natural Language Reasoning, A SurveyACM Computing Surveys (ACM Comput. Surv.), 2023 Fei Yu Hongbo Zhang Prayag Tiwari Benyou Wang ReLM LRM 270 94 0 26 Mar 2023
tasksource: A Dataset Harmonization Framework for Streamlined NLP Multi-Task Learning and Evaluation Damien Sileo 166 13 0 14 Jan 2023
TestAug: A Framework for Augmenting Capability-based NLP TestsInternational Conference on Computational Linguistics (COLING), 2022 Guanqun Yang Mirazul Haque Qiaochu Song Wei Yang Xueqing Liu ELM 136 0 0 14 Oct 2022
FOLIO: Natural Language Reasoning with First-Order LogicConference on Empirical Methods in Natural Language Processing (EMNLP), 2022 Simeng Han Hailey Schoelkopf Yilun Zhao Zhenting Qi Martin Riddell ... Yingbo Zhou Caiming Xiong Rex Ying Arman Cohan Dragomir R. Radev ReLM LRM 309 149 0 02 Sep 2022
Penguins Don't Fly: Reasoning about Generics through Instantiations and ExceptionsConference of the European Chapter of the Association for Computational Linguistics (EACL), 2022 Emily Allaway Jena D. Hwang Chandra Bhagavatula Kathleen McKeown Doug Downey Yejin Choi LRM 136 23 0 23 May 2022
Systematicity, Compositionality and Transitivity of Deep NLP Models: a Metamorphic Testing PerspectiveFindings (Findings), 2022 Edoardo Manino Julia Rozanova Danilo S. Carvalho André Freitas Lucas C. Cordeiro 157 10 0 26 Apr 2022
Curriculum: A Broad-Coverage Benchmark for Linguistic Phenomena in Natural Language UnderstandingNorth American Chapter of the Association for Computational Linguistics (NAACL), 2022 Zeming Chen Qiyue Gao ELM 193 4 0 13 Apr 2022
An Analysis of Negation in Natural Language Understanding CorporaAnnual Meeting of the Association for Computational Linguistics (ACL), 2022 Md Mosharaf Hossain Dhivya Chinnappa Eduardo Blanco 183 48 0 16 Mar 2022
Neuro-symbolic Natural Logic with Introspective Revision for Natural Language InferenceTransactions of the Association for Computational Linguistics (TACL), 2022 Yufei Feng Xiaoyu Yang Xiao-Dan Zhu Michael A. Greenspan LRM NAI 338 13 0 09 Mar 2022
Decomposing Natural Logic Inferences in Neural NLI Julia Rozanova Deborah Ferreira Marco Valentino Mokanrarangan Thayaparan André Freitas NAI LRM 170 5 0 15 Dec 2021
A Logic-Based Framework for Natural Language Inference in Dutch Lasha Abzianidze Konstantinos Kogkalidis 209 4 0 07 Oct 2021
Weakly Supervised Explainable Phrasal Reasoning with Neural Fuzzy Logic Zijun Wu Zi Xuan Zhang Atharva Naik Zhijian Mei Mauajama Firdaus Lili Mou LRM NAI 149 14 0 18 Sep 2021
Transformers in the loop: Polarity in neural models of languageAnnual Meeting of the Association for Computational Linguistics (ACL), 2021 Lisa Bylinina Alexey Tikhonov 113 0 0 08 Sep 2021
Investigating Transfer Learning in Multilingual Pre-trained Language Models through Chinese Natural Language InferenceFindings (Findings), 2021 Hai Hu He Zhou Zuoyu Tian Yiwen Zhang Yina Ma Yanting Li Yixin Nie Kyle Richardson 151 12 0 07 Jun 2021
NeuralLog: Natural Language Inference with Joint Neural and Logical Reasoning Zeming Chen Qiyue Gao Lawrence S. Moss FedML NAI 213 47 0 29 May 2021
Supporting Context Monotonicity Abstractions in Neural NLI Models Julia Rozanova Deborah Ferreira Mokanarangan Thayaparan Marco Valentino André Freitas 84 10 0 17 May 2021
Does Putting a Linguist in the Loop Improve NLU Data Collection?Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021 Alicia Parrish William Huang Omar Agha Soo-hwan Lee Nikita Nangia Alex Warstadt Karmanya Aggarwal Emily Allaway Tal Linzen Samuel R. Bowman 180 48 0 15 Apr 2021
Exploring Transitivity in Neural NLI Models through VeridicalityConference of the European Chapter of the Association for Computational Linguistics (EACL), 2021 Hitomi Yanaka K. Mineshima Kentaro Inui 165 23 0 26 Jan 2021
Attentive Tree-structured Network for Monotonicity Reasoning Zeming Chen LRM 66 2 0 03 Jan 2021
Exploring End-to-End Differentiable Natural Logic Modeling Yufei Feng Zióu Zheng Quan Liu Michael A. Greenspan Xiao-Dan Zhu 161 15 0 08 Nov 2020
ANLIzing the Adversarial Natural Language Inference Dataset Adina Williams Tristan Thrush Douwe Kiela AAML 363 48 0 24 Oct 2020
Beyond Leaderboards: A survey of methods for revealing weaknesses in Natural Language Inference data and models Viktor Schlegel Goran Nenadic Riza Batista-Navarro ELM 171 18 0 29 May 2020
Do Neural Models Learn Systematicity of Monotonicity Inference in Natural Language?Annual Meeting of the Association for Computational Linguistics (ACL), 2020 Hitomi Yanaka K. Mineshima D. Bekki Kentaro Inui NAI 145 52 0 30 Apr 2020
Neural Natural Language Inference Models Partially Embed Theories of Lexical Entailment and Negation Atticus Geiger Kyle Richardson Christopher Potts 219 4 0 30 Apr 2020
Syntactic Data Augmentation Increases Robustness to Inference HeuristicsAnnual Meeting of the Association for Computational Linguistics (ACL), 2020 Junghyun Min R. Thomas McCoy Dipanjan Das Emily Pitler Tal Linzen 216 189 0 24 Apr 2020
Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESuppositionAnnual Meeting of the Association for Computational Linguistics (ACL), 2020 Paloma Jeretic Alex Warstadt Suvrat Bhooshan Adina Williams ReLM AI4CE 305 126 0 07 Apr 2020
Posing Fair Generalization Tasks for Natural Language InferenceConference on Empirical Methods in Natural Language Processing (EMNLP), 2019 Atticus Geiger Ignacio Cases L. Karttunen Christopher Potts 180 49 0 03 Nov 2019
Probing Natural Language Inference Models through Semantic FragmentsAAAI Conference on Artificial Intelligence (AAAI), 2019 Kyle Richardson Hai Hu L. Moss Ashish Sabharwal 209 150 0 16 Sep 2019
Can neural networks understand monotonicity reasoning? Hitomi Yanaka K. Mineshima D. Bekki Kentaro Inui Satoshi Sekine Lasha Abzianidze Johan Bos LRM 202 86 0 15 Jun 2019