Improving Text-to-SQL Evaluation Methodology

23 June 2018

Catherine Finegan-Dollak

Jonathan K. Kummerfeld

Papers citing "Improving Text-to-SQL Evaluation Methodology"

50 / 175 papers shown

Title
Semantic Parsing for Conversational Question Answering over Knowledge Graphs Laura Perez-Beltrachini Parag Jain Emilio Monti Mirella Lapata 30 10 0 28 Jan 2023
EHRSQL: A Practical Text-to-SQL Benchmark for Electronic Health Records Gyubok Lee Hyeonji Hwang Seongsu Bae Yeonsu Kwon W. Shin Seongjun Yang Minjoon Seo Jong-Yeup Kim E. Choi 21 18 0 16 Jan 2023
On the Structural Generalization in Text-to-SQL Jieyu Li Lu Chen Ruisheng Cao Su Zhu Hongshen Xu Zhi Chen Hanchong Zhang Kai Yu 30 1 0 12 Jan 2023
Towards Knowledge-Intensive Text-to-SQL Semantic Parsing with Formulaic Knowledge Longxu Dou Yan Gao Xuqi Liu Mingyang Pan Dingzirui Wang Wanxiang Che Dechen Zhan Min-Yen Kan Jian-Guang Lou 26 13 0 03 Jan 2023
Uncontrolled Lexical Exposure Leads to Overestimation of Compositional Generalization in Pretrained Models Najoung Kim Tal Linzen P. Smolensky 21 30 0 21 Dec 2022
Diverse Demonstrations Improve In-context Compositional Generalization Itay Levy Ben Bogin Jonathan Berant 27 128 0 13 Dec 2022
Real-World Compositional Generalization with Disentangled Sequence-to-Sequence Learning Hao Zheng Mirella Lapata OOD CoGe DRL 11 5 0 12 Dec 2022
On the Compositional Generalization Gap of In-Context Learning Arian Hosseini Ankit Vani Dzmitry Bahdanau Alessandro Sordoni Aaron C. Courville 19 24 0 15 Nov 2022
Characterizing Intrinsic Compositionality in Transformers with Tree Projections Shikhar Murty Pratyusha Sharma Jacob Andreas Christopher D. Manning 14 39 0 02 Nov 2022
Augmenting Task-Oriented Dialogue Systems with Relation Extraction Andrew Lee Zhe Chen Kevin Leach Jonathan K. Kummerfeld 13 1 0 24 Oct 2022
Towards Generalizable and Robust Text-to-SQL Parsing Chang Gao Bowen Li Wenxuan Zhang W. Lam Binhua Li Fei Huang Luo Si Yongbin Li 33 8 0 23 Oct 2022
Searching for Better Database Queries in the Outputs of Semantic Parsers A. Osokin Irina Saparina R. B. Yarullin 29 0 0 13 Oct 2022
Benchmarking Long-tail Generalization with Likelihood Splits Ameya Godbole Robin Jia ALM 24 8 0 13 Oct 2022
Translate First Reorder Later: Leveraging Monotonicity in Semantic Parsing Francesco Cazzaro Davide Locatelli A. Quattoni X. Carreras 27 4 0 10 Oct 2022
Compositional Generalisation with Structured Reordering and Fertility Layers Matthias Lindemann Alexander Koller Ivan Titov CoGe 32 7 0 06 Oct 2022
State-of-the-art generalisation research in NLP: A taxonomy and review Dieuwke Hupkes Mario Giulianelli Verna Dankers Mikel Artetxe Yanai Elazar ... Leila Khalatbari Maria Ryskina Rita Frieske Ryan Cotterell Zhijing Jin 114 93 0 06 Oct 2022
Improving Compositional Generalization in Math Word Problem Solving Yunshi Lan Lei Wang Jingwu Jiang Ee-Peng Lim 76 7 0 03 Sep 2022
Recent Advances in Text-to-SQL: A Survey of What We Have and What We Expect Naihao Deng Yulong Chen Yue Zhang 27 44 0 22 Aug 2022
Deep Learning Driven Natural Languages Text to SQL Query Conversion: A Survey Ayush Kumar P. Nagarkar Prabhav Nalhe Sanjeev Vijayakumar 11 17 0 08 Aug 2022
A Survey of Intent Classification and Slot-Filling Datasets for Task-Oriented Dialog Stefan Larson Kevin Leach 33 20 0 26 Jul 2022
Learning Math Reasoning from Self-Sampled Correct and Partially-Correct Solutions Ansong Ni J. Inala Chenglong Wang Oleksandr Polozov Christopher Meek Dragomir R. Radev Jianfeng Gao ReLM AIMat LRM 19 42 0 28 May 2022
Few-shot Subgoal Planning with Language Models Lajanugen Logeswaran Yao Fu Moontae Lee Honglak Lee LRM 20 26 0 28 May 2022
Evaluating the Impact of Model Scale for Compositional Generalization in Semantic Parsing Linlu Qiu Peter Shaw Panupong Pasupat Tianze Shi Jonathan Herzig Emily Pitler Fei Sha Kristina Toutanova AI4CE LRM 25 52 0 24 May 2022
Workflow Discovery from Dialogues in the Low Data Regime Amine El Hattami Stefania Raimondo I. Laradji David Vazquez Pau Rodríguez López C. Pal 49 10 0 24 May 2022
SeqZero: Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Models Jingfeng Yang Haoming Jiang Qingyu Yin Danqing Zhang Bing Yin Diyi Yang 46 47 0 15 May 2022
Measuring and Improving Compositional Generalization in Text-to-SQL via Component Alignment Yujian Gan Xinyun Chen Qiuping Huang Matthew Purver 22 21 0 04 May 2022
SUBS: Subtree Substitution for Compositional Semantic Parsing Jingfeng Yang Le Zhang Diyi Yang 16 21 0 03 May 2022
Compositional Generalization and Decomposition in Neural Program Synthesis Kensen Shi Joey Hong Manzil Zaheer Pengcheng Yin Charles Sutton 32 5 0 07 Apr 2022
Structurally Diverse Sampling for Sample-Efficient Training and Comprehensive Evaluation Shivanshu Gupta Sameer Singh Matt Gardner 45 7 0 16 Mar 2022
Evaluating the Text-to-SQL Capabilities of Large Language Models Nitarshan Rajkumar Raymond Li Dzmitry Bahdanau LMTD ELM 21 98 0 15 Mar 2022
A Survey on Artificial Intelligence for Source Code: A Dialogue Systems Perspective Erfan Al-Hossami Samira Shaikh 26 6 0 10 Feb 2022
Unobserved Local Structures Make Compositional Generalization Hard Ben Bogin Shivanshu Gupta Jonathan Berant CoGe 25 33 0 15 Jan 2022
Improving Compositional Generalization with Latent Structure and Data Augmentation Linlu Qiu Peter Shaw Panupong Pasupat Pawel Krzysztof Nowak Tal Linzen Fei Sha Kristina Toutanova CoGe 23 57 0 14 Dec 2021
Weakly Supervised Text-to-SQL Parsing through Question Decomposition Tomer Wolfson Daniel Deutch Jonathan Berant ReLM 13 15 0 12 Dec 2021
Compositional Generalization for Natural Language Interfaces to Web APIs Saghar Hosseini Ahmed Hassan Awadallah Yu-Chuan Su 17 6 0 09 Dec 2021
Dyna-bAbI: unlocking bAbI's potential with dynamic synthetic benchmarking Ronen Tamari Kyle Richardson Aviad Sar-Shalom Noam Kahlon Nelson F. Liu Reut Tsarfaty Dafna Shahaf 37 5 0 30 Nov 2021
Learning to Generalize Compositionally by Transferring Across Semantic Parsing Tasks Wang Zhu Peter Shaw Tal Linzen Fei Sha 32 7 0 09 Nov 2021
Grounded Graph Decoding Improves Compositional Generalization in Question Answering Yu Gai Paras Jain Wendi Zhang Joseph E. Gonzalez D. Song Ion Stoica BDL OOD 29 8 0 05 Nov 2021
On The Ingredients of an Effective Zero-shot Semantic Parser Pengcheng Yin John Wieting Avirup Sil Graham Neubig 50 15 0 15 Oct 2021
LAGr: Labeling Aligned Graphs for Improving Systematic Generalization in Semantic Parsing Dora Jambor Dzmitry Bahdanau 105 9 0 14 Oct 2021
Disentangled Sequence to Sequence Learning for Compositional Generalization Hao Zheng Mirella Lapata CoGe DRL 25 39 0 09 Oct 2021
How BPE Affects Memorization in Transformers Eugene Kharitonov Marco Baroni Dieuwke Hupkes 163 32 0 06 Oct 2021
Visually Grounded Concept Composition Bowen Zhang Hexiang Hu Linlu Qiu Peter Shaw Fei Sha CoGe 68 4 0 29 Sep 2021
COVR: A test-bed for Visually Grounded Compositional Generalization with real images Ben Bogin Shivanshu Gupta Matt Gardner Jonathan Berant CoGe 34 29 0 22 Sep 2021
Prefix-to-SQL: Text-to-SQL Generation from Incomplete User Questions Naihao Deng Shuaichen Chang Peng Shi Tao Yu Rui Zhang LMTD 30 4 0 15 Sep 2021
End-to-End Entity Resolution and Question Answering Using Differentiable Knowledge Graphs Armin Oliya Amir Saffari Priyanka Sen Tom Ayoola 13 23 0 13 Sep 2021
Expanding End-to-End Question Answering on Differentiable Knowledge Graphs with Intersection Priyanka Sen Amir Saffari Armin Oliya 14 21 0 13 Sep 2021
Natural SQL: Making SQL Easier to Infer from Natural Language Specifications Yujian Gan Xinyun Chen Jinxia Xie Matthew Purver J. Woodward J. Drake Qiaofu Zhang 78 91 0 11 Sep 2021
Finding needles in a haystack: Sampling Structurally-diverse Training Sets from Synthetic Data for Compositional Generalization I. Oren Jonathan Herzig Jonathan Berant 24 31 0 06 Sep 2021
The paradox of the compositionality of natural language: a neural machine translation case study Verna Dankers Elia Bruni Dieuwke Hupkes CoGe 162 75 0 12 Aug 2021