Improving Text-to-SQL Evaluation Methodology

23 June 2018

Catherine Finegan-Dollak

Jonathan K. Kummerfeld

Papers citing "Improving Text-to-SQL Evaluation Methodology"

50 / 175 papers shown

Title
(Un)solving Morphological Inflection: Lemma Overlap Artificially Inflates Models' Performance Omer Goldman David Guriel Reut Tsarfaty 87 27 0 12 Aug 2021
Break, Perturb, Build: Automatic Perturbation of Reasoning Paths Through Question Decomposition Mor Geva Tomer Wolfson Jonathan Berant ReLM LRM 20 21 0 29 Jul 2021
Learning Algebraic Recombination for Compositional Generalization Chenyao Liu Shengnan An Zeqi Lin Qian Liu Bei Chen Jian-Guang Lou Lijie Wen Nanning Zheng Dongmei Zhang CoGe 194 36 0 14 Jul 2021
Question Answering over Knowledge Graphs with Neural Machine Translation and Entity Linking Daniel Diomedi Aidan Hogan 6 14 0 06 Jul 2021
Draw Me a Flower: Processing and Grounding Abstraction in Natural Language R. Lachmy Valentina Pyatkin Avshalom Manevich Reut Tsarfaty 21 18 0 27 Jun 2021
Text-to-SQL in the Wild: A Naturally-Occurring Dataset Based on Stack Exchange Data Moshe Hazoom Vibhor Malik Ben Bogin 19 48 0 09 Jun 2021
Turing: an Accurate and Interpretable Multi-Hypothesis Cross-Domain Natural Language Database Interface Peng-Tao Xu Wenjie Zi H. Shahidi Ákos Kádár Keyi Tang Wei Yang Jawad Ateeq Harsh Barot Meidan Alon Yanshuai Cao 10 2 0 08 Jun 2021
One Semantic Parser to Parse Them All: Sequence to Sequence Multi-Task Learning on Semantic Parsing Datasets Marco Damonte Emilio Monti AIMat 17 6 0 08 Jun 2021
Structured Reordering for Modeling Latent Alignments in Sequence Transduction Bailin Wang Mirella Lapata Ivan Titov BDL 8 20 0 06 Jun 2021
Unlocking Compositional Generalization in Pre-trained Models Using Intermediate Representations Jonathan Herzig Peter Shaw Ming-Wei Chang Kelvin Guu Panupong Pasupat Yuan Zhang AI4CE 8 67 0 15 Apr 2021
Learning to Synthesize Data for Semantic Parsing Bailin Wang Wenpeng Yin Xi Victoria Lin Caiming Xiong 20 49 0 12 Apr 2021
Paired Examples as Indirect Supervision in Latent Decision Models Nitish Gupta Sameer Singh Matt Gardner Dan Roth 23 7 0 05 Apr 2021
FeTaQA: Free-form Table Question Answering Linyong Nan Chia-Hsuan Hsieh Ziming Mao Xi Victoria Lin Neha Verma ... Isabel Trindade Renusree Bandaru Jacob Cunningham Caiming Xiong Dragomir R. Radev LMTD 15 143 0 01 Apr 2021
Semantic Parsing to Manipulate Relational Database For a Management System M. Mushtaq 14 0 0 18 Feb 2021
An Investigation Between Schema Linking and Text-to-SQL Performance Yasufumi Taniguchi H. Nakayama Takahiro Kubo Jun Suzuki 15 5 0 03 Feb 2021
Few-Shot Semantic Parsing for New Predicates Zhuang Li Lizhen Qu Shuo Huang Gholamreza Haffari NAI 20 9 0 26 Jan 2021
Code Generation from Natural Language with Less Prior and More Monolingual Data Sajad Norouzi Keyi Tang Yanshuai Cao 9 19 0 01 Jan 2021
Iterative Utterance Segmentation for Neural Semantic Parsing Yinuo Guo Zeqi Lin Jian-Guang Lou Dongmei Zhang 30 3 0 13 Dec 2020
Tracking Interaction States for Multi-Turn Text-to-SQL Semantic Parsing Runze Wang Zhenhua Ling Jing-Bo Zhou Yu Hu 16 24 0 09 Dec 2020
Sequence-Level Mixed Sample Data Augmentation Demi Guo Yoon Kim Alexander M. Rush 16 98 0 18 Nov 2020
Character-level Representations Improve DRS-based Semantic Parsing Even in the Age of BERT Rik van Noord Antonio Toral Johan Bos 11 4 0 09 Nov 2020
"What Do You Mean by That?" A Parser-Independent Interactive Approach for Enhancing Text-to-SQL Yuntao Li Bei Chen Qian Liu Yan Gao Jian-Guang Lou Yan Zhang Dongmei Zhang 11 16 0 09 Nov 2020
Structure-Grounded Pretraining for Text-to-SQL Xiang Deng Ahmed Hassan Awadallah Christopher Meek Oleksandr Polozov Huan Sun Matthew Richardson LMTD 13 150 0 24 Oct 2020
Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both? Peter Shaw Ming-Wei Chang Panupong Pasupat Kristina Toutanova CoGe 25 182 0 24 Oct 2020
Did You Ask a Good Question? A Cross-Domain Question Intention Classification Benchmark for Text-to-SQL Yusen Zhang Xiangyu Dong Shuaichen Chang Tao Yu Peng Shi Rui Zhang OOD 43 15 0 23 Oct 2020
Compositional Generalization via Semantic Tagging Hao Zheng Mirella Lapata 14 34 0 22 Oct 2020
On the Potential of Lexico-logical Alignments for Semantic Parsing to SQL Queries Tianze Shi Chen Zhao Jordan L. Boyd-Graber Hal Daumé Lillian Lee 24 78 0 21 Oct 2020
DuoRAT: Towards Simpler Text-to-SQL Models Torsten Scholak Raymond Li Dzmitry Bahdanau H. D. Vries C. Pal AI4TS 22 26 0 21 Oct 2020
ColloQL: Robust Cross-Domain Text-to-SQL Over Search Queries Karthik Radhakrishnan A. Srikantan Xi Victoria Lin 19 4 0 19 Oct 2020
Knowledge Graph-based Question Answering with Electronic Health Records Junwoo Park Youngwoo Cho Haneol Lee Jaegul Choo E. Choi 36 33 0 19 Oct 2020
Improving Compositional Generalization in Semantic Parsing I. Oren Jonathan Herzig Nitish Gupta Matt Gardner Jonathan Berant 21 63 0 12 Oct 2020
Semantic Evaluation for Text-to-SQL with Distilled Test Suites Ruiqi Zhong Tao Yu Dan Klein 9 125 0 06 Oct 2020
GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing Tao Yu Chien-Sheng Wu Xi Victoria Lin Bailin Wang Y. Tan Xinyi Yang Dragomir R. Radev R. Socher Caiming Xiong LMTD 19 247 0 29 Sep 2020
Span-based Semantic Parsing for Compositional Generalization Jonathan Herzig Jonathan Berant ReLM LRM 17 100 0 13 Sep 2020
Photon: A Robust Cross-Domain Text-to-SQL System Jichuan Zeng Xi Victoria Lin Caiming Xiong R. Socher M. Lyu Irwin King S. Hoi 25 52 0 30 Jul 2020
Towards Ecologically Valid Research on Language User Interfaces H. D. Vries Dzmitry Bahdanau Christopher D. Manning 204 51 0 28 Jul 2020
Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized Architectures Daniel Furrer Marc van Zee Nathan Scales Nathanael Scharli CoGe 13 113 0 17 Jul 2020
Latent Compositional Representations Improve Systematic Generalization in Grounded Question Answering Ben Bogin Sanjay Subramanian Matt Gardner Jonathan Berant ReLM OOD BDL LRM 14 28 0 01 Jul 2020
TableQA: a Large-Scale Chinese Text-to-SQL Dataset for Table-Aware SQL Generation Ningyuan Sun Xuefeng Yang Yunfeng Liu LMTD 14 34 0 10 Jun 2020
Efficient Deployment of Conversational Natural Language Interfaces over Databases Anthony Colas Trung Bui Franck Dernoncourt Moumita Sinha Doo Soon Kim 17 3 0 31 May 2020
Quda: Natural Language Queries for Visual Data Analytics Siwei Fu Kai Xiong Xiaodong Ge Siliang Tang Wei-Neng Chen Yingcai Wu 12 26 0 07 May 2020
Syntactic Question Abstraction and Retrieval for Data-Scarce Semantic Parsing Wonseok Hwang Jinyeong Yim Seunghyun Park Minjoon Seo 20 0 0 01 May 2020
A Methodology for Creating Question Answering Corpora Using Inverse Data Annotation Jan Deriu Katsiaryna Mlynchyk Philippe Schlapfer Álvaro Rodrigo D. V. Grunigen Nicolas Kaiser Kurt Stockinger Eneko Agirre Mark Cieliebak 14 15 0 16 Apr 2020
Bootstrapping a Crosslingual Semantic Parser Tom Sherborne Yumo Xu Mirella Lapata 28 25 0 06 Apr 2020
Revisit Systematic Generalization via Meaningful Learning Ning Shi Boxin Wang Wei Wang Xiangyu Liu Zhouhan Lin 14 2 0 14 Mar 2020
Scalable Neural Methods for Reasoning With a Symbolic Knowledge Base William W. Cohen Haitian Sun R. A. Hofer M. Siegler 22 61 0 14 Feb 2020
Measuring Compositional Generalization: A Comprehensive Method on Realistic Data Daniel Keysers Nathanael Scharli Nathan Scales Hylke Buisman Daniel Furrer ... Tibor Tihon Dmitry Tsarkov Xiao Wang Marc van Zee Olivier Bousquet CoGe 21 347 0 20 Dec 2019
RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers Bailin Wang Richard Shin Xiaodong Liu Oleksandr Polozov Matthew Richardson 11 565 0 10 Nov 2019
Byte-Pair Encoding for Text-to-SQL Generation Samuel G. Müller Andreas Vlachos AIMat 13 2 0 20 Oct 2019
Model-based Interactive Semantic Parsing: A Unified Framework and A Text-to-SQL Case Study Ziyu Yao Yu-Chuan Su Huan Sun Wen-tau Yih 15 77 0 11 Oct 2019