Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning

Victor ZhongCaiming XiongRichard Socher

article2017arXiv1,629 citations

Proposes Seq2SQL and the large-scale WikiSQL benchmark, demonstrating how reinforcement learning with database execution feedback significantly improves natural language translation into structured SQL queries over standard sequence-to-sequence models.

Listen

Relational databases underpin critical business functions across finance, healthcare, and customer management, yet accessing them typically requires technical expertise in structured query languages like SQL. Translating conversational, natural language questions directly into executable database queries can bridge this gap and democratize data access. However, traditional semantic parsing systems struggle because they are often restricted to narrow domains, require rigid hand-crafted grammars, and struggle with the flexible, unordered nature of query filtering conditions.

The article demonstrates an end-to-end deep learning framework, named Seq2SQL, designed to translate natural language questions directly into valid SQL queries across unfamiliar database schemas. It also introduces WikiSQL, a large-scale, crowd-annotated benchmark dataset specifically developed to train and evaluate such systems.

The researchers developed a modular neural network tailored to SQL's structural components—predicting the aggregation operation, selecting the target column, and generating filtering conditions. To address the fact that filtering conditions can appear in any order and still produce identical results, the model incorporates reinforcement learning with in-the-loop query execution, rewarding the system based on actual database output rather than strict string matching. The approach was evaluated against existing state-of-the-art semantic parsers on the new WikiSQL dataset, which spans 80,654 hand-annotated question-query pairs across 24,241 distinct tables extracted from Wikipedia.

The evaluation yielded several key findings:

The Seq2SQL model significantly improved execution accuracy to 59.4%, outperforming a state-of-the-art sequence-to-sequence baseline by 23.5 percentage points (up from 35.9%) and an augmented pointer network baseline by 6.1 percentage points (up from 53.3%).
Incorporating reinforcement learning directly improved execution accuracy by 2.3 percentage points over a supervised counterpart, specifically resolving issues where filtering conditions were logically correct but ordered differently than the reference query.
Structuring the model around SQL components reduced the generation of invalid database queries from 7.9% to 4.8% by preventing references to non-existent columns.
Limiting the model's vocabulary to table schema and question inputs via pointer mechanisms substantially improved precision when handling rare entities, names, and dates.

These results show that natural language interfaces can generalize to unseen database structures without requiring direct access to underlying table contents, thereby preserving data privacy and reducing deployment complexity. For organizations, this approach lowers technical barriers, accelerates decision-making timelines, and cuts the operational overhead needed to support ad-hoc reporting and business intelligence requests.

Organizations evaluating conversational database interfaces should adopt modular neural architectures that leverage SQL structure and execution-based feedback rather than relying on generic translation models. Before enterprise deployment, teams should conduct pilot studies on internal databases to evaluate query generation against domain-specific vocabularies and ensure execution environments are properly isolated.

While the model demonstrates high performance on single-table queries, its confidence and applicability are bounded by the dataset's scope. The system currently targets single-table lookups with basic aggregations and conditions; it does not evaluate complex relational operations such as multi-table joins, nested queries, or multi-turn conversational context. Further validation on complex enterprise schemas is necessary before deploying the technology for mission-critical operations.

Salesforce natural language questions neural networks optimization Seq2SQL neural semantic parser reinforcement learning

arXiv: 1709.00103 salesforce/WikiSQL

Paper: Sequence Level Training with Recurrent Neural Networks, Marc'Aurelio Ranzato et al. (2015). Introduces policy-gradient reinforcement learning methods for sequence generation to overcome exposure bias and optimize non-differentiable evaluation metrics, providing the algorithmic foundation used in Seq2SQL.
Paper: Semantic Parsing on Freebase from Question-Answer Pairs, Jonathan Berant et al. (2013). Demonstrates how to train semantic parsers to generate executable database queries using execution feedback from question-answer pairs rather than full logical supervision.
Paper: Neural Machine Translation by Jointly Learning to Align and Translate, Dzmitry Bahdanau et al. (2015). Introduces the attentional sequence-to-sequence framework that serves as the baseline architecture adapted and enhanced in Seq2SQL.
Paper: Learning to Map Sentences to Logical Form: Structured Classification with Probabilistic Categorial Grammars, Luke S. Zettlemoyer et al. (2005). Establishes foundational statistical methods for mapping natural language questions into structured logical forms for database querying.
Paper: Sequence to Sequence Learning with Neural Networks, Ilya Sutskever et al. (2014). Provides the seminal neural encoder-decoder paradigm for translating variable-length input sequences into structured target sequences.

Paper: Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task, Tao Yu et al. (2018). Substantially expands on WikiSQL by introducing a complex, cross-domain benchmark spanning multi-table databases to evaluate generalized text-to-SQL semantic parsing.
Paper: Program Synthesis with Large Language Models, Jacob Austin et al. (2021). Extends the concept of natural language code generation and execution verification from domain-specific SQL queries to full general-purpose programming languages.

Cover for Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning

Abstract

A significant amount of the world's knowledge is stored in relational databases. However, the ability for users to retrieve facts from a database is limited due to a lack of understanding of query languages such as SQL. We propose Seq2SQL, a deep neural network for translating natural language questions to corresponding SQL queries. Our model leverages the structure of SQL queries to significantly reduce the output space of generated queries. Moreover, we use rewards from in-the-loop query execution over the database to learn a policy to generate unordered parts of the query, which we show are less suitable for optimization via cross entropy loss. In addition, we will publish WikiSQL, a dataset of 80654 hand-annotated examples of questions and SQL queries distributed across 24241 tables from Wikipedia. This dataset is required to train our model and is an order of magnitude larger than comparable datasets. By applying policy-based reinforcement learning with a query execution environment to WikiSQL, our model Seq2SQL outperforms attentional sequence to sequence models, improving execution accuracy from 35.9% to 59.4% and logical form accuracy from 23.4% to 48.3%.

Topics

natural language questions neural networks optimization query execution reinforcement learning sql structured queries execution environments complex queries

Keywords

Seq2SQL neural semantic parser reinforcement learning semantic parsing structured query generation relational databases Seq2Seq model

1 Introduction
2 Model
2.1 Augmented Pointer Network
2.2 Seq2SQL
3 WikiSQL
3.1 Evaluation
4 Experiments
4.1 Result
4.2 Analysis
5 Related Work
6 Conclusion
References
A Collection of WikiSQL
B Attentional Seq2Seq Neural Semantic Parser Baseline
C Predictions by Seq2SQL

Knowls

Knowl 1 — Seq2SQL Model Architecture and Input Representation
model/method
Seq2SQL is a neural network architecture designed to translate natural language questions into structured SQL queries over relational database tables without requiring access to table content during generation. Instead of generating unconstrained text, Seq2SQL leverages the syntactic structure of SQL to prune the output space by decomposing query synthesis into three specialized components: an aggregation operator classifier, a SELECT column pointer module, and a WHERE clause pointer decoder.

Given a question consisting of a word sequence $x^q$ , a table schema containing $N$ column names where each column $j \in \{1, \dots, N\}$ has word sequence $x^c_j = [x^c_{j,1}, x^c_{j,2}, \dots, x^c_{j,T_j}]$ , and a fixed vocabulary of SQL keywords $x^s$ (such as SELECT, WHERE, COUNT, MIN, MAX), Seq2SQL defines an augmented input sequence $x$ : $x = [\langle\text{col}\rangle; x^c_1; \dots; x^c_N; \langle\text{sql}\rangle; x^s; \langle\text{question}\rangle; x^q]$ where $[\cdot;\cdot]$ denotes sequence concatenation demarcated by sentinel boundary tokens $\langle\text{col}\rangle$ , $\langle\text{sql}\rangle$ , and $\langle\text{question}\rangle$ .

The sequence $x$ is encoded using a two-layer bidirectional Long Short-Term Memory (LSTM) network. Word tokens are represented by concatenating fixed GloVe word embeddings and character $n$ -gram embeddings (the mean of embeddings of all character $n$ -grams in the word). The encoder yields a hidden state representation $h^{\text{enc}}_t$ for each token position $t$ in the augmented input sequence.
Knowl 2 — Aggregation Operator Classification in Seq2SQL
model/method
In Seq2SQL, predicting the aggregation operator of the query (such as COUNT, MIN, MAX, or a NULL operator corresponding to no aggregation) is formulated as an attention-driven classification task over the encoded input sequence.

For each token position $t$ with encoder hidden state $h^{\text{enc}}_t$ , a scalar attention score $\alpha^{\text{inp}}_t$ is computed via a learnable parameter row vector $W^{\text{inp}}$ : $\alpha^{\text{inp}}_t = W^{\text{inp}} h^{\text{enc}}_t$ These scores are normalized across all positions via softmax to generate attention weights $\beta^{\text{inp}}$ : $\beta^{\text{inp}} = \text{softmax}(\alpha^{\text{inp}})$ An aggregated question-schema representation vector $\kappa^{\text{agg}}$ is computed as the weighted sum of encoder hidden states: $\kappa^{\text{agg}} = \sum_t \beta^{\text{inp}}_t h^{\text{enc}}_t$ A multi-layer perceptron (MLP) maps $\kappa^{\text{agg}}$ to scores $\alpha^{\text{agg}}$ over the set of possible aggregation operations: $\alpha^{\text{agg}} = W^{\text{agg}} \tanh(V^{\text{agg}} \kappa^{\text{agg}} + b^{\text{agg}}) + c^{\text{agg}}$ where $W^{\text{agg}}$ and $V^{\text{agg}}$ are learnable weight matrices, and $b^{\text{agg}}$ and $c^{\text{agg}}$ are bias vectors. The operator probability distribution is given by $\beta^{\text{agg}} = \text{softmax}(\alpha^{\text{agg}})$ , and the module is supervised during training using cross-entropy loss $\mathcal{L}^{\text{agg}}$ .
Knowl 3 — SELECT Column Pointer Module in Seq2SQL
model/method
In Seq2SQL, selecting the target column for the SELECT clause is cast as a pointer matching problem between an attention-weighted question representation and column name encodings.

Each column $j \in \{1, \dots, N\}$ with token sequence $x^c_j = [x^c_{j,1}, \dots, x^c_{j,T_j}]$ is encoded with an LSTM: $h^c_{j,t} = \text{LSTM}(\text{emb}(x^c_{j,t}), h^c_{j,t-1})$ The final encoder state $h^c_{j,T_j}$ serves as the fixed-dimensional column representation $e^c_j = h^c_{j,T_j}$ .

A question-conditioned representation $\kappa^{\text{sel}}$ is obtained by applying a separate attention mechanism over the input encoder states $h^{\text{enc}}_t$ : $\alpha^{\text{inp,sel}}_t = W^{\text{inp,sel}} h^{\text{enc}}_t, \quad \beta^{\text{sel}} = \text{softmax}(\alpha^{\text{inp,sel}}), \quad \kappa^{\text{sel}} = \sum_t \beta^{\text{sel}}_t h^{\text{enc}}_t$ where $W^{\text{inp,sel}}$ is a learnable parameter row vector untied from the aggregation module.

The score for selecting column $j$ is computed by an MLP conditioned on both the input context and the column representation: $\alpha^{\text{sel}}_j = W^{\text{sel}} \tanh(V^{\text{sel}} \kappa^{\text{sel}} + V^c e^c_j)$ where $W^{\text{sel}}$ is a weight vector and $V^{\text{sel}}, V^c$ are projection weight matrices. The distribution over candidate columns is $\beta^{\text{sel}} = \text{softmax}(\alpha^{\text{sel}})$ , and the module is optimized via cross-entropy loss $\mathcal{L}^{\text{sel}}$ .
Knowl 4 — Policy Gradient Reinforcement Learning for WHERE Clause Generation
model/method
In relational queries, multiple condition predicates in a WHERE clause are unordered: swapping their sequence (for example, WHERE age > 18 AND gender = 'male' vs. WHERE gender = 'male' AND age > 18) produces identical database execution results. Optimizing sequence generation using standard cross-entropy loss against a single ground-truth string penalizes semantically correct queries that generate conditions in a different order.

Seq2SQL addresses this order-invariance problem by generating the WHERE clause with an attentional pointer decoder and optimizing the policy with reinforcement learning based on in-the-loop query execution rewards.

At decoder step $s$ , given decoder hidden state $g_s$ from a unidirectional LSTM, pointer attention scores over encoder states $h^{\text{enc}}_t$ are computed as: $\alpha^{\text{ptr}}_{s,t} = W^{\text{ptr}} \tanh(U^{\text{ptr}} g_s + V^{\text{ptr}} h^{\text{enc}}_t)$ Tokens $y_s$ are sampled from the distribution $p_y(y_s; \Theta) = \text{softmax}(\alpha^{\text{ptr}}_s)$ .

The loss $\mathcal{L}^{\text{whe}}$ is defined as the negative expected execution reward: $\mathcal{L}^{\text{whe}} = -\mathbb{E}_{y \sim p_y}[R(q(y), q_g)]$ where $q(y)$ is the complete generated SQL query, $q_g$ is the ground-truth query, and $R(q(y), q_g)$ is the execution reward. The policy gradient with respect to model parameters $\Theta$ is estimated via a single Monte Carlo sample $y$ : $\nabla_\Theta \mathcal{L}^{\text{whe}} \approx -R(q(y), q_g) \nabla_\Theta \sum_{t=1}^T \log p_y(y_t; \Theta)$

During training, Seq2SQL first pretrains the WHERE clause decoder using teacher forcing with cross-entropy loss, and subsequently fine-tunes the network using policy gradient RL.
Knowl 5 — Query Execution Reward Function for Reinforcement Learning
equation
The reinforcement learning reward $R(q(y), q_g)$ measures the execution validity and semantic correctness of a generated SQL query $q(y)$ against the execution result of the ground truth query $q_g$ over the database table:

$R(q(y), q_g) = \begin{cases} -2, & \text{if } q(y) \text{ is not a valid SQL query} \\ -1, & \text{if } q(y) \text{ is a valid SQL query and executes to an incorrect result} \\ +1, & \text{if } q(y) \text{ is a valid SQL query and executes to the correct result} \end{cases}$

This reward directly penalizes invalid SQL syntax and nonexistent schema references, penalizes semantically incorrect queries, and positively rewards queries that return the exact result set.

Knowl 6 — WikiSQL Dataset

definition

WikiSQL is a large-scale semantic parsing dataset designed for natural language to SQL translation across diverse relational table schemas. It contains 80,654 hand-annotated pairs of natural language questions and corresponding SQL queries distributed over 24,241 HTML tables extracted from Wikipedia.

Key characteristics of WikiSQL include:

Cross-schema splits: Tables are randomly partitioned into train, development, and test splits such that no table appears in more than one split, testing cross-schema generalization.
Standardized query space: Queries follow the structure: $\text{SELECT } [\text{agg\_op}] \text{ agg\_col FROM table [WHERE } \text{cond1\_col cond1\_op cond1 AND } \dots \text{]}$ where $\text{agg\_op} \in \{\emptyset, \text{COUNT}, \text{MIN}, \text{MAX}\}$ and $\text{cond\_op} \in \{=, >, <\}$ .

Dataset	Size	Logical Form (LF)	Schema (Tables)
WikiSQL	80,654	yes	24,241
GeoQuery880	880	yes	8
ATIS	5,871	yes	141
Free917	917	yes	81
Overnight	26,098	yes	8
WebQuestions	5,810	no	2,420
WikiTableQuestions	22,033	no	2,108

Knowl 7 — WikiSQL Dataset Construction and Filtering Pipeline
algorithm
WikiSQL is constructed through a two-phase crowd-sourcing procedure involving synthetic query synthesis, template-based question generation, natural language paraphrasing, and multi-worker quality verification:
Input: Set of raw HTML tables extracted from Wikipedia
Output: WikiSQL dataset of (Question, SQL Query, Table) triples
for each candidate table $T$ :
    Check filtering criteria for $T$ :
        Discard if rows have unequal cell counts
        Discard if any cell content exceeds 50 characters
        Discard if any header cell is empty
        Discard if $T$ has fewer than 5 rows or fewer than 5 columns
        Discard if $>40\%$ of cells in any row contain identical content
    Remove summary statistics row (last row of table $T$ )

    Generate 6 candidate SQL queries on $T$ :
        Select agg_op $\in \{\emptyset, \text{COUNT}\}$ (or $\{\text{MIN}, \text{MAX}\}$ if target column is numeric)
        Sample condition columns, operators ( $=$ or additionally $\{>, <\}$ if numeric), and values from $T$
        Prune unnecessary conditions that do not change execution results
        Keep only queries that return a non-empty result set

    for each generated query $Q$ :
        Instantiate crude template question $Q_{\text{template}}$ from $Q$
        Obtain human paraphrase $Q_{\text{para}}$ via Amazon Mechanical Turk
        Obtain verification labels from 2 separate workers on whether $Q_{\text{para}}$ matches $Q_{\text{template}}$

        Filter paraphrase:
            Retain only if at least one verifier marks $Q_{\text{para}}$ as correct
            Retain only if character-level edit distance between $Q_{\text{para}}$ and $Q_{\text{template}}$ is $> 10$
Knowl 8 — Evaluation Metrics for Text-to-SQL
definition
On WikiSQL, models are evaluated on $N$ test instances using two complementary metrics:
1. Execution Accuracy ( $\text{Acc}_{\text{ex}}$ ): The fraction of generated queries that, when executed on the target database table, return the identical result set as the ground-truth query: $\text{Acc}_{\text{ex}} = \frac{N_{\text{ex}}}{N}$ where $N_{\text{ex}}$ is the number of queries producing the correct execution result. $\text{Acc}_{\text{ex}}$ evaluates functional correctness but may produce false positives if an incorrect query happens to return the same values on a particular table state.
2. Logical Form Accuracy ( $\text{Acc}_{\text{lf}}$ ): The fraction of generated queries that have an exact string match with the ground-truth query string: $\text{Acc}_{\text{lf}} = \frac{N_{\text{lf}}}{N}$ where $N_{\text{lf}}$ is the count of exact string matches. $\text{Acc}_{\text{lf}}$ prevents false positives from spurious table overlap, but produces false negatives when a query is functionally equivalent but reorders condition clauses in the WHERE statement.

Knowl 9 — Empirical Performance on WikiSQL Benchmark

data/table

Models evaluated on the WikiSQL benchmark include the attentional sequence-to-sequence neural semantic parser of Dong and Lapata (2016) augmented with table schemas, an Augmented Pointer Network baseline, Seq2SQL trained without reinforcement learning (cross-entropy teacher forcing on all three submodules), and the full Seq2SQL model with policy gradient reinforcement learning.

Model	Dev $\text{Acc}_{\text{lf}}$	Dev $\text{Acc}_{\text{ex}}$	Test $\text{Acc}_{\text{lf}}$	Test $\text{Acc}_{\text{ex}}$
Baseline (Dong Lapata, 2016)	23.3%	37.0%	23.4%	35.9%
Augmented Pointer Network	44.1%	53.8%	43.3%	53.3%
Seq2SQL (no RL)	48.2%	58.1%	47.4%	57.1%
Seq2SQL	49.5%	60.8%	48.3%	59.4%

The Augmented Pointer Network improves test execution accuracy over the Seq2Seq baseline by +17.4% by restricting output vocabulary to the input sequence and SQL keywords. Decomposing query generation into structural modules (Seq2SQL without RL) provides an additional +3.8% test execution accuracy, and execution-based policy gradient RL adds a further +2.3%, reaching 59.4% execution accuracy and 48.3% logical form accuracy.

Knowl 10 — Impact of Structural Decomposition and Execution RL on Query Validity and Clause Order

empirical result

Experimental analysis demonstrates specific error reductions attributable to Seq2SQL's structural decomposition and reinforcement learning:

Invalid Query Reduction: Direct classification of aggregation and pointer matching of column names reduces the rate of generated invalid SQL queries from 7.9% in the Augmented Pointer Network to 4.8% in Seq2SQL (without RL). Invalid queries primarily occurred when multi-token column names (e.g., 'Miles (km)') were corrupted during sequential token decoding.
Improved Aggregation Accuracy: A dedicated classification head increases performance when identifying the COUNT operator:

Order-Invariant Condition Learning: Under standard cross-entropy supervision, models penalize valid condition orderings that deviate from the ground truth. Fine-tuning with policy gradient RL allows Seq2SQL to learn correct filtering logic regardless of reference clause order (e.g., generating Party = Democratic AND First elected = 1992 when ground truth is First elected = 1992 AND Party = Democratic), eliminating spurious condition hallucinations caused by rigid sequential teacher forcing.

Coverage note — No substantial contributed material was omitted; standard hyperparameter details (such as Adam optimizer learning rate decay and dropout rates) and general related work comparisons are fully covered or contextualized within the extracted knowls.

References

1.I. Androutsopoulos, G.D. Ritchie, and P. Thanisch. Natural language interfaces to databases - an introduction. 1995.
2.Yoav Artzi and Luke S. Zettlemoyer. Bootstrapping semantic parsers from conversations. In EMNLP, 2011.
3.Yoav Artzi and Luke S. Zettlemoyer. Weakly supervised learning of semantic parsers for mapping instructions to actions. TACL, 1:49–62, 2013.
4.Thorsten Beck, Asli Demirguc-Kunt, and Ross Levine. A new database on the structure and development of the financial sector. The World Bank Economic Review, 14(3):597–605, 2000.
5.Irwan Bello, Hieu Pham, Quoc V. Le, Mohammad Norouzi, and Samy Bengio. Neural combinatorial optimization with reinforcement learning. ICLR, 2017.
6.Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. Semantic parsing on freebase from question-answer pairs. In EMNLP, 2013.
7.Chandra Bhagavatula, Thanapon Noraset, and Doug Downey. Methods for exploring and mining tables on wikipedia. In IDEA@KDD, 2013.
8.Qingqing Cai and Alexander Yates. Large-scale semantic parsing via schema matching and lexicon extension. In ACL, 2013.
9.Li Dong and Mirella Lapata. Language to logical form with neural attention. ACL, 2016.
10.Alessandra Giordani and Alessandro Moschitti. Translating questions to SQL queries with generative parsers discriminatively reranked. In COLING, 2012.
11.Jiatao Gu, Zhengdong Lu, Hang Li, and Victor O. K. Li. Incorporating copying mechanism in sequence-to-sequence learning. ACL, 2016.
12.Kelvin Guu, Panupong Pasupat, Evan Zheran Liu, and Percy Liang. From language to programs: Bridging reinforcement learning and maximum marginal likelihood. In ACL, 2017.
13.Kazuma Hashimoto, Caiming Xiong, Yoshimasa Tsuruoka, and Richard Socher. A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks. arXiv, cs.CL 1611.01587, 2016.
14.Richard Hillestad, James Bigelow, Anthony Bower, Federico Girosi, Robin Meili, Richard Scoville, and Roger Taylor. Can electronic medical record systems transform health care? potential health benefits, savings, and costs. Health affairs, 24(5):1103–1117, 2005.
15.Sepp Hochreiter and Jurgen Schmidhuber. Long short-term memory. Neural computation, 1997.
16.Srinivasan Iyer, Ioannis Konstas, Alvin Cheung, Jayant Krishnamurthy, and Luke Zettlemoyer. Learning a neural semantic parser from user feedback. In ACL, 2017.
17.Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv, abs/1412.6980, 2014.
18.Chen Liang, Jonathan Berant, Quoc V. Le, Ken Forbus, and Ni Lao. Neural symbolic machines: Learning semantic parsers on freebase with weak supervision. In ACL, 2017.
19.Percy Liang, Michael I. Jordan, and Dan Klein. Learning dependency-based compositional semantics. Computational Linguistics, 39:389–446, 2011.
20.Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David McClosky. The Stanford CoreNLP natural language processing toolkit. In Association for Computational Linguistics (ACL) System Demonstrations, pp. 55–60, 2014.
21.Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. ICLR, 2017.
22.Lili Mou, Zhengdong Lu, Hang Li, and Zhi Jin. Coupling distributed and symbolic execution for natural language queries. In ICML, 2017.
23.Arvind Neelakantan, Quoc V. Le, Martın Abadi, Andrew McCallum, and Dario Amodei. Learning a natural language interface with neural programmer. In ICLR, 2017.
24.Eric WT Ngai, Li Xiu, and Dorothy CK Chau. Application of data mining techniques in customer relationship management: A literature review and classification. Expert systems with applications, 36(2):2592–2602, 2009.
25.Panupong Pasupat and Percy Liang. Compositional semantic parsing on semi-structured tables. In ACL, 2015.
26.Jeffrey Pennington, Richard Socher, and Christopher D. Manning. Glove: Global vectors for word representation. In EMNLP, 2014.
27.Ana-Maria Popescu, Oren Etzioni, and Henry Kautz. Towards a theory of natural language interfaces to databases. In Proceedings of the 8th International Conference on Intelligent User Interfaces, pp. 149–157. ACM, 2003.
28.Patti J. Price. Evaluation of spoken language systems: The ATIS domain. 1990.
29.Siva Reddy, Mirella Lapata, and Mark Steedman. Large-scale semantic parsing without question-answer pairs. TACL, 2:377–392, 2014.
30.John Schulman, Nicolas Heess, Theophane Weber, and Pieter Abbeel. Gradient estimation using stochastic computation graphs. In NIPS, 2015.
31.Min Joon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. Bidirectional attention flow for machine comprehension. ICLR, 2017.
32.Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15:1929–1958, 2014.
33.Janez Starc and Dunja Mladenic. Joint learning of ontology and semantic parser from text. Intelligent Data Analysis, 21:19–38, 2017.
34.Richard S Sutton, David A McAllester, Satinder P Singh, and Yishay Mansour. Policy gradient methods for reinforcement learning with function approximation. In Advances in neural information processing systems, pp. 1057–1063, 2000.
35.Lappoon R. Tang and Raymond J. Mooney. Using multiple clause constructors in inductive logic programming for semantic parsing. In ECML, 2001.
36.Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. Pointer networks. In NIPS, 2015.
37.Yushi Wang, Jonathan Berant, and Percy Liang. Building a semantic parser overnight. In ACL, 2015.
38.Yuk Wah Wong and Raymond J. Mooney. Learning synchronous grammars for semantic parsing with lambda calculus. In ACL, 2007.
39.Caiming Xiong, Victor Zhong, and Richard Socher. Dynamic coattention networks for question answering. ICLR, 2017.
40.Pengcheng Yin, Zhengdong Lu, Hang Li, and Ben Kao. Neural enquirer: Learning to query tables. In ACL, 2016.
41.John M. Zelle and Raymond J. Mooney. Learning to parse database queries using inductive logic programming. In AAAI/IAAI, Vol. 2, 1996.
42.Luke S. Zettlemoyer and Michael Collins. Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars. In Uncertainty in Artificial Intelligence, 2005.
43.Luke S. Zettlemoyer and Michael Collins. Online learning of relaxed ccg grammars for parsing to logical form. In EMNLP-CoNLL, 2007.

Citation

MLA

Zhong, V., et al. “Seq2SQL: Generating Structured Queries from Natural Language Using Reinforcement Learning”. arXiv, 2017, http://arxiv.org/abs/1709.00103v7.

APA

Zhong, V., Xiong, C., & Socher, R. (2017). Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning. arXiv. http://arxiv.org/abs/1709.00103v7

Chicago

Zhong, V., C. Xiong, and R. Socher. 2017. “Seq2SQL: Generating Structured Queries from Natural Language Using Reinforcement Learning”. arXiv. http://arxiv.org/abs/1709.00103v7.

Harvard

Zhong, V., Xiong, C. and Socher, R. (2017) “Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning”, arXiv [Preprint]. Available at: http://arxiv.org/abs/1709.00103v7.

Vancouver

1. Zhong V, Xiong C, Socher R (2017) Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning. arXiv

BibTeX

@article{zhong2017seq2sql,
  title = {Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning},
  author = {Zhong, Victor and Xiong, Caiming and Socher, Richard},
  year = {2017},
  journal = {arXiv},
  url = {http://arxiv.org/abs/1709.00103v7},
  eprint = {1709.00103}
}

Metadata:arXiv

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: Authors

Model	Precision	Recall	F1
Augmented Pointer Network	66.3%	64.4%	65.4%
Seq2SQL	72.6%	66.2%	69.2%

Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Seq2SQL Model Architecture and Input Representation

Knowl 2 — Aggregation Operator Classification in Seq2SQL

Knowl 3 — SELECT Column Pointer Module in Seq2SQL

Knowl 4 — Policy Gradient Reinforcement Learning for WHERE Clause Generation

Knowl 5 — Query Execution Reward Function for Reinforcement Learning