Track Awesome Qa Updates Weekly

😎 A curated list of the Question Answering (QA)

🏠 Home · 🔍 Search · 🔥 Feed · 📮 Subscribe · ❤️ Sponsor · 😺 seriousran/awesome-qa · ⭐ 706 · 🏷️ Computer Science

[ Daily / Weekly / Overview ]

Jan 11 - Jan 17, 2021

Recent QA Models

• DilBert: Delaying Interaction Layers in Transformer-based Encoders for Efficient Open Domain Question Answering (2020)
  • paper: https://arxiv.org/pdf/2010.08422.pdf
  • github: https://github.com/wissam-sib/dilbert (⭐15)

May 11 - May 17, 2020

Recent QA Models

• UnifiedQA: Crossing Format Boundaries With a Single QA System (2020)
  • Demo: https://unifiedqa.apps.allenai.org/

• ProQA: Resource-efficient method for pretraining a dense corpus index for open-domain QA and IR. (2020)
  • paper: https://arxiv.org/pdf/2005.00038.pdf
  • github: https://github.com/xwhan/ProQA (⭐44)

• TYDI QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages (2020)
  • paper: https://arxiv.org/ftp/arxiv/papers/2003/2003.05002.pdf

• Retrospective Reader for Machine Reading Comprehension
  • paper: https://arxiv.org/pdf/2001.09694v2.pdf

• TANDA: Transfer and Adapt Pre-Trained Transformer Models for Answer Sentence Selection (AAAI 2020)
  • paper: https://arxiv.org/pdf/1911.04118.pdf

Recent Language Models

• ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators, Kevin Clark, et al., ICLR, 2020.

• TinyBERT: Distilling BERT for Natural Language Understanding, Xiaoqi Jiao, et al., ICLR, 2020.

• MINILM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers, Wenhui Wang, et al., arXiv, 2020.

• T5: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer, Colin Raffel, et al., arXiv preprint, 2019.

• ERNIE: Enhanced Language Representation with Informative Entities, Zhengyan Zhang, et al., ACL, 2019.

• XLNet: Generalized Autoregressive Pretraining for Language Understanding, Zhilin Yang, et al., arXiv preprint, 2019.

• ALBERT: A Lite BERT for Self-supervised Learning of Language Representations, Zhenzhong Lan, et al., arXiv preprint, 2019.

• RoBERTa: A Robustly Optimized BERT Pretraining Approach, Yinhan Liu, et al., arXiv preprint, 2019.

• DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter, Victor sanh, et al., arXiv, 2019.

• SpanBERT: Improving Pre-training by Representing and Predicting Spans, Mandar Joshi, et al., TACL, 2019.

• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, Jacob Devlin, et al., NAACL 2019, 2018.

Arxiv

• Investigating the Successes and Failures of BERT for Passage Re-Ranking, Harshith Padigela, et al., arXiv preprint, May 2019.

• BERT with History Answer Embedding for Conversational Question Answering, Chen Qu, et al., arXiv preprint, May 2019.

• Understanding the Behaviors of BERT in Ranking, Yifan Qiao, et al., arXiv preprint, Apr 2019.

• BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis, Hu Xu, et al., arXiv preprint, Apr 2019.

• End-to-End Open-Domain Question Answering with BERTserini, Wei Yang, et al., arXiv preprint, Feb 2019.

• A BERT Baseline for the Natural Questions, Chris Alberti, et al., arXiv preprint, Jan 2019.

• Passage Re-ranking with BERT, Rodrigo Nogueira, et al., arXiv preprint, Jan 2019.

• SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering, Chenguang Zhu, et al., arXiv, Dec 2018.

Dec 16 - Dec 22, 2019

AAAI 2020

• TANDA: Transfer and Adapt Pre-Trained Transformer Models for Answer Sentence Selection, Siddhant Garg, et al., AAAI 2020, Nov 2019.

ACL 2019

• Towards Scalable and Reliable Capsule Networks for Challenging NLP Applications, Wei Zhao, et al., ACL 2019, Jun 2019.

• Cognitive Graph for Multi-Hop Reading Comprehension at Scale, Ming Ding, et al., ACL 2019, Jun 2019.

• Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index, Minjoon Seo, et al., ACL 2019, Jun 2019.

• Unsupervised Question Answering by Cloze Translation, Patrick Lewis, et al., ACL 2019, Jun 2019.

• Improving Question Answering over Incomplete KBs with Knowledge-Aware Reader, Wenhan Xiong, et al., ACL 2019, May 2019.

• Matching Article Pairs with Graphical Decomposition and Convolutions, Bang Liu, et al., ACL 2019, May 2019.

• Episodic Memory Reader: Learning what to Remember for Question Answering from Streaming Data, Moonsu Han, et al., ACL 2019, Mar 2019.

• Textbook Question Answering with Multi-modal Context Graph Understanding and Self-supervised Open-set Comprehension, Daesik Kim, et al., ACL 2019, Nov 2018.

EMNLP-IJCNLP 2019

• Language Models as Knowledge Bases?, Fabio Petron, et al., EMNLP-IJCNLP 2019, Sep 2019.

• LXMERT: Learning Cross-Modality Encoder Representations from Transformers, Hao Tan, et al., EMNLP-IJCNLP 2019, Dec 2019.

• Answering Complex Open-domain Questions Through Iterative Query Generation, Peng Qi, et al., EMNLP-IJCNLP 2019, Oct 2019.

• KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoning, Bill Yuchen Lin, et al., EMNLP-IJCNLP 2019, Sep 2019.

• Mixture Content Selection for Diverse Sequence Generation, Jaemin Cho, et al., EMNLP-IJCNLP 2019, Sep 2019.

• A Discrete Hard EM Approach for Weakly Supervised Question Answering, Sewon Min, et al., EMNLP-IJCNLP, 2019, Sep 2019.

Dataset

• ELI5: Long Form Question Answering, Angela Fan, et al., ACL 2019, Jul 2019

• CODAH: An Adversarially-Authored Question Answering Dataset for Common Sense, Michael Chen, et al., RepEval 2019, Jun 2019.

Datasets / Subtypes of QA

• CODAH Dataset (⭐19)

• ELI5 (⭐261)
  • Paper: https://arxiv.org/abs/1907.09190

Sep 16 - Sep 22, 2019

ACL 2019

• Overview of the MEDIQA 2019 Shared Task on Textual Inference, Question Entailment and Question Answering, Asma Ben Abacha, et al., ACL-W 2019, Aug 2019.

• SemEval-2019 Task 10: Math Question Answering, Mark Hopkins, et al., ACL-W 2019, Jun 2019.

• Natural Questions: a Benchmark for Question Answering Research, Tom Kwiatkowski, et al., TACL 2019, Jan 2019.

Feb 18 - Feb 24, 2019

Publications / Subtypes of QA

• Papers
  • "Learning to Skim Text", Adams Wei Yu, Hongrae Lee, Quoc V. Le, 2017. : Show only what you want in Text
  • "Deep Joint Entity Disambiguation with Local Neural Attention", Octavian-Eugen Ganea and Thomas Hofmann, 2017.
  • "BI-DIRECTIONAL ATTENTION FLOW FOR MACHINE COMPREHENSION", Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, Hananneh Hajishirzi, ICLR, 2017.
  • "Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks", Matthew Francis-Landau, Greg Durrett and Dan Klei, NAACL-HLT 2016.
    • https://GitHub.com/matthewfl/nlp-entity-convnet (⭐57)
  • "Entity Linking with a Knowledge Base: Issues, Techniques, and Solutions", Wei Shen, Jianyong Wang, Jiawei Han, IEEE Transactions on Knowledge and Data Engineering(TKDE), 2014.
  • "Introduction to “This is Watson", IBM Journal of Research and Development, D. A. Ferrucci, 2012.
  • "A survey on question answering technology from an information retrieval perspective", Information Sciences, 2011.
  • "Question Answering in Restricted Domains: An Overview", Diego Mollá and José Luis Vicedo, Computational Linguistics, 2007
  • "Natural language question answering: the view from here", L Hirschman, R Gaizauskas, natural language engineering, 2001.
  • Entity Disambiguation / Entity Linking

Datasets / Subtypes of QA

• WikiQA
  • A publicly available set of question and sentence pairs for open-domain question answering.

Jan 28 - Feb 03, 2019

Datasets / Subtypes of QA

• AI2 Science Questions v2.1(2017)
  • It consists of questions used in student assessments in the United States across elementary and middle school grade levels. Each question is 4-way multiple choice format and may or may not include a diagram element.
  • Paper: http://ai2-website.s3.amazonaws.com/publications/AI2ReasoningChallenge2018.pdf

• DeepMind Q&A Dataset; CNN/Daily Mail (⭐1.3k)
  • Hermann et al. (2015) created two awesome datasets using news articles for Q&A research. Each dataset contains many documents (90k and 197k each), and each document companies on average 4 questions approximately. Each question is a sentence with one missing word/phrase which can be found from the accompanying document/context.
  • Paper: https://arxiv.org/abs/1506.03340

• MS MARCO
  • This is for real-world question answering.
  • Paper: https://arxiv.org/abs/1611.09268

• NarrativeQA (⭐391)
  • It includes the list of documents with Wikipedia summaries, links to full stories, and questions and answers.
  • Paper: https://arxiv.org/pdf/1712.07040v1.pdf

• NewsQA (⭐232)
  • A machine comprehension dataset
  • Paper: https://arxiv.org/pdf/1611.09830.pdf

• SQuAD1.0
  • Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.
  • Paper: https://arxiv.org/abs/1606.05250

• SQuAD2.0
  • SQuAD2.0 combines the 100,000 questions in SQuAD1.1 with over 50,000 new, unanswerable questions written adversarially by crowdworkers to look similar to answerable ones. To do well on SQuAD2.0, systems must not only answer questions when possible, but also determine when no answer is supported by the paragraph and abstain from answering.
  • Paper: https://arxiv.org/abs/1806.03822

• Story cloze test
  • 'Story Cloze Test' is a new commonsense reasoning framework for evaluating story understanding, story generation, and script learning. This test requires a system to choose the correct ending to a four-sentence story.
  • Paper: https://arxiv.org/abs/1604.01696

• TriviaQA
  • TriviaQA is a reading comprehension dataset containing over 650K question-answer-evidence triples. TriviaQA includes 95K question-answer pairs authored by trivia enthusiasts and independently gathered evidence documents, six per question on average, that provide high quality distant supervision for answering the questions.
  • Paper: https://arxiv.org/abs/1705.03551

Nov 12 - Nov 18, 2018

Facebook AI Research's publication within 5 years / Subtypes of QA

• 2018
  • Embodied Question Answering, Abhishek Das, Samyak Datta, Georgia Gkioxari, Stefan Lee, Devi Parikh, and Dhruv Batra, CVPR, 2018
  • Do explanations make VQA models more predictable to a human?, Arjun Chandrasekaran, Viraj Prabhu, Deshraj Yadav, Prithvijit Chattopadhyay, and Devi Parikh, EMNLP, 2018
  • Neural Compositional Denotational Semantics for Question Answering, Nitish Gupta, Mike Lewis, EMNLP, 2018

Oct 15 - Oct 21, 2018

Systems / Subtypes of QA

• IBM Watson - Has state-of-the-arts performance.

• Facebook DrQA - Applied to the SQuAD1.0 dataset. The SQuAD2.0 dataset has released. but DrQA is not tested yet.

• MIT media lab's Knowledge graph - Is a freely-available semantic network, designed to help computers understand the meanings of words that people use.

Lectures / Subtypes of QA

• Question Answering - Natural Language Processing - By Dragomir Radev, Ph.D. | University of Michigan | 2016.

Slides / Subtypes of QA

• Question Answering with Knowledge Bases, Web and Beyond (⭐33) - By Scott Wen-tau Yih & Hao Ma | Microsoft Research | 2016.

• Question Answering - By Dr. Mariana Neves | Hasso Plattner Institut | 2017.

Datasets / Subtypes of QA

• MultiRC
  • A dataset of short paragraphs and multi-sentence questions
  • Paper: http://cogcomp.org/page/publication_view/833

Sep 24 - Sep 30, 2018

Analysis and Parsing for Pre-processing in QA systems / Subtypes of QA

• Morphological analysis

• Named Entity Recognition(NER)

• Homonyms / Polysemy Analysis

• Syntactic Parsing (Dependency Parsing)

• Semantic Recognition

Aug 13 - Aug 19, 2018

Types of QA / Subtypes of QA

• Knowledge-based QA

• Table/List-based QA

• Text-based QA

• Community-based QA

• Visual QA

Google AI's publication within 5 years / Subtypes of QA

• 2017
  • "Analyzing Language Learned by an Active Question Answering Agent", Christian Buck and Jannis Bulian and Massimiliano Ciaramita and Wojciech Gajewski and Andrea Gesmundo and Neil Houlsby and Wei Wang, NIPS, 2017.
  • "Learning Recurrent Span Representations for Extractive Question Answering", Kenton Lee and Shimi Salant and Tom Kwiatkowski and Ankur Parikh and Dipanjan Das and Jonathan Berant, ICLR, 2017.
  • Identify the same question
    • "Neural Paraphrase Identification of Questions with Noisy Pretraining", Gaurav Singh Tomar and Thyago Duque and Oscar Täckström and Jakob Uszkoreit and Dipanjan Das, SCLeM, 2017.

Facebook AI Research's publication within 5 years / Subtypes of QA

• 2017
  • DrQA
    • Reading Wikipedia to Answer Open-Domain Questions, Danqi Chen, Adam Fisch, Jason Weston & Antoine Bordes, ACL, 2017.

Jul 30 - Aug 05, 2018

Types of QA

• Single-turn QA: answer without considering any context

• Conversational QA: use previsous conversation turns

Most QA systems have roughly 3 parts / Subtypes of QA

• Fact extraction
  
  1. Entity Extraction
     
     1. Named-Entity Recognition(NER)
  2. Relation Extraction
     

• Understanding the question

• Generating an answer

Events / Subtypes of QA

• Wolfram Alpha launced the answer engine in 2009.

• IBM Watson system defeated top Jeopardy! champions in 2011.

• Apple's Siri integrated Wolfram Alpha's answer engine in 2011.

• Google embraced QA by launching its Knowledge Graph, leveraging the free base knowledge base in 2012.

• Amazon Echo | Alexa (2015), Google Home | Google Assistant (2016), INVOKE | MS Cortana (2017), HomePod (2017)

Dataset Collections / Subtypes of QA

• NLIWOD's Question answering datasets (⭐93)

• karthinkncode's Datasets for Natural Language Processing (⭐920)

Datasets / Subtypes of QA

• Children's Book Test

• It is one of the bAbI project of Facebook AI Research which is organized towards the goal of automatic text understanding and reasoning. The CBT is designed to measure directly how well language models can exploit wider linguistic context.

• GraphQuestions (⭐86)
  • On generating Characteristic-rich Question sets for QA evaluation.

• LC-QuAD
  • It is a gold standard KBQA (Question Answering over Knowledge Base) dataset containing 5000 Question and SPARQL queries. LC-QuAD uses DBpedia v04.16 as the target KB.

• Qestion-Answer Dataset by CMU
  • This is a corpus of Wikipedia articles, manually-generated factoid questions from them, and manually-generated answers to these questions, for use in academic research. These data were collected by Noah Smith, Michael Heilman, Rebecca Hwa, Shay Cohen, Kevin Gimpel, and many students at Carnegie Mellon University and the University of Pittsburgh between 2008 and 2010.

The DeepQA Research Team in IBM Watson's publication within 5 years / Subtypes of QA

• 2015
  • "Automated Problem List Generation from Electronic Medical Records in IBM Watson", Murthy Devarakonda, Ching-Huei Tsou, IAAI, 2015.
  • "Decision Making in IBM Watson Question Answering", J. William Murdock, Ontology summit, 2015.
  • "Unsupervised Entity-Relation Analysis in IBM Watson", Aditya Kalyanpur, J William Murdock, ACS, 2015.
  • "Commonsense Reasoning: An Event Calculus Based Approach", E T Mueller, Morgan Kaufmann/Elsevier, 2015.

• 2014
  • "Problem-oriented patient record summary: An early report on a Watson application", M. Devarakonda, Dongyang Zhang, Ching-Huei Tsou, M. Bornea, Healthcom, 2014.
  • "WatsonPaths: Scenario-based Question Answering and Inference over Unstructured Information", Adam Lally, Sugato Bachi, Michael A. Barborak, David W. Buchanan, Jennifer Chu-Carroll, David A. Ferrucci*, Michael R. Glass, Aditya Kalyanpur, Erik T. Mueller, J. William Murdock, Siddharth Patwardhan, John M. Prager, Christopher A. Welty, IBM Research Report RC25489, 2014.
  • "Medical Relation Extraction with Manifold Models", Chang Wang and James Fan, ACL, 2014.

MS Research's publication within 5 years / Subtypes of QA

• 2018
  • "Characterizing and Supporting Question Answering in Human-to-Human Communication", Xiao Yang, Ahmed Hassan Awadallah, Madian Khabsa, Wei Wang, Miaosen Wang, ACM SIGIR, 2018.
  • "FigureQA: An Annotated Figure Dataset for Visual Reasoning", Samira Ebrahimi Kahou, Vincent Michalski, Adam Atkinson, Akos Kadar, Adam Trischler, Yoshua Bengio, ICLR, 2018

• 2017
  • "Multi-level Attention Networks for Visual Question Answering", Dongfei Yu, Jianlong Fu, Tao Mei, Yong Rui, CVPR, 2017.
  • "A Joint Model for Question Answering and Question Generation", Tong Wang, Xingdi (Eric) Yuan, Adam Trischler, ICML, 2017.
  • "Two-Stage Synthesis Networks for Transfer Learning in Machine Comprehension", David Golub, Po-Sen Huang, Xiaodong He, Li Deng, EMNLP, 2017.
  • "Question-Answering with Grammatically-Interpretable Representations", Hamid Palangi, Paul Smolensky, Xiaodong He, Li Deng,
  • "Search-based Neural Structured Learning for Sequential Question Answering", Mohit Iyyer, Wen-tau Yih, Ming-Wei Chang, ACL, 2017.

• 2016
  • "Stacked Attention Networks for Image Question Answering", Zichao Yang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Smola, CVPR, 2016.
  • "Question Answering with Knowledge Base, Web and Beyond", Yih, Scott Wen-tau and Ma, Hao, ACM SIGIR, 2016.
  • "NewsQA: A Machine Comprehension Dataset", Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, Kaheer Suleman, RepL4NLP, 2016.
  • "Table Cell Search for Question Answering", Sun, Huan and Ma, Hao and He, Xiaodong and Yih, Wen-tau and Su, Yu and Yan, Xifeng, WWW, 2016.

• 2015
  • "WIKIQA: A Challenge Dataset for Open-Domain Question Answering", Yi Yang, Wen-tau Yih, and Christopher Meek, EMNLP, 2015.
  • "Web-based Question Answering: Revisiting AskMSR", Chen-Tse Tsai, Wen-tau Yih, and Christopher J.C. Burges, MSR-TR, 2015.
  • "Open Domain Question Answering via Semantic Enrichment", Huan Sun, Hao Ma, Wen-tau Yih, Chen-Tse Tsai, Jingjing Liu, and Ming-Wei Chang, WWW, 2015.

• 2014
  • "An Overview of Microsoft Deep QA System on Stanford WebQuestions Benchmark", Zhenghao Wang, Shengquan Yan, Huaming Wang, and Xuedong Huang, MSR-TR, 2014.
  • "Semantic Parsing for Single-Relation Question Answering", Wen-tau Yih, Xiaodong He, Christopher Meek, ACL, 2014.

Google AI's publication within 5 years / Subtypes of QA

• 2018
  • Google QA
    • "QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension", Adams Wei Yu, David Dohan, Minh-Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, Quoc V. Le, ICLR, 2018.
    • "Ask the Right Questions: Active Question Reformulation with Reinforcement Learning", Christian Buck and Jannis Bulian and Massimiliano Ciaramita and Wojciech Paweł Gajewski and Andrea Gesmundo and Neil Houlsby and Wei Wang, ICLR, 2018.
    • "Building Large Machine Reading-Comprehension Datasets using Paragraph Vectors", Radu Soricut, Nan Ding, 2018.
  • Sentence representation
    • "An efficient framework for learning sentence representations", Lajanugen Logeswaran, Honglak Lee, ICLR, 2018.
  • "Did the model understand the question?", Pramod K. Mudrakarta and Ankur Taly and Mukund Sundararajan and Kedar Dhamdhere, ACL, 2018.

• 2014
  • "Great Question! Question Quality in Community Q&A", Sujith Ravi and Bo Pang and Vibhor Rastogi and Ravi Kumar, ICWSM, 2014.

Books / Subtypes of QA

• Natural Language Question Answering system Paperback - Boris Galitsky (2003)

• New Directions in Question Answering - Mark T. Maybury (2004)

• Part 3. 5. Question Answering in The Oxford Handbook of Computational Linguistics - Sanda Harabagiu and Dan Moldovan (2005)

• Chap.28 Question Answering in Speech and Language Processing - Daniel Jurafsky & James H. Martin (2017)

Links / Subtypes of QA

• Building a Question-Answering System from Scratch— Part 1

• Qeustion Answering with Tensorflow By Steven Hewitt, O'REILLY, 2017

• Why question answering is hard