Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks — part 4/8

Cleaned-text code points [41818, 53884). Bibliography omitted here; source.md includes it.

Mohaddeseh Bastan, Mahnaz Koupaee, Youngseo Son,
Richard Sicoli, and Niranjan Balasubramanian. 2020.
Author’s sentiment prediction. In International Con-
ference on Computational Linguistics (COLING).
Luisa Bentivogli, Peter Clark, Ido Dagan, and Danilo
Giampiccolo. 2008. The sixth pascal recognizing
textual entailment challenge. In Text Analysis Con-
ference (TAC).
Jonathan Bragg, Arman Cohan, Kyle Lo, and Iz Beltagy.
2021. Flex: Unifying evaluation for few-shot nlp. In
Advances in Neural Information Processing Systems
(NeurIPS).
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie
Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind
Neelakantan, Pranav Shyam, Girish Sastry, Amanda
Askell, Sandhini Agarwal, and et al. 2020. Language
models are few-shot learners. In Advances in Neural
Information Processing Systems (NeurIPS).
Rich Caruana. 1997. Multitask learning. Machine
learning, 28(1):41–75.
Kushal Chawla, Jaysa Ramirez, Rene Clever, Gale
Lucas, Jonathan May, and Jonathan Gratch. 2021.
CaSiNo: A Corpus of Campsite Negotiation Dia-
logues for Automatic Negotiation Systems. In Con-
ference of the North American Chapter of the Associ-
ation for Computational Linguistics (NAACL).
Ronan Collobert and Jason Weston. 2008. A unified
architecture for natural language processing: Deep
neural networks with multitask learning. In Interna-
tional Conference on Machine Learning (ICML).
Ido Dagan, Oren Glickman, and Bernardo Magnini.
2005. The pascal recognising textual entailment chal-
lenge. In Machine Learning Challenges Workshop.
Avia Efrat and Omer Levy. 2020. The Turking Test: Can
Language Models Understand Instructions? arXiv
preprint arXiv:2010.11982.
Nancy Fulda, Nathan Tibbetts, Zachary Brown, and
David Wingate. 2017. Harvesting common-sense
navigational knowledge for robotics from uncurated
text corpora. In Conference on Robot Learning (IJ-
CAI).
Aditya Gupta, Jiacheng Xu, Shyam Upadhyay, Diyi
Yang, and Manaal Faruqui. 2021. Disfl-qa: A bench-
mark dataset for understanding disfluencies in ques-
tion answering. In Annual Meeting of the Association
for Computational Linguistics (ACL) - Findings.
Kazuma Hashimoto, Caiming Xiong, Yoshimasa Tsu-
ruoka, and Richard Socher. 2017. A joint many-task
model: Growing a neural network for multiple nlp
tasks. In Conference on Empirical Methods in Natu-
ral Language Processing (EMNLP).
William Hersh, Chris Buckley, TJ Leone, and David
Hickam. 1994. Ohsumed: An interactive retrieval
evaluation and new large test collection for research.
In Conference of the Association for Computing Ma-
chinery Special Interest Group in Information Re-
trieval (SIGIR).
Joel Hestness, Sharan Narang, Newsha Ardalani, Gre-
gory Diamos, Heewoo Jun, Hassan Kianinejad,
Md Patwary, Mostofa Ali, Yang Yang, and Yanqi
Zhou. 2017. Deep Learning Scaling is Predictable,
Empirically. arXiv preprint arXiv:1712.00409.
Daniel Khashabi, Sewon Min, Tushar Khot, Ashish
Sabharwal, Oyvind Tafjord, Peter Clark, and Han-
naneh Hajishirzi. 2020. UnifiedQA: Crossing Format
Boundaries With a Single QA System. In Conference
on Empirical Methods in Natural Language Process-
ing (EMNLP) - Findings.
Hyeonji Kim, Byeong-Hoon So, Wook-Shin Han, and
Hongrae Lee. 2020. Natural language to sql: Where
are we today? Proceedings of the VLDB Endowment,
13(10):1737–1750.
Andrew K Lampinen, Ishita Dasgupta, Stephanie CY
Chan, Kory Matthewson, Michael Henry Tessler, An-
tonia Creswell, James L McClelland, Jane X Wang,
and Felix Hill. 2022. Can Language Models Learn
from Explanations in Context? arXiv preprint
arXiv:2204.02329.
Logan Lebanoff, John Muchovej, Franck Dernoncourt,
Doo Soon Kim, Lidan Wang, Walter Chang, and Fei
Liu. 2020. Understanding points of correspondence
between sentences for abstractive summarization. In
Annual Meeting of the Association for Computational
Linguistics (ACL) - Student Research Workshop.
Brian Lester, Rami Al-Rfou, and Noah Constant. 2021.
The power of scale for parameter-efficient prompt
tuning. In Conference on Empirical Methods in Nat-
ural Language Processing (EMNLP).
Chin-Yew Lin. 2004. ROUGE: A Package for Auto-
matic Evaluation of Summaries. In ACL Workshop
on Text Summarization Branches Out.
Bryan McCann, Nitish Shirish Keskar, Caiming Xiong,
and Richard Socher. 2018. The natural language
decathlon: Multitask learning as question answering.
arXiv preprint arXiv:1806.08730.
Sewon Min, Mike Lewis, Luke Zettlemoyer, and Han-
naneh Hajishirzi. 2022a. MetaICL: Learning to learn
in context. In NAACL-HLT.
So Yeon Min, Devendra Singh Chaplot, Pradeep Raviku-
mar, Yonatan Bisk, and Ruslan Salakhutdinov. 2022b.
FILM: Following Instructions in Language with Mod-
ular Methods. In International Conference on Learn-
ing Representations (ICLR).
Swaroop Mishra, Daniel Khashabi, Chitta Baral, Yejin
Choi, and Hannaneh Hajishirzi. 2022a. Reframing
instructional prompts to gptk’s language. In Annual
Meeting of the Association for Computational Lin-
guistics (ACL) - Findings.
5094

Swaroop Mishra, Daniel Khashabi, Chitta Baral, and
Hannaneh Hajishirzi. 2022b. Cross-Task Generaliza-
tion via Natural Language Crowdsourcing Instruc-
tions. In Annual Meeting of the Association for Com-
putational Linguistics (ACL).
Courtney Napoles, Keisuke Sakaguchi, and Joel
Tetreault. 2017. Jfleg: A fluency corpus and bench-
mark for grammatical error correction. In Confer-
ence of the European Chapter of the Association for
Computational Linguistics (EACL).
Jekaterina Novikova, Ondˇrej Dušek, and Verena Rieser.
2017. The e2e dataset: New challenges for end-
to-end generation. In Annual SIGdial Meeting on
Discourse and Dialogue.
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Car-
roll L Wainwright, Pamela Mishkin, Chong Zhang,
Sandhini Agarwal, Katarina Slama, Alex Ray, et al.
2022. Training Language Models to Follow In-
structions with Human Feedback. arXiv preprint
arXiv:2203.02155.
Mihir Parmar, Swaroop Mishra, Mirali Purohit, Man
Luo, Murad Mohammad, and Chitta Baral. 2022. In-
BoXBART: Get instructions into biomedical multi-
task learning. In Findings of the Association for Com-
putational Linguistics: NAACL 2022, pages 112–128,
Seattle, United States. Association for Computational
Linguistics.
Ellie Pavlick, Matt Post, Ann Irvine, Dmitry Kachaev,
and Chris Callison-Burch. 2014. The Language De-
mographics of Amazon Mechanical Turk. Transac-
tions of the Association for Computational Linguis-
tics (TACL).
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine
Lee, Sharan Narang, Michael Matena, Yanqi Zhou,
Wei Li, and Peter J Liu. 2020. Exploring the lim-
its of transfer learning with a unified text-to-text
transformer. Journal of Machine Learning Research
(JMLR).
Melissa Roemmele, Cosmin Adrian Bejan, and An-
drew S Gordon. 2011. Choice of plausible alter-
natives: An evaluation of commonsense causal rea-
soning. In AAAI spring symposium: logical formal-
izations of commonsense reasoning.
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhaga-
vatula, and Yejin Choi. 2020. WINOGRANDE: an
adversarial winograd schema challenge at scale. In
Conference on Artificial Intelligence (AAAI).
Victor Sanh, Albert Webson, Colin Raffel, Stephen
Bach, Lintang Sutawika, Zaid Alyafeai, Antoine
Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey,
M Saiful Bari, Canwen Xu, Urmish Thakker,
Shanya Sharma Sharma, Eliza Szczechla, Taewoon
Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti
Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han
Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong,
Harshit Pandey, Rachel Bawden, Thomas Wang, Tr-
ishala Neeraj, Jos Rozen, Abheesht Sharma, An-
drea Santilli, Thibault Fevry, Jason Alan Fries, Ryan
Teehan, Teven Le Scao, Stella Biderman, Leo Gao,
Thomas Wolf, and Alexander M Rush. 2022. Multi-
task Prompted Training Enables Zero-Shot Task Gen-
eralization. In International Conference on Learning
Representations (ICLR).
Igor Shalyminov, Alessandro Sordoni, Adam Atkinson,
and Hannes Schulz. 2020. Fast domain adaptation
for goal-oriented dialogue using a hybrid generative-
retrieval transformer. In IEEE International Con-
ference on Acoustics, Speech and Signal Processing
(ICASSP).
Mohit Shridhar, Jesse Thomason, Daniel Gordon,
Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke
Zettlemoyer, and Dieter Fox. 2020. ALFRED: A
Benchmark for Interpreting Grounded Instructions
for Everyday Tasks. In IEEE Conference on Com-
puter Vision and Pattern Recognition(CVPR).
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao,
Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch,
Adam R. Brown, Adam Santoro, Aditya Gupta,
Adrià Garriga-Alonso, et al. 2022. Beyond the
imitation game: Quantifying and extrapolating the
capabilities of language models. arXiv preprint
arXiv:2206.04615.
Simon Stepputtis, Joseph Campbell, Mariano Phielipp,
Stefan Lee, Chitta Baral, and Heni Ben Amor. 2020.
Language-Conditioned Imitation Learning for Robot
Manipulation Tasks. In Advances in Neural Informa-
tion Processing Systems (NeurIPS).
Chen Sun, Abhinav Shrivastava, Saurabh Singh, and
Abhinav Gupta. 2017. Revisiting Unreasonable Ef-
fectiveness of Data in Deep Learning Era. In Inter-
national Conference on Computer Vision (ICCV).
Eleni Triantafillou, Tyler Zhu, Vincent Dumoulin, Pas-
cal Lamblin, Utku Evci, Kelvin Xu, Ross Goroshin,
Carles Gelada, Kevin Swersky, Pierre-Antoine Man-
zagol, et al. 2019. Meta-Dataset: A Dataset of
Datasets for Learning to Learn from Few Examples.
In International Conference on Learning Representa-
tions (ICLR).
Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu,
Adams Wei Yu, Brian Lester, Nan Du, Andrew M.
Dai, and Quoc V Le. 2022. Finetuned Language
Models are Zero-Shot Learners. In International
Conference on Learning Representations (ICLR).
Nathaniel Weir, Xingdi Yuan, Marc-Alexandre Côté,
Matthew Hausknecht, Romain Laroche, Ida Momen-
nejad, Harm Van Seijen, and Benjamin Van Durme.
2022. One-Shot Learning from a Demonstration
with Hierarchical Latent Language. arXiv preprint
arXiv:2203.04806.
Orion Weller, Nicholas Lourie, Matt Gardner, and
Matthew Peters. 2020. Learning from Task Descrip-
tions. In Conference on Empirical Methods in Natu-
ral Language Processing (EMNLP).
5095

Thomas Wolf, Lysandre Debut, Victor Sanh, Julien
Chaumond, Clement Delangue, Anthony Moi, Pier-
ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz,
Joe Davison, Sam Shleifer, Patrick von Platen, Clara
Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le
Scao, Sylvain Gugger, Mariama Drame, Quentin
Lhoest, and Alexander M. Rush. 2020. Transform-
ers: State-of-the-Art Natural Language Processing.
In Conference on Empirical Methods in Natural Lan-
guage Processing (EMNLP) - System Demonstra-
tions.
Tianbao Xie, Chen Henry Wu, Peng Shi, Ruiqi Zhong,
Torsten Scholak, Michihiro Yasunaga, Chien-Sheng
Wu, Ming Zhong, Pengcheng Yin, Sida I Wang, et al.
2022. UnifiedSKG: Unifying and Multi-Tasking
Structured Knowledge Grounding with Text-to-Text
Language Models. arXiv preprint arXiv:2201.05966.
Hanwei Xu, Yujun Chen, Yulun Du, Nan Shao, Yang-
gang Wang, Haiyu Li, and Zhilin Yang. 2022. Zero-
Prompt: Scaling Prompt-Based Pretraining to 1,000
Tasks Improves Zero-Shot Generalization. arXiv
preprint arXiv:2201.06910.
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale,
Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and
Colin Raffel. 2021. mT5: A Massively Multilingual
Pre-trained Text-to-Text Transformer. In Conference
of the North American Chapter of the Association for
Computational Linguistics (NAACL).
Qinyuan Ye, Bill Yuchen Lin, and Xiang Ren. 2021.
CrossFit: A Few-shot Learning Challenge for Cross-
task Generalization in NLP. In Conference on Em-
pirical Methods in Natural Language Processing
(EMNLP).
Qinyuan Ye and Xiang Ren. 2021. Learning to Generate
Task-Specific Adapters from Task Description. In
Annual Meeting of the Association for Computational
Linguistics (ACL).
Ruiqi Zhong, Kristy Lee, Zheng Zhang, and Dan Klein.
2021. Adapting Language Models for Zero-shot
Learning by Meta-tuning on Dataset and Prompt Col-
lections. In Conference on Empirical Methods in
Natural Language Processing (EMNLP) - Findings.
5096