Invited Speakers
Associate Professor, Researcher*
Computer Science & Cornell Tech
Cornell University, HuggingFace*
Brett Helsel Professor
Paul G. Allen School of Computer Science & Engineering
University of Washington
Gabilan Assistant Professor
Viterbi School of Engineering
University of Southern California
Alexander Rush
Title: Model Criticism for Long-Form Text GenerationAbstract: Language models have demonstrated the ability to generate highly fluent text; however, they still require additional scaffolding to maintain coherent high-level structure (e.g., story progression). Here, we propose to apply a statistical tool, model criticism in latent space, to evaluate the high-level structure of the generated text. Model criticism compares the distributions between real and generated data in a latent space obtained according to an assumptive generative process. Different generative processes identify specific failure modes of the underlying model. We perform experiments on three representative aspects of high-level discourse---coherence, coreference, and topicality---and find that transformer-based language models are able to capture topical structures but have a harder time maintaining structural coherence or modeling coreference structures.
Bio: Alexander "Sasha" Rush is a Professor at Cornell Tech and researcher at Hugging Face. His work is at the intersection of natural language processing and probabilistic deep learning with applications in text generation and efficient inference. He has written several popular open-source software projects supporting NLP research and data science, as well as pedagogical implementations of popular libraries. He is the secretary of ICLR and developed the MiniConf software used to run ML/NLP virtual conferences during COVID. His work has received paper and demo awards at major NLP, visualization, and hardware conferences, an NSF Career Award, and a Sloan Fellowship.
William W. Cohen
Title: Memory is the Second Most Important Thing: Language Models that RetrieveAbstract: Since its inception one of the goals of AI has been intelligent systems that understand language and reason with information. In the early days of AI, information was represented in symbolic forms that had certain properties: in particular, knowledge was stored in discrete modular components that could be independently assessed and corrected, and combined in many ways to answer complex questions. Current AI systems built on large language models have excellent performance on many tasks, but store knowledge opaquely in their parameters. I will survey approaches over the last few years to bring large language models closer to symbolic AI systems by adding the ability to explicitly store and retrieve information from external sources.
Bio: William Cohen is a Principal Scientist at Google, and is based in Google's Pittsburgh office. He received his bachelor's degree in Computer Science from Duke University in 1984, and a PhD in Computer Science from Rutgers University in 1990. From 1990 to 2000 Dr. Cohen worked at AT&T Bell Labs and later AT&T Labs-Research, and from April 2000 to May 2002 Dr. Cohen worked at Whizbang Labs, a company specializing in extracting information from the web. From 2002 to 2018, Dr. Cohen worked at Carnegie Mellon University in the Machine Learning Department, with a joint appointment in the Language Technology Institute, as an Associate Research Professor, a Research Professor, and a Professor. Dr. Cohen also was the Director of the Undergraduate Minor in Machine Learning at CMU and co-Director of the Master of Science in ML Program. Dr. Cohen is a past president of the International Machine Learning Society. In the past he has also served as an action editor for the the AI and Machine Learning series of books published by Morgan Claypool, for the journal Machine Learning, the journal Artificial Intelligence, the Journal of Machine Learning Research, and the Journal of Artificial Intelligence Research. He was General Chair for the 2008 International Machine Learning Conference, held July 6-9 at the University of Helsinki, in Finland; Program Co-Chair of the 2006 International Machine Learning Conference; and Co-Chair of the 1994 International Machine Learning Conference. Dr. Cohen was also the co-Chair for the 3rd Int'l AAAI Conference on Weblogs and Social Media, which was held May 17-20, 2009 in San Jose, and was the co-Program Chair for the 4rd Int'l AAAI Conference on Weblogs and Social Media. He is a AAAI Fellow, and was a winner of the 2008 the SIGMOD "Test of Time" Award for the most influential SIGMOD paper of 1998, and the 2014 SIGIR "Test of Time" Award for the most influential SIGIR paper of 2002-2004. Dr. Cohen's research interests include question answering, machine learning for NLP tasks, and neuro-symbolic reasoning. He has a long-standing interest in statistical relational learning and learning models, or learning from data, that display non-trivial structure. He holds seven patents related to learning, discovery, information retrieval, and data integration, and is the author of more than 200 publications. Dr. Cohen is also a Consulting Professor at the School of Computer Science at Carnegie Mellon University.
Yejin Choi
Title: David V.S. Goliath: the Art of Leaderboarding in the Era of Extreme-Scale Neural ModelsAbstract: Scale appears to be the winning recipe in today's leaderboards. And yet, extreme-scale neural models are still brittle to make errors that are often nonsensical and even counterintuitive. In this talk, I will argue for the importance of knowledge as well as inference-time reasoning algorithms, and demonstrate how smaller models developed in academia can still have an edge over larger industry-scale models, if powered with knowledge and/or reasoning algorithms. I will first introduce "symbolic knowledge distillation", a new framework to distill larger neural language models into smaller (commonsense) models, which leads to a machine-authored commonsense KB that wins, for the first time, over a human-authored KB in all criteria: scale, accuracy, and diversity. Moreover, I’ll demonstrate new, nonobvious applications of symbolic knowledge distillation --- (1) abstractive summarization and (2) generic knowledge induction, where the recurring theme is smaller models winning over models that are orders of magnitude larger. Next, I will highlight how we can make better lemonade out of neural language models by shifting our focus to unsupervised, inference-time reasoning algorithms. I will demonstrate how unsupervised models powered with algorithms can match or even outperform supervised approaches on hard reasoning tasks such as nonmonotonic reasoning (such as counterfactual and abductive reasoning), or complex language generation tasks that require logical constraints.
Bio: Yejin Choi is Brett Helsel professor at the Paul G. Allen School of Computer Science & Engineering at the University of Washington and also a senior research director at AI2 overseeing the project Mosaic. Her research investigates a wide variety problems across NLP and AI including commonsense knowledge and reasoning, neural language (de-)generation, language grounding with vision and experience, and AI for social good. She is a MacArthur Fellow and a co-recipient of the NAACL Best Paper Award in 2022, the ICML Outstanding Paper Award in 2022, the ACL Test of Time award in 2021, the CVPR Longuet-Higgins Prize (test of time award) in 2021, the NeurIPS Outstanding Paper Award in 2021, the AAAI Outstanding Paper Award in 2020, the Borg Early Career Award (BECA) in 2018, the inaugural Alexa Prize Challenge in 2017, IEEE AI's 10 to Watch in 2016, and the ICCV Marr Prize (best paper award) in 2013. She received her Ph.D. in Computer Science at Cornell University and BS in Computer Science and Engineering at Seoul National University in Korea.
Swabha Swayamdipta
Title: Generating Datasets for Robust GeneralizationAbstract: Datasets are the cornerstone of AI. Yet building new datasets for modern AI systems is both expensive as well as fraught with issues due to annotation biases, resulting in models that struggle with robustness and generalization. In this talk, I will argue that the generative capability of language models offers a promising path forward. I will present two algorithms for dataset generation. The first approach involves overgenerating ambiguous instances for the natural language inference task via GPT-3 prompting, followed by efficient filtering and finally, a cheaper human labeling process. The resulting dataset, WaNLI, greatly improves out-of-distribution generalization across multiple benchmarks, while avoiding known biases. Next, I will describe how we can generate counterfactuals for existing instances, using a smaller language model (GPT-2) and controlled generation. Augmenting the training data with such “NeuroCounterfactuals” results in improved out-of-domain generalization. Overall, both algorithms highlight that involving generative models in the data creation process can promote faster AI progress.
Bio: Swabha Swayamdipta is an Assistant Professor of Computer Science and a Gabilan Assistant Professor at the University of Southern California. Her research interests are in natural language processing and machine learning, with a primary interest in the estimation of dataset quality, the semi-automatic collection of impactful data, and evaluating how human biases affect dataset construction and model decisions. At USC, Swabha leads the Data, Interpretability, Language and Learning (DILL) Lab. She received her PhD from Carnegie Mellon University, followed by a postdoc at the Allen Institute for AI. Her work has received outstanding paper awards at ICML 2022, NeurIPS 2021 and an honorable mention for the best overall paper at ACL 2020.
Yanning Shen
Title: Fairness-aware Graph Neural Networks: Explainable Normalization and Attention DesignAbstract: We live in an era of big data and "small world", where a large amount of data resides on highly connected networks representing a wide range of physical, biological, and social interdependencies, e.g., social networks and smart grids. Learning from graph/network data is hence expected to bring significant science and engineering advances along with consequent improvements in quality of life. Node representation learning has demonstrated its effectiveness for various applications on graphs. Particularly, recent developments in graph neural networks and contrastive learning have led to promising results in node representation learning for a number of tasks, such as node classification and link prediction, as well as natural language processing. Despite the success of graph neural networks, fairness is largely under-explored in the field, which may lead to biased results toward underrepresented groups in the networks. To this end, this talk will first introduce novel fainess-enhancement techniques for Graph Neural Networks. Furthermore, theoretical analysis is provided to prove that the schemes can reduce intrinsic bias. Experimental results on real networks are presented to demonstrate that the proposed framework can enhance fairness while providing comparable accuracy to state-of-the-art alternative approaches for node classification and link prediction tasks.
Bio: Yanning Shen is an assistant professor with the EECS department at the University of California, Irvine. Her research interests span the areas of machine learning, network science, data science, and signal processing. She received her Ph.D. degree from the University of Minnesota (UMN) in 2019. She was a finalist for the Best Student Paper Award at the 2017 CAMSAP conference and the 2017 Asilomar Conference. She was selected as a Rising Star in EECS by Stanford University in 2017. She received the UMN Doctoral Dissertation Fellowship in 2018, the Microsoft Academic Grant Award for AI Research in 2021, the Google Research Scholar Award in 2022, and the Hellman Fellowship in 2022. She is also an honoree of the MIT Technology Review Innovators under 35 Asia Pacific in 2022.
Bolei Zhou
Title: Human-AI Interaction via Emergent LearningAbstract: AI has been deployed in many real-world applications, ranging from image recognition, content creation, to autonomous driving. While AI models achieve a high level of autonomy, it is difficult to establish a trustworthy relationship without the meaningful interactions between humans and AI. I will talk about utilizing the interpretable concepts emerged in the learned neural representations to facilitate human-AI interaction, with applications to interactive image generation, ML interpretability, and human-AI shared control.
Bio: Bolei Zhou is an Assistant Professor in the Computer Science Department at the University of California, Los Angeles (UCLA). He earned his Ph.D. from MIT in 2018. He has been a faculty member at The Chinese University of Hong Kong (CUHK) for the past 3 years. His research interest lies at the intersection of computer vision and machine autonomy, focusing on enabling interpretable and trustworthy human-AI interaction. He has developed many widely used interpretation methods such as CAM and Network Dissection, as well as computer vision benchmarks Places and ADE20K.
