SoCal ML & NLP Symposium 2021

University of California, San Diego | Mar 22-23, 2021

Welcome

The goal of the Southern California Machine Learning and Natural Language Processing Symposium is to gather researchers from the southern California region with broad expertise in machine learning and natural language processing. The symposium will provide a participatory environment where attendees from a variety of fields can share and discuss their latest findings.

This year SoCal ML and NLP will be hosted jointly as a two-day virtual conference.

Important Dates

SoCal ML Symposium: Mar 22, 2021
SoCal NLP Symposium: Mar 23, 2021
ML Submissions: Feb 16, 2021, 11:59 PM Anywhere On Earth [ML Submission Portal]
NLP Submission deadline: Feb 16, 2021, 11:59 PM Anywhere On Earth [NLP submission Portal]
Submission format: Approx 1-2 page submissions, Neurips style -- though longer submissions are also acceptable

Symposium Details

The planned 2020 SoCal ML & NLP Symposia were postponed as a result of the pandemic. We will instead host a (slightly off-cycle) virtual conference organized by researchers at UC San Diego. The joint SoCal ML & NLP Virtual Symposium will include invited talks from academia and industry, contributed work, poster presentations, and open discussion. We welcome all students, postdocs, and faculty members from universities in the region, including University of Southern California (USC), UC Los Angeles, UC Irvine, UC Santa Barbara, UC Riverside, Caltech, UC San Diego, and other schools to join us.

How to Attend

Each morning, before the corresponding conference day, we will post a single Zoom link here. Anyone with the link will be able to join. Poster sessions will be organized using breakout rooms. Each poster presenter will be assigned their own breakout room during their poster session. Conference attendees will be able to jump between the main session and poster breakout rooms as they please. Note: You must have Zoom 5.3.0 or later installed in order to self-select breakout rooms. Instructions for joining a breakout room: (1) Click the breakout rooms button at the bottom of your Zoom window. (2) Click on the join button shown in blue to the right of the breakout room you want to join. [On some operating systems, the join button will not appear and instead one must click on the listed number of participants in the desired room.]

Venue

Date: Mar 22-23, 2021

Location: Zoom! Check here for Zoom link on day of conference.

Parking information: You can park at home :D

Schedule

Mar 22 (ML day)

Mar 23 (NLP day)

Spotlights must be recorded by Mar 19 as an MP4 file and shared with the organizers. Please upload your video to the Google Drive folder linked in your paper acceptance email.

09:15am - 09:30am Opening remarks

Keynotes
09:30am - 10:30am Nanyun (Violet) Peng
10:30am - 11:30am William Wang
11:30am - 12:30pm Matt Gardner

12:30pm - 12:00pm Lunch

01:00pm - 01:50pm Social (Discussion around networking and career paths for under-represented minorities)
Panelists:
Ana Marasović: Ana Marasović (she/her) is a postdoctoral researcher at the University of Washington and at the AllenNLP team of the Allen Institute for Artificial Intelligence (AI2). Her interests broadly lie in the fields of natural language processing and explainable AI. She is currently working on developing and evaluating models with advanced reasoning abilities that provide readable explanations of their decision process. Dr. Marasović did her PhD in the Heidelberg University NLP Group where she worked on learning with limited labeled data for discourse-oriented tasks. Her advisor was Anette Frank. Prior to receiving her PhD in Janurary 2019, she completed B.Sc. (2013) and M.Sc. (2015) in Mathematics at the University of Zagreb.
Luca Soldaini: Luca Soldaini (he/him) is an applied scientist at Amazon Alexa Search in Manhattan Beach, California. His research efforts are currently focused on building ranking and generative models for natural language understanding tasks, such as open-domain question answering. Dr. Soldaini obtained a B.Eng. in Computer Engineering from the University of Florence in Italy, and he has a Ph.D. in Computer Science from Georgetown University. During his doctoral studies, he investigated natural language processing techniques to improve access to medical literature for both medical professionals and lay people.

Session 1 (paper ID 01-10)
02:00pm - 02:30pm Spotlights (3-min presentations)
02:30pm - 03:30pm Poster Session 1

Session 2 (paper ID 11-20)
03:30pm - 04:00pm Spotlights (3-min presentations)
04:00pm - 05:00pm Poster Session 2

Invited Speakers

Yizhou Sun

Title: Bringing Additional Symbolic Knowledge for Knowledge Graph Reasoning

Abstract: Knowledge graph has received tremendous attention recently, due to its wide applications, such as search engines and Q&A systems. Knowledge graph embedding, which aims at representing entities as low-dimensional vectors, and relations as operators on these vectors, has been widely studied and successfully applied to many tasks, such as knowledge completion. However, most of the existing knowledge graph embedding approaches treat knowledge graph as a complete, error-free, and flat data structure to store knowledge. In this talk, I will introduce two recent techniques developed in our lab to bring additional knowledge for better knowledge graph embedding. First, external knowledge represented as first-order logic is brought into knowledge graph embedding, which is able to address the uncertainty in knowledge graph and handle missing facts. Second, a unified embedding framework that incorporates ontological view KG into widely studied instance view KG will be introduced, which can seamlessly bring instance world and concept world together. Both techniques can significantly enhance the quality of KG embedding, on different downstream tasks, which also show a promising future direction in better knowledge graph reasoning.

Bio: Yizhou Sun is an associate professor at department of computer science of UCLA. She received her Ph.D. in Computer Science from the University of Illinois at Urbana-Champaign in 2012. Her principal research interest is on mining graphs/networks, and more generally in data mining, machine learning, and network science, with a focus on modeling novel problems and proposing scalable algorithms for large-scale, real-world applications. She is a pioneer researcher in mining heterogeneous information network, with a recent focus on deep learning on graphs/networks. Yizhou has over 100 publications in books, journals, and major conferences. Tutorials of her research have been given in many premier conferences. She received 2012 ACM SIGKDD Best Student Paper Award, 2013 ACM SIGKDD Doctoral Dissertation Award, 2013 Yahoo ACE (Academic Career Enhancement) Award, 2015 NSF CAREER Award, 2016 CS@ILLINOIS Distinguished Educator Award, 2018 Amazon Research Award, and 2019 Okawa Foundation Research Grant.

Matt Gardner

Title: Contrastive evaluation and learning in NLP

Abstract: Any dataset created by humans will almost unavoidably have spurious correlations between inputs and outputs. This means that when we collect data and split it into train and test sets, models that maximize the likelihood of the data will tend to find these spurious correlations, and they will use them to perform better than they should at test time. I will show that this problem is pervasive in natural language processing, extending even to traditional NLP tasks such as dependency parsing, and I will briefly demonstrate one method to partially solve this problem in our evaluations, by generalizing the long-standing notion of a "minimal pair". Solving the problem during training is more challenging. As a start, I will present work that leverages consistency on related examples during training to improve compositional reasoning in neural module networks. This is admittedly a very narrow solution to the problem, but it hints at how we might approach a more general solution.

Bio: Matt is a senior research scientist at the Allen Institute for AI on the AllenNLP team. His research focuses primarily on getting computers to read and answer questions, dealing both with open domain reading comprehension and with understanding question semantics in terms of some formal grounding (semantic parsing). He is particularly interested in cases where these two problems intersect, doing some kind of reasoning over open domain text. He is the original architect of the AllenNLP toolkit, and the instigator of the NLP Highlights podcast.

Yuandong Tian

Title: Understanding and Employing Learned Representation in Supervised, Self-supervised Learning and Decision-Making Process

Abstract: How to learn good latent representations is an important topic in the modern era of machine learning. Deep models excel because from raw data they learn a good representation on which the tasks become easier. Understanding the learned representations leads to strong model interpretability and better algorithms, and using a good representation makes the decision-making process more efficient. In this talk, I will cover our recent works on understanding representation in supervised and self-supervised learning by opening the black-box of the deep network, as well as strong performance in black-box optimization, if a good task-specific representation is learned during the optimization process.

Bio: Yuandong Tian is a Research Scientist and Manager in Facebook AI Research, working on deep reinforcement learning and representation learning. He is the lead scientist and engineer for ELF OpenGo and DarkForest Go projects. Prior to that, he was in Google Self-driving Car team in 2013-2014. He received a Ph.D in Robotics Institute, Carnegie Mellon University in 2013. He is the recipient of 2013 ICCV Marr Prize Honorable Mentions.

Nanyun (Violet) Peng

Title: Controllable Text Generation Beyond Auto-regressive Models

Abstract: Recent advances in large auto-regressive language models have demonstrated appealing results in generating natural languages and significantly improved the performances for applications such as machine translation and summarization. However, when the generation tasks are open-ended and the content is under-specified, existing techniques struggle to generalize to novel scenarios and generate long-term coherent and creative content. This happens because the generation models are trained to capture the surface patterns (i.e. sequences of words) following the left-to-right order, instead of capturing underlying semantics and discourse structures. In this talk, I will present our recent works on controllable text generation that go beyond the prevalent auto-regressive formulation. We explore hierarchical generation and insertion-based generation, with applications to creative story generation and image captioning.

Bio: Nanyun (Violet) Peng is an Assistant Professor of Computer Science at the University of California, Los Angeles. Prior to that, she spent three years at the University of Southern California's Information Sciences Institute as an Assistant Research Professor. She received her Ph.D. in Computer Science from Johns Hopkins University, Center for Language and Speech Processing. Her research focuses on the robustness and generalizability of NLP models, with applications to creative language generation and low-resource information extraction.

William Wang

Title: Learning to Reason with Text and Tables

Abstract: A key challenge for Artificial Intelligence is to design intelligent agents that can reason with heterogeneous representations. In this talk, I will describe our recent work on teaching machines to reason in semi-structured tables and unstructured text data. More specifically, I will introduce: (1) TabFact, a large benchmark dataset for table-based fact-checking; (2) HybridQA and OTT-QA, multi-hop question answering frameworks on tables and text; (3) How one can utilize TabFact to facilitate logical natural language generation with LogicNLG. I will also describe some other work at UCSB's NLP Group on learning to reason with multiple modalities.

Bio: William Wang is the Duncan and Suzanne Mellichamp Chair in Artificial Intelligence and Designs, and an Assistant Professor in the Department of Computer Science at the University of California, Santa Barbara. He is Director of UC Santa Barbara's Natural Language Processing group, and Center for Responsible Machine Learning. He received his PhD from Carnegie Mellon University. He has broad interests in machine learning and natural language processing, including statistical relational learning, information extraction, computational social science, and vision. He has published more than 100 papers at leading NLP/AI/ML/Vision conferences and journals, and received best paper awards (or nominations) at ASRU 2013, CIKM 2013, EMNLP 2015, and CVPR 2019, a DARPA Young Faculty Award (Class of 2018), IEEE Intelligent Systems AI's 10 to Watch (2020), an NSF CAREER Award (2021), and other faculty research awards from Google, Facebook, IBM, Amazon, JP Morgan Chase, Adobe, and Intel. His work and opinions appear at major tech media outlets such as Wired, VICE, Scientific American, Fortune, Fast Company, NPR, etc.

Phebe Vayanos

Title: Integer optimization for predictive and prescriptive analytics in high stakes domains

Abstract: Data-driven predictive and prescriptive analytics tools are increasingly being used to assist decision-making in high stakes domains (e.g., to prioritize people experiencing homelessness for scarce housing resources, to identify individuals at risk of suicide, and to design public health interventions). The deployment of such algorithms in these domains that can impact people’s lives and societal outcomes creates an urgent need for algorithms that are fair and interpretable and that leverage the available data to its full extent to yield the most accurate decisions. In this presentation, we discuss our recent works that leverage tools from integer optimization and causal inference to design optimal, interpretable, and fair decision-support tools that are suitable to deploy in high stakes settings.

Bio: Phebe Vayanos is an Assistant Professor of Industrial & Systems Engineering and Computer Science at the University of Southern California. She is also an Associate Director of CAIS, the Center for Artificial Intelligence in Society, an interdisciplinary research initiative between the schools of Engineering and Social Work at USC. Her research is focused on Operations Research and Artificial Intelligence and in particular on optimization and machine learning. Her work is motivated by problems that are important for social good, such as those arising in public housing allocation, public health, and biodiversity conservation. Prior to joining USC, she was lecturer in the Operations Research and Statistics Group at the MIT Sloan School of Management, and a postdoctoral research associate in the Operations Research Center at MIT. She holds a PhD degree in Operations Research and an MEng degree in Electrical & Electronic Engineering, both from Imperial College London. She served as a member of the ad hoc INFORMS AI Strategy Advisory Committee and is an elected member of the Committee on Stochastic Programming (COSP). She is a recipient of the INFORMS Diversity, Equity, and Inclusion Ambassador Program Award.

Accepted Work

ML Day: Poster Session #1

A Dynamic Factor Approach To Estimate Animal Spirits and Policy Impact On The COVID-19 Stock Market
Diana Joy Xiuyao Yang
GPT-GNN: Generative Pre-Training of Graph Neural Networks
Ziniu Hu
Connecting Interpretability and Robustness in Decision Trees through Separation
Michal Moshkovitz, Yao-Yuan Yang and Kamalika Chaudhuri
A Closer Look at Accuracy vs. Robustness
Yao-Yuan Yang, Cyrus Rashtchian, Hongyang Zhang, Russ Salakhutdinov and Kamalika Chaudhuri
Close Category Generalization for Out-of-Distribution Classification
Yao-Yuan Yang, Cyrus Rashtchian, Russ Salakhutdinov and Kamalika Chaudhuri
Local Inferential Privacy through Data Shuffling
Casey Meehan, Kamalika Chaudhuri, Amrita Roy-Chowdhury and Somesh Jha
Explainable k-Means Clustering: Theory and Practice
Michal Moshkovitz, Cyrus Rashtchian, Nave Frost and Sanjoy Dasgupta
Unexpected Effects of Online no-Substitution k-means Clustering
Michal Moshkovitz
GPU-Accelerated Primal Learning for Extremely Fast Large-Scale Classification
John Halloran
LDLE: Low Distortion Local Eigenmaps
Dhruv Kohli, Alexander Cloninger and Gal Mishne

ML Day: Poster Session #2

Variational Beam Search for Learning with Distribution Shifts
Aodong Li, Alex Boyd, Padhraic Smyth and Stephan Mandt
Online Adversarial Purification based on Self-supervised Learning
Changhao Shi, Chester Holtz and Gal Mishne
Data Profiling for Adversarial Training: On the Ruin of Problematic Data
Chengyu Dong, Liyuan Liu and Jingbo Shang
On Heuristic and Unstable Fairness Surrogates
Harry Bendekgey and Erik Sudderth
Takenout
N/A
AReN: Assured ReLU NN Architecture for Model Predictive Control of LTI Systems
James Ferlez and Yasser Shoukry
An Integer Linear Programming Framework for Mining Constraints from Data
Tao Meng and Kai-Wei Chang
Takenout
N/A
Bounding the Performance of Human-Machine Classifier Ensembles
Gavin Kerrigan, Disi Ji, Padhraic Smyth and Mark Steyvers
Hierarchical Autoregressive Modeling for Neural Video Compression
Ruihan Yang, Yibo Yang, Joseph Marino and Stephan Mandt

ML Day: Poster Session #3

Successive Subspace Learning: An Overview
Mozhdeh Rouhsedaghat, Masoud Monajatipoor, Zohreh Azizi and C.-C. Jay Kuo
Learning Consistent Deep Generative Models from Sparsely Labeled Data via Prediction Constraints
Gabriel Hope, Madina Abdrakhmanova, Xiaoyin Chen, Michael Hughes and Erik Sudderth
Risk Neutral Density Estimation with a Functional Linear Model
Idriss Tsafack and Marine Carrasco
Takenout
N/A
Provable Generalization of SGD-trained Neural Networks of Any Width in the Presence of Adversarial Label Noise
Spencer Frei, Yuan Cao and Quanquan Gu
Dynamic Social Media Monitoring for Fast-Evolving Online Discussions
Maya Srikanth, Anqi Liu, Nicholas Adams-Cohen, Jian Cao, Michael Alvarez and Animashree Anandkumar
No-Substitution $k$-means Clustering with Low Center Complexity and Memory
Robi Bhattacharjee and Jacob Imola
Consistent Non-Parametric Methods for Adaptive Robustness
Robi Bhattacharjee and Kamalika Chaudhuri
Sample Complexity of Robust Linear Classification on Separated Data
Robi Bhattacharjee, Kamalika Chaudhuri and Somesh Jha
Discrete Variational Optimization with Stochastic Gumbel Annealing
Yibo Yang, Robert Bamler and Stephan Mandt

NLP Day: Poster Session #1

Problems with Flesch-Kincaid as a Text Simplification Evaluation Metric
Teerapaun Tanprasert and David Kauchak.
LOGAN: Local Group Bias Detection by Clustering
Jieyu Zhao and Kai-Wei Chang
Generating Syntactically Controlled Paraphrases without Using Annotated Parallel Pairs
Kuan-Hao Huang and Kai-Wei Chang
Medical Question Understanding and Answering for Older Adults
Khalil Mrini, Chen Chen, Ndapa Nakashole, Nadir Weibel and Emilia Farcas
Nice Try, Kiddo: Ad Hominems in Dialogue Systems
Emily Sheng, Kai-Wei Chang, Prem Natarajan and Nanyun Peng
Unsupervised Vision-and-Language Representation Pre-training Without Parallel Images and Captions
Liunian Harold Li, Haoxuan You, Zhecan Wang, Alireza Zareian, Shih-Fu Chang and Kai-Wei Chang
COVIDLies: Detecting COVID-19 Misinformation on Social Media
Tamanna Hossain, Robert Logan IV, Arjuna Ugarte, Yoshitomo Matsubara, Sean Young and Sameer Singh
InsNet: Efficient Training of Insertion-based Generators
Sidi Lu and Nanyun Peng
SentiBERT: A Transferable Transformer-Based Architecture for Compositional Sentiment Semantics
Da Yin, Tao Meng and Kai-Wei Chang

NLP Day: Poster Session #2

Building a Motivated Deception Corpus: Improving the Quality of Deceptive Text Through Gaming
Dan Barsever and Emre Neftci
X-class Text Classification with Extremely Weak Supervision
Zihan Wang, Dheeraj Mekala and Jingbo Shang
Evaluating Source Values in Transfer Learning
Md Rizwan Parvez and Kai-Wei Chang
Studying Entity Ambiguity in Open Domain Question Answering
Anthony Chen, Pallavi Gudipati, Shayne Longpre, Xiao Ling and Sameer Singh
Memorization of Conspiracy Theories in Language Models
Sharon Levy, Michael Saxon and William Yang Wang
An Effective Vision-and-Language Model for Chest X-Ray Disease Diagnosis
Masoud Monajatipoor, Mozhdeh Rouhsedaghat, Haoxin Zheng, Liunian Harold Li, Fabien Scalzo and Kai-Wei Chang
Modeling Disclosive Transparency with GPT-2
Michael Saxon, Sharon Levy, Xinyi Wang, Alon Albalak and William Wang
Plot-guided Adversarial Example Construction for Evaluating Open-domain Story Generation
Sarik Ghazarian, Zixi Liu, Akash Sm, Ralph Weischedel, Aram Galstyan and Nanyun Peng
MELINDA: A Multimodal Dataset for Biomedical Experiment Method Classification
Te-Lin Wu, Shikhar Singh, Sayan Paul, Gully Burns and Nanyun Peng
AUTOPROMPT: Eliciting Knowledge from Language Models with Automatically Generated Prompts
Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace and Sameer Singh